Opened 13 years ago

Last modified 8 years ago

#327 new Enhancement

Proposal of a new auto bad-block detection

Reported by: erdgeist Owned by:
Priority: Normal Milestone: None Set
Component: libtransmission Version: 0.80
Severity: Normal Keywords: patch-needed hash sha1
Cc:

Description

The source file attached allows three new features to be implemented into libtransmission:

1) putting blame on a per-block basis for each corrupt piece 2) assembling pieces from multiple corrupt pieces as long as each block appears correctly in at least one piece 3) redownloading random blocks for a failed piece

These features are achieved by building hash trees on multiple versions of pieces. If, while linearly scanning through all blocks, the multi_sha1 function encounters two or more different versions of one block, it copies the sha1 state and calculates each hash independently.

Feature 1) Since it knows, which path any given hash took through different versions of a piece, it can exactly point to corrupt blocks in each version (provided it finds a path). If blocks are associated with peer ids, you can put the blame on exactly the right culprit not on everyone contributing to the piece.

Feature 2) On torrents with lots of fakers trying to disturb distribution it is sufficient to receive one correct version of each block in order to reassemble a complete piece just by following the path, sha1 took through them.

Feature 3) If hash-failed pieces are not deleted but copied and blocks are redownloaded over the copy, an on-the-fly check can be introduced, that re-checks after each new block.

multi_sha1 expects all pieces to be on a word boundary and was tested on a big and on one little endian machine (FreeBSD x86 and MacOS X, ppc). It is licensed under a beerware license, which pratically puts it into public domain and should be compatible to any version you want to use it under.

Attachments (1)

multi_sha1.c (12.1 KB) - added by erdgeist 13 years ago.

Download all attachments as: .zip

Change History (11)

Changed 13 years ago by erdgeist

comment:1 Changed 13 years ago by erdgeist

  • Type changed from Bug Report to Enhancement

comment:2 Changed 13 years ago by jah

this would probably also fix ticket 238.

comment:3 Changed 13 years ago by erdgeist

  • Summary changed from Propsal of a new auto bad-block detection to Proposal of a new auto bad-block detection

comment:4 Changed 11 years ago by charles

  • Keywords patch-needed added

comment:5 Changed 10 years ago by charles

Well it's been three years now. As much as I like the idea of this, it's not clear to me how useful this would be in practice. There doesn't seem to be a strong need for it so far.

erdgeist, if you want to submit a full patch to integrate this into Transmission I would be very very likely to use it. But it's not a feature that I feel strongly enough to write & debug myself.

comment:6 Changed 10 years ago by x190

We do need this feature IMO, particularly if it helps get rid of the bad guys faster.

comment:7 follow-up: Changed 10 years ago by charles

x190: you've +1'ed several nontrivial tickets this morning. Are you volunteering to submit patches for any of them? :)

comment:8 in reply to: ↑ 7 Changed 10 years ago by x190

Replying to charles:

x190: you've +1'ed several nontrivial tickets this morning. Are you volunteering to submit patches for any of them? :)

I was under the impression that the more interest that was shown in a ticket the more attention it might get from the developers. *confused*

comment:9 Changed 10 years ago by charles

You're right, voting on tickets does help their odds of success.

However this ticket's had two votes in three years, which makes me wonder how useful it really would be in practice. Also the overhead for this patch seems very high IMO, which is why I wanted to talk erdgeist into doing it. ;)

comment:10 Changed 8 years ago by x190

https://trac.transmissionbt.com/ticket/5076

Patch still needed.

"Feature 1) Since it knows, which path any given hash took through different versions of a piece, it can exactly point to corrupt blocks in each version (provided it finds a path). If blocks are associated with peer ids, you can put the blame on exactly the right culprit not on everyone contributing to the piece."

Note: See TracTickets for help on using tickets.