Opened 11 years ago

Closed 7 years ago

Last modified 7 years ago

#4005 closed Enhancement (wontfix)

Support torrents whose piece size is not a power of 2

Reported by: jordan Owned by: jordan
Priority: Normal Milestone: Sometime
Component: libtransmission Version: 2.20
Severity: Normal Keywords:
Cc:

Description

It's rare for a piece size to not be a power of 2, but it happens. Transmission currently doesn't support it because it currently requires a piece size that's cleanly divisible by some uniform block size.

Attachments (1)

4005TestTorrents.zip (4.8 KB) - added by cfpp2p 7 years ago.

Download all attachments as: .zip

Change History (37)

comment:1 Changed 9 years ago by MechMK1

Is it specified in the BitTorrent? protocol that piece size has to be a power of 2? If not, those torrents are simply broken.

comment:2 Changed 7 years ago by mike.dld

TPB statistics (3,495,213 torrent files) on torrents with piece size not power of two:

Creator Count
Unknown (no "created by" field) 136
Azureus 2300
AzureusBitTryant 2
Bitflu 11
BitsOnWheels 5
BitSpirit 153
BTARENA.org 4
createtorrent 1
DiHaTi 2
Enhanced-CTorrent 8
FlashGet 8
Halite 1
Hermes 1
http://pornoshara.tv 2
m 1
py3createtorrent 22
qBittorrent 2
qMakeTorrent 32
RHash 2
ruTorrent (PHP Class - Adrien Gibrat) 1
ShAQ 2
thepiratebay.org/user/-jonny- 96
TNT 1
TorrentBitch/libtorrent 4
Torrent PHP Class - Adrien Gibrat 1
Transmission 3
TX 1
uTorrent 2
www.btarena.com 1
Total: 2807

and

Piece size /16K Count
916 0.056 1
2961 0.181 1
4198 0.256 1
5120 0.312 1
7123 0.435 1
13409 0.818 1
21107 1.288 1
21504 1.312 22
26144 1.596 3
26857 1.639 1
31352 1.914 1
31814 1.942 1
49152 3.000 311
81920 5.000 1
98304 6.000 436
99476 6.072 1
108855 6.644 1
196608 12.000 393
230582 14.074 2
327680 20.000 2
360286 21.990 6
393216 24.000 399
450358 27.488 6
458752 28.000 8
507039 30.947 1
543008 33.143 1
546272 33.342 1
548160 33.457 1
562948 34.360 2
635790 38.806 1
637318 38.899 1
703686 42.950 123
786432 48.000 545
873821 53.334 1
975261 59.525 1
1024000 62.500 18
1024768 62.547 1
1099510 67.109 31
1179666 72.001 9
1233338 75.277 1
1374388 83.886 9
1572864 96.000 180
1717986 104.858 2
1966080 120.000 1
3145728 192.000 245
3801189 232.006 23
10132107 618.415 1
16771216 1023.634 1
23216671 1417.033 1
131072000 8000.000 3
536871936 32768.062 1
1842799616 112475.562 1

comment:3 Changed 7 years ago by jordan

  • Resolution set to wontfix
  • Status changed from new to closed

mike.dld, I'm in awe of your stats-collecting skills.

comment:4 follow-up: Changed 7 years ago by cfpp2p

  • Resolution wontfix deleted
  • Status changed from closed to reopened

ticket:5734#comment:3

Support torrents whose piece size is not a power of 2

This is partially true, so should this ticket be labeled 'won't fix', 'already fixed' or 'not thoroughly tested'.

Light testing indicates any piece size is OK at or below 16383, or any of mike.dld's comment:2 bold multiples of 16K might all work. *16383*, *49152* 3.000 311 and *131072000* 8000.000 3 accepted, downloaded and/or seeded correctly.

Last edited 7 years ago by cfpp2p (previous) (diff)

comment:5 follow-up: Changed 7 years ago by cfpp2p

There seems to be some rather unusual behaviors I wouldn't want someone to instigate for piece sizes greater than or equal to 4GB. Something like this stops it.

metainfo.c line 531

if (!tr_variantDictFindInt (infoDict, TR_KEY_piece_length, &i)
    || (i < 1) || ( i >= 4294967296))

comment:6 in reply to: ↑ 5 Changed 7 years ago by cfpp2p

Replying to myself:

There seems to be some rather unusual behaviors...

The problem arises when the type conversion of i ( int64_t i ) is downcast to pieceSize ( uint32_t pieceSize )

line 533 inf->pieceSize = i;

The result for integers greater than 4gb is that piece size will be set to zero for many values of i. Then transmission will crash at line 560.

if ((uint64_t) inf->pieceCount != (inf->totalSize + inf->pieceSize - 1) / inf->pieceSize)

Otherwise transmission will continue on and proceed with the improperly overflowed piece size following the rules of ticket:5734#comment:3

Testing confirms this.

comment:7 in reply to: ↑ 4 ; follow-ups: Changed 7 years ago by jordan

Replying to cfpp2p:

ticket:5734#comment:3

Resolution wontfix deleted Status changed from closed to reopened

Support torrents whose piece size is not a power of 2

This is partially true, so should this ticket be labeled 'won't fix', 'already fixed' or 'not thoroughly tested'.

... That was its resolution until you reopened the ticket, so I'm confused what you're requesting here? :)

ticket:5734#comment:5

There seems to be some rather unusual behaviors I wouldn't want someone to instigate for piece sizes greater than or equal to 4GB

Could you file a separate ticket for this, please? This deserves to get fixed but it isn't about the powers-of-two question.

comment:8 in reply to: ↑ 7 Changed 7 years ago by cfpp2p

Replying to jordan:

Could you file a separate ticket for this, please? This deserves to get fixed but it isn't about the powers-of-two question.

ticket:5736

Does this look OK?

comment:9 in reply to: ↑ 7 ; follow-up: Changed 7 years ago by cfpp2p

Replying to jordan:

... That was its resolution until you reopened the ticket, so I'm confused what you're requesting here? :)

Non powers of two is partially supported so I guess this ticket could be closed as won't fix.

comment:10 in reply to: ↑ 9 Changed 7 years ago by x190

Replying to cfpp2p:

Replying to jordan:

... That was its resolution until you reopened the ticket, so I'm confused what you're requesting here? :)

Non powers of two is partially supported so I guess this ticket could be closed as won't fix.

Well then, wouldn't it be more like "partially invalid" or "sometimes worksforme"? *lol*

See also #5734.

Last edited 7 years ago by x190 (previous) (diff)

comment:11 follow-up: Changed 7 years ago by suitessay

I've been seeing a recent uptick in the number of torrents encountered with piece lengths that aren't multiples of 16K. They fail to open in Transmission, with an invalid data error.

Some examples: piece lengthi423792e piece lengthi333744e piece lengthi330192e

comment:12 Changed 7 years ago by pathetic_loser

Non powers of two is partially supported so I guess this ticket could be closed as won't fix.

Partially? Hmm, if some program "partially works", its a bug, isn't it? And I do not get why someone thinks pieces should be multiple of 16K. Are there any requirements in standards about piece size to be multiple of 16K?

comment:13 Changed 7 years ago by MechMK1

As far as I can tell, BitTorrent? specification does not require piece size to be a multiple of 16k.

For the sake of the argument, we should support these torrents. However, I'm not sure how much time this task would take and if it's not better to use this somewhere else better

comment:14 Changed 7 years ago by mike.dld

IsoHunt statistics (10,226,795 torrent files) on torrents with piece size not power of two:

Creator Count
Unknown (no "created by" field) 2916
Area51 17
Azureus 5736
AzureusBitTyrant 22
BitComet 4
Bitflu 7
BitsOnWheels 5
BitSpirit 192
Blunderthorm 1
BTARENA.org 5
buhaypirata.net 1
buildtorrent 23
ceasers-palace.info 1
[CFBTAutoMake] 1
Coiso 1
createtorrent 1
DiHaTi 24
E-Hentai Galleries feat EHTracker 260
Emerald-Speeds 224
Enhanced-CTorrent 2
FlashGet 51
GGRREEKK (hdreactor.org) 5
Halite 13
Hermes 1
http://myto.us.to 13
http://pornoshara.tv 1
Jonny 1
Lith 1
MakeTorrent 4
mazuki 8
ME 1
Mononoke BT 1
Movie Torrent 1
Nyaa 2
\( -o _ o- )/ 4
potvin 1
qBittorrent 1
qemist.info 3
qMakeTorrent 5686
RHash 3
ruTorrent (PHP Class - Adrien Gibrat) 1
SEEDPEER.COM 58
Sturmtruppen-bt 1
thepiratebay.org/user/-jonny- 80
TheGuardian [torrents-heaven.info] 1
TNT 2009 2
TorrentBitch 5
Torrent-Outpost.com 2
Torrent PHP Class - Adrien Gibrat 342
Torrent RW PHP Class 1
Transmission 1
uTorrent 48
Vip Torrent 1
www.JAM-HOT.com 1
WWW.MEGANOVA.ORG 4
xxxxxxxxxxxxxxx 1
Total: 15793

and (sorry, the table is really huge)

Piece size /16K Count
916 0.056 1
1599 0.098 1
2111 0.129 1
2961 0.181 1
5120 0.312 1
7123 0.435 1
13409 0.818 1
16400 1.001 32
16496 1.007 1
16528 1.009 1
16736 1.021 2
16784 1.024 1
16976 1.036 1
17312 1.057 1
17504 1.068 1
17808 1.087 1
17840 1.089 1
17872 1.091 1
17920 1.094 1
18592 1.135 1
18880 1.152 1
19408 1.185 1
20112 1.228 1
20144 1.229 1
20448 1.248 1
21107 1.288 1
21504 1.312 1
21824 1.332 1
21968 1.341 1
22048 1.346 1
22304 1.361 1
22320 1.362 1
22640 1.382 1
22672 1.384 1
23232 1.418 1
23424 1.430 1
23728 1.448 1
24448 1.492 1
25296 1.544 1
25424 1.552 1
25472 1.555 1
25648 1.565 1
25792 1.574 1
25888 1.580 1
26096 1.593 1
26144 1.596 896
26288 1.604 1
26576 1.622 1
26857 1.639 1
27216 1.661 1
28144 1.718 1
28752 1.755 1
28864 1.762 1
29840 1.821 1
30016 1.832 1
30208 1.844 1
30896 1.886 2
30928 1.888 1
31048 1.895 1
31184 1.903 1
31352 1.914 1
31360 1.914 1
31680 1.934 1
31814 1.942 1
32496 1.983 1
32560 1.987 1
33312 2.033 1
33584 2.050 1
33840 2.065 1
34192 2.087 1
34496 2.105 1
34624 2.113 1
35552 2.170 1
35648 2.176 1
36416 2.223 1
36560 2.231 1
36928 2.254 1
37232 2.272 1
37488 2.288 1
37600 2.295 1
38080 2.324 1
38320 2.339 1
38496 2.350 1
38976 2.379 1
39680 2.422 1
40464 2.470 1
42112 2.570 1
42176 2.574 1
42320 2.583 1
42816 2.613 1
43392 2.648 1
43632 2.663 1
44560 2.720 2
44928 2.742 1
46288 2.825 1
46576 2.843 2
47312 2.888 1
47632 2.907 1
48480 2.959 1
48880 2.983 1
49152 3.000 708
51200 3.125 1
51216 3.126 1
51264 3.129 1
51488 3.143 1
51632 3.151 1
52832 3.225 1
54240 3.311 1
54464 3.324 1
55520 3.389 1
55824 3.407 1
55840 3.408 1
57200 3.491 1
57344 3.500 1
57728 3.523 1
58048 3.543 1
58208 3.553 1
58576 3.575 1
58800 3.589 1
59248 3.616 1
59280 3.618 1
59504 3.632 2
59984 3.661 1
60352 3.684 1
61424 3.749 1
61680 3.765 1
63040 3.848 1
63952 3.903 1
64128 3.914 1
64640 3.945 1
64688 3.948 1
64800 3.955 1
65344 3.988 1
65520 3.999 1
65535 4.000 1
69152 4.221 1
69392 4.235 1
69632 4.250 1
69696 4.254 1
69920 4.268 1
70928 4.329 1
72464 4.423 1
74320 4.536 1
75072 4.582 1
75264 4.594 1
76976 4.698 1
77184 4.711 1
78112 4.768 1
78624 4.799 1
78880 4.814 1
78928 4.817 1
79072 4.826 1
79680 4.863 1
80032 4.885 1
80912 4.938 1
81920 5.000 3
83248 5.081 1
83696 5.108 1
84592 5.163 1
84960 5.186 1
85936 5.245 1
86016 5.250 1
88864 5.424 1
89984 5.492 1
93600 5.713 1
93744 5.722 1
95152 5.808 1
97456 5.948 1
97472 5.949 1
97760 5.967 1
98304 6.000 1156
98608 6.019 1
98736 6.026 1
99476 6.072 1
101296 6.183 1
104400 6.372 1
105872 6.462 1
106496 6.500 1
108112 6.599 1
108855 6.644 1
110992 6.774 1
113504 6.928 1
116976 7.140 1
118864 7.255 1
118896 7.257 1
121792 7.434 1
122496 7.477 1
122528 7.479 1
123392 7.531 1
129152 7.883 1
129472 7.902 1
130912 7.990 1
131440 8.022 1
132912 8.112 1
134304 8.197 1
136960 8.359 1
145344 8.871 1
145408 8.875 1
151008 9.217 1
153600 9.375 1
159744 9.750 1
163088 9.954 1
163344 9.970 1
165072 10.075 1
171488 10.467 1
172592 10.534 1
173376 10.582 1
175056 10.685 1
175744 10.727 1
177408 10.828 1
179312 10.944 1
186800 11.401 1
192512 11.750 1
196608 12.000 1290
199456 12.174 1
201760 12.314 1
203648 12.430 1
205920 12.568 1
216016 13.185 1
216224 13.197 1
216480 13.213 1
220592 13.464 1
222480 13.579 1
225392 13.757 1
230582 14.074 82
231104 14.105 1
231232 14.113 1
237872 14.519 1
245760 15.000 1
247808 15.125 1
248576 15.172 1
270544 16.513 1
274432 16.750 1
292864 17.875 1
310416 18.946 1
316480 19.316 1
327680 20.000 11
328416 20.045 1
350208 21.375 1
358608 21.888 1
360286 21.990 19
360448 22.000 1
362496 22.125 1
376832 23.000 1
383504 23.407 1
393216 24.000 1792
395552 24.143 1
396432 24.196 1
405504 24.750 1
415280 25.347 1
428032 26.125 1
432704 26.410 1
450358 27.488 9
458752 28.000 13
475136 29.000 1
489472 29.875 1
507904 31.000 3
517312 31.574 1
524281 32.000 1
528832 32.277 1
532000 32.471 1
562948 34.360 3
579202 35.352 1
635790 38.806 1
641280 39.141 1
680464 41.532 1
681984 41.625 21
688128 42.000 2
703686 42.950 239
737280 45.000 1
757312 46.223 1
759824 46.376 1
786432 48.000 1096
807504 49.286 1
819200 50.000 2
873821 53.334 1
904792 55.224 1
966656 59.000 1
1024000 62.500 1210
1024768 62.547 1
1048288 63.982 1
1048575 64.000 1
1097152 66.965 1
1099510 67.109 75
1179666 72.001 1181
1194304 72.895 3
1233338 75.277 1
1294336 79.000 1
1310738 80.001 73
1374388 83.886 10
1441792 88.000 16
1500000 91.553 2
1523696 92.999 1
1554064 94.853 1
1572864 96.000 446
1717986 104.858 1
1728512 105.500 1
1759120 107.368 2
1966080 120.000 8
2048000 125.000 20
2048576 125.035 1
2072576 126.500 1
2097197 128.003 8
2097271 128.007 10
2490368 152.000 7
2949120 180.000 1
3145728 192.000 642
3342387 204.003 2
3801189 232.006 4068
3801192 232.006 7
3891200 237.500 1
4250000 259.399 1
5120000 312.500 1
6357063 388.004 3
6357087 388.006 4
6357106 388.007 22
6488161 396.006 26
6619250 404.007 4
7077993 432.006 48
7209061 440.006 30
7274611 444.007 13
7274612 444.007 11
7471150 456.003 51
7471215 456.007 51
7602293 464.007 22
7852528 479.280 1
10132107 618.415 1
10485760 640.000 2
13836824 844.533 28
14350016 875.855 23
23216671 1417.033 1
100000000 6103.516 2
103809024 6336.000 1
629769216 38438.062 1
2147483647 131072.000 1
Last edited 7 years ago by mike.dld (previous) (diff)

comment:15 Changed 7 years ago by x190

This is the test:

uint32_t
tr_getBlockSize (uint32_t pieceSize)
{
  uint32_t b = pieceSize;

  while (b > MAX_BLOCK_SIZE)
    b /= 2u;

  if (!b || (pieceSize % b)) /* not cleanly divisible */
    return 0;

  return b;
}
Still, it means we are invalidating several thousand torrents.
e.g. "3801189	232.006	4068"
3801189/2 = 1900594.5 Invalid!

See also https://trac.transmissionbt.com/ticket/5734#comment:8

Last edited 7 years ago by x190 (previous) (diff)

comment:16 follow-ups: Changed 7 years ago by cfpp2p

It doesn't seem to be holding true that all piece sizes cleanly divisible by 16384 will be valid. The names of the test torrents indicate pieceSize / 16384} with no remainder. r14324

32768-OK.torrent 32769-OK.torrent 32852-OK.torrent 35474-INVALID.torrent 35477-INVALID.torrent 35479-INVALID.torrent 35480-OK.torrent

Changed 7 years ago by cfpp2p

comment:17 in reply to: ↑ 16 Changed 7 years ago by x190

Replying to cfpp2p:

It doesn't seem to be holding true that all piece sizes cleanly divisible by 16384 will be valid.

From ticket summary: "...piece size that's cleanly divisible by some uniform block size." 16384 has nothing to do with it, other than being the max block size of choice, nor does being a power of 2, although that is recommended by the protocol. The test is /2 without a remainder until at or below MAX_BLOCK_SIZE.

Last edited 7 years ago by x190 (previous) (diff)

comment:18 Changed 7 years ago by cfpp2p

I was thinking back to my comment:4 statement where I said "multiples of 16K might all work" which I now know that by looking at the math loop and following it through manually, you can produce torrents that will be invalid.

Last edited 7 years ago by cfpp2p (previous) (diff)

comment:19 Changed 7 years ago by x190

"32769-OK.torrent" cfpp2p, what gives? 32769/16384 = 2.00006103515625 and therefore should fail:

if (!b || (pieceSize % b)) /* not cleanly divisible */
    return 0;

Is that due to all the decimal places? Will this cause issues elsewhere in the code?

Last edited 7 years ago by x190 (previous) (diff)

comment:20 follow-up: Changed 7 years ago by cfpp2p

The names of the test torrents indicate pieceSize DIVIDED-BY 16384} with no remainder.

So 32769 X 16384 = 536887296 is the pieceSize for the 32769-OK.torrent

Lets try the 35474-INVALID.torrent

35474 X 16384 = 581206016

Now start diving by 2 until we reach below 16384

In the final iteration before we get below 16384 we'll have 17737 17737 / 2 = 8868.5 results in remainder once below 16384 so invalid

I'm interested in what is the lowest multiple of 16384 that will result in an invalid torrent. There's going to be lots of blocks of invalid torrent pieceSizes interspersed with lots of valid ones. Someone could chart it out and it would be interesting.

Last edited 7 years ago by cfpp2p (previous) (diff)

comment:21 in reply to: ↑ 20 Changed 7 years ago by x190

Replying to cfpp2p:

The names of the test torrents indicate pieceSize DIVIDED-BY 16384} with no remainder.

So 32769 X 16384 = 536887296 is the pieceSize for the 32769-OK.torrent

mea culpa!

comment:22 Changed 7 years ago by cfpp2p

mea culpa!

It's an interesting situation... I've updated comment:20

Last edited 7 years ago by cfpp2p (previous) (diff)

comment:23 follow-up: Changed 7 years ago by x190

35480-OK.torrent is consistently crashing r14324 when asked to verify data included. Add torrent, move data to Desktop, verify data, CRASH!

Exception Type:  EXC_ARITHMETIC (SIGFPE)
Exception Codes: EXC_I386_DIV (divide by zero)
Crashed Thread:  4

Thread 4 Crashed:
0   org.m0k.transmission          	0x0000000100077942 tr_cpBlockAdd + 60
1   org.m0k.transmission          	0x00000001000778f6 tr_cpPieceAdd + 45
2   org.m0k.transmission          	0x000000010008f4dc verifyThreadFunc + 1065
3   org.m0k.transmission          	0x0000000100072ed5 ThreadFunc + 15
4   libSystem.B.dylib             	0x00007fff83e58fd6 _pthread_start + 331
5   libSystem.B.dylib             	0x00007fff83e58e89 thread_start + 13


If you can confirm, I will open a new ticket. If confirmed, do you know why it crashes?

Last edited 7 years ago by x190 (previous) (diff)

comment:24 follow-up: Changed 7 years ago by cfpp2p

32852-OK.torrent now to is 32852-CRASH.torrent, same crash as 35480-OK.torrent

comment:25 in reply to: ↑ 24 Changed 7 years ago by x190

Replying to cfpp2p:

32852-OK.torrent now to is 32852-CRASH.torrent, same crash as 35480-OK.torrent

Test code to fix EXC_I386_DIV (divide by zero): torrent.h->Line 189:

    uint16_t                   blockCountInPiece;
    uint16_t                   blockCountInLastPiece;
CHANGE TO: uint32_t
Last edited 7 years ago by x190 (previous) (diff)

comment:26 in reply to: ↑ 16 Changed 7 years ago by x190

Replying to cfpp2p:

It doesn't seem to be holding true that all piece sizes cleanly divisible by 16384 will be valid. The names of the test torrents indicate pieceSize / 16384} with no remainder. r14324

35474-INVALID.torrent 35477-INVALID.torrent 35479-INVALID.torrent

Test code to fix: torrent.c->Line 801:

/**
 * Decide on a block size. Constraints:
 * (1) most clients decline requests over 16 KiB
 * (2) pieceSize must be a multiple of block size
 */
uint32_t
tr_getBlockSize (uint32_t pieceSize)
{
  if (pieceSize <= MAX_BLOCK_SIZE)
  {
    tr_logAddInfo ("Block size in tr_getBlockSize is %d", pieceSize);
    return pieceSize;
  }
  
  if ((pieceSize % MAX_BLOCK_SIZE) == 0)
  {
    tr_logAddInfo ("Block size in tr_getBlockSize is %d", MAX_BLOCK_SIZE);
    return MAX_BLOCK_SIZE;
  }
  
  uint32_t b = MAX_BLOCK_SIZE;
  
  while ((pieceSize % b) != 0)
    b--;
    
  tr_logAddInfo ("Block size in tr_getBlockSize is %d", b);
  
  return b;
}

Last edited 7 years ago by x190 (previous) (diff)

comment:27 in reply to: ↑ 23 ; follow-up: Changed 7 years ago by suitessay

Replying to x190:

If you can confirm, I will open a new ticket. If confirmed, do you know why it crashes?

In torrentInitFromInfo(), assigning 65536 to the 16-bit blockCountInPiece field is overflowing to 0. tr_torBlockPiece() then tries to divide by it.

comment:28 follow-ups: Changed 7 years ago by jordan

  • Resolution set to wontfix
  • Status changed from reopened to closed

15,793 torrents out of 10,226,795 is 0.001%, so I'm re-closing this as wontfix.

Please move the crasher discussion to another ticket. The crasher discussion really is a separate issue, related only by the fact that both topics are about piece size.

comment:29 in reply to: ↑ 27 Changed 7 years ago by cfpp2p

Replying to suitessay:

Replying to x190:

If you can confirm, I will open a new ticket. If confirmed, do you know why it crashes?

In torrentInitFromInfo(), assigning 65536 to the 16-bit blockCountInPiece field is overflowing to 0. tr_torBlockPiece() then tries to divide by it.

It's the call from completion.c tr_cpBlockAdd()

const tr_piece_index_t piece = tr_torBlockPiece (cp->tor, block);

where the crash initiates.

Someone needs to create a new ticket. This is off topic now.

Replying to jordan:

Please move the crasher discussion to another ticket. The crasher discussion really is a separate issue, related only by the fact that both topics are about piece size.

x190 said he would comment:23

I'm a little short of time...

comment:30 in reply to: ↑ 28 Changed 7 years ago by cfpp2p

Replying to jordan:

Please move the crasher discussion to another ticket. The crasher discussion really is a separate issue, related only by the fact that both topics are about piece size.

I think that if I did it would so similar to #5736 it could be considered duplicate so I added a comment to that existing ticket:5736#comment:2

Last edited 7 years ago by cfpp2p (previous) (diff)

comment:31 in reply to: ↑ 28 Changed 7 years ago by x190

Replying to jordan:

15,793 torrents out of 10,226,795 is 0.001%, so I'm re-closing this as wontfix.

Many of which will generate complaints here and on the forum. e.g. comment:11.

Please move the crasher discussion to another ticket. The crasher discussion really is a separate issue, related only by the fact that both topics are about piece size.

See #5754.

comment:32 in reply to: ↑ 11 ; follow-ups: Changed 7 years ago by x190

Replying to suitessay:

I've been seeing a recent uptick in the number of torrents encountered with piece lengths that aren't multiples of 16K. They fail to open in Transmission, with an invalid data error.

Some examples: piece lengthi423792e piece lengthi333744e piece lengthi330192e

Please try using the code from comment:26 and comment:25 above. All these examples should load and it would be interesting to see if they will work properly for downloading and seeding.

comment:33 in reply to: ↑ 32 Changed 7 years ago by cfpp2p

Replying to x190:

Please try using the code from comment:26 and comment:25 above. All these examples should load and it would be interesting to see if they will work properly for downloading and seeding.

The code of comment:26 looks OK to me. It looks to set the block size so that some exactly even multiple of it will equal piece size, which is correct. I'll test it when I get a chance. Thanks.

comment:34 follow-up: Changed 7 years ago by cfpp2p

I tested comment:26 code and basically it works but I found that in some cases we get very small block sizes. This results in undue overhead and requests. The logic works but building upon the old code is inefficient and results in finding the smallest block size that could be used instead of the largest in some cases. As an enhancement I've posted ticket:5755 which addresses this issue. Building upon old code is partially the reason comment:26 code is less than optimal. ticket:5755 code can easily be altered to allow any block size down to 1, but I don't think tiny block sizes is a good way to go.

comment:35 in reply to: ↑ 34 Changed 7 years ago by x190

Replying to cfpp2p:

I tested comment:26 code and basically it works but I found that in some cases we get very small block sizes. This results in undue overhead and requests. The logic works but building upon the old code is inefficient and results in finding the smallest block size that could be used instead of the largest in some cases. As an enhancement I've posted ticket:5755 which addresses this issue. Building upon old code is partially the reason comment:26 code is less than optimal. ticket:5755 code can easily be altered to allow any block size down to 1, but I don't think tiny block sizes is a good way to go.

I have updated comment:26 code to reflect your suggestions. The v 2.84 original code is just plain wrong, as it unnecessarily returns smaller block sizes in many cases, when MAX_BLOCK_SIZE could have been used. We can also accommodate that .001% while seldom seeing block sizes <2K.

comment:36 in reply to: ↑ 32 Changed 7 years ago by suitessay

Replying to x190:

Please try using the code from comment:26 and comment:25 above. All these examples should load and it would be interesting to see if they will work properly for downloading and seeding.

Verified to work (with the caveat of comment:34) for several previously invalid examples.

Note: See TracTickets for help on using tickets.