2.30 Final - 100% CPU Load with Larger Torrents

[gunzip]

see also: ​https://forum.transmissionbt.com/viewtopic.php?f=2&t=11695

testing with a 8GB multi-file torrent (650 files) CPU Load pulses to 100%. the profile again indicates countArray() in bitfield.c is the cause:

$ opreport -l -g /usr/local/bin/transmission-daemon
Profiling through timer interrupt
samples  %        linenr info                 symbol name
79854    97.4269  bitfield.c:49               countArray
354       0.4319  (no location information)   __udivdi3
337       0.4112  bitfield.c:61               countRange
178       0.2172  torrent.c:2322              tr_torGetPieceBlockRange
151       0.1842  completion.c:137            tr_cpSizeWhenDone
123       0.1501  peer-mgr.c:2823             rechokeDownloads
103       0.1257  completion.c:198            tr_cpMissingBlocksInPiece
94        0.1147  bitfield.c:136              tr_bitfieldIsValid
81        0.0988  bitfield.h:89               tr_bitfieldHasAll
64        0.0781  bitfield.c:120              tr_bitfieldCountRange
62        0.0756  bitfield.h:95               tr_bitfieldHasNone
49        0.0598  bitfield.h:89               tr_bitfieldHasAll
48        0.0586  JSON_parser.c:662           JSON_parser_char
48        0.0586  peer-mgr.c:1046             pieceListLookup
40        0.0488  peer-mgr.c:930              comparePieceByWeight
33        0.0403  bitfield.c:184              tr_bitfieldEnsureBitsAlloced
33        0.0403  torrent.h:284               tr_torPieceCountBytes
31        0.0378  bitfield.c:303              tr_bitfieldAdd
31        0.0378  bitfield.h:101              tr_bitfieldHas
24        0.0293  torrent.h:284               tr_torPieceCountBytes
22        0.0268  completion.h:103            tr_cpHasAll
19        0.0232  completion.c:115            tr_cpHaveValid
15        0.0183  completion.h:126            tr_cpPieceIsComplete
12        0.0146  bencode.c:1042              bencWalk
11        0.0134  bitfield.c:220              tr_bitfieldSetTrueCount
10        0.0122  bitfield.c:237              tr_bitfieldIncTrueCount
10        0.0122  json.c:140                  tr_jsonParse
9         0.0110  bitfield.c:152              get_bytes_needed
8         0.0098  bencode.c:1367              jsonStringFunc
7         0.0085  completion.h:126            tr_cpPieceIsComplete
6         0.0073  bencode.c:380               dictIndexOf
5         0.0061  bencode.c:754               tr_bencDictAdd
5         0.0061  verify.c:43                 verifyTorrent

Linux Debian, transmission-daemon 2.30 (12439)

[jordan]

gunzip, could you confirm that your copy is being built with -DNDEBUG defined?

[gunzip]

Replying to jordan:

gunzip, could you confirm that your copy is being built with -DNDEBUG defined?

i didn't see anything in the console output during ./configure and make re DNDEBUG. the most i could find is this code snippet in configure.ac

...
if test m4_substr(peer_id_prefix,6,1) = "0"; then
  supported_build=yes
  CPPFLAGS="$CPPFLAGS -DNDEBUG"
else
  supported_build=no
  if test "x$GCC" = "xyes" ; then
    CFLAGS="$CFLAGS -g -O0"
    CXXFLAGS="$CXXFLAGS -g -O0"
  fi
fi
...

is there someway to test for this?

[KyleK]

The peer_id_prefix in configure.ac is defined as '-TR230X-', the 'X' marking it as beta build.

This has immediate influence on how Transmission is built (in this case with DEBUG enabled and any compiler optimizations disabled).

[jordan]

Thansk, KyleK

Closing as a duplicate of #4246

[livings124]

Shouldn't it not use 100% CPU regardless of debug setting?

[gunzip]

Replying to livings124:

Shouldn't it not use 100% CPU regardless of debug setting?

Yes i'm wondering the same thing. What is going on in bitfield.c that causes these calls in the profile to be two orders of magnitude greater than everything else. It does seem disproportionate.

[livings124]

Reopening so that we can figure this out without losing track of this.

[jordan]

Reclosing because there's nothing to figure out. The assertions are expensive, which is why they're disabled in the stable releases but active in the nightlies.

After bitfield.c has had more burn-in time we can remove some of these assertions, but there's no mystery about them. They're known, and are in there intentionally.