Opened 10 years ago

Closed 10 years ago

#4514 closed Bug (invalid)

cpu usage pegs and holds on magnet/dht download

Reported by: reardon Owned by: jordan
Priority: Normal Milestone: None Set
Component: libtransmission Version: 2.33+
Severity: Normal Keywords:
Cc:

Description

[using r21918]

Not sure where to even start on this one. I don't have past experience with magnets.

I just tried a public domain torrent via a magnet link. The link has no instantiating trackers, just the hash. I also have 1000 private trackers loaded and running, about 10 of which are active at any one time.

CPU usage (~2GHz dualcore server) was sitting at 1-2%.

As soon as I added the magnet link, usage went to 100%.

There were not a ton of hanging TCP sessions. I presume most of the new traffic was DHT (ie UDP). The logs show a DHT start announcement and a "done" about 4 minutes later, with about 500 peers found.

I have a 60peer limit for the torrent. About 40 are from DHT, another 10 from PEX.

Why the crazy CPU spike? It basically goes for as long as the torrent is running (I let it go 20min). transmission-daemon itself becomes very unresponsive, even over localhost...ie it's not a network b/w issue.

Curiously, 'top' shows the process running at 100%, but 'ps -m -o cpu' shows only one thread in the process as active, and only at 9%.

Change History (5)

comment:1 Changed 10 years ago by jordan

Reardon, I haven't seen that behavior.

Any chance you could run it in perf to see where the cycles are being spent?

comment:2 Changed 10 years ago by reardon

Need to setup stock kernel. Will try. But what profile info would be useful? With "-g" call stack?

comment:3 Changed 10 years ago by reardon

here's the relevant log portion, and the top fragment from 'perf report'. I ran this with 700 paused torrents (so peers were cached) and then added magnet. it was the only torrent running.

[03:49:30.150] Transmission 2.40b2 (12921) started (session.c:718)
[03:49:30.150] Cache Maximum cache size set to 64.00 MiB (4096 blocks) (cache.c:249)
[03:49:30.150] RPC Server Adding address to whitelist: 127.0.0.1 (rpc-server.c:805)
[03:49:30.150] RPC Server Serving RPC and Web requests on port 127.0.0.1:51214/transmission/ (rpc-server.c:999)
[03:49:30.150] RPC Server Password required (rpc-server.c:1006)
[03:49:30.150] Bound socket 16 to port 54387 on 0.0.0.0 (net.c:373)
[03:49:30.150] Bound socket 17 to port 54387 on :: (net.c:373)
[03:49:30.150] Port Forwarding Stopped (port-forwarding.c:181)
[03:49:30.150] DHT Initializing DHT (tr-dht.c:276)
[03:49:30.150] DHT Reusing old id (tr-dht.c:305)
[03:49:30.150] DHT Bootstrapping from 62 IPv4 nodes (tr-dht.c:153)
[03:49:30.150] DHT Bootstrapping from 40 IPv6 nodes (tr-dht.c:156)
[03:49:30.150] DHT DHT initialized (tr-dht.c:330)
...
[03:50:37.158] originalmagnettorrent Starting IPv4 DHT announce (poor, 14 nodes) (tr-dht.c:574)
[03:50:38.158] originalmagnettorrent Learned 1 IPv4 peers from DHT (tr-dht.c:534)
[03:50:40.158] originalmagnettorrent Starting IPv6 DHT announce (poor, 15 nodes) (tr-dht.c:574)
[03:50:41.158] originalmagnettorrent Learned 5 IPv6 peers from DHT (tr-dht.c:534)
[03:50:41.158] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:42.159] originalmagnettorrent Learned 2 IPv6 peers from DHT (tr-dht.c:534)
[03:50:42.159] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:42.159] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:43.159] originalmagnettorrent Learned 5 IPv6 peers from DHT (tr-dht.c:534)
[03:50:43.159] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:43.159] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:43.159] originalmagnettorrent Learned 6 IPv6 peers from DHT (tr-dht.c:534)
[03:50:45.159] SO_SNDBUF size is 16384 (fdlimit.c:692)
[03:50:45.159] SO_RCVBUF size is 87380 (fdlimit.c:694)
[03:50:49.159] originalmagnettorrent Learned 152 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 6 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 6 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 1 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 11 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 36 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 52 IPv4 peers from DHT (tr-dht.c:534)
[03:50:49.159] originalmagnettorrent Learned 11 IPv4 peers from DHT (tr-dht.c:534)
[03:50:50.159] originalmagnettorrent Learned 75 IPv4 peers from DHT (tr-dht.c:534)
[03:50:55.159] Saved "/home/.../.config/transmission-daemon/torrents/renamedmagnettorrent.577d58aa66beaceb.torrent" (bencode.c:1721)
[03:51:11.160] renamedmagnettorrent Learned 114 IPv4 peers from DHT (tr-dht.c:534)
[03:51:35.162] renamedmagnettorrent Learned 143 IPv4 peers from DHT (tr-dht.c:534)
[03:51:49.163] renamedmagnettorrent IPv6 DHT announce done (tr-dht.c:545)
[03:52:38.166] renamedmagnettorrent Learned 139 IPv4 peers from DHT (tr-dht.c:534)
[03:53:20.168] renamedmagnettorrent Learned 86 IPv4 peers from DHT (tr-dht.c:534)
[03:55:34.176] Saved "/home/.../.config/transmission-daemon/resume/renamedmagnettorrent.577d58aa66beaceb.resume" (bencode.c:1721)
[03:55:34.176] Saved "/home/.../.config/transmission-daemon/stats.json" (bencode.c:1721)
[03:55:44.177] renamedmagnettorrent IPv4 DHT announce done (tr-dht.c:542)
#
# Overhead          Command                         Shared Object  Symbol
# ........  ...............  ....................................  ......
#
    80.52%  transmission-da  /usr/local/bin/transmission-daemon    [.] countArray
                |          
                |--100.00%-- tr_bitfieldIsValid
                |          |          
                |          |--50.01%-- tr_bitfieldEnsureBitsAlloced
                |          |          |          
                |          |          |--100.00%-- tr_bitfieldAdd
                |          |          |          |          
                |          |          |          |--99.99%-- tr_cpBlockAdd
                |          |          |          |          peerCallbackFunc
                |          |          |          |          publish
                |          |          |          |          fireGotBlock
                |          |          |          |          clientGotBlock
                |          |          |          |          readBtPiece
                |          |          |          |          canRead
                |          |          |          |          canReadWrapper
                |          |          |          |          |          
                |          |          |          |          |--92.98%-- tr_peerIoTryRead
                |          |          |          |          |          tr_peerIoFlush
                |          |          |          |          |          phaseOne
                |          |          |          |          |          tr_bandwidthAllocate
                |          |          |          |          |          bandwidthPulse
                |          |          |          |          |          event_base_loop
                |          |          |          |          |          
                |          |          |          |          |--5.71%-- event_read_cb
                |          |          |          |          |          event_base_loop
                |          |          |          |          |          
                |          |          |          |           --1.31%-- utp_on_read
                |          |          |          |                     UTP_ProcessIncoming(UTPSocket*, unsigned char const*, unsigned long, bool)
                |          |          |          |                     UTP_IsIncomingUTP
                |          |          |          |                     tr_utpPacket
                |          |          |          |                     event_callback
                |          |          |          |                     event_base_loop
                |          |          |           --0.01%-- [...]
                |          |           --0.00%-- [...]
                |          |          
                |          |--49.99%-- tr_bitfieldSetTrueCount
                |          |          |          
                |          |          |--100.00%-- tr_bitfieldIncTrueCount
                |          |          |          tr_bitfieldAdd
                |          |          |          |          
                |          |          |          |--99.99%-- tr_cpBlockAdd
                |          |          |          |          peerCallbackFunc
                |          |          |          |          publish
                |          |          |          |          fireGotBlock
                |          |          |          |          clientGotBlock
                |          |          |          |          readBtPiece
                |          |          |          |          canRead
                |          |          |          |          canReadWrapper
                |          |          |          |          |          
                |          |          |          |          |--92.98%-- tr_peerIoTryRead
                |          |          |          |          |          tr_peerIoFlush
                |          |          |          |          |          phaseOne
                |          |          |          |          |          tr_bandwidthAllocate
                |          |          |          |          |          bandwidthPulse
                |          |          |          |          |          event_base_loop
                |          |          |          |          |          
                |          |          |          |          |--5.71%-- event_read_cb
                |          |          |          |          |          event_base_loop
                |          |          |          |          |          
                |          |          |          |           --1.31%-- utp_on_read
                |          |          |          |                     UTP_ProcessIncoming(UTPSocket*, unsigned char const*, unsigned long, bool)
                |          |          |          |                     UTP_IsIncomingUTP
                |          |          |          |                     tr_utpPacket
                |          |          |          |                     event_callback
                |          |          |          |                     event_base_loop
                |          |          |           --0.01%-- [...]
                |          |           --0.00%-- [...]
                |           --0.00%-- [...]
                 --0.00%-- [...]

     4.20%  transmission-da  /usr/local/bin/transmission-daemon    [.] cache_block_compare

...

comment:4 Changed 10 years ago by jordan

  • Component changed from Transmission to libtransmission
  • Owner set to jordan
  • Version changed from 2.33 to 2.33+

comment:5 Changed 10 years ago by jordan

  • Resolution set to invalid
  • Status changed from new to closed

These assertions will be compiled out altogether in the stable releases; this overhead is just for beta testing.

Nevertheless, we're calling it twice per tr_bitfieldAdd() call -- indirectly, by the functions tr_bitfieldEnsureBitsAlloced() and tr_bitfieldSetTrueBitCount() which are called in pairs. So we can cut the overhead in half by removing the redundant call. (r12927)

Note: See TracTickets for help on using tickets.