Opened 13 years ago

Closed 13 years ago

Last modified 12 years ago

#1829 closed Enhancement (fixed)

High CPU use in refillPulse()

Reported by: qqlapraline Owned by: charles
Priority: Normal Milestone: 1.51
Component: libtransmission Version: 1.50
Severity: Normal Keywords: Slow, CPU
Cc:

Description (last modified by charles)

Platform: Syabas (NMT)

Description: The version compiled for Syabas platform is significantly slower than the previous version (1.42 build 7496). It consumes a lot of CPU (not memory).

Change History (50)

comment:1 Changed 13 years ago by livings124

comment:2 Changed 13 years ago by qqlapraline

Unfortunately, as stated, it's running on a NMT box: I won't be able to go much further a "top" or "ps" on this platform.

comment:3 Changed 13 years ago by Ger Teunis

That binary is created by me. Some more information regarding the build: Build flags: -O2 -frename-registers -march=4kc And binaries are stripped after that.

I do not have a profiler on the MIPS media box (NMT).

comment:4 Changed 13 years ago by qqlapraline

What kind of information do you need exactly ? May be GDB can be helpful ?

comment:5 Changed 13 years ago by charles

  • Summary changed from High ressources consumption to High CPU use

Sounds good to me.

I don't know what kind of profiling tools NMT has, if any. Probably the most low-level profiling is one you could do by hand in gdb -- start it running inside gdb, and when you see it reaching a high CPU use, hit ctrl-c and get a backtrace. Then continue running and take another sample a few seconds later if the CPU use is still high.

You need to use a NON-STRIPPED executable when doing this. Stripping it tears out most of what makes the gdb backtraces useful.

Ger, do you have any other suggestions on better ways to do profiling on NMT?

comment:6 Changed 13 years ago by Ger Teunis

I currently do not see the high cpu load, some do some don't. But I am willing to provide qqlapraline a non-stripped build so he can use gdb to do a manual profiling session?

comment:7 Changed 13 years ago by qqlapraline

Ger, I'll be waiting for your non-stripped build. Just post a URL here.

comment:8 Changed 13 years ago by qqlapraline

By the way, if you find a MIPS version of GDB first, it will save me some time...

comment:9 Changed 13 years ago by qqlapraline

I think I got one here: http://www.macraigor.com/full_gnu.htm#For%20Linux

I'll try it and see what happens..

comment:10 Changed 13 years ago by qqlapraline

Well, I've downloaded it, send it to my HDX by ftp and here is the result of GDB launch: ./mips-elf-gdbtui: line 1: syntax error: "(" unexpected

comment:11 Changed 13 years ago by charles

qqlapraline: sounds like maybe you should run Ger's version :)

The session should look like this:

$ gdb transmission (gdb) handle SIGPIPE nostop noprint nopass (gdb) r then run transmission for a little while until the cpu load is high, and press ctrl-c (gdb) thread apply all bt gdb will spit out the first backtrace that you'll provide (gdb) continue repeat as necessary

comment:12 Changed 13 years ago by charles

$ gdb transmission
(gdb) handle SIGPIPE nostop noprint nopass
(gdb) r
// then run transmission for a little while until the cpu load is high, and press ctrl-c
(gdb) thread apply all bt
// gdb will spit out the first backtrace that you'll provide
(gdb) continue
// repeat as necessary

comment:13 Changed 13 years ago by qqlapraline

Well, I got the GDB error by launching it only...

comment:14 Changed 13 years ago by repattila

Shark sample: http://transmission.pastebin.com/m7a9fb400

At the time transmission running with 11 transfers (6 with peers) global max peer count 300, cpu usage round 8%, with spikes of 12+, on a 2.1GHz Macbbook. I had higher cpu usage before (15% avarage) with other torrents (count 12), but don't have those anymore, still no big difference in means of speed, encrypted peer count (very low) or file number (avarage) in torrents.

comment:15 Changed 13 years ago by qqlapraline

You may have a point here. I usually have many encrypted peers (private trackers). Can this change anything ?

comment:16 Changed 13 years ago by charles

repattila: d'oh, I forgot that the official 1.50 release has debugging information stripped out of it. Would it be possible for you to download a nightly build and re-run the shark sample?

comment:17 Changed 13 years ago by repattila

Running r7896: http://transmission.pastebin.com/m5f9f6980

Same torrents as before and about the same cpu load.

comment:18 Changed 13 years ago by charles

repattila: much better; thanks

It looks like most of the time in that shark trace is being spent in refillPulse(). That's something that'll be worked on for 1.51.

It would be nice to hear more in the way of profiling from the NMT users. I scanned the NMT forum's Transmission thread for more information on this subject -- and as an aside, I'm very happy to see NMT users using Transmission so much, and grateful to Ger for providing builds -- but the bug reports there are so vague as to be useless...

comment:19 Changed 13 years ago by charles

  • Component changed from Daemon to libtransmission
  • Milestone changed from None Set to 1.51
  • Status changed from new to assigned
  • Summary changed from High CPU use to High CPU use in refillPulse()

Other users have also reported that refillPulse() is eating a lot of CPU cycles, so for now I'm going to work from the assumption that the same behavior is taking place on the NMT. As I mentioned above (maybe too grumpily ;) I'd be happy for any profiling information of Transmission on NMT.

Refining this ticket's topic so that it's a discrete problem that can be fixed. If there are other issues affecting CPU load please open another ticket and I'll dig into them as well.

comment:20 Changed 13 years ago by charles

There is an experimental fix for the refillPulse() issue in the nightlies starting in r7909. Feedback on this would be great...

comment:21 Changed 13 years ago by repattila

Looking better here, cpu load around 4% (no peaks) with torrents like before. Thx for the quick response.

Made a shark report too if its relevant: http://transmission.pastebin.com/m7195be21

comment:22 Changed 13 years ago by charles

Sounds good. Thanks repattila :)

comment:23 Changed 13 years ago by DaveF1

That's made quite a big difference on my NSLU2 - now running ~20-40% CPU (with odd peaks of 90%+) instead of constant 95-100%. T-remote is now far more responsive as is the web interface (doesn't take as long to load).

Running 7914 on NSLU2 Unslung 6.8

comment:24 Changed 13 years ago by DaveF1

Oops, spoke too soon there. After a couple of hours running I'm back up to a fairly constant 95-100% CPU with odd dips to ~20%. Memory usage seems to creep up over time and d/l speeds seem to drop off. Anything I can do to help? Dave

comment:25 Changed 13 years ago by charles

Yes: submit a profile of what Transmission is doing when it's eating 100% of your CPU. I'm not clairvoyant :)

I don't know what profilers (if any) are available on the NSLU2. If nothing's available you might have to do brute-force profiling with gdb... see the above comments.

comment:26 Changed 13 years ago by DaveF1

Can you give me some pointers ... I tried running transmission-daemon under gdb but can't connect to it to start paused torrents. I worked out how to pass command line arguments when running T under gdb but it doesn't seem to create 3 processes as it does normally (I'm a bit of a newbie when it comes to profiling) Gdb 6.8 from ipkg (optware).

comment:27 Changed 13 years ago by charles

You'll want to run the daemon in the foreground

% gdb transmission-daemon
(gdb) handle SIGPIPE nostop noprint nopass
(gdb) run --foreground --your-daemon-args-go-here
// then run transmission for a little while until the cpu load is high, and press ctrl-c
(gdb) thread apply all bt
// gdb will spit out the first backtrace that you'll provide
(gdb) continue
// repeat as necessary

comment:28 Changed 13 years ago by DaveF1

Sorry, I can't get Transmission-daemon running under gdb. It appears to run (once I've continued through a breakpoint in libc.so.6) but looking at the process using top it isn't actually doing anything. There only appears to be one process when running under gdb but there are 3 normally - I take it its not running properly?

comment:29 Changed 13 years ago by Ger Teunis

Sorry guys; wasn't tracking this one actively. Dowload the nonstripped trunk build (revision 7929) here http://members.tele2.nl/ger.teunis/downloads/transmission-trunk-nmt.zip

comment:30 follow-up: Changed 13 years ago by DaveF1

Ger - I've noticed similar high CPU usage on NSLU2 to the original poster's NMT device. Do you have a nonstripped trunk build for NSLU2 (armb)? Thanks Dave

comment:31 in reply to: ↑ 30 Changed 13 years ago by Ger Teunis

Replying to DaveF1:

Ger - I've noticed similar high CPU usage on NSLU2 to the original poster's NMT device. Do you have a nonstripped trunk build for NSLU2 (armb)? Thanks Dave

Sorry, but I cannot provide that build for you. It took me a great while to get the MIPS (read NMT) build up and running. Perhaps somebody else can provide you this build.

comment:32 Changed 13 years ago by qqlapraline

Here are a few tools I've found to profile. Unfortunately, as stated, I don't have any platform where I could build a MIPS version of these...Hopefully, somebody will have time to do so.

OProfile: http://www.linux-mips.org/wiki/Oprofile (On MIPS ? http://oprofile.sourceforge.net/news/) KernProf?: http://oss.sgi.com/projects/kernprof/ VProf: http://sourceforge.net/projects/vprof/ TSProf: http://freshmeat.net/projects/tsprof/

comment:33 Changed 13 years ago by qqlapraline

After many, many (many !) tests, I think I have issues only with public trackers. With private trackers, I don't have CPU usage issues. Hope this helps. And I still can't get any profile information....

comment:34 Changed 13 years ago by charles

Some questions that DaveF and qqlapraline can answer even without a profiler. Note that it's a given that CPU load will be high when verifying local data, so please don't verify local data while testing to answer these questions:

  1. Is the CPU load high when only seeding?
  1. Is the CPU load high when only downloading?
  1. Is the CPU load high when only downloading from seeds?
  1. Is the CPU load high when only downloading, from both seeds and peers?
  1. Is the CPU load high when neither seeding nor downloading?

comment:35 Changed 13 years ago by KyleK

I'm experiencing the same on my NAS. I really want to do profiling, but I simply can't find a way to do it on the device. oprofile compiles but there's no kernel support. Valgrind doesn't support ARM at all. KernProf? needs a modified kernel as well. VProf uses functions that are disabled on the device. And to top it all off, gdb gives me an "generic error" when I try to start Transmssion. I also tried to compile Transmission with gprof support (-pg), but that didn't work either (can't remember the error message right now).

If anyone has any other ideas, let me here them :)

comment:36 Changed 13 years ago by okouais

Same problem on my NAS (Qnap TS409Pro). Transmission 1.50 installed via optware. Use with Transmission Remote Gui (Windows) and a ssh session to check the %cpu via "top" command. Tested on 10 torrents from two private trackers and one public.

The CPU load is high when only downloading (about 20-40%) from both seeds and peers. Cpu usage and download speed seems to be linked. But there is some random 95% peaks. When only seeding, I think cpu usage is normal (less than 5%).

Any ideas?

comment:37 follow-up: Changed 13 years ago by livings124

okouais: As stated in this ticket, feedback on the current source would be useful.

comment:38 follow-up: Changed 13 years ago by charles

KyleK, okouais: even without profiling, possibly the most helpful thing you could do would be to compare for me (1) the CPU load when downloading in a swarm where you're only connected to seeds, compared to (2) the CPU load when downloading in a swarm where you're connected to both seeds and other downloaders

comment:39 in reply to: ↑ 38 ; follow-up: Changed 13 years ago by okouais

Replying to charles:

KyleK, okouais: even without profiling, possibly the most helpful thing you could do would be to compare for me (1) the CPU load when downloading in a swarm where you're only connected to seeds, compared to (2) the CPU load when downloading in a swarm where you're connected to both seeds and other downloaders

Test : 1 torrent downloading from 1 seed and 1 peer : average cpu load<5% with frequent random peaks > 15-20% 1 torrent downloading from 3 seeds and no peer : average cpu load<3% with almost NO random peaks 1 torrent downloading from 3 seeds and 1 peer : average cpu load<5% with frequent random peaks > 15-20%

Cpu load peaks seems to be linked to peer exchange ...

comment:40 in reply to: ↑ 37 ; follow-up: Changed 13 years ago by okouais

Replying to livings124:

okouais: As stated in this ticket, feedback on the current source would be useful.

Do you mean I have to open a new ticket ?

comment:41 in reply to: ↑ 40 Changed 13 years ago by livings124

Replying to okouais:

Replying to livings124:

okouais: As stated in this ticket, feedback on the current source would be useful.

Do you mean I have to open a new ticket ?

No, I mean your tests should be based off of current source, not 1.50 release version.

comment:42 Changed 13 years ago by okouais

Sorry for my ignorance, but it's the first time I post in Trac. I'm not a developper. I'm only a Transmission user. I only have basic Linux knowledges. I don't know how to compile current source for my device (Qnap) so I only can run the released version. Sorry again.

comment:43 in reply to: ↑ 39 Changed 13 years ago by charles

  • Description modified (diff)

Test :

1 torrent downloading from 1 seed and 1 peer : average cpu load<5% with frequent random peaks > 15-20%

1 torrent downloading from 3 seeds and no peer : average cpu load<3% with almost NO random peaks

1 torrent downloading from 3 seeds and 1 peer : average cpu load<5% with frequent random peaks > 15-20%

This is very helpful. Thanks!

comment:44 Changed 13 years ago by DaveF1

My CPU usage seems to run higher than okouais - 1 torrent U/L to 6, D/L from 11, CPU ~13-15% odd peaks to 20% and occasional peaks to 95%, Memory usage starts at 38% with torrents paused, increases to 57% with 1 torrent running, gradually increases to 101% over several days (OK my NSLU2 only has 32M memory). When the Memory usage is high the CPU usage seems to run high - ie runs at fairly constant 90%+.

Any suggestions for a seeds only torrent? I'm only using public trackers.

comment:45 Changed 13 years ago by charles

There's another experimental improvement for the refillPulse() issue in the nightlies starting in r7909. Feedback on this would be great...

comment:46 Changed 13 years ago by DaveF1

Charles - I think the changes you made 7 days ago (according to the revision log) are what appeared to make a big difference when I first posted. I've had some slow, large torrents running for several months which have a bad seed/peer ratio. I definitely noticed a speed increase when I started using 7909 and the overall processor usage seems to be down.

I've tried downloading a ubuntu release from the site mentioned in the "slow speeds" page of the wiki - this saturates my d/l (about 300Kb) with CPU running at an average of 80% (62% memory).

I'm not having much joy with profiling either, haven't yet tried compiling to run under gprof although I've downloaded and installed it.

comment:47 Changed 13 years ago by charles

The revision in my last message was wrong... that's what I get for cutting and pasting. :)

What I was really looking for was feedback on r7942.

comment:48 Changed 13 years ago by KyleK

With the latest SVN build I have a steady CPU usage of about 20% (with rare spikes to 40%). This is on a NAS (500MHz ARM CPU, 64MB RAM).

Before this latest checkin, Transmission would jump to 99% CPU within a minute after starting, and stay there as long as the torrent(s) would download. When seeding, all was fine.

So, I'd say this is a very welcome improvement :)

(On a side note: I had noticed very high mem usage with versions 1.50 and before, but this seems to have gone with this update as well)

comment:49 Changed 13 years ago by charles

  • Resolution set to fixed
  • Status changed from assigned to closed

Backported to the 1.5x branch in r7949.

comment:50 Changed 12 years ago by sim

(deleted spam)

Last edited 12 years ago by charles (previous) (diff)
Note: See TracTickets for help on using tickets.