Opened 11 years ago

Closed 11 years ago

#3531 closed Bug (invalid)

Pausing 500 torrents uses 650 MB of memory

Reported by: Tantali Owned by: livings124
Priority: Normal Milestone: None Set
Component: Mac Client Version: 2.04
Severity: Normal Keywords:
Cc:

Description

I have 1000 seeding torrents (of which approx. 99% are stalled, so not much activity altogether). Starting them takes mere seconds, but pausing them is almost impossible. It takes up more than 7,5GB VRAM and at least half an hour. At that point I have to force quit due to lack of space for more VRAM, so I have no idea how much VRAM it theoretically needs. Transmission does continue to download though, so I doubt it freezes (except for the download-window itself).

All I can assume is that while pausing the torrents there is a lot of resume-data being written, but not that (memory-)efficient...

Just thought I should let you know. (this has been around for a long while and still is present in r11187)

Change History (31)

comment:1 Changed 11 years ago by Tantali

If I restart after the force quit, about 250 torrents where still seeding, so that leads me to believe theoretically 10GB VRAM is needed (give or take a couple of GB)

comment:2 Changed 11 years ago by livings124

  • Component changed from Mac Client to Transmission

comment:3 Changed 11 years ago by charles

I'm not seeing this behavior. Stopping 200 torrents takes about 10 seconds on my machine and takes no memory worth speaking of.

comment:4 Changed 11 years ago by Tantali

Well, it seems to depend on a number of factors... Just to give some more info: It were 1000 torrents of about 100MB each, all folders with mp3-files in them. They were set to seed unlimited and were seeding for a couple of hours before being paused. The total seeded amount was not more than a couple of MB, so not much activity altogether.

I've also noticed this problem in other conditions, but in a much lesser extent. Generally speaking, when pausing the torrents, I see the memory-usage increasing non-stop up until Transmission finally paused all torrents. At that moment, the memory is released instantly. I'm assuming that the pausing of each torrent allocates an amount of memory, because force quitting and restarting Transmission during the pausing results in a number of paused torrents and a number of still seeding torrents.

I might not be getting the bigger picture, but I don't see why Transmission keeps the allocated memory for each paused torrent until all torrents are finally paused...

comment:5 Changed 11 years ago by charles

This patch might be interesting to test: http://pastebin.org/1165529

I'll check it in after the code freeze thaws out after 2.10 is released

comment:6 Changed 11 years ago by charles

Tantali, does this still occur in 2.11?

comment:7 Changed 11 years ago by Tantali

Yes, I'm afraid so... I think that less memory is used, but I can only test it with 500 torrents anymore. The RAM-usage went up by 1 gig approximately and was released once all torrents were paused. I have no idea to what extent this RAM-usage might be affected by circumstances, though. I let the 500 torrents seed for an hour or so.

This still seems like a lot of ram-usage for a fairly sequential process, though.

comment:8 Changed 11 years ago by charles

  • Type changed from Enhancement to Bug

comment:9 Changed 11 years ago by charles

  • Version changed from 2.04+ to 2.04

comment:10 Changed 11 years ago by charles

  • Component changed from Transmission to Mac Client

In the GTK+ client, it takes about three seconds to pause 300 torrents. Here is the before and after of memory use:

 3887 20   0  252m  48m  14m S  6.3  1.6  58:10.61 transmission-gtk    
 3887 20   0  253m  47m  14m S  7.0  1.6  58:13.01 transmission-gtk    

This is either an issue with your system, or an issue with the Mac client. Bumping back to the Mac client component so livings can take a look. The Mac client has some of its own per-torrent configuration files that get saved when torrents are stopped, so that might benefit from testing.

WRT the daemon and the GTK+ client, this ticket is a "worksforme".

comment:11 Changed 11 years ago by livings124

Have things improved in 2.12? I'm not seeing any issues on my end.

comment:12 Changed 11 years ago by livings124

  • Resolution set to worksforme
  • Status changed from new to closed

There's not much I can do with this as things are. Please reopen if you can supply new info.

comment:13 Changed 11 years ago by Tantali

  • Resolution worksforme deleted
  • Status changed from closed to reopened

Just tested this again in the latest nightly r11531 and it's still present.

In full detail: I start Transmission and start around 500 completed torrents with mp3-files in it (100MB average per torrent). I let it seed for about 2 hours. Almost nothing has been uploaded (130KB, basically nothing). I pause them all at once and have to wait for about 20 seconds, while the RAM-usage rises from around 200MB to 750MB, which results in Transmission being unresponsive + 100% CPU-usage. After that it instantly frees RAM to around 400MB, which then slowly descents to about 250MB. The same behavior is seen when I restart the torrents after that.

I'm using Snow Leopard 10.6.5, I have 4GB RAM (and around 1GB of that is free) and plenty of diskspace, with almost no fragmentation. Restarting and pausing the torrents a few times, I keep seeing the same behavior, although it's not always present at the restarting. The maximum allocated RAM isn't always the same either. In this case it varies from 400 to 800MB.

comment:14 Changed 11 years ago by Tantali

I also noticed that it takes around 4 seconds (with CPU 100%) before the RAM actually starts to rise.

comment:15 Changed 11 years ago by charles

I'm confused. Earlier you were reporting that it took half an hour and 7,5 GB VRAM to pause 1000 torrents. Now you're saying 20 seconds and 650 MB to pause 500 torrents...?

comment:16 Changed 11 years ago by Tantali

That was the case when I still had 2GB RAM and not that much diskspace. Now I've upgraded both of those. On top of that, I don't have 1000 torrents no more, so I can only test for the 500... But the numbers I've provided here are correct. Why all this happens and whether the latest nightlies have improved this behavior or not... I haven't got a clue.

comment:17 Changed 11 years ago by charles

  • Summary changed from Pausing 1000 torrents results in insane amount of VRAM to Pausing 500 torrents uses 650 MB of memory

comment:18 Changed 11 years ago by charles

r11544 libtransmission/ (bencode.c fdlimit.c fdlimit.h): (trunk libT) #3531 "Pausing 500 torrents uses 650 MB of memory" -- on OS X, when saving a benc/json file, send a hint to the OS to not cache the file.

Tantali, does this make things better, worse, or no change? https://build.transmissionbt.com/job/trunk-mac/

comment:19 Changed 11 years ago by Tantali

Nothing changed

comment:20 Changed 11 years ago by livings124

Have things improved in the latest nightly?

comment:21 Changed 11 years ago by Tantali

Nope, sorry... r11595 still has the same behaviour. And it's hard to tell whether it improved a little bit or not, as the ram-usage isn't always the same.

comment:22 Changed 11 years ago by livings124

I'm not sure how to approach this. The change from the start of this ticket to recently shows an improvement in memory usage. What is an end goal of this ticket that's measurable?

Last edited 11 years ago by livings124 (previous) (diff)

comment:23 Changed 11 years ago by Tantali

I would say a non-linear memory increase for the amount of torrents to be paused, better said Transmission should have a memory use independent of the amount of torrents to be paused. I haven't got a clue of how feasible this is, but to me it seems unnecessary that Transmission allocates new memory for each torrent it pauses, while this is a fairly sequential process.

comment:24 Changed 11 years ago by livings124

I don't think that's truly reasonable. How would Transmission achieve this?

comment:25 Changed 11 years ago by Tantali

Well, after Transmission pauses torrents, the memory is released. So, to put it very simplistic, if I were to manually pause each torrent separately the amount of RAM never would increase relative to the amount of torrents. Now, I don't have a clue about all the mechanics behind this, but to me this seems perfectly reasonable. Once a torrent is paused, the memory isn't needed anymore, so why keep it until all torrents are paused?

comment:26 Changed 11 years ago by livings124

What about the memory to store info on the torrent such as icon and file list, plus activity to determine total speed (each torrent needs to be pinged) and other stats. A transfer still exists in the app when it's paused, and the app needs to account for this.

comment:27 Changed 11 years ago by Tantali

My point is, before pausing and after pausing torrents the total amount of memory is roughly the same. So the info you talk about is still there whether or not you solve this ticket.

So basically I don't think that the process of pausing needs an amount of memory that's linear with the amount of torrents to be paused.

comment:28 Changed 11 years ago by livings124

Paused torrents still scrape. Also, are the unpaused transfers actually downloading or uploading?

comment:29 Changed 11 years ago by Tantali

The paused torrents are uploading only, I haven't tested with downloading torrents.

I am assuming that unpaused torrents scrape as well, so what is your point exactly? This ticket is about the extra memory it requires to go from the unpaused to the paused state. Before and after this process the amount of memory is already roughly the same. What I find strange is why it needs an extra amount of memory dependent on the amount of torrents to pause, during this (normally) short process.

comment:30 Changed 11 years ago by livings124

Pausing causes trackers to be sent a message saying the transfer is being stopped.

comment:31 Changed 11 years ago by livings124

  • Resolution set to invalid
  • Status changed from reopened to closed

Closing for lack of clear goal. Activity goes up when pausing because of the mass of torrents that need to announce that they're pausing. (don't be put off by the resolution of "invalid," there aren't a lot of options)

Note: See TracTickets for help on using tickets.