Opened 12 years ago

Closed 12 years ago

Last modified 12 years ago

#1565 closed Bug (fixed)

Download speeds are inconsistent in situations with very high bandwidth

Reported by: jusid Owned by: charles
Priority: Normal Milestone: 1.41
Component: libtransmission Version: 1.40
Severity: Major Keywords:
Cc:

Description

I noticed that recent version of Transmissions downloads very poor if number of peers is very small (1-5). I tested versions 1.34, 1.40 and 1.41b2 on linux. Version 1.22 and trunk (!) version work well.

I had only 1 seeder for a torrent. Download started at good speed ~500Kb/s then, after about a minute, the speed dropped to 0. After 1-2 minutes the download continues at the same good speed and speed drops again after some time. As result, Transmission is downloading from a peer not persistently and resulting speed is very poor. I repeated this test with test torrent in my LAN. Transmission was downloading from single peer (uTorrent 1.8) in my LAN. The download was not persistent as well.

As I mention before, trunk version of Transmission does not have this bug. Please investigate this issue, since it can be easily reproduced.

Change History (30)

comment:1 Changed 12 years ago by jusid

I forgot to mention that I tested that using transmission daemon.

comment:2 Changed 12 years ago by jusid

  • Component changed from Transmission to libtransmission
  • Owner set to charles

comment:3 Changed 12 years ago by livings124

  • Resolution set to invalid
  • Status changed from new to closed

This ticket is a bit vague, but regardless, trunk is the newest developmental code. If this isn't a problem in trunk, there's nothing to fix.

comment:4 Changed 12 years ago by livings124

  • Milestone changed from 1.41 to None Set

comment:5 follow-up: Changed 12 years ago by jusid

  • Resolution invalid deleted
  • Status changed from closed to reopened

AFAIK there will be lot of releases from 1.4x branch, where this issue still exists. It will be great if needed fixes will be merged to 1.4x branch from trunk before 1.41 release.

You can easily reproduce this problem by seeding some test torrent inside your LAN and trying to download it using transmission daemon...

comment:6 in reply to: ↑ 5 Changed 12 years ago by charles

Replying to jusid:

You can easily reproduce this problem by seeding some test torrent inside your LAN and trying to download it using transmission daemon...

Okay, I've tried connecting two copies of 1.41b2 together and got a sustained transfer rate of > 2 MB/s. I don't see the problem.

comment:7 follow-up: Changed 12 years ago by jusid

Charles, thanks for testing this issue.

I reproduced it with uTorrent 1.8 as seed. Maybe this problem occurs only with uTorrent?

Can you test with uTorrent as well?

comment:8 Changed 12 years ago by livings124

  • Resolution set to invalid
  • Status changed from reopened to closed

jusid: How thorough is your test? It sounds like it's just the luck of the draw. And did you build from the 1.4x branch, which has fixes since 1.41b2?

This ticket is way too vague to be useful. Also, the only differences between trunk and branch's engines are major changes that won't be backported to the 1.4x branch. It sounds to me like this ticket really should say "1.50 will give me faster speeds than the current release because of new core features." Closing for these reasons.

comment:9 in reply to: ↑ 7 Changed 12 years ago by charles

Replying to jusid:

I reproduced it with uTorrent 1.8 as seed. Maybe this problem occurs only with uTorrent?

Can you test with uTorrent as well?

No, I don't want to install Wine.

comment:10 Changed 12 years ago by charles

  • Resolution invalid deleted
  • Status changed from closed to reopened

I'll run a uTorrent 1.8 -> Transmission { 1.41b2, trunk } test to make sure. I agree that this ticket is too vague, but on the other hand this is the kind of performance issue that I've been trying to get Transmission to be more stable at.

jusid: just to make sure I understand you correctly: you ran a test with uTorrent and Transmission both on your local LAN, so that they were both in your control, with no speed limits and full bandwidth available on your local network, and still the Transmission leech downloaded slowly from the uTorrent seed. Is that correct? Since it was all on your LAN there were no variables outside your control, is that correct?

comment:11 Changed 12 years ago by charles

  • Summary changed from Download from a peer is not persistent to Inconsistent download speeds

comment:12 Changed 12 years ago by jusid

Yes, you are correct, all my tests was performed in my LAN and both parties were under my full control.

I'll investigate more and test Transmission 1.41b2 to Transmission 1.41b2 scenario.

comment:13 Changed 12 years ago by jusid

I found that revision 7234 fixes this speed problem in trunk. All revisions before 7234 have this problem.

comment:14 follow-up: Changed 12 years ago by jusid

As I said, before version 1.22 does not have this problem. So, I started to investigate when exactly this problem began. The guilty revision is 5980.

Summary: this problem was introduced in rev. 5980 and fixed by rev. 7234.

comment:15 Changed 12 years ago by livings124

  • Resolution set to duplicate
  • Status changed from reopened to closed

This feature is too big to be backported to the 1.4x branch, but it will be in 1.50. Marking as duplicate of #1549, which implements r7234.

comment:16 in reply to: ↑ 14 Changed 12 years ago by mezz

Replying to jusid:

As I said, before version 1.22 does not have this problem. So, I started to investigate when exactly this problem began. The guilty revision is 5980.

Summary: this problem was introduced in rev. 5980 and fixed by rev. 7234.


Nice job for find when problem was introduced and when was solved.

comment:17 Changed 12 years ago by charles

  • Resolution duplicate deleted
  • Status changed from closed to reopened

comment:18 Changed 12 years ago by charles

I don't understand what r5980 and r7234 have to do with one another. Their diffs don't really seem to overlap much at all.

I would really like to have more information on what's going on -- either from user comments, or from personal testing -- before closing this ticket. Several users are reporting problems with 1.41b2's transfer rates. If these two revisions actually are the culprits -- though, right now, I can't imagine why -- then it's at least worh investigating.

jusid: what do I need to do, in order to reproduce your tests?

comment:19 Changed 12 years ago by jusid

After 2 days of building and testing I can explain all :)

The problem occurs at least with uTorrent client as seeder when download speed is relatively high.

Before r7234, Transmission did not report that it supports fast extensions to other peers. However, Transmission had some fast extensions code for long time (for example BT_REJECT message).

Here is scenario for Transmission before r7234:

During download, Transmission calculates the number of download requests based on download speed in ratePulse() function. At some point the number of requests becomes to high and uTorrent can not handle them. Since Transmission did not report that it support fast extensions, uTorrent do not send BT_REJECT message and silently drops these requests. They remain in peerAskedFor list and Transmission can not send new download requests. The download just stops for SENT_REQUEST_TTL_SECS (240 seconds). After that timeout, peerAskedFor list is cleared and download starts again for some time until uTorrent will be flooded with requests again. In 100MBit LAN Transmission downloads from uTorrent for 3-5 seconds and then stops for 240 seconds...

After r7234, Transmission properly reports that it supports fast extensions and uTorrent sends reject messages for requests that it can not handle at this moment. Rejected requests are deleted from peerAskedFor list and download do not stop. But download speed is not good enough (~2 Mb/s). uTorrent can download from uTorrent at 5MB/s speed under the same conditions.

Let's investigate further...

In r6101 ratePulse() code was changed and requests flood cases started to happen too often. Currently the number of requests is calculated for 30 (!) seconds ahead and they are sent in one batch. In my tests good results were achieved when calculating requests for 1 second ahead. It prevents too often requests flood and higher speed (after r7234 of course).

r5980 also introduced download speed drop. Download requests were delayed for HIGH_PRIORITY_INTERVAL_SECS seconds in protocolSendRequest() function. It is not good and results in download speed drops. Download requests must have IMMEDIATE_PRIORITY_INTERVAL_SECS, since they already sent in batch and it is not wise to delay them.

After these tweaks trunk version of Transmission was able to download at full speed of 5 MB/s.

I hope my investigation will be useful :)

comment:20 Changed 12 years ago by charles

...wow. I'm glad I reopened this ticket. :)

Very nice work.

comment:21 Changed 12 years ago by charles

jusid: I see your point about changing ratePulse() in the case of fast peers, but unconditionally changing it to a one-second calculation would make slow peers even slower.

It might be better to add to ratePulse() an upper limit to what peer->maxActiveRequests can be. In your tests for getting 5 MB/s, what is a good value for peer->maxActiveRequests?

comment:22 follow-up: Changed 12 years ago by jusid

When peer->maxActiveRequests is 300-350, 5 MB/s can be achieved. IMO it is not good idea to hard code some limit here.

Are you sure that slow peers will suffer from this change? If yes, the number of seconds can depend on peer speed.

comment:23 in reply to: ↑ 22 Changed 12 years ago by charles

Replying to jusid:

When peer->maxActiveRequests is 300-350, 5 MB/s can be achieved. IMO it is not good idea to hard code some limit here.

Are you sure that slow peers will suffer from this change? If yes, the number of seconds can depend on peer speed.

Yes, it does seem to make things worse when you're in a swarm of slow peers. It cuts down on the number of requests that are sent in a batch.

Hardcoding a number like 500 seems like a easy, but dirty, fix. It's unlikely that anyone will complain about that kind of a speed cap anytime soon. :)

But you're right that adjusting the number based on the peer's speed would be better. Judging from the tests you've run, do you think that a linear scale from a 30 second period with slow peers, to a 2 second period for peers of over 1 MiB/s, would be work?

comment:24 Changed 12 years ago by jusid

It is better to make scale from 30 to 1 sec for speeds from 0 to 1MiB/s. Peers with speed above 1MiB/s should have 1 sec period.

comment:25 Changed 12 years ago by charles

  • Milestone changed from None Set to 1.41
  • Summary changed from Inconsistent download speeds to Download speeds are inconsistent in situations with very high bandwidth

jusid: thinking over this ticket, and how to try to get the fix into the 1.4x branch (for 1.41) instead of having to wait for 1.50 for the Fast Extensions, I remembered the reqq key that uTorrent provides. That key tells peers how many requests they can have pending before the peer starts throwing away the excess requests. I created a separate ticket for that and checked the fix into trunk and into the 1.4x branch.

I've also checked in code to make outgoing requests an immediate' priority instead of a high' priority, as per your suggestion in comment #9. These changes are in r7322 (1.4x) and r7323 (trunk).

From your description. I think these two changes should prevent the weaknesses you found, even without support for the Fast Extensions protocol. If your fingers aren't too worn out from your previous tests, could you try checking out a copy of the 1.4x branch to see how it behaves at these high speeds?

comment:26 Changed 12 years ago by charles

actually this change is now in 1.41b3, if you want to try it there. :)

comment:27 Changed 12 years ago by jusid

I tested 1.4x branch and this speed problem was gone! Download speed is up to 5MB/s in my tests without drops due to requests flood. :)

Charles, many thanks for spending your time for fixing this issue! This ticket can be closed now.

comment:28 Changed 12 years ago by charles

  • Resolution set to fixed
  • Status changed from reopened to closed

comment:29 Changed 12 years ago by charles

jusid: thanks for the great research and investigation. Without your help this bug wouldn't've been fixed until at least 1.50.

Note: See TracTickets for help on using tickets.