Opened 12 years ago

Closed 12 years ago

#2442 closed Bug (worksforme)

Transmission speed limiting is incorrect by off-by-one type error

Reported by: krx Owned by:
Priority: Normal Milestone: None Set
Component: Transmission Version: 1.75
Severity: Normal Keywords:
Cc:

Description

When there is a call to transmission-remote -l, and the output is given, the end-line with upload and download sum-speeds seems to be incorrect by a factor of 0.5. For exampe, speed is set as follows:

-------------------------------------

root@bgt-h:~# transmission-remote -si
VERSION
  Daemon version: 1.75 (9117)
  RPC version: 6
  RPC minimum version: 1

TRANSFER
  Download directory: /mmc/transmission/downloads
  Listenport: 51413
  Portforwarding enabled: Yes
  Peer exchange allowed: Yes
  Encryption: preferred

LIMITS
  Peer limit: 511
  Upload speed limit: 25 KB/s  (Enabled limit: 25 KB/s; Disabled turtle limit: 45 KB/s)
  Download speed limit: 50 KB/s  (Enabled limit: 50 KB/s; Disabled turtle limit: 120 KB/s)
  Turtle schedule: 00:00 - 08:00  Sun Mon Tue Wed Thu Fri Sat


The report (truncated) always shows as follows:
-------------------------------------

root@bgt-h:~# transmission-remote -l



ID     Done       Have  ETA           Up    Down  Ratio  Status       Name
Sum:          179.2 GB              12.5    25.0

When looking to the router WAN interface bandwidth usage, it seams, that the correct speed limits are obeyed. Therefore I suppose, that the reporting is incorrect. Yet, I do not have definite proof for this, therefore both hypotehis should be tested.

Change History (9)

comment:1 Changed 12 years ago by krx

P.S. Intermediate individual torrents speeds are also divided (or throtled) by a factor of half.



Yet, this issue does not happen always. When I run transmission-remote -l few times in a row with some minutes random interval, sometimes the output exceeds 50% of the current speed limit (values are not factored, but as expected, are random).

comment:2 follow-up: Changed 12 years ago by charles

Thank you for taking the time to report this bug and helping to make Transmission better. Please answer these questions:

  • Is this reproducible?
  • If so, what specific steps should we take to recreate this bug?

This will help us to find and resolve the problem.

comment:3 in reply to: ↑ 2 Changed 12 years ago by krx

  • Severity changed from Normal to Minor

Replying to charles:

Thank you for taking the time to report this bug and helping to make Transmission better. Please answer these questions:

  • Is this reproducible?

Yes, it is. I see it almost every time in command line. Should not be hard to spot.

  • If so, what specific steps should we take to recreate this bug?

I have had an insight into this.

I think, that incorrect speeds are reported when there is specific condition. I believe that condition to be downloading at full speed allowed by limit, or near it. E.g. when Transmission downloads/uploads BELOW the speed limit, "transmission -l" reporting is working OK and telling the exact momentary speed. When Transmission downloads/uploads at the limit speed and is throtled, "transmission -l" show HALF the real speed.

Both download and upload reporting bug OR normal behaviour happens independently of each other. That means, that if the specific condition is not met, either the download or the upload, or even both of them are reported correctly.

I think this insight to be true, because "transmission -l" is reporting OK just after startup, when the peers connections have not been established and the speeds are slow. After the Transmission kicks-in fully after a few minutes of the startup, the speeds reporting column starts to report speeds in the buggy way. If the is concurent fast download/upload (better multiple from local/national provider subnets) through WAN link is started, which competes with the Transmission for the limited link bandwidth, this forces the Transmission to go even below set limits, the reporting is OK again.

Hope this is true, and will help to escape diging through the pile of code ;-)

comment:4 Changed 12 years ago by charles

  • Resolution set to worksforme
  • Status changed from new to closed

I'm not sure what to do about this ticket. You've explained it fairly well, but I can't reproduce it and neither could a volunteer in #transmission who tested it out... :/

If you feel like parsing json in your head, maybe run transmission-remote --debug and see if the raw values are in error...

comment:5 Changed 12 years ago by krx

  • Priority changed from Low to Normal
  • Resolution worksforme deleted
  • Severity changed from Minor to Normal
  • Status changed from closed to reopened
  • Summary changed from Transmission-remote -l endline sum-speed report is incorrect by half to Transmission speed limiting is incorrect by of-by-one type error

I have addition available. I was right about the incorrect reported speed, but I was unable to see the system. Now:

Speed limits are as follows:

LIMITS
  Peer limit: 511
  Upload speed limit: 25 KB/s  (Enabled limit: 25 KB/s; Disabled turtle limit: 27 KB/s)
  Download speed limit: 45 KB/s  (Enabled limit: 45 KB/s; Disabled turtle limit: 100 KB/s)
  Turtle schedule: 00:00 - 08:00  Sun Mon Tue Wed Thu Fri Sat

Real link speed is 512/256 Kbps during daytime speed limit and 1024/256 Kbps during "Turtle schedule" in night-time (which is a bit inversed logic, but it is so). That translates to: 62,5 /31,25 during day-time, and 125/31,25 KB/s during night time. I have tested the link for speed with www.speedtest.net, it is real to few percent accuracy. There are no any intrusive concurent download/upload processes running during measuring activity, except SSH session to router. As you may see, the set limits are well bellow link saturation threshold.

Now the torrents situation with any activity is as follows:

  34    76%     3.2 GB  21 hrs       2.0     5.3   0.38  Up & Down
  35   100%     1.1 GB  Done         0.0     0.0   0.35  Stopped
  36   100%     1.1 GB  Done         0.0     0.0   0.28  Stopped
  37   100%     1.1 GB  Done         0.0     0.0   0.25  Stopped
  38   100%     7.9 GB  Done         0.0     0.0   0.02  Stopped
  39   100%     9.0 GB  Done         0.0     0.0   0.05  Stopped
  40   100%     6.9 GB  Done         0.0     0.0   0.87  Stopped
  41   100%     6.0 GB  Done         0.0     0.0   0.55  Stopped
  42   100%     8.0 GB  Done         0.0     0.0   0.22  Stopped
  43   100%    16.4 GB  Done         0.0     0.0   0.43  Stopped
  44   100%     2.3 GB  Done         0.0     0.0   0.76  Stopped
  45   100%     2.0 GB  Done         0.0     0.0   0.62  Stopped
  46     5%   899.1 MB  25 days      1.5     5.7   0.11  Up & Down
  47   100%   697.7 MB  Done         0.0     0.0   0.53  Stopped
  48    14%   659.2 MB  Unknown      0.0     1.0   0.04  Up & Down
  49   100%   982.8 MB  Unknown      0.0     0.0   0.56  Seeding
  50    56%     2.5 GB  17 hrs      15.2    21.7   0.33  Up & Down
  51   100%   699.7 MB  Done         0.0     0.0   0.60  Stopped
  52   100%   633.8 MB  Unknown      0.0     0.0   0.19  Idle
  53   100%   667.5 MB  Done         0.0     0.0   0.09  Stopped
  54   100%     2.2 GB  Done         0.0     0.0   0.23  Stopped
Sum:          188.1 GB              18.7    33.7

Under these circumstances, link is FULLY saturated. I have ping-reply from my router to different locations in excess of 1600-2200 ms. That pretty much kill many normal communications with the host. If I quit transmission, latency drops to 9-15 ms. This is definetly bug with transmission speed limiting.

Now further. As you may see, there are 4 active torrents in upload/download mode (I have striped 1-33 torrents, as they are stopped). If you will divide upload limit of 25 KB/s by its momentary transmission reported speed of 18,7 KB/s (rounded!), you will get ~1,(3). The same is for download: 45/33,7 = ~1,(3).

There is definetly some situation with bad math, and actual enforcing (or rather not enforcing limits).

Now that I have error factor for 4 active torrents to be of 1,(3), I can actually recalculate the corrected speed limits, which will transmission will be capable to enforce with compliance with factual available bandwidth. Therefore during daytime: 62,5/1,3=46,875, and 31,25/1,(3)=23,4375 KB/s. As you see, my old UPLOAD limit of 25 KB/S is well over the capacity of upload bandwidth available. Therefore, I issue transmission-remote -u 20, and voula! -> ping latency drops to quite normal ~30-200 ms.

Because the factor of correction is so tightly mathematical, I believe that the error type here is called "off-by-one". That is why I was not able to indentify it in the previous encouters as such, because the incorrect limit momentary factor depends of the active torrents, separattely in the upload/download mode, and in the larger torrents queue this "virtual" factor changes rapidly.

Maybe because I use "inverse" turtle mode during daytime, I was able to notice this behaviour quickly, as it might be, that this limiting error is only in the "turtle" limit section. If the problem is apparent during night time only, it might be possible, that normal users do not even notice link saturation and thus increased latency on the router/hosts...

I do not have time now to check for this assumption, but maybe this information is enough to reconsider this incident for review. Hope this helps! ;-)

comment:6 Changed 12 years ago by livings124

  • Summary changed from Transmission speed limiting is incorrect by of-by-one type error to Transmission speed limiting is incorrect by off-by-one type error

comment:7 follow-up: Changed 12 years ago by charles

I've read this bug report three times now and still don't understand what it's saying.

The speed reporting code doesn't know anything at all about the limits or turtle mode; it just reports how many bytes have been placed in the peer's socket's sndbufs.

I also don't understand what "off by one" has to do with this.

comment:8 in reply to: ↑ 7 Changed 12 years ago by krx

Replying to charles:

I've read this bug report three times now and still don't understand what it's saying.

I am sorry for that. I do not know, how I might cleart this up, or rephrase. Please read only my last reply, and disregard the original posting with the follow-ups. Maybe this will be more clear.

The speed reporting code doesn't know anything at all about the limits or turtle mode; it just reports how many bytes have been placed in the peer's socket's sndbufs.

Yes, I understand that now. I thought, that maybe there is some feedback in this, but I was wrong. At first I have thought, that only the reporting is buggy. But now I think, that both the reporting and the limiting are bad.

The limits are not beeing enforced correctly, that is 100% sure, at least in my particullar case. But I think, that the second problem is that the bytes which are over the limit are not beeing accounted for in the reporting code. That is the only idea I have, why reporting does not show the true issue.

I do not know how the limiting code works. I think it counts all active torrents, separatelly for download/upload direction, and assign each torrent some % of all available limited bandwidth, based also on the torrent individual activity (popularity). Therefore, if the active torrent count is incorrect by -1, then there will be incorrect limit enforced, and the error will be the more visible, the less torrents are running at the given moment.

I know, I should not do any suggestions, because I do not know the code. I do not have any new symptoms to report. And as the symptoms are a bit ellusive, I have to make some ideas how the problem is born, to be able to articulate those ellusive symptoms.

I also don't understand what "off by one" has to do with this.

It means, that somewhere in the code, which is responsible for limiting, there is some math involved, that there is +1 (less likely) or -1 (more likely) done incorrectly, or maybe FOR/WHILE/UNTIL cycle runs one more or one less time as really needed.

Therefore:

  1. Limits set in the settings.json/or set by any interface are not beeing enforced.
  2. The actual speed in both directions is OVER the limits, set in the file/during session.
  3. The "overspeed" is dynamic, not static.
  4. The "overspeed" dynamics is dependent on the active torrent count in the particular direction (up/down).
  5. The reporting is incorrect, as it does not sees the "overspeed". Therefore it misleads during bug identification, and should be disregarded, and substituted by OS/hardware speed reporting.

IMHO, Limiting bug should be encountered first, the reporting - second.

That it is all about it. If there are particular questions to any points, I might clarify them.

I have two work-arounds: I set lower limits, that those, I might have set, if this had not been encountered, and I keep active torrent count high, as the more active torrents are running, the less there is drift in the "overspeed", due to lower divider error. Therefore, if this problem is evident only for me, you can close it, as it is counter-productive to the development. Maybe the bug will represent itself to anyone else and will be appended with usefull insights.

Another idea: perhaps the bug is platform dependend.

comment:9 Changed 12 years ago by charles

  • Resolution set to worksforme
  • Status changed from reopened to closed
Note: See TracTickets for help on using tickets.