DHT announce status display

[livings124]

​http://forum.transmissionbt.com/viewtopic.php?f=1&t=7617&p=37725#p37724

...you might want to consider exporting the tor->dhtAnnounceInProgress and tor->dhtAnnounceAt fields in the trackers pane (I'm thinking of a single field, say DHT announce due in, that either gives a time in minutes or says in progress).

[Elbandi]

could you make it like in screenshot? PEX and DHT are a "fake" trackers. Not only in GUI, but in rpc calls too (torrent-get - trackers).

And pls add to the trackers rpc "query" some statistic info: like next update, peers count from that tracker, stb

[chantra]

A potential patch, but to be honest, I have little knowledge of transmission code and had to modify the tr_stat structure. :s

for the GUI result, see screenshots

[charles]

chantra: looks good. :)

[chantra]

new patch:

• Adding peers'origin summary in Peer Tab
• fixing code indentation

[jch]

I like the screenshots very much indeed. Clear and unobtrusive.

I'd like to suggest that we also need to provide a handle on the DHT status (one of good, firewalled, etc., as obtained from tr_dhtStatus); this is essential for debugging connectivity problems.

--Juliusz

[jch]

I've tested this patch, but still not looked at it in detail. However, I've noticed that there's a discrepancy between the summary counts at the bottom of the peers window and the actual counts of peers. Are we computing something different for the summary display and the status field?

I'd suggest simply dropping the peer summary; on small screens, it makes the peers window less useful, and I don't think the information it provides is useful for most people.

The DHT display in the trackers pane, on the other hand, is exactly what I had in mind. Thanks.

--Juliusz

[User294]

Please not drop peers summary. This could be useful sometimes.

Wanna a good use case? Try to set up Transmission in a bit tricky network environment. Then try to seed something NEW (i.e. create torrent file and become a first seeder). Preferrably without using tracker. Then you will have some idea how painful it could be with Transmission when things are starting to be less ideal.

At first, you can't setup different ports for DHT and TCP. So, if you're having any network configuration which is anyhow unusual, you're in loss. And those who closed my bug about this have underestimated amount of non-PnP NATs in the world. Threre is many ISPs running NATs due to lacl of IPs (IPs are getting expensive due to IPv4 address space limits). At very most you can arrange ports forward with administrators and there is no warranty TCP and UDP ports will be same. But you're guys refusing to fix this. Very user-friendly, for sure.

Then you're figuring out that trackers are not responding or even hate your IP. Surely, that's great experience (for losers who as lucky as I was).

Then you're figuring out that with Transmission you can't even bootstrap DHT unless there is a tracker. And what if you failed to find one that works for you? You're loser, then. It will not work at all. I have to admit bootstrapping implemented in "very user friendly" manner. So once things are non-ideal you're doomed to miserably fail and no, user-friendly case is only for ideal conditions. In non-ideal it simply does not works at all rather than allow you to bootstrap from known IP:port of another machine, etc (which is like an emergency button - rarely needed but critical to have in these rare cases when you're got out of luck).

Ok, if you even managed to bootstrap DHT anyhow, you will have hard times to check if DHT is up and running. Neither log reports anything related to DHT, nor there is any proper status indicators, etc. So, you can't see DHT port state and even can't check if DHT is running at all. Unless you'll restort to using sniffers and indirect methods, etc. Very user-friendly for sure.

Ok, but if you still managed to reach this point? (highly unlikely, but...). Then, you can start seeding. And right now there is bare minimum of information about peers which allows to understand if you have to seed further or could stop seeding and swarm will live on it's own. Actually if you're going to remove something like info about peers, this will make transmission use for initial seeding almost impossible. Are you're creating client which virtually can't be used by initial seeders at all? And why I should "enjoy" by UI designed to run on small screen on my 17" 1280x1024 LCD? That's weird. Maybe you need to have simplified low-res version of UI? oO But it will be almost not suitable for serious client's use like creating new torrent and seeding it even if conditions are happened to be far from ideal ones.

Sorry for flame, I really had a bad day. Especially after figuring out how Transmission dev's are handling my bugs. Actually, I never seen worse handling of my bugs :(. Thank you very much, sirs.

[livings124]

I'm sorry us not satisfying your every whim does not meet your satisfaction. I have never seen a worse case of a user feeling such entitlement. But feel free to submit a patch of your own.

[jch]

Dear User294,

You're raising no less than three different points in your report:

1. different ports for TCP and UDP. That's #2265, and it won't be fixed unless you can provide us with a good use case. (And no, yelling about tricky network environments with no further details does not count as a use case.)

2. bootstrapping improvements. As you'll notice, I've reopened #2280, and I'm working on it right now. I'm planning the ability to add bootstrap nodes manually to file under config/, which will not obfuscate the UI for people who don't need the feature. Since I'm planning to make some other improvements to the DHT's bootstrapping, it will take some time.

3. monitoring UI for the DHT. We're open to any suggestion, since we haven't taken a decision yet.

Your input on these points is welcome. However, we don't like being yelled at, so please chill out.

--Juliusz

[jch]

Ping to charles.

I suggest taking a decision on this one, either way. It's been rotting for almost half a year.

--Juliusz

[livings124]

I'm thinking this might be good for post-1.8.

[livings124]

#2754 might be a part of this.

[jch]

Ping?

[livings124]

#3457 is a duplicate of this.

[gnu]

I am happy to improve the display of DHT status, but the feedback I got on the patch I posted at #3457 wasn't clear.

I see zero point in making separate status structures for each different source of possible peers. It seems like bad design to me. Modularity suggests having the GUIs report on each source of peers by using the same data structure. And it works.

Except all the different GUIs are going go need to make sure they can handle these non standards tracker entries in trackerstats, you had to fix a segfault just to make GTK client play nice with the new trackerstats.

Yes, I think it's cleaner to post a one-line patch to a GUI, which defends it against zero-pointers for peer sources that don't have a URL, than to define a new duplicative data structure and then write new code in every GUI to handle it.

DHT results won't give you seeder, leecher, or download numbers so the UI will have to have special code to cover that up as well. This is all a non issue if DHT trackers are exposed a different way.

Did you run the patched code I posted? The UI doesn't need special code to avoid reporting seeders/leechers/downloads. The existing trackerstats data structure already knows how to represent trackers that don't report those things. I'll post a screenshot.

And further, the comments in the patch also talk about later implementing BEP 33 (Bit Torrent protocol revision for DHT Scrape) that DOES allow DHT's to provide seeder and leecher counts. Why invent a separate data structure to remove these fields, then be forced to go back to the main data structure after implementing them?

This kind of discussion is what I was looking for when I did just a bit of coding, got it working, and posted it. Now I need direction from someone with authority to merge the code. What in specific should I improve to make it acceptable for merging?

[gnu]

Screenshot of DHT tracker status display for a running torrent

[gnu]

Screenshot of DHT tracker display when a DHT isn't bootstrapped yet

[gnu]

Hi, I've been hoping to contribute this feature to Transmission, but it's been 7 weeks since I asked "What in specific should I improve to make it acceptable for merging?" and I've gotten no answer.

What a great way to discourage your volunteers.

[charles]

I suspect the reason this ticket's been slow going is that none of the developers are sure what the Right Thing would look like.

IMO Chantra's patch doesn't Keep It Simple enough for Transmission -- the five people reading this ticket probably care about these details, but not many other end users ever will.

gnu's diff's GUI has a lighter touch which, of the two, I prefer. Longinus00 in #3457 pointed out that it's a bit awkward to shoeshorn DHT into trackers, and he's right. But it fits even worse on the other tabs, and IMO adding a new tab would be overkill. So what possibilities does that leave us? :)

[Rolcol]

Replying to charles:

So what possibilities does that leave us? :)

Some suggestions:

• Rename the trackers tab to something like "Peer Discovery"
• Just include it in the trackers with no other changes
• Do nothing and just have a party with hula girls

[charles]

I don't really like "Peer Discovery" because it takes something that users already know & understand ("Trackers") and replaces it with something more jargony.

I like the idea of partying with hula girls, but I don't think that's sufficient to close this ticket. :)

I kind of like leaving them in the Trackers tab, but they're not trackers. You can't edit them, so the add/edit/remove buttons in the tracker tab don't apply. You can't reannounce them manually, so users probably don't care when the last announce was. And the "show backup" and "show more info" checkboxes don't only kind-of-sort-of apply to them, too. This is probably the best of the three choices, but it's still not great.

....What about adding a new line to the Activity section of the Information tab for DHT giving something like "%'d IPv4, %'d IPv6"? We would have to jigger the Information tab around a little more to take #3549 into account, but still, the Activity section seems like a nice predictable place to put DHT peer counts.

[gnu]

Replying to charles:

I kind of like leaving them in the Trackers tab, but they're not trackers. You can't edit them, so the add/edit/remove buttons in the tracker tab don't apply. You can't reannounce them manually, so users probably don't care when the last announce was. And the "show backup" and "show more info" checkboxes don't only kind-of-sort-of apply to them, too. This is probably the best of the three choices, but it's still not great.

I could add the ability to edit them (at least turn them off and on for this torrent), and the ability to reannounce them manually. This pane would actually be a good place to add a general "query this tracker again" command. Currently I find it odd that I can see the trackers and their timestamps in the pane, but I have to go back to the big Transmission window's pop-up menu to do the "Ask tracker for more peers" thing. These abilities would also apply to local peer discovery, which would fit here as well.

The "show more info" checkbox will apply to the DHTs once we implement DHT scrape.

....What about adding a new line to the Activity section of the Information tab for DHT giving something like "%'d IPv4, %'d IPv6"? We would have to jigger the Information tab around a little more to take #3549 into account, but still, the Activity section seems like a nice predictable place to put DHT peer counts.

I find the Information tab useless compared to the other tabs, myself. I wish the Peers display came up by default, since it actually tells me something, and that something changes every few seconds. Also, isn't the Information tab full already?. I suppose you could bury the DHT info there, but to me the DHT really does act more like a tracker than anything else. Indeed it replaces a tracker, which is why I think people won't be surprised to find DHT info in the Tracker pane.

[User294]

charles

I kind of like leaving them in the Trackers tab, but they're not trackers.

...

so users probably don't care when the last announce was.

Are you sure? At least such indication could help to understand that DHT works and that it works properly without reading huge logs (there is no other obvious indications in UI anyway). Actually, it's hard to say I like how logging works in, say, GTK version at all. Especially if DHT haves some troubles due to say, partially firewalled environment, so it "partially" fails to bootstrap, etc - things can go crazy enough in logs and user can run into a major troubles like unresponsive client with current logs implementation (I'm probably should file one or two bugs about this).

which is why I think people won't be surprised to find DHT info in the Tracker pane.

And looks like we're not only 5 peoples who want to have a better information on how things are performing in overall for certain torrent. Just look at ​http://bayimg.com/image/faegeaaco.jpg - this image referenced by TPB from their blog when they explain what the hell DHT and magnet links are. Some other (popular?) client shows DHT, PEX and LPD as "trackers" and statistics shown here as well. As for me, I see nothing wrong with it. It's good to see all peer counts and all announce details in one place. I don't even know which client displayed on screen shot but I do know they implemented thing in a good way. It's both quite small. unobtrusive and informative. Single look to simple list in single place gives idea on how various subsystems are performing. And after all, TPB explains things to millions of their users in this way. So I bet you're incorrect about just 5 users reading this ticket :)

gnu

but to me the DHT really does act more like a tracker than anything else

Furthermore, you're definitely not the first person to implement things in this way. Look, some screen shot referenced in TPB blog explaining what are magnet links and DHT provides screenshot of some client. And this client implements DHT, LPD and PEX peer counts as "fake trackers", quite similar to you. As for me, that's simple for users and convenient since you can get overall state of torrent by looking for all counts in single tab. Even while DHT, PEX and LPD are not trackers, grouping them all in one place seems to be reasonable.

[charles]

I like both of those arguments. Maybe it should go in the tracker tab, after all. Longinus00?

[Longinus00]

I don't mind it in the tracker tab, I just don't like stuffing it in trackerStats.

[gnu]

I found trackerStats to be a pretty undocumented, ad-hoc structure anyway, so I'd be happy to update it to something more straightforward. Is your objection that the name of the structure is wrong, or that it contains the wrong members, or merely that it's the same structure currently used?

If I agree not to use trackerStats, my question is, what structure *should* it use to report tracker and/or DHT and/or LPD and/or PEX statistics? Having a different structure for each of N sources of peers would mean that each of the M back ends has to implement all the possible structures. Is there any point in making an NxM combinatorial problem out of this relatively simple status reporting feature?

[Longinus00]

My problem with using trackerstats is that many of its fields are meaningless when describing lpd, pex and dht. By reusing trackerstats you're forcing all other clients to play nice with invalid/nonsense tiers, trackerids, scrape times, etc. just because the gtk client happens to ignore it. By introducing a new place to store all this info other clients are able to choose if they want to implement displaying these statistics and where they choose to display them. A single structure whose members are the other peer sources should suffice and be extendable.

The fact that transmission doesn't have an indication of dht connectivity/nodes, as decided in #2279, like other clients leads me to believe that much of the proposed information is perhaps necessary. A more "transmissiony" approach might be to condense all the other peer sources into a single field saying something to the effect of "Seeing xxx peers from from other peer sources/Private torrent, other peer sources disabled". This could be accomplished simply by walking through the atom list and counting from dht, lpd and pex. A further benefit is that this could easily be put outside(below?) the trackerlist and won't make the tracker manipulation buttons behave inconsistently.

[gnu]

So I'm still interested in adding this feature to Transmission, but I'm still having trouble moving forward when somebody who's basically opposed to the feature (Longinus00) is setting the criteria for how to implement it.

Longinus00 wants "other clients are able to choose if they want to implement displaying these statistics". But clients don't actually choose this, DEVELOPERS of clients choose it. Regardless of what I do, other client developers could choose to suppress or display this information. My approach is to make it as easy as possible for them to display it (ideally it works without any attention from them). Longinus00 apparently wants me to make it as hard as possible for them to display the info.

My approach will be to go ahead and code the feature the way I'd like to code the feature, ignoring future input from Longinus00. If the team doesn't like the resulting code, then I guess I'll just be informing myself about DHT usage in my own torrents, and will have wasted my time trying to provide a feature to the rest of the Transmission users. Let this be a wakeup call to team members - in this case you-all have not been very welcoming to an incoming volunteer. Maybe you have enough developers already that you don't mind driving some away. If I had a thinner skin I'd have been long gone.

[charles]

I agree that trackerstats is pretty ad-hoc. It was mostly designed to give the fields necessary for the current tracker display. If you want to do cleanup work there as a basis for the rest of this, IMO that would be progress.

[Longinus00]

Replying to gnu:

So I'm still interested in adding this feature to Transmission, but I'm still having trouble moving forward when somebody who's basically opposed to the feature (Longinus00) is setting the criteria for how to implement it.

Longinus00 wants "other clients are able to choose if they want to implement displaying these statistics". But clients don't actually choose this, DEVELOPERS of clients choose it. Regardless of what I do, other client developers could choose to suppress or display this information. My approach is to make it as easy as possible for them to display it (ideally it works without any attention from them). Longinus00 apparently wants me to make it as hard as possible for them to display the info.

My approach will be to go ahead and code the feature the way I'd like to code the feature, ignoring future input from Longinus00. If the team doesn't like the resulting code, then I guess I'll just be informing myself about DHT usage in my own torrents, and will have wasted my time trying to provide a feature to the rest of the Transmission users. Let this be a wakeup call to team members - in this case you-all have not been very welcoming to an incoming volunteer. Maybe you have enough developers already that you don't mind driving some away. If I had a thinner skin I'd have been long gone.

I've laid out various reasons why I think overloading trackerStats is a bad idea. So far I'm not seeing any counterpoints to my specific concerns.

If the end goal is to replace trackerStats then the rpc commands should be overhauled completely to get rid of a few warts in the current specification (#3276 for instance).