Opened 11 years ago

Closed 9 years ago

Last modified 9 years ago

#3219 closed Enhancement (fixed)

Don't scrape paused torrents

Reported by: niol Owned by: charles
Priority: Normal Milestone: 2.40
Component: libtransmission Version: 1.93
Severity: Normal Keywords: privacy paused scrape peer
Cc: alexandre.rossi@…

Description

Hi,

Using transmission as a daemon, I had expected that when all torrents are paused, the daemon does not have any network activity.

However, when I do a netstat on my machine, I can see :

  • what looks likes scrapes,
  • what looks likes incoming connections concerning my formerly announced torrents.

About scrapping paused torrents, there was #2623. Although solved as wontfix, it was fixed in changeset:10074 and the fix was reverted in changeset:10095 and I do not understand why (but I do not know the source enough to be sure). The fix still works.

About incoming connections, I think that the connection should be closed if the torrent info hash provided in the handshake does not refer to a running torrent. Would that be OK regarding the bittorrent protocol?

Maybe there is also DHT stuff running on paused torrents. But when I read

if( tor->isRunning && tr_torrentAllowsDHT(tor) )

it seems OK.

Thanks in advance for your comments.

Attachments (3)

opt-scrape-paused.diff (7.6 KB) - added by ijuxda 10 years ago.
Patch to svn trunk r11506
010-opt_scrape_paused.patch (5.7 KB) - added by hbt 10 years ago.
updated to 2.31 (left out gtk support, since I only use the daemon)
pause-scrape.diff (6.7 KB) - added by jordan 9 years ago.

Download all attachments as: .zip

Change History (33)

comment:1 Changed 11 years ago by charles

  • Summary changed from Ensure no activity for paused torrents to Don't scrape paused torrents

We can't do anything about incoming connections on your formerly announce torrents. You can tell a tracker when you're done with a particular torrent, but there's no way to guarantee that information is propagated to the other peers in the swarm. If there was a mechanism for that, it would probably be used as a vector for a DOS attack. :/

Accordingly, I'm changing this ticket's summary to address only the other issue, scrape-when-paused. IIRC we removed when we were debugging an announce bug and we readded it after we found and fixed the bug.

I agree with the idea of not scraping paused torrents. However IIRC this generates useful information for the Mac client, so I'd like livings to weigh in on this.

comment:2 Changed 11 years ago by niol

Thanks for your answer.

About incoming connections, I'm just asking if it is possible to simply close them when they are related to a paused torrent. On my computer, they stay ESTABLISHED and then time out.

On the scraping of paused torrents, could it be disabled when using the daemon only (i.e. for people who do not shut down transmission when they are not using it)?

comment:3 follow-up: Changed 11 years ago by Longinus00

Why wouldn't you want to scrape a torrent? If you don't scrape when paused then why scrape at all, announcing gives just as much information as scraping right?

comment:4 Changed 11 years ago by livings124

The whole reason for scrapes is to show basic information when paused. Scrapes would be a lot less useful if it only applied to active transfers. I'm on the "scrape when paused" bandwagon on this issue.

comment:5 in reply to: ↑ 3 ; follow-up: Changed 11 years ago by charles

Replying to Longinus00:

Why wouldn't you want to scrape a torrent?

If you have a lot of torrents to announce, the announces will go through faster if they're not competing with scrapes.

If you don't scrape when paused then why scrape at all, announcing gives just as much information as scraping right?

Announcing gives even more. However there's a limit to how frequently you can announce and how frequently you can scrape, so if you stagger it s.t. the scrapes come halfway between announces, you can keep the torrent's seed/leech stats better up-to-date.

Last edited 10 years ago by charles (previous) (diff)

comment:6 Changed 11 years ago by charles

  • Resolution set to wontfix
  • Status changed from new to closed

comment:7 Changed 11 years ago by niol

Does the wontfix means "never in transmission" or "not willing to do the necessary work to make this configurable or daemon only", because if it's the latter, maybe I can work on this. But I do not want to start work on a patch that has no chance of getting accepted.

comment:8 Changed 11 years ago by livings124

More than likely the former.

comment:9 Changed 10 years ago by User294

  • Resolution wontfix deleted
  • Status changed from closed to reopened

As for me, I dislike idea to scrape paused torrents, too. I do not need any stuff from tracker if torrent is paused (pretty obvious, huh?) and as for me, it's strange and unwanted activity and even could somewhat f...k up my privacy in some cases. For example, I may want to load torrent paused, look to it's content and maybe delete it if I figure out I do not want or need content. Right now Transmission will inform tracker that I have loaded torrent, potentially violating my privacy (wtf I should inform tracker that I added their torrent when I haven't started download yet and maybe will decide to abandon this at all?!).

Once I pause torrent, I do not expect or want any network activity associated with this torrent, i.e. IMHO, it should not be published to DHT, no requests to tracker, no pex activity, etc. I will be evil enough to reopen this. Or maybe I should try to ask for "Stop" feature which completely cancels all network activity associated with particular torrent, etc?

comment:10 Changed 10 years ago by charles

livings124 wants to keep scrapes in there so that it can show basic information on paused torrents.

Also, I think pausing scraped torrents will be useful in the future when the queue is able to look at S/L counts and use that to better guess which torrents should be unpaused from the queue.

I definitely see the other side to this though... there have been many times that I've started up Transmission with all torrents paused on some public network to test something that didn't require communications, and it bothers me that tracker network connections happen against my wishes. In addition to being annoying, I also wonder about privacy issues since some public torrents have so many trackers in their lists.

For the queue's purposes, I think we will need a separate state to differentiate between stopped and queued torrents. My preference would be to not scrape truly stopped torrents.

livings124, what is the basic information in paused torrents that you show in the mac gui? Is it visible from the main page, or only from the inspector? if the latter, could we trigger a scrape when the inspector is opened?

comment:11 Changed 10 years ago by Longinus00

What if I want to check the health of a torrent to determine which one to seed manually? This is also going to wreck havoc on the rpc clients as many (all?) of them show seeders and leechers as columns in a list display just like every other client I can think of (vuze, deluge, utorrent, etc.).

comment:12 Changed 10 years ago by livings124

charles: The basic information for scrapes shown is seeders, leechers, and number of downloads. It is only viewable in the inspector. I suppose we could defer this until needed, but that would slow down the display and possibly lead to stale information. Scrapes were designed to be lightweight, so I don't see these scrapes as a major issue.

comment:13 Changed 10 years ago by charles

  • Resolution set to invalid
  • Status changed from reopened to closed

Bah :)

I see both sides of this argument. I guess it probably is better overall to leave scraping enabled even for paused torrents.

comment:14 Changed 10 years ago by User294

Then maybe it's possible to have another torrent state like "stopped" when no any potentially unwanted activity takes place? I'm really want to trust my client that it will do what I'm asking to. Without trying to fool me that things are stopped while there is some unwanted activity related to certain torrent takes place while I explicitly asked to stop it. I can propose to have "pause" state which keeps some track of swarm state while suspends UL/DL activity (so it can quickly resume and suits your queue management needs) and separate "stop" state where torrent is completely disregarded, swarm state not tracked, all activity cancelled and yes, queue management should just never ever touch these torrents at all - they're stopped and it's up to user to start them, etc if desired. I.e. idea is to have "stop" mode which is similar to temporary deletion of torrent from UI/dirs without performing actual torrent deletion so easy to "restore" it by starting torrent. What do you think about it?

comment:15 Changed 10 years ago by livings124

Having a "stopped" state as well as a "paused" state is something we will not implement. It is extremely confusing and useless to all but a handful of users. Scrape activity is a trivial amount of bandwidth and part of the protocol.

comment:16 Changed 10 years ago by User294

extremely confusing and useless

So, using of any multimedia player is confusing? It's kinda simple, well-known and familiar logic: in pause mode, song state (position, etc) is kept. In stop mode all activity is stopped and it just shuts up, losing state and ceasing all activity. Offered logic resembles standard player logic well, isn't it? I have never heard someone blaming players for being confusing. Also some P2P apps are implementing similar logic. For example, aMule and eMule to name the two.

Scrape activity is a trivial amount of bandwidth

But can f..k up privacy in a non-trivial amount, to say at least. What if someone renamed child porn torrent to some innocent-looking .torrent filename? I do not want to download CP just to take a look what is inside of torrent. With Transmission I have to add it to see what's inside, just to make decision. And when I added it, even if I will specify that adding torrent should happen in paused mode (so I can peacefully look into it and decide if I want to download it), it has turned out this does not warrants that I would not scrape some tracker, reporting myself as a "some CP downloader/distributor" to it (while I'm explicitly do not want to be one of them, bah!).

and part of the protocol.

Actually, tracker is not a mandatory part of protocol for public torrents since DHT invention :P and magnets are yet another step in protocol evolution.

Well, looks like this is not going to be constructive, so I'll have to patch things myself for my local installation to make a client just trustworthy and predictable. Surely I can live with it.

comment:17 Changed 10 years ago by niol

Well, looks like this is not going to be constructive, so I'll have to patch things myself for my local installation to make a client just trustworthy and predictable. Surely I can live with it.

I can also live with applying at least changeset:10074 to my personal build of transmission.

Changed 10 years ago by ijuxda

Patch to svn trunk r11506

comment:18 Changed 10 years ago by ddxx0n

It would be very advisable to introduce a 'no scrape on pause option' for another reason not mentioned here:

Two trackers I am on (and I guess another rising number of others) use a slot system, meaning you can only have a limited number of active torrents at once. The slots expire after some time of inactivity and can then be re-used.

This collides with the constant scraping very badly: The trackers think that even paused torrents are active and allocate slots for them, thus it is impossible to tell transmission which torrents to enable (= which torrents should use the slots) and which to queue and pause for later download.

The only way around this is to use the patch above for my local build, and I must confess it totally escapes me why it hasn't been applied yet to trunk since I don't see any possible regressions.

Changed 10 years ago by hbt

updated to 2.31 (left out gtk support, since I only use the daemon)

comment:19 Changed 9 years ago by hbt

The patch only prevents scraping when transmission is started in paused mode. Once a torrent is unpaused and paused again, it will continue to scrape.

comment:20 Changed 9 years ago by aki

  • Resolution invalid deleted
  • Status changed from closed to reopened
  • Version changed from 1.93 to 1.93+

Okay, so if you think stop/pause will confuse users, how about adding an option in preferences dialog? By default it works as it is now and the 'a handful of users' also will be happy to disable scraping.

Apart from privacy issues, I have system running whole day but use transmission-remote in cron to unpause only during wee hours at night when my ISP allows free connectivity. Scraping will take significant amount of bandwidth if running whole day!(my limit is only 1GB/month during day).

Thank you.

comment:21 Changed 9 years ago by livings124

  • Resolution set to wontfix
  • Status changed from reopened to closed
  • Version changed from 1.93+ to 1.93

With multiscrape support in 2.3, this is even less necessary. An option for this is still confusing, as well as unnecessary.

comment:22 in reply to: ↑ 5 Changed 9 years ago by x190

Replying to charles:

Replying to Longinus00:

Why wouldn't you want to scrape a torrent?

If you have a lot of torrents to announce, the announces will go through faster if they're not competing with scrapes.

If you don't scrape when paused then why scrape at all, announcing gives just as much information as scraping right?

Announcing gives even more. However there's a limit to how frequently you can announce and how frequently you can scrape, so if you stagger it s.t. the scrapes come halfway between announces, you can keep the torrent's seed/leech stats better up-to-date.

Are we failing here? Most of my scrapes show as occurring at the same time or within 2-3 mins. of announces. This would be wasting limited web task slots and possibly annoying trackers. Ongoing scrapes for paused torrents does seem senseless.

Changed 9 years ago by jordan

comment:23 follow-up: Changed 9 years ago by jordan

  • Milestone changed from None Set to 2.40
  • Resolution wontfix deleted
  • Status changed from closed to reopened

There are valid for not scraping paused torrents -- it's arguably a minor privacy issue, and might reduce the number of short-term HTTP connections.

But these aren't overwhelming arguments and it's still an esoteric feature. If it were in the GUI, the main result would probably be confused users.

It would be useful as a "hidden" option in settings.json.

comment:24 Changed 9 years ago by jordan

  • Resolution set to fixed
  • Status changed from reopened to closed

comment:25 Changed 9 years ago by Username

Why not make this a default? Once torrent is paused, most users do not really care about it state, especially peers numbers and such. Sending non-requested parasitic traffic for nothing is very strange idea. Not to mention that if you'll run torrent for a while, tracker could die and there will be number of stuck or failing requests.

comment:26 Changed 9 years ago by livings124

Username: There are over 20 comments above explaining this. Just be happy there's a hidden option at all.

comment:27 Changed 9 years ago by x190

What is the default expected to be? Will this apply to the Mac Client? Sorry, but the answer is not immediately clear to me. Might be helpful to know re: forum. Thanks.

comment:28 Changed 9 years ago by Username

There are over 20 comments above explaining this.

And as for me, none of them provides a valid usecase. As the user I'm unable to understand why I need something from paused torrents at all.

Just be happy there's a hidden option at all.

Sounds a bit arrogant and strange to me. At the end of day, if something would annoy me enough, I can patch it myself in a private manner. Sure, I'm crappy coder (to say the least). But that's enough to eliminate annoying (mis)features, etc. At the end of day, I just failing to understand what problem you're trying to solve by scraping paused torrents. It like kicking the dead. Looks strange and pretty useless anyway.

comment:29 in reply to: ↑ 23 Changed 9 years ago by elpres

Replying to jordan:

It would be useful as a "hidden" option in settings.json.

Now that 2.41 is out, does this option also work in the MacOS version? According to ConfigFiles, the MacOS GUI version doesn't evaluate settings.json, only org.m0k.transmission.plist. Can this option be added manually to the plist, and if so, what would be the correct spelling?

comment:30 Changed 9 years ago by x190

elpres: If you have Xcode installed you can do this: (See ticket summary)

https://trac.transmissionbt.com/changeset/10074

Note: See TracTickets for help on using tickets.