Opened 7 years ago

Last modified 6 years ago

#5852 new Enhancement

Metatorrents - torrents pointing to other torrents

Reported by: d33tah Owned by:
Priority: Normal Milestone: None Set
Component: Transmission Version: 2.84
Severity: Normal Keywords:
Cc:

Description

I have an idea for a torrent extension and I wanted to create a reference implementation for Transmission, but wanted to ask Transmission developers first what they think of it.

My idea is to add support for "metatorrents", which would in their metadata have information about which torrents to download and what files to pick from them so that if you want to create "my favorite mp3s" collection, you could use better seeds instead of adding redundancy.

I created a forum thread, but received no attention and IRC channel operator mikedld suggested to open a trac ticket. What do you think of the idea? I already started a bit of the implementation and did quite a lot of thinking, so I can provide the details if you're interested.

Change History (2)

comment:2 Changed 6 years ago by d33tah

Here's the discussion I had on ##bittorrent on irc.freenode.net:

13:13:02   d33tah $ hello
13:13:16   d33tah $ i was thinking of an enhancement to bittorrent protocol that i call "metatorrents"
13:14:14   d33tah $ the idea is to enable torrents to reference other torrents so that you could borrow seeds from them - think of software with patches bundled with them, movies with subtitles, "my music collection" torrents that reference other discographies... what do you think of that?
13:15:41 The_8472 $ http://bittorrent.org/beps/bep_0038.html
13:16:11   d33tah $ what's the status of this bep?
13:16:21   d33tah $ ah, "draft". how could i help it improve?
13:17:04 The_8472 $ the status doesn't mean much. some draft BEPs are widely deployed, others are niche-implementation-only or mere proposals...
13:17:43   d33tah $ i'd like this one to happen and i already put some thinking on it. how could i make it succeed?
13:18:13 The_8472 $ it seems like you could already implement it.
13:18:56   d33tah $ okay, so i guess i'm back to square one - i'm talking with transmission project about getting some support on that ;)
13:20:08 The_8472 $ My guess would be that BT Inc. uses it for some commercial distribution, it makes sense if you control the distribution of a large amount of content. normal end users would probably not bother to ever use that feature during torrent creation
13:21:01 The_8472 $ since arvid co-authored it you might want to check libtorrent's source whether it supports it or not
13:22:00   d33tah $ i guess i'll contact him and cbrown
13:22:43   d33tah $ as for distribution - this could be helped by the trackers. it should be possible to implement it in the way that works on a per-piece basis and if you indexed the pieces, you could add some hints on finding peers to the .torrent file
13:23:41 The_8472 $ pieces are unlikely to be aligned. and trackers know nothing about torrent contents anyway.
13:24:22   d33tah $ hm, i meant the torrent websites.
13:24:56   d33tah $ i know that none of them does that right now but it could probably be changed
13:24:57 The_8472 $ in most cases they have insufficient knowledge to add that information
13:25:10 The_8472 $ it needs to be done on creation time
13:25:11   d33tah $ well the .torrent file contains the pieces - if the size matches, they have the hash
13:25:24   d33tah $ and if it's repeated, you can see how much the files have in common
13:26:06 The_8472 $ pieces of identical files in two different torrents neither have to be aligned nor of the same size. i.e. their hashes will be completely different
13:26:48 The_8472 $ you can only calculate this kind of identity if you have the content itself. which the websites do not
13:26:59   d33tah $ you're right, but in many cases they're not. i did this kind of research once on a small scale by pulling a lot of torrent files and trying to cross-reference them this way and i found similarities.
13:27:30   d33tah $ sometimes it's just a metadata change that happens at the end of the file
13:28:08   d33tah $ what do you think about it, The_8472?
13:29:49 The_8472 $ well, yes, for a limited subset it may be possible. you'd still have to get site admins and client devs to implement it though.
13:30:29 The_8472 $ and since it's only a hint there is the whole headache of verifying untrusted information with as little overhead as possible. you don't want your client to start downloading 100 unrelated torrents just because the file claimed it's related
13:30:43   d33tah $ The_8472: that's something that could be done separately from implementing this feature.
13:31:07   d33tah $ hmmm... what you just said sounds like an UX problem, doesn't it?
13:32:10 The_8472 $ well yeah, letting a human do the dirty work is one solution
13:33:36   d33tah $ it could be as simple as showing a window: "this torrent seems to reference 100 other torrents and this information could be used to speed up the download process. would you like to activate this experimental feature?"
13:34:21 The_8472 $ that's a terrible way to do it, but sure
13:35:27   d33tah $ why's that terrible?
13:37:34 The_8472 $ a) promising speedup [user will just click yes] b) not providing more detailed information c) calling things "experimental" with no idea how to improve it sounds like an eternal provisorium
13:38:04 The_8472 $ d) no fine-grained control
13:38:54   d33tah $ a,b,c can be easily fixed. this was a message proposed on the quick ;)
13:39:07   d33tah $ i'm not an UX expert, i'm just trying to imagine this thing working.
13:39:29 The_8472 $ well, you asked why it's terrible
13:39:36   d33tah $ yeah, sure, i understand, thanks for feedback
13:40:08   d33tah $ hopefully i'll be able to find somebody to implement it with - as soon as any of the popular clients would support this feature out of the box, i guess that this could be used as leverage
Note: See TracTickets for help on using tickets.