Progressive downloads for streaming playback

[Waldorf]

This was posted in the ​forum.

Quote:
The reason for this feature would be to be able to use the BitTorrent? P2P protocol for streaming broadcast and also online video viewing, making it possible to start playing back the video file before the download is complete. One practical example would be if a multimedia-player integrated the Transmission BitTorrent? client as backend for downloading torrents files, you point the player to a torrent and Transmission starts to download the file progressivly and then the player automaticly start the playback once it has cached/buffered to certain procentage of the video, (similar to how Flash Video and the Flash Video player works).

Note: This "progressive download" method should of course be optional.

---

The ability to download progressively is very welkom indeed. I feel that also integrating a player would be overkill. Although an integrated player might feel bloat, if well implemented it shouldn't be.

If the backend supports it, then it should be the front-end responsibility to (automatically?) flag it for streaming.

[charles]

Adding a media player is very bloaty. The code it takes to do a media player right would double Transmission's footprint, at a minimum.

Also, having a begin-to-end download algorithm would create very unhealthy swarms, and is not a particularly good fit for bittorrent.

[Gamester17]

I am not asking for a media-player to be integrated into libtransmission but for libtransmission to have this "progressive download" feature so that libtransmission can in turn be integrated into third-party media-players and media-center software (I am personally thinking of XBMC, Boxee, and Plex/CenterStage?). libtransmission would only be the backend feeding the stream, the third-party audio/video-player would be the actual media-player that plays the audio/video stream (and deal with any other compression like ZIP or RAR if the files are archives as XBMC, Boxee, and Plex can already do).

So once again, I would like to request support for "progressive downloads" for "streaming playback" in libtransmission. I have rewritten this request to specificly suggested workarounds for the potential issue of unhealthy swarms.

In general, BitTorrent?'s non-contiguous download methods have prevented it from supporting "progressive downloads" (downloading sequentially) or "streaming playback". But recent comments by Bram Cohen suggests that streaming torrent downloads will soon be commonplace.

The reason for this feature would be to be able to use the BitTorrent? P2P protocol for streaming broadcast and also online video viewing, making it possible to start playing back the video file before the download is complete. One practical example would be if a multimedia-player integrated the libtransmission BitTorrent? client as backend for downloading torrents files, you point the player to a torrent and libtransmission starts to download the file progressivly and then the player automaticly start the playback once it has cached/buffered to certain procentage of the video, (similar to how Flash Video and the Flash Video player works).

Note! This "progressive download" method should of course be optional.

I believe that both BitComet??, FoxTorrent?? and µTorrent (a.k.a. uTorrent, which is also the Mainline BitTorrent? client since version 6.0) do already support progressive downloads algorithm as an option. There are also media-players such as BitLet?? and WeStream??, that prioritizes the bits at the beginning of each song/video, otherwise it will be impossible to stream anything before the entire track is downloaded, (however, like most other BitTorrent? clients, both BitLet?? and WeStream?? also gives a high priority to rare pieces in order to guarantee an optimal swarm speed, so that unhealthy swarms are not created). So integrating a BitTorrent? client with this "progressive download" method support should enable XBMC (and other audio/video-players and media centers alike) to begin playback of audio and video torrents without fully downloading them, depending on your download speed it would just cache 5% to 10% before starting the playback (given that it downloaded enough bits from the beginning of the file as well as enough rare pieces in order to guarantee an optimal swarm speed).

Additional peer2peer protocols/clients that feature progressive download support:

1-Click Media "1-Click Media is based in Paris, France and provides peer-to-peer plug-in to top TV channel portal. 1-Click Media guarante Speed, Bandwidth savings and Browser experience". 1-Click Media is an hybrid mixing BitTorrent? tracker and a HTTP webseed for each video (that way the video is always available even if there is not enough peers seeding online). ​http://www.1-click.com

Peer Impact can stream movies with its progressive download algorithm and they prioritise the first chucks Peer Impact also needs about a 200kbps to stream the movies that start within 30 seconds after the download is initiated in most cases . ​http://www.peerimpact.com/mediaCenter/index.html

Also Neocast is also streaming movies and doing live p2p ​http://neokastblog.com

PS! There is also some interesting research going on in the streaming space at this current time. ​https://www.tribler.org/test_streaming/index.php?n=Site.Introduction

[charles]

This is *not* going to be implemented in libtransmission, for reasons discussed many times in many places.

​http://www.azureuswiki.com/index.php/Sequential_downloading_is_bad

​http://forum.utorrent.com/viewtopic.php?id=18294

​http://forum.utorrent.com/viewtopic.php?pid=13269

clearly there's a demand for this sort of thing, so if you want to fork libtransmission and make your own client that does this, then best of luck to you. However -- for the umpteenth time -- this is never going to be a feature in Transmission.

[slacka]

The reasons discussed in those threads are all based on a dumb implementation. I agree 100% that sequentially downloading a torrent is bad for the health of the swarm and would never suggest it. However, this is not what uTorrent 3.0 implemented.

Instead they chose to continue to use the rarest first and choke algorithms for the majority of the file, but also created a small streaming buffer at the front of the file where the pieces get a higher priority. This maintains the health of the swarm, while enabling users the flexibility to watch or preview without waiting for the download to finish.

This method has been enabled in the mainline client for over 1 year now with no detrimental effects. But this is obviously this since it is no different than allowing users to prioritize the files in a torrent with many individual files, a feature that has been around since the beginning.

It’s crazy that I’m still running uTorrent using Wine, when I would be perfectly happy with transmission if it just supported this one feature.

Please reconsider with an implementation like described above.

[shaforostoff]

slacka: ktorrent has the feature you want: just open the video the embedded media player and the sequential download will start. a port of ktorrent to qt5 is in the progress right now

[pva0xd]

Documenting here in case somebody finds this bug as I did. Have not checked if that works, but I found patches: ​https://github.com/midenok/transmission-patches

As for issue at hand, probably, I need to read all the reasoning better... Yet I see that for the file I download 70% of peers use utorrent that supports sequential download.

[cfpp2p]

Replying to pva0xd:

Documenting here in case somebody finds this bug as I did. Have not checked if that works, but I found patches: ​https://github.com/midenok/transmission-patches

As for issue at hand, probably, I need to read all the reasoning better... Yet I see that for the file I download 70% of peers use utorrent that supports sequential download.

A patch like that looks to take no precautions whatsoever for preventing harm to the swarm. ​https://github.com/midenok/transmission-patches/commit/c92ea7c761a90346ff3d0fdeeb89a308fb040051#diff-8c3b89b4f93c549d0e6a76381537e699


"A torrent with multiple webseeds, a large swarm of seeders and very few or no leechers is best. Don't abuse. Try the "Weak" mode first before any of the stronger modes.

When streaming the file(s) of a torrent, the buffer at the front of each file helps maintain the swarm health "

​https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/def2dc6d/#da45
​https://github.com/cfpp2p/transmission/wiki/Streaming-of-torrent-files---how-to