Opened 12 years ago

Closed 11 years ago

Last modified 11 years ago

#1483 closed Enhancement (fixed)

New feature: a directory to store incomplete downloads

Reported by: kysucix Owned by: charles
Priority: Normal Milestone: 1.80
Component: Transmission Version: 1.40
Severity: Normal Keywords: daemon patch
Cc: tom@…, loial@…, woodzu123@…, kresh@…, thenewme91@…, Kilominha, transmission@…

Description

Patch that enables 2 new options in transmission-daemon:

  • -c --complete-dir <path> : enabling this option will move every file of a completed torrent in the specified directory (now available only in linux)
  • -d --watch-dir <path> : this option add a watchdir to the daemon. Every torrent created inside that directory will be automatically added to the daemon.

Attachments (6)

watchdir.c (4.1 KB) - added by kysucix 12 years ago.
watchdir.h (993 bytes) - added by kysucix 12 years ago.
watchdir.2.c (84 bytes) - added by kysucix 12 years ago.
deleteme
watchdir-movefiles.diff (10.5 KB) - added by kysucix 12 years ago.
sync patch with current trunk
move-completed.diff (5.8 KB) - added by casper 12 years ago.
no_incomplete_with_differing_download_folder.patch (855 bytes) - added by KyleK 11 years ago.

Download all attachments as: .zip

Change History (74)

comment:1 Changed 12 years ago by wereHamster

  • Cc tom@… added

I'll add automatic blocklist reloading once this patch is merged. Added myself to the CC list so I know when that is.

comment:2 follow-up: Changed 12 years ago by charles

  • Milestone changed from 1.41 to 1.50
  • Version set to 1.40

kysucix: you probably ought to add the torrent via RPC -- instead of directly with the ctor and tr_torrentNew() -- so that the clients will get a tr_rpc_callback notifying them that the torrent's been added.

Changing milestone from 1.41 to 1.50. 1.4x is strictly a bugfix branch.

comment:3 in reply to: ↑ 2 Changed 12 years ago by kysucix

  • Status changed from new to assigned

Replying to charles:

kysucix: you probably ought to add the torrent via RPC -- instead of directly with the ctor and tr_torrentNew() -- so that the clients will get a tr_rpc_callback notifying them that the torrent's been added.

Ok Now it add torrent through RPC using request_exec. Let me know Silvano

Changed 12 years ago by kysucix

Changed 12 years ago by kysucix

Changed 12 years ago by kysucix

deleteme

comment:4 Changed 12 years ago by kysucix

  • Keywords patch added

comment:5 Changed 12 years ago by kysucix

  • Keywords daemon added; damon removed

comment:6 Changed 12 years ago by charles

  • Milestone changed from 1.50 to None Set
  • Owner kysucix deleted
  • Status changed from assigned to new

Changed 12 years ago by kysucix

sync patch with current trunk

comment:7 Changed 12 years ago by loial

I'm interested in this added functionality. Would it be possible to add a (aptly named) patch to the current 1.40 version? I've tried applying the watchdir-movefiles.diff to v1.40 and that did not succeed without problems (no wonder since it is aimed at the current trunk as of 30-11-2008).

comment:8 Changed 12 years ago by loial

  • Cc loial@… added

comment:9 Changed 12 years ago by kysucix

1.41 and other releases will contains just bugfix. *Maybe* this patch will be on 1.50.

comment:10 Changed 12 years ago by charles

This is a good patch that we should use.

Before this goes into trunk, though, it would be nice to have a mac implementation of the watchdir code.

comment:11 Changed 12 years ago by kysucix

I found pnotify, a portable filesystem event notifier. http://mark.heily.com/pnotify/

It's an effort to wrap different watch syscall(inotify,kqueue,...) to just one. But even the test application does not work :|

I have a shell access on a freebsd machine, but I don't know wether kqueue has the same interface on macosx. any hint?

I'll try to make a freebsd implementation, but it's not so immediate as inotify

Otherwise what about applying this patch and wait later for a kqueue(or Spotlight syscall) implementation?

comment:12 Changed 12 years ago by kysucix

I thought It would be possible to use kqueue, but I did some test and it does not have the correct features.

So we should use fileSystem event API (macosx only):

http://developer.apple.com/documentation/Darwin/Conceptual/FSEvents_ProgGuide/Introduction/chapter_2_section_1.html

comment:13 follow-up: Changed 12 years ago by livings124

The mac version already has a watch directory using kqueue.

comment:14 in reply to: ↑ 13 Changed 12 years ago by charles

Replying to livings124:

The mac version already has a watch directory using kqueue.

No. The mac version of the gui client already has a watch directory using kqueue. There is no such generic feature so that the daemon can use it

comment:15 Changed 12 years ago by livings124

What I'm getting at is that it's possible to use kqueue for this functionality, not that it doesn't need to be implemented with Mac support.

comment:16 Changed 12 years ago by turbo

See #1647 (and #920)

comment:17 Changed 12 years ago by turbo

... and #1768 and, in some degree #1496

comment:18 Changed 12 years ago by turbo

Regarding the completedir:

Consider #920 and #1796 - Download into a 'secret' directory, then move it (either manually with #920 or from script (via #920) from #1796).

That's how I do it...

comment:19 Changed 12 years ago by charles

  • Owner set to charles

kysucix: I know you've been waiting for this patch to make it into trunk for quite awhile already. Any chance you could refresh the patch to bring it in sync with current trunk?

comment:20 Changed 12 years ago by charles

Experimental commit to trunk in r7971 to add a Watch directory to the daemon...

comment:21 Changed 12 years ago by KyleK

I saw that you removed the inotify() functionality in favor of an opendir() approach in one of the latest commits. May I suggest that this is only used as the last resort? (in case inotify is not available).

My reasoning is two-fold: My NAS does support inotify(), and so do most current Linux distributions. It is a far more efficient way to react on folder/file changes than to do it in intervals.

Also, the opendir() approach will basically prevent my NAS drives to go to standby.

In case there are now active downloads, and I haven't placed any torrent files in the watchfolder, the drive still won't be able to spin down and conserve power, since it is accessed again and again.

Let me know what you think.

comment:22 Changed 12 years ago by charles

  • Summary changed from [PATCH] transmission-daemon watchdir and completedir to [PATCH] transmission-daemon completedir

I'm splitting this ticket into its two separate pieces.

This current ticket is for move-when-torrent-completes; #1882 is for the watchdir.

comment:23 Changed 12 years ago by charles

  • Summary changed from [PATCH] transmission-daemon completedir to patch to move completed torrents to a user-specified directory

comment:24 Changed 12 years ago by turbo

I'm about to add the completeDir functionality to my #920 patch... But considering #1796, which should come (go) first: The move or the script?

comment:25 Changed 12 years ago by turbo

  • Summary changed from patch to move completed torrents to a user-specified directory to [PATCH] Move completed torrents to a user-specified directory

comment:26 Changed 12 years ago by livings124

  • Component changed from Daemon to Transmission
  • Summary changed from [PATCH] Move completed torrents to a user-specified directory to Move completed torrents to a user-specified directory

Once #920 is integrated into libtransmission, this functionality in the core would make much more sense.

comment:27 Changed 12 years ago by charles

I agree. I'm doing this feature now because I need it for my own computer :)

comment:28 Changed 12 years ago by charles

Whoops, that last comment was meant for a different ticket. I'm not currently working on this ticket.

comment:29 Changed 12 years ago by simonbcn

  • Version changed from 1.40 to 1.60

This patch works in Transmission 1.60 (Linux)?

comment:30 Changed 12 years ago by simonbcn

  • Version changed from 1.60 to 1.40

comment:31 Changed 12 years ago by woodzu123

  • Cc woodzu123@… added

comment:32 Changed 12 years ago by casper

  • Owner changed from charles to casper
  • Status changed from new to assigned

Looks like #920 has been commited. Still, there's no "move completed"-option in the daemon. Couldn't we simply call the move function on torrent completion if a setting is set? I'll implement that as I really need it.

comment:33 Changed 12 years ago by casper

  • Cc kresh@… added

comment:34 Changed 12 years ago by casper

attached patch "move-completed.diff" adds the option to libtransmission. to be used against transmission 1.71 release.

if config setting "completed-dir" is nonempty, a torrent gets moved there on completion using "tr_torrentSetLocation".

Changed 12 years ago by casper

comment:35 follow-up: Changed 12 years ago by teras

Any news on this patch? I tried to apply it on current source tree, and while it patches right (with some offsets of course), the parameters are not visible in the command line.

comment:36 Changed 12 years ago by livings124

#2209 is tangentially related to this.

comment:37 in reply to: ↑ 35 Changed 12 years ago by casper

Replying to teras:

Any news on this patch? I tried to apply it on current source tree, and while it patches right (with some offsets of course), the parameters are not visible in the command line.

The command line option for move-completed is not implemented yet, but the configuration setting is. Apply the patch, start transmission-daemon, stop it. Then there should be a setting in settings.json.

comment:38 Changed 12 years ago by neutrino

Hello,

thank you for the patch! the patch works well so far, i have found 1 bug.

if you delete the torrent via webgui ("Trash Data & Remove From List"), crash the program. :(

Version: Transmission 1.73 (8831) on Debian Lenny i386 Tranmission running as root, so permission problems are excluded.

The Log: [15:02:29.826] Found new .torrent file "test.torrent" in watchdir "/storange/transmission-daemon/import/" [15:02:29.827] test: Couldn't read resume file [15:02:29.828] test: Queued for verification [15:02:29.830] test: Verifying torrent [15:02:29.833] Found new .torrent file "test.torrent" in watchdir "/storange/transmission-daemon/import/" [15:02:30.113] test: Got 2 peers from tracker [15:01:48.164] test: State changed from "Incomplete" to "Complete" [15:01:48.172] test: moving "/storange/transmission-daemon/downloads/test/rzr-c5kg.001" to "/storange/complete/test/rzr-c5kg.001" [15:01:48.174] test: moving "/storange/transmission-daemon/downloads/test/rzr-c5kg.nfo" to "/storange/complete/test/rzr-c5kg.nfo" [15:01:48.176] test: moving "/storange/transmission-daemon/downloads/test/rzr-c5kg.sfv" to "/storange/complete/test/rzr-c5kg.sfv" [15:01:48.179] test: Queued for verification [15:01:48.180] test: Verifying torrent transmission-daemon: torrent.c:2349: tr_torrentCheckMoveCompleted: Assertion `tr_isTorrent( tor )' failed. Aborted

[14:59:23.142] test2.torrent" in watchdir "/storange/transmission-daemon/import/" [14:59:23.143] test2: Couldn't read resume file [14:59:23.144] test2: Queued for verification [14:59:23.145] test2: Verifying torrent [14:59:23.147] test2.torrent" in watchdir "/storange/transmission-daemon/import/" [14:59:23.371] test2: Got 1 peers from tracker [14:59:35.381] test2: State changed from "Incomplete" to "Complete" [14:59:35.384] test2: moving "/storange/transmission-daemon/downloads/test2/vcf-transporter.3.subfix.nfo" to "/storange/complete/test2/vcf-transporter.3.subfix.nfo" [14:59:35.385] test2: moving "/storange/transmission-daemon/downloads/test2/vcf-transporter3-subs.rar" to "/storange/complete/test2/vcf-transporter3-subs.rar" [14:59:35.385] test2: moving "/storange/transmission-daemon/downloads/test2/vcf-transporter3-subs.sfv" to "/storange/complete/test2/vcf-transporter3-subs.sfv" [14:59:35.386] test2: Queued for verification [14:59:35.388] test2: Verifying torrent [15:00:42.269] test2: Queued for verification [15:00:42.270] test2: Verifying torrent

Segmentation fault

comment:39 Changed 12 years ago by livings124

Ideally this feature would be the same as the Mac client's implementation - specify a location for incomplete transfers (as opposed to a separate "complete" location).

comment:40 Changed 12 years ago by KyleK

I had applied this patch to my copy of Transmission and it was working great, but ever since issue #2137 was fixed with changeset [8878], it breaks functionality. While complete torrents are still moved to the specified directory, Transmission for some reason doesn't know about the move and starts redownloading the torrent from scratch.

Before, they'd be moved and seeding would still continue.

comment:41 Changed 12 years ago by cbhl

  • Cc thenewme91@… added

comment:42 Changed 12 years ago by charles

I think the crash reported with http://trac.transmissionbt.com/attachment/ticket/1483/move-completed.diff is because, as a tr_timer func, tr_torrentCheckMoveCompleted needs to return an int and that int needs to be zero.

comment:43 Changed 12 years ago by atommix

  • Cc atommix added

This patch will be added to the trunk when it is stable enough?

comment:44 Changed 12 years ago by Kilominha

is it possible to add multiple directories and from these choose which one to move the completed file. For example I use webgui and I have checked a box which is called (specify directory for completed file) when this is checked, for each .torrent I add I get a windows which lets me specify one of my directories I have pre-chosen in the configuraion.

This can also work with watchdirectory, if the checkbox is checked, I get a popup window in webgui/qt/gtk "choose finished dir" and after it is clicked it start the download.

comment:45 Changed 12 years ago by Kilominha

  • Cc Kilominha added

comment:46 Changed 12 years ago by palfrey

Any plans for support for this feature for the Gtk client?

comment:47 Changed 12 years ago by palfrey

  • Cc transmission@… added

comment:48 Changed 12 years ago by charles

  • Milestone changed from None Set to 1.80

comment:49 Changed 12 years ago by charles

Partially added to trunk by r9328 for 1.80.

Remaining items:

  • mac client integration
  • qt client integration
  • daemon & remote integration
  • web client integration?

comment:50 Changed 12 years ago by charles

RPC, Qt client support added in r9330

comment:51 Changed 12 years ago by charles

transmission-remote support added in r9331

Unless I'm missing something because I'm tired, all that's left now is debugging, integrating with the Mac client, and possibly the web client

comment:52 follow-up: Changed 12 years ago by Rolcol

I think the behavior should be to remove the .part extension after the individual file has finished instead of when the entire torrent finishes.

comment:53 in reply to: ↑ 52 Changed 12 years ago by Rolcol

Replying to Rolcol:

I think the behavior should be to remove the .part extension after the individual file has finished instead of when the entire torrent finishes.

Ugh... I seem to be getting mixed results in this. Nevermind.

comment:54 Changed 12 years ago by charles

I think the behavior should be to remove the .part extension after the individual file has finished instead of when the entire torrent finishes.

That's exactly what #629 does, but it doesn't have anything to do with this ticket :)

comment:55 Changed 12 years ago by charles

13:53 -!- Rolcol [n=Rolcol@oalug/member/rolcol] has joined #transmission 13:56 < Rolcol> charles: I think the reason I comlained in the extension ticket was because when it's in libtransission, it doesn't send te message to the OS saying the file has changed and it doesn't automatically update Finder. 13:56 < Rolcol> (OSX) 15:03 -!- Rolcol [n=Rolcol@oalug/member/rolcol] has quit Lost terminal?

comment:56 Changed 12 years ago by Rolcol

Oops. Sorry for the incorrect ticket.

comment:57 Changed 11 years ago by livings124

  • Summary changed from Move completed torrents to a user-specified directory to Support an incomplete directory

r9342 updates the Mac client to use libtransmission's incomplete functionality.

comment:58 Changed 11 years ago by charles

  • Resolution set to fixed
  • Status changed from assigned to closed

comment:59 Changed 11 years ago by atommix

  • Cc atommix removed

comment:60 Changed 11 years ago by KyleK

While most of my downloads go to the default download folder, I sometimes specify a specific directory when adding a torrent, so the data is stored to a different hard drive.

With this new enhancement, data will first be downloaded to the Incomplete folder and then moved to the download directory (as expected). But, copying from one hard drive to another is rather resource-intensive, especially on embedded devices.

That's why I created a patch that ensures, that the incomplete folder is only used when the download folder is in fact the default location.

If necessary this could be made an option as well.

comment:61 Changed 11 years ago by Rolcol

  • Resolution fixed deleted
  • Status changed from closed to reopened

This feature broke the "Open Folder" feature for unfinished torrents.

<Rolcol> charles: "Open folder" is still broken for the incomplete feature.
..
<charles> Rolcol: I'd forgotten about that and can't look at it right now... you might want to reopen the ticket

comment:62 follow-up: Changed 11 years ago by charles

  • Resolution set to fixed
  • Status changed from reopened to closed

"Open Folder" fixed in r9498

comment:63 in reply to: ↑ 62 Changed 11 years ago by Rolcol

Replying to charles:

"Open Folder" fixed in r9498

Thank you, wise one.

comment:64 Changed 11 years ago by Rolcol

After testing this, I think the latest revision created a new bug. "Open folder" actually launches the item. If the download is a video, it opens up totem and plays it.

comment:65 Changed 11 years ago by charles

  • Resolution fixed deleted
  • Status changed from closed to reopened

reopening to give proper credit

comment:66 Changed 11 years ago by charles

  • Owner changed from casper to charles
  • Status changed from reopened to new

comment:67 Changed 11 years ago by charles

  • Resolution set to fixed
  • Status changed from new to closed

comment:68 Changed 11 years ago by charles

  • Summary changed from Support an incomplete directory to New feature: a directory to store incomplete downloads
Note: See TracTickets for help on using tickets.