Opened 11 years ago

Last modified 18 months ago

#532 assigned Enhancement

Collect unwanted, but received, blocks in a separate file

Reported by: Waldorf Owned by: charles
Priority: Normal Milestone: Sometime
Component: libtransmission Version: 1.03
Severity: Normal Keywords: patch
Cc: starburst, wfaulk, colrol@…, thenewme91@…, violetmaze@…, bruno.bierbaumer@…, bondiblueos9@…, transmissionbt@…, chris@…, groundzerox15@…, linus@…, alex.bock@…

Description

12/06/07 15:43:15 eisa01 said:
It's possible to store the extra data in a separate transmission.dat file or something. That's what µTorrent does, works flawlessly. I guess you could put this in the Application Support folder on macs too, which makes it even better for the user.
µTorrent stores it in the folder of the files being downloaded.


The idea is to, as overly discussed in #514, not actually create the files we didn't want in the first place. Instead of creating the deselected files we could keep these pieces in a cache of some kind. ...

Attachments (24)

sparse.c (375 bytes) - added by wfaulk 10 years ago.
simple C program to demonstrate sparse files
fix532.diff (6.9 KB) - added by ody 10 years ago.
Patch to store unwanted data in temporary files
transmission-532-r6847.diff (6.8 KB) - added by nox 10 years ago.
Patch against r6847
piece_temp.patch (30.3 KB) - added by ijuxda 7 years ago.
Patch to svn trunk r11625
piece_temp.2.patch (24.0 KB) - added by jordan 7 years ago.
piece_temp.3.patch (32.9 KB) - added by ijuxda 7 years ago.
Patch to svn/trunk r11625
piece_temp.4.patch (35.9 KB) - added by ijuxda 7 years ago.
patch to svn/trunk r11666
Screen shot 2011-01-17 at 10.47.14 PM.png (43.8 KB) - added by x190 7 years ago.
Inspector->Info Data File: N/A
piece_temp.5.patch (36.0 KB) - added by ijuxda 7 years ago.
Patch to svn/trunk r11744
piece_temp.6.patch (36.2 KB) - added by ijuxda 7 years ago.
Patch to svn/trunk r11822
piece_temp.7.patch (36.1 KB) - added by ijuxda 7 years ago.
Patch to svn trunk r11842
pieceTMP_8.patch (32.6 KB) - added by cfpp2p 6 years ago.
patch for r13255 v250+
pieceTMP_8-v242.patch (35.4 KB) - added by cfpp2p 6 years ago.
pieceTMP_8-v242.patch for MAC users without necessary SDK to build anything post v2.42
peiceTMP_9.patch (40.1 KB) - added by cfpp2p 6 years ago.
allows changing 'pieces' directory
peiceTMP_9b.patch (40.1 KB) - added by cfpp2p 6 years ago.
fixes minor compile issue (MAC and possibly others) thanks! to bb
peiceTMP_9c.patch (40.3 KB) - added by cfpp2p 6 years ago.
fix minor verification issues
pieceTMP_10.patch (41.6 KB) - added by cfpp2p 6 years ago.
100% done but skipped files fixed
pieceTMP_rc1.patch (41.9 KB) - added by cfpp2p 6 years ago.
fixes verify of 100% done skipped moved to downloads
pieceTMP_rc1a.patch (41.9 KB) - added by cfpp2p 6 years ago.
fixes rare problem with multiple files in a single piece
pieceTMP_rc1b.patch (41.8 KB) - added by cfpp2p 6 years ago.
patch for r13456
toggleInspectorCompletedFiles.patch (869 bytes) - added by x190 5 years ago.
tag275_pcTMP_rc2.patch (42.0 KB) - added by cfpp2p 5 years ago.
patch for 2.75 tag
branch27x_pcTMP_rc2.patch (41.2 KB) - added by cfpp2p 5 years ago.
patch for 2.75 branch
532-collect-unwanted-blocks-r14572.patch (38.7 KB) - added by mike.dld 3 years ago.

Download all attachments as: .zip

Change History (252)

comment:1 Changed 11 years ago by starburst

  • Cc eisa01 starburst added

comment:2 Changed 10 years ago by charles

  • Component changed from Transmission to libtransmission
  • Owner set to charles

comment:3 Changed 10 years ago by wfaulk

I get the impression that this bug is being considered only as tidiness, but large amounts of disk space can be wasted on filesystems that do not have sparse file support with the current method.

My generalized explanations in other bugs seemed to have been unclear, so I'll provide a specific example:

Imagine a torrent that contains three 500MB files. Assume that the blocks are 1MB in size. The user is only interested in the "middle" one. (That is, the one whose blocks lie between the blocks of the other two.) As such, he marks the first and last files as "do not download". Chances are that the block at the beginning and the block at the end of the middle file extend into the first and last files.

With the current method, the unwanted remainder of the block that ends the first file causes Transmission to create the first file, lseek() to the very end of the file, and write the 1MB or less of data. Under filesystems that support sparse files, this creates a file that consumes that 1MB or less of data. However, under filesystems that do not support sparse files, such as HFS, the default MacOSX filesystem, this creates a file that consumes 500MB of data.

To see this in action, I will attach a small C program named sparse. It creates a new file, seeks to 1GB into the file, then writes one byte. Under a sparsefile-supporting filesystem (ext2, for example), this will consume 1B of space. Under a sparsefile-nonsupporting filesystem, this will consume 1GB of space.

% uname -a
Linux hzlw2016 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:27:17 EDT 2006 i686 i686 i386 GNU/Linux
% ls -l sparsefiletest
ls: sparsefiletest: No such file or directory
% df -k .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda2             20641788   8655364  10937784  45% /
% ./sparse
% ls -l sparsefiletest
-rwx--xr--  1 wfaulk users 1073741825 Mar 17 18:02 sparsefiletest
% df -k .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda2             20641788   8655376  10937772  45% /

comment:4 Changed 10 years ago by wfaulk

  • Cc wfaulk added

Oops. Clicked wrong button. Continued:

% uname -a
Darwin mass.internal.beaglebros.com 9.2.0 Darwin Kernel Version 9.2.0: Tue Feb  5 16:13:22 PST 2008; root:xnu-1228.3.13~1/RELEASE_I386 i386
% ls -l sparsefiletest
ls: sparsefiletest: No such file or directory
% df -k .
Filesystem   1024-blocks      Used Available Capacity  Mounted on
/dev/disk0s2   122552320 120098208   2198112    99%    /
% ./sparse
% ls -l sparsefiletest
----------  1 wfaulk  wheel  1073741825 Mar 17 17:55 sparsefiletest
% df -k .
Filesystem   1024-blocks      Used Available Capacity  Mounted on
/dev/disk0s2   122552320 121146788   1149532   100%    /

Changed 10 years ago by wfaulk

simple C program to demonstrate sparse files

comment:5 Changed 10 years ago by charles

  • Milestone changed from Sometime to None Set

comment:6 Changed 10 years ago by jidanni

All I know is when I'm through downloading, I have to use e.g. find -printf '%S\t%s\t%p\n' to get some idea of which were the files I wanted, and which are just neighboring scraps mostly filled with nulls.

comment:8 Changed 10 years ago by jidanni

See bug 913 for a new idea to chmod to readonly mode the completed indiviual files.

comment:9 Changed 10 years ago by charles

See also: #987

Changed 10 years ago by ody

Patch to store unwanted data in temporary files

comment:10 Changed 10 years ago by charles

  • Milestone changed from None Set to Sometime
  • Status changed from new to assigned

comment:11 Changed 10 years ago by charles

  • Milestone changed from Sometime to 1.40

comment:12 Changed 10 years ago by charles

  • Version changed from 1.02+ to 1.03

comment:13 follow-up: Changed 10 years ago by eisa01

Just want to add that it's probably not a good idea to store it as one file in Application Support on macs. If people are downloading lots of partial torrents to external drives, they will lose a noticeable bit of HD space on their main drive. It's better to store the unused data in the folder for the torrent.

comment:14 in reply to: ↑ 13 Changed 10 years ago by ody

Replying to eisa01:

Just want to add that it's probably not a good idea to store it as one file in Application Support on macs. If people are downloading lots of partial torrents to external drives, they will lose a noticeable bit of HD space on their main drive. It's better to store the unused data in the folder for the torrent.

The impact of my patch in Application Support is at most twice the piece size of the torrent for each contiguous set of checked files - pretty small. The partial data in Application Support is deleted when the torrent is removed.

If by "the folder for the torrent" you mean the place where the torrent's checked files are being downloaded to, the whole point of this bug is that that's bad for usability; we're trying to prevent files that aren't checked from manifesting themselves in a readily user-obvious way in the filesystem.

comment:15 follow-up: Changed 10 years ago by wfaulk

I would argue that eisa01 is correct. For one thing, while usability is an important issue, drive space is the reason I submitted this bug originally. And assuming that the partial files are saved with a filename distinct from the filenames in the torrent, it shouldn't be a huge deal. Also, you could make the first character be a dot, which would hide them from the finder by default.

comment:16 in reply to: ↑ 15 Changed 10 years ago by ody

Replying to wfaulk:

I would argue that eisa01 is correct. For one thing, while usability is an important issue, drive space is the reason I submitted this bug originally. And assuming that the partial files are saved with a filename distinct from the filenames in the torrent, it shouldn't be a huge deal. Also, you could make the first character be a dot, which would hide them from the finder by default.

The patch solves the drive space issue as well as possible. It's a significant improvement over the previous situation, where if a piece spanned two files, the first of which wasn't downloaded, the first would be created at full size. When you consider the typical piece size of torrents, the disk space used by the files created by this patch is generally going to be on the order of tens of megs, which there should be plenty of room for in a user's home directory.

Creating hidden files in the torrent's folder doesn't strike me as a better solution - that would make the folder's total size mysteriously larger than the sum of the sizes of the visible files in it. I don't see either solution (storing unwanted data in Application Support or the torrent folder) as being significantly better than the other, really.

comment:17 Changed 10 years ago by wfaulk

Fair enough. Maybe a preference in a finalized version would make sense?

comment:18 Changed 10 years ago by charles

Too often preferences are added not because they make sense, but because the programmers couldn't come to a consensus and finally just pushed the choice onto the user.

IMO it's a bad idea to make this feature dependent on a preference option.

comment:19 Changed 10 years ago by wfaulk

For the record, by no means am I saying that this patch is awful if the partials are saved in Application Support. My preference is with the downloaded files, but in AS is still 1000x better than the current situation.

comment:20 Changed 10 years ago by starburst

I think having the partials in Application Support is a good solution.

It also has the advantage over saving them in the folder that when you would copy the folder you don't copy hidden partial files without realising it.

Changed 10 years ago by nox

Patch against r6847

comment:21 follow-ups: Changed 10 years ago by koriat

hey devs if your here i have a solution. lets say I have a torrent, piece size of 512 KB, and only one 4mb file selected for download. at most, that should be 2 pieces downloaded that contain data for unwanted files, one each at the beginning and end of the torrent. transmission then splits these blocks in half, places the end of the first block as the first bytes of data in the wanted file, and the beginning of the last block as the last bits of data in the wanted file. this extra data is then placed into the correct location for the unwanted files, creating 3total files that are all seperate from each other. if they wanted file is already downloaded, the extra 2 files can be deleted without affecting the middle file. the wanted file is fully cpomplete, so no more blocks need to be downloaded, thus NOT downloading the unwanted data.

please correct me if I amwrong about my assumptions that the files are now seperate once the wanted file is completely downloaded.

comment:22 in reply to: ↑ 21 Changed 10 years ago by jidanni

Replying to koriat: Sounds like what already happens, except you don't want to delete the "crust" files at the edges, as one day you might change your mind, and wish to get more of the torrent.

Idea: add a .tmp suffix:

So perhaps for incomplete files, attach .tmp to their file name, so one wouldn't run mplayer on them by accident. Wait, if a piece is a tail piece, it might have several minutes of mplayerable stuff, therefore how about e.g., .tmp1 and .tmp2 suffixes to distinguish them. Anyway, all the .tmp suffixes should then get removed when the file is complete.

comment:23 Changed 10 years ago by charles

  • Milestone changed from 1.40 to Sometime

comment:24 Changed 10 years ago by eisa01

  • Cc eisa01 removed

comment:25 Changed 10 years ago by charles

  • Keywords patch added

comment:26 Changed 10 years ago by charles

  • Milestone changed from Sometime to 1.50

comment:27 Changed 9 years ago by charles

  • Milestone changed from 1.50 to 1.60

comment:28 Changed 9 years ago by Rolcol

  • Cc colrol@… added

comment:29 in reply to: ↑ 21 Changed 9 years ago by wfaulk

Replying to koriat:

extra data is then placed into the correct location for the unwanted files

Very late reply. Sorry.

That is exactly what happened when the bug was reported and is the cause of the bug to begin with. Please reread wfaulk. But, in summary, if you create a file that has 1 byte at the end of a 1GB file, a full 1GB of space is used up. (Assuming the default filesystem under MacOSX.)

please correct me if I amwrong about my assumptions that the files are now seperate once the wanted file is completely downloaded.

You would no longer be able to upload or verify those two bounding blocks. Otherwise true.

comment:30 Changed 9 years ago by charles

  • Milestone changed from 1.60 to Sometime

comment:31 follow-up: Changed 9 years ago by thosmos

  • Cc thomas@… added

As a user I've been bothered by this problem a lot. I've been downloading large torrents with many folders of music. I would like to selectively download just the ones I want without getting a bunch of corrupt songs along with them, as I had done with µTorrent. This seems like an essential feature to me, and I'm surprised it's still unimplemented. I would be fine with ANY of the suggested solutions here. I actually don't care if the full or partial files are created temporarily either in place or somewhere else, or in a hidden folder, or in the Application Support folder, just as long as they don't show up as valid files in my Completed Transmissions folder when the torrent is finished (they can show up there if they are differentiated by a .tmp file ending as someone suggested). I eagerly look forward to having some version of this feature implemented. It's tempting me to try to find another torrent solution in the mean time. Other than this problem, I've enjoyed using Transmission and it seems like an otherwise well thought-out app.

comment:32 in reply to: ↑ 31 Changed 9 years ago by ody

Is there a particular reason for the delay in integrating this? The patch almost certainly doesn't apply cleanly anymore, and this does seem to be an oft-requested feature.

comment:33 Changed 9 years ago by charles

ody: did you just volunteer to update the patch? ;)

comment:34 Changed 9 years ago by livings124

  • Priority changed from Low to Normal

comment:35 Changed 9 years ago by charles

ticket #2101 has been marked as a duplicate of this ticket.

comment:36 Changed 9 years ago by urdh

  • Cc Sigurdhsson@… added

comment:37 Changed 9 years ago by charles

This should probably be implemented at the same time as #629 so that they don't step on each other's toes.

comment:38 Changed 9 years ago by cbhl

  • Cc thenewme91@… added

comment:39 Changed 9 years ago by porg

General question: Would it be possible (and efficient?) to solve the issue from the other side (that of the bittorrent protocol)? Is it possible to create torrents with multiple files in such a fashion, that a single torrent piece never spans multiple files? Example: A file is 510mb in size, each piece 50mb. You then get 10 pieces with 50mb each, and one remaining piece with 10mb. The problem which I see then, is that there are sheer many different remaining piece sizes, probably inefficient for the protocol?

If it cannot be solved from the protocol side, hence the possibility remaining that pieces can span files, I propose to give the user the choice via preferences, whether to store "remainder-files" within "Application Support" or in the "Torrent Download Folder", and set the default to "Torrent Download Folder". Reason: Data integrity likely higher, if people move torrent data folders.

comment:40 Changed 9 years ago by wfaulk

BitComet? did/does padding files. You can debate its efficiency.

comment:41 Changed 9 years ago by livings124

Padding files is universally despised - we will not be adding that.

comment:42 follow-up: Changed 9 years ago by jamesisin

I hope this matter is taken seriously. In my test the additional (unwanted) file-space was at least 697.6 MB (I won't know until the selected file has finished whether another 700 MB of drive space will be used at the other end of this file--and this is on a torrent whose pieces come in at 4.0 MB each).

comment:43 in reply to: ↑ 42 Changed 9 years ago by urdh

Replying to jamesisin:

I hope this matter is taken seriously. In my test the additional (unwanted) file-space was at least 697.6 MB (I won't know until the selected file has finished whether another 700 MB of drive space will be used at the other end of this file--and this is on a torrent whose pieces come in at 4.0 MB each).

That seems really strange. Shouldn't the absolute extreme of unwanted data be just one byte short of the peice size?

comment:44 follow-ups: Changed 9 years ago by jamesisin

No, it creates a data-lacking file of which the piece is a part (in this case a 700 MB avi). Presumably the 4 MB piece contains the tail end of this file and the front end of the file I selected. And just as presumably the piece that contains the tail end of the file I selected with also contain the beginning of the next 700 MB avi file. So, in this example, the extra crud amounts to near 1.4 GB. Ouch.

comment:45 in reply to: ↑ 44 Changed 9 years ago by urdh

Replying to jamesisin:

No, it creates a data-lacking file of which the piece is a part (in this case a 700 MB avi). Presumably the 4 MB piece contains the tail end of this file and the front end of the file I selected. And just as presumably the piece that contains the tail end of the file I selected with also contain the beginning of the next 700 MB avi file. So, in this example, the extra crud amounts to near 1.4 GB. Ouch.

Oh I see, I think I misunderstood your comment.

comment:46 in reply to: ↑ 44 ; follow-ups: Changed 9 years ago by charles

Replying to jamesisin:

No, it creates a data-lacking file of which the piece is a part (in this case a 700 MB avi). Presumably the 4 MB piece contains the tail end of this file and the front end of the file I selected. And just as presumably the piece that contains the tail end of the file I selected with also contain the beginning of the next 700 MB avi file. So, in this example, the extra crud amounts to near 1.4 GB. Ouch.

I think this is incorrect. Transmission uses sparse allocation for unwanted files, so even though it shows up as a 700 MB file it probably isn't using up 700 MB on your disk.

comment:47 in reply to: ↑ 46 Changed 9 years ago by porg

A little help for charles's problem determination:

On Mac OS X's default file system, HFS+ you do not have support for sparse files, hence you get disk consuming files, which are almost completely zero, except the one torrent segment which they loaded.

If you are on another file system, you can determine the real disk usage of the potential sparse file with the GNU command line program: du -B1 --apparent-size yourfile

comment:48 Changed 9 years ago by jamesisin

My system is using ext3, so I guess I get a savings...

Here is the output for the command above compared with ls:

$ du -B1 --apparent-size Billy\ Bites\ Yer\ Bum.avi 734011392 Billy Bites Yer Bum.avi

$ ls -al total 2620 drwxr-xr-x 2 4096 2009-11-30 01:03 . drwxr-xr-x 5 4096 2009-11-30 01:49 .. -rw-r--r-- 1 734011392 2009-11-30 04:21 Billy Bites Yer Bum.avi

And the properties dialog for that file (Nautilus) shows 700 MB (734011392 bytes). All identical.

But finally the 'sparse' size:

$ du -s -B1 -h Billy\ Bites\ Yer\ Bum.avi 2.6M Billy Bites Yer Bum.avi

However, I wonder if this blank space is 'free' space or 'available' space (in System Monitor, for instance).

This is a complicated issue and one without an apparent solution. The BitComet? (padding files) seems to have lasted all of eleven seconds (from v .85 to v .86).

The ideal solution would, I suppose, be for Transmission to store these pieces somewhere as raw data (rather than as the file from which the piece was taken, maybe a .pieces hidden file) and to then purge those extras when the torrent is deleted (and perhaps include a Purge button for manually removing any extras which were not deleted through the normal method of removing torrent files). This raw data (.pieces file) cannot be stored in the torrent location for obvious reasons and would have to be stored in some other Transmission accessible and controllable location (again presumably giving users the ability to determine that location manually).

Even this seems a little unwieldy as solutions go.

comment:49 in reply to: ↑ 46 Changed 9 years ago by KyleK

Replying to jamesisin:

I think this is incorrect. Transmission uses sparse allocation for unwanted files, so even though it shows up as a 700 MB file it probably isn't using up 700 MB on your disk.

Someone already mentioned HFS+, and I just wanted to add that on embedded machines, sparse allocation might not necessarily be available either. I know my NAS doesn't support it.

comment:50 Changed 9 years ago by charles

  • Summary changed from Collect unwanted, but received, pieces in a separate file to Collect unwanted, but received, blocks in a separate file

comment:51 follow-up: Changed 8 years ago by ierotuweoit

  • Version changed from 1.03 to 1.82

I'd just like to say that I really think this is something to be resolved, I can count many gigabytes of data left in .part-files of completed transfers. I agree with the Application Support approach to storing the left-over parts of blocks.

(I'm using an account from bugmenot.com because I don't want to go through the hassle of registering an account *just* so I can state my opinion on this matter)

Kind regards, Jørgen.

comment:52 Changed 8 years ago by livings124

  • Version changed from 1.82 to 1.03

comment:53 Changed 8 years ago by livings124

#3419 overlaps with this.

comment:54 in reply to: ↑ 51 Changed 8 years ago by x190

Replying to ierotuweoit:

I'd just like to say that I really think this is something to be resolved, I can count many gigabytes of data left in .part-files of completed transfers. I agree with the Application Support approach to storing the left-over parts of blocks.

(I'm using an account from bugmenot.com because I don't want to go through the hassle of registering an account *just* so I can state my opinion on this matter)

Kind regards, Jørgen.

+1 for this idea!

comment:55 Changed 8 years ago by charles

Ticket #3827 has been closed as a duplicate of this ticket.

comment:57 Changed 7 years ago by ijuxda

Here's a preliminary implementation loosely based on nox's patch.

  • Data from unwanted pieces is written to files (from here on referred to as "temporary piece files") in a directory under pieces in the config dir in the form:
    <torrent hash string>/<piece number>.dat
    
  • A session setting controls whether this feature is used.
  • If the real target file exists, then it is used regardless of the setting.
  • When a file's DND status is set to false (i.e. the file is "wanted") any existing temporary piece files are deleted and the corresponding pieces in the torrent are removed (referred to as "invalidation").
  • When the setting is toggled off all temporary piece files for all torrents are invalidated.

Removing the pieces from the torrent was chosen over writing the piece data back to minimize disk activity.

Comments and testing welcome.

Last edited 7 years ago by jordan (previous) (diff)

comment:58 follow-up: Changed 7 years ago by jordan

ijuxda: there's a lot to look at in this patch, but just as a first thought, what's the use case for being able to toggle this feature via TR_PREFS_KEY_PIECE_TEMP_ENABLED?

comment:59 Changed 7 years ago by jordan

In tr_torrentRemovePieceTemp(), why call deleteLocalFile() outside of the readdir() loop? Does deleting a file invalidate readdir() on some platform?

comment:60 in reply to: ↑ 58 Changed 7 years ago by ijuxda

Replying to jordan:

ijuxda: there's a lot to look at in this patch, but just as a first thought, what's the use case for being able to toggle this feature via TR_PREFS_KEY_PIECE_TEMP_ENABLED?

If the filesystem supports sparse files there are no appreciable space savings from using this feature, and the invalidation penalty when toggling DND can be avoided.

In tr_torrentRemovePieceTemp(), why call deleteLocalFile() outside of the readdir() loop? Does deleting a file invalidate readdir() on some platform?

Being that the answer is "maybe", and that I cannot test all possible platforms, I decided to err on the side of caution.
http://stackoverflow.com/questions/1676522/delete-files-while-reading-directory-with-readdir

Changed 7 years ago by ijuxda

Patch to svn trunk r11625

comment:62 Changed 7 years ago by jordan

I'm still finding my way through this code, but have a few more thoughts on this patch:

  • I still think it would be better to always leave this feature enabled. Few if any users will link their filesystem's sparse file support to a performance hit when toggling DND files. In addition it adds more configurations that will have to be tested, debugged, and maintained. It may complicate future bug reports, yadda yadda. Removing the toggle would eliminate tr_sessionSetPieceTempEnabled(), tr_sessionIsPieceTempEnabled(), TR_PREFS_KEY_PIECE_TEMP_ENABLED, and tr_torrentInvalidatePieceTemp().
  • The directory names underneath the pieces/ directory should probably follow the same naming schemes as in torrents/ and resume/ by using tr_metainfoGetBasename() when assigning tor->pieceTempDir in torrent.c
  • A lot of the new methods are package-visible when they could be class-visible instead.
  • We still could batch pieces together in tr_torrentInitFileDLs() s.t. there aren't a lot of redundant calls to the piece invalidation code.

Attached is a thinking-out-loud patch that addresses these points.

Changed 7 years ago by jordan

comment:63 follow-up: Changed 7 years ago by jordan

Other things not addressed in the above patch:

  • When toggling a file's DND flag, we should try to preserve what we've already downloaded by moving it from the destination file to the piece file, or vice versa.
  • When the user specifies that they don't want to download a file, if that file exists we should close it and remove it.
  • Looking at this code again, it's a little annoying how messy it is using the negative logic of the DND flag. We should probably shoot it and replace it with a "doDownload" flag instead... :)

comment:64 in reply to: ↑ 63 ; follow-up: Changed 7 years ago by ijuxda

Replying to jordan:

When toggling a file's DND flag, we should try to preserve what we've already downloaded by moving it from the destination file to the piece file, or vice versa.

I can try to implement this using the "piece overlap" calculation ideas from #3865, if you think the amount of disk IO required is acceptable.

When the user specifies that they don't want to download a file, if that file exists we should close it and remove it.

In my opinion this ability is better provided by the delete menu-entry approach in #3865; I would consider file/piece dnd status to control what is given to or requested from peers, and the "delete" functionality to control what exists on the disk. Also, if clicking the checkbox in the file list (in the gtk client) deleted the file, it might be too easy to do this by mistake.

Looking at this code again, it's a little annoying how messy it is using the negative logic of the DND flag. We should probably shoot it and replace it with a "doDownload" flag instead... :)

I don't mind the double negative. Another name possibly better than "dnd" would be "wanted", as in "this file is wanted by the user". It would appear to already be in use in this way in some parts of the code and would at least avoid the camel case or underscore required for "do download".

P.S. To avoid needless formatting changes in future patches, is there a coding style guide somewhere I can follow, or should I just try to mimic the style of your patch and other recent commits?

Last edited 7 years ago by ijuxda (previous) (diff)

comment:65 in reply to: ↑ 64 Changed 7 years ago by jordan

Replying to ijuxda:

Replying to jordan:

When toggling a file's DND flag, we should try to preserve what we've already downloaded by moving it from the destination file to the piece file, or vice versa.

I can try to implement this using the "piece overlap" calculation ideas from #3865, if you think the amount of disk IO required is acceptable.

disk IO is faster than bandwidth, so it's worth a try.

When the user specifies that they don't want to download a file, if that file exists we should close it and remove it.

In my opinion this ability is better provided by the delete menu-entry approach in #3865; I would consider file/piece dnd status to control what is given to or requested from peers, and the "delete" functionality to control what exists on the disk. Also, if clicking the checkbox in the file list (in the gtk client) deleted the file, it might be too easy to do this by mistake.

Good point.

P.S. To avoid needless formatting changes in future patches, is there a coding style guide somewhere I can follow, or should I just try to mimic the style of your patch and other recent commits?

You've been doing fine. My guess is the confusion is coming because I've slowly been changing the code from camelcase to gnomeish monocase and underscores.

(One of these days we need to get rid of the crazy parenthesis spacing too ;)

comment:66 Changed 7 years ago by x190

Ijuxda: Your work on this ticket is much appreciated. As OS X users, this is something that has been on my and many forum posters 'wishlist' for a long time.

comment:67 Changed 7 years ago by jidanni

I think it is just fabulous that you guys are working on this. However the excitement is just too much and I now wish to remove myself from the Cc: list for this bug. However all I found was a way to "Add to Cc: jidanni@…" But no way to remove myself, or even see if I am actually on the Cc list, even thought it sure feels that way. Therefore could the (buggy) Trac Owner kindly remove me. Thanks.

comment:68 follow-up: Changed 7 years ago by jordan

There's no jidanni on the CC list. There's a starburst, wfaulk, colrol, thomas, Sigurd, and thenewme... are you one of them? :)

comment:69 in reply to: ↑ 68 Changed 7 years ago by jidanni

Replying to jordan: Nope. I can't see past the first few, but I'm only jidanni. Odd, I'm not the reporter, not on the Cc list. Then why am I getting these notices? I don't want to unsubscribe to all my bugs here. Just this one.

comment:70 Changed 7 years ago by jidanni

I'll try adding myself to the CC list, then removing it back...

comment:71 Changed 7 years ago by jidanni

  • Cc jidanni@… added

comment:72 Changed 7 years ago by jidanni

  • Cc jidanni@… removed

OK, I'm going to remove myself from the Cc list and resist the urge to comment, lest I get added back to some unlisted cc list...

Changed 7 years ago by ijuxda

Patch to svn/trunk r11625

comment:74 follow-up: Changed 7 years ago by jidanni

Should I file a bug to get me unsubscribed from this bug?

comment:75 in reply to: ↑ 74 ; follow-up: Changed 7 years ago by jordan

Replying to jidanni:

Should I file a bug to get me unsubscribed from this bug?

If I knew how to unsubscribe you I already would have, since you've already told me about it. filing a new bug would only serve to tell me again :)

Last edited 7 years ago by jordan (previous) (diff)

comment:78 in reply to: ↑ 77 ; follow-ups: Changed 7 years ago by ijuxda

Replying to x190:

Okay, I just checked out r11667 from svn complete with xcodeproj so if someone can give me the steps to obtain and apply this patch in Xcode, I should be able to test on macosx. EDIT: Just found the d/l button for the patch, so all I need is Xcode application instructions.

Save the patch file to the source directory containing r11667. In a terminal go to that directory and issue the command

patch -p1 < ./piece_temp.3.patch

then build as you normally would. I don't know for sure if osx has patch or that its shell behaves like bash, but I assume given its posix compliance the above command should not be too far off.

comment:79 in reply to: ↑ 78 Changed 7 years ago by gunzip

Replying to ijuxda:

Save the patch file to the source directory containing r11667. In a terminal go to that directory and issue the command

patch -p1 < ./piece_temp.3.patch

then build as you normally would ...

tried your patch with r11667 and so far it seems to be working as advertised, i.e., there are no de-selected files showing up in the torrent download directory. poking around i noticed a new directory "pieces" in the config-dir where you hid the overlappinging non-needed parts. pretty neat.

i'll play around with it more and see how it goes -- but so far so good.

OS Linux Debian

comment:80 in reply to: ↑ 78 Changed 7 years ago by x190

Replying to ijuxda:

Replying to x190: Save the patch file to the source directory containing r11667. In a terminal go to that directory and issue the command

patch -p1 < ./piece_temp.3.patch

then build as you normally would. I don't know for sure if osx has patch or that its shell behaves like bash, but I assume given its posix compliance the above command should not be too far off.

Thanks! That appeared to work and build was successful altho I got 88 warnings. Does this patch work on currently added torrents or just new ones? So far, I haven't seen any data moved around on a seeding torrent with unwanted .parts. It just hit "Seeding complete" and no unwanted parts were moved. Will try with new torrent.

SL 10.6.6

comment:81 in reply to: ↑ 75 ; follow-up: Changed 7 years ago by jidanni

Replying to jordan:

Should I file a bug to get me unsubscribed from this bug?

OK, I instead filed a bug with the real perpetrators, http://trac.edgewall.org/ticket/9971 .

comment:82 Changed 7 years ago by x190

I got it working with a new selection farther down the list on a multi-folder/file torrent. :D Potential disk space saved -- 5 GB!!!! Well done, ijuxda. Puts .dat pieces in ~/Library/Application? Support/Transmission/Pieces?. I'm thinking it's okay that it doesn't mess with previously d/led data unwanted or otherwise? Details will emerge with further testing. Again, good work! :)

SL 10.6.6 2006 Intel

Last edited 7 years ago by x190 (previous) (diff)

comment:83 follow-up: Changed 7 years ago by x190

One small bug. When a user-requested "Verify Local Data" is done on a completed (100% selected) torrent, it does not find the pieces which are partially stored in Pieces/Torrentname?.xxxxxxxxxxx/.dat and therefore must re-download them to complete the file.

comment:84 in reply to: ↑ 81 ; follow-up: Changed 7 years ago by jidanni

Replying to jordan: OK Jordan, they tell you how to unsubscribe me from this bug in http://trac.edgewall.org/ticket/9971 . Thanks.

comment:85 Changed 7 years ago by urdh

  • Cc Sigurdhsson@… removed

comment:86 in reply to: ↑ 84 ; follow-up: Changed 7 years ago by jordan

Replying to jidanni:

Replying to jordan: OK Jordan, they tell you how to unsubscribe me from this bug in http://trac.edgewall.org/ticket/9971 . Thanks.

jidanni, I would be happy to unsubscribe you but I don't see the instructions you're referring to.

That link refers to two more of trac's tickets that don't list a workaround. It also mentions changing always_notify_updater in trac.in, but that's a systemwide setting that would affect lots of trac users, not just you.

I'm not sure what you want me to do with this information.

Last edited 7 years ago by jordan (previous) (diff)

comment:87 in reply to: ↑ 83 ; follow-up: Changed 7 years ago by ijuxda

Thank you x190 and gunzip for testing.

Replying to x190:

I'm thinking it's okay that it doesn't mess with previously d/led data unwanted or otherwise?

If the destination file already exists, it just uses that. The delete feature in #3865 could be used to save space for previously downloaded torrents, but the patch in that ticket needs to be updated once the implementation in this ticket is finalized.

Replying to x190:

One small bug. When a user-requested "Verify Local Data" is done on a completed (100% selected) torrent, it does not find the pieces which are partially stored in Pieces/Torrentname?.xxxxxxxxxxx/.dat and therefore must re-download them to complete the file.

Thanks for pointing that out. I'll check out the verify code to try and find out what is happening.

comment:88 in reply to: ↑ 87 ; follow-up: Changed 7 years ago by jordan

Replying to ijuxda:

Thanks for pointing that out. I'll check out the verify code to try and find out what is happening.

Right, I missed that bug too. The verify code is a mostly self-contained chunk of code in verifyTorrent() that walks straight through the files using tr_torrentFindFile(). That logic would need to be redone to take piece files into account.

comment:89 in reply to: ↑ 88 Changed 7 years ago by x190

Replying to jordan:

Replying to ijuxda:

Thanks for pointing that out. I'll check out the verify code to try and find out what is happening.

Right, I missed that bug too. The verify code is a mostly self-contained chunk of code in verifyTorrent() that walks straight through the files using tr_torrentFindFile(). That logic would need to be redone to take piece files into account.

You may be away ahead of me on this, but I assume the piece(s) in question would also fail the jit verification check if they were requested by peers during the d/ling or seeding process.

Last edited 7 years ago by x190 (previous) (diff)

Changed 7 years ago by ijuxda

patch to svn/trunk r11666

comment:90 follow-up: Changed 7 years ago by ijuxda

Version 4:

  • Verify and resume should now work as expected.
  • Fixed a bug that would cause writes past the end of piece files.
  • Switched back to camel case for function names to match surrounding functions.

Replying to x190:

You may be away ahead of me on this, but I assume the piece(s) in question would also fail the jit verification check if they were requested by peers during the d/ling or seeding process.

I haven't noticed anything bad happening during download tests yet, but I'll check that.

comment:91 in reply to: ↑ 90 Changed 7 years ago by ijuxda

Replying to ijuxda:

Replying to x190:

You may be away ahead of me on this, but I assume the piece(s) in question would also fail the jit verification check if they were requested by peers during the d/ling or seeding process.

I haven't noticed anything bad happening during download tests yet, but I'll check that.

The code ends up using either the cached data or tr_ioRead (which respects temporary piece files), so jit verification should work without problems as far as I can see.

comment:92 follow-up: Changed 7 years ago by gunzip

@ ijuxda: can you petition the developers to get your patch piece_temp.4.patch included in a nightly build? it looks very promising but a lot of users may not be aware of the patch or can't be bothered to apply it.

if there are any unresolved issues they are more likely to come out when a larger sample of people use it. if all goes well it may make it into the 2.20 release.

comment:93 Changed 7 years ago by x190

I agree that "piece_temp.4.patch" works great. The verify issue I reported has been fixed and ijuxda's patch worked perfectly, even throughout the issue I reported in #3900.

comment:94 in reply to: ↑ 92 Changed 7 years ago by ijuxda

Replying to gunzip:

@ ijuxda: can you petition the developers to get your patch piece_temp.4.patch included in a nightly build?

As far as inclusion is concerned, I can do no more than post patches and make comments in tickets; it's up to whoever has write access to the svn repository or makes the packages.

Changed 7 years ago by x190

Inspector->Info Data File: N/A

comment:95 Changed 7 years ago by x190

This may be a Mac Client thing but according to Inspector->Info, T doesn't know where the data is although it still d/ls correctly. Individual files are still properly linked. "Move Data File To..." solves the problem even tho the location has not changed. EDIT: Further testing shows that this may be more related to #3900 as it only applies to existing torrents while newly added ones are handled correctly by "piece_temp.4.patch".

Last edited 7 years ago by x190 (previous) (diff)

comment:96 Changed 7 years ago by ijuxda

Are you sure that #3900 and the inspector issues are not due to any of my changes? I.e. did you revert the changes from my patch, recompile, and test again?

comment:97 follow-up: Changed 7 years ago by x190

I believe #3900 is valid with or without your patch. Bottom line, your patch should be committed, then it's benefits can be enjoyed and tested while bugs are tracked down in a less confusing environment.

comment:98 in reply to: ↑ 97 Changed 7 years ago by gunzip

Replying to x190:

... Bottom line, your patch should be committed, then it's benefits can be enjoyed and tested while bugs are tracked down in a less confusing environment.

Yes it would be useful to get this in the builds somehow. "piece_temp.4.patch" no longer works in r11702 as the source files are changing:

...
patching file libtransmission/inout.c
Hunk #6 FAILED at 187.
Hunk #7 succeeded at 277 (offset 1 line).
1 out of 7 hunks FAILED -- saving rejects to file libtransmission/inout.c.rej
...

It's now rather difficult to test the latest nightlies and this patch, and the situation will become more confusing in the future.

comment:99 follow-up: Changed 7 years ago by jordan

This patch can't go in the nightlies now because we of the 2.20 freeze, and the beta test cycle we'll be kicking off this week -- we have to have 2.20 ready in time for the Ubuntu freeze.

Sorry for breaking the patch but it'll have to wait until after 2.20 is out the door.

Changed 7 years ago by ijuxda

Patch to svn/trunk r11744

comment:100 follow-ups: Changed 7 years ago by ijuxda

"piece_temp.4.patch" no longer works in r11702 as the source files are changing

I'm not sure what happened in the meantime, but I did not actually have to change any code to have it apply to r11744 (as far as git merge is concerned).

Anyway, you should be able to use piece_temp.5.patch if you want the feature for testing or your own convenience.

comment:101 in reply to: ↑ 100 Changed 7 years ago by x190

Replying to ijuxda:

"piece_temp.4.patch" no longer works in r11702 as the source files are changing

I'm not sure what happened in the meantime, but I did not actually have to change any code to have it apply to r11744 (as far as git merge is concerned).

Anyway, you should be able to use piece_temp.5.patch if you want the feature for testing or your own convenience.

Much appreciated! Now testing on macosx with r11744. Hopefully, Jordan will get this in to trunk before he gets the urge to reformat the source code again. :)

comment:102 in reply to: ↑ 100 Changed 7 years ago by gunzip

Replying to ijuxda:

Anyway, you should be able to use piece_temp.5.patch if you want the feature for testing or your own convenience.

Yes that did the trick and the patch is working again.

comment:103 Changed 7 years ago by x190

Unfortunately, piece_temp.5.patch is busticated again in r11812. ijuxda, can you fix again, please? :)

Changed 7 years ago by ijuxda

Patch to svn/trunk r11822

comment:104 Changed 7 years ago by x190

Thanks for the quick fix, ijuxda! I'm back in business with r11823. :)

comment:105 follow-up: Changed 7 years ago by x190

One more patch fix please and now that 2.20 is released, a commit s.v.p.(it's only been 3 yrs and 2 months)! :)

comment:106 in reply to: ↑ 105 Changed 7 years ago by gunzip

Replying to x190:

One more patch fix please and now that 2.20 is released, a commit s.v.p.(it's only been 3 yrs and 2 months)! :)

Exactly! piece_temp.6.patch is broken in 2.20 .. i hope ijuxda has time to redo this and his udp patch one more time.

Changed 7 years ago by ijuxda

Patch to svn trunk r11842

comment:108 Changed 7 years ago by thosmos

  • Cc thomas@… removed

comment:109 Changed 7 years ago by x190

This 'piece_temp.7.patch' has been well tested and works perfectly with no errors of any kind. Why has it not been committed?

comment:110 in reply to: ↑ 86 ; follow-up: Changed 7 years ago by thosmos

Replying to jordan:

Replying to jidanni:

Replying to jordan: OK Jordan, they tell you how to unsubscribe me from this bug in http://trac.edgewall.org/ticket/9971 . Thanks.

jidanni, I would be happy to unsubscribe you but I don't see the instructions you're referring to.

That link refers to two more of trac's tickets that don't list a workaround. It also mentions changing always_notify_updater in trac.in, but that's a systemwide setting that would affect lots of trac users, not just you.

Jordan, I just tried to unsubscribe from this bug and, as jidanni also discovered, was not able to do so simply by removing my email from the CC list. My only question is, why the __BLEEP__ is always_notify_updater enabled in trac.ini in the first place? If you want updaters to be notified they can easily add themselves to the CC list. Taking away this kind of choice for a mere commenter is absurd and inappropriate. I've been using uTorrent for so long, due precisely to this long-standing bug, that I now can't remember why I wanted to use Transmission in the first place, and this bug lock-in just adds to my aversion. Keep up the good work though. Maybe once both issues are fixed I'll come back to Transmission.

T

comment:111 in reply to: ↑ 110 Changed 7 years ago by jordan

Replying to thosmos:

My only question is, why the __BLEEP__ is always_notify_updater enabled in trac.ini in the first place?

Because it's enabled by default, and in principle it's a good idea.

IMO the problem is this good idea is implemented in a bad way. always_notify_updater should add one to the CC list, rather than some invisible uneditable list. That's the idea under consideration at http://trac.edgewall.org/ticket/9971 so you might want to give them a vote ;)

comment:112 in reply to: ↑ 99 ; follow-up: Changed 7 years ago by gunzip

Replying to jordan:

This patch can't go in the nightlies now because we of the 2.20 freeze, and the beta test cycle we'll be kicking off this week -- we have to have 2.20 ready in time for the Ubuntu freeze.

Sorry for breaking the patch but it'll have to wait until after 2.20 is out the door.

Since 2.20 is now "out the door" and initial testing looked very promising, just curious why piece_temp.7.patch hasn't been included in the trunk yet. Is there still some outstanding issue(s) that hasn't been mentioned in this thread that would keep this out?

comment:113 in reply to: ↑ 112 ; follow-ups: Changed 7 years ago by jordan

Replying to gunzip:

just curious why piece_temp.7.patch hasn't been included in the trunk yet. Is there still some outstanding issue(s) that hasn't been mentioned in this thread that would keep this out?

First, neither livings nor I have reviewed this code in depth.

Second, we're waiting to see how 2.21 is received so that we'll know whether to go into development mode for 2.30 or bugfix mode for 2.22. We haven't created a 2.2x branch in SVN yet, because premature branching forces us to commit each bugfix twice. (That minor headache is easier to handle in git, but that's a different story. ;)

Third, the .config/pieces/ folder will cause problems on embedded devices, where it's common for .config to reside on a small system partition. Partial data needs to be stored on a large partition. One way to do this without user intervention is to store it in the same folder as the rest of the torrent's data.

comment:114 in reply to: ↑ 113 ; follow-up: Changed 7 years ago by gunzip

Replying to jordan:

First, neither livings nor I have reviewed this code in depth.

Second, we're waiting to see how 2.21 is received so that we'll know whether to go into development mode for 2.30 or bugfix mode for 2.22. We haven't created a 2.2x branch in SVN yet, because premature branching forces us to commit each bugfix twice. (That minor headache is easier to handle in git, but that's a different story. ;)

Third, the .config/pieces/ folder will cause problems on embedded devices, where it's common for .config to reside on a small system partition. Partial data needs to be stored on a large partition. One way to do this without user intervention is to store it in the same folder as the rest of the torrent's data.

OK, thanks for the response as it seems these will not be insurmountable problems to resolve in the near future. In meantime the patch is still holding up with recent builds so i'll stick with that for now.

comment:115 in reply to: ↑ 114 ; follow-up: Changed 7 years ago by jordan

Replying to gunzip:

OK, thanks for the response as it seems these will not be insurmountable problems to resolve in the near future. In meantime the patch is still holding up with recent builds so i'll stick with that for now.

I hear you, but "near future" are your words, not mine. The milestone on this ticket is currently "Sometime."

comment:116 in reply to: ↑ 113 Changed 7 years ago by ijuxda

Replying to jordan:

Third, the .config/pieces/ folder will cause problems on embedded devices, where it's common for .config to reside on a small system partition. Partial data needs to be stored on a large partition. One way to do this without user intervention is to store it in the same folder as the rest of the torrent's data.

I would tend to agree with ody (comment:14, comment:16) and starburst (comment:20) as to why this is not a good solution (or at least not a preferable solution). I'm open to other ideas though.

comment:117 Changed 7 years ago by veldt

Store the unwanted pieces in the download dir, in origtorrentdirname.tmp/ (or better ".origtorrentdirname.tmp/" or otherwise hidden.)

The global download dir, above/beside the folder people will be dragging to their burner/permanent storage.

Last edited 7 years ago by veldt (previous) (diff)

comment:118 in reply to: ↑ 115 ; follow-up: Changed 7 years ago by x190

Replying to jordan:

I hear you, but "near future" are your words, not mine. The milestone on this ticket is currently "Sometime."

Such a response without providing a valid explanation is incomprehensible. :(

comment:119 in reply to: ↑ 118 Changed 7 years ago by jordan

Replying to x190:

Replying to jordan:

I hear you, but "near future" are your words, not mine. The milestone on this ticket is currently "Sometime."

Such a response without providing a valid explanation is incomprehensible. :(

I have already given specifics above, but I see this thread has a potential for heating up into an argument, so let's take a deep breath together and keep our heads cool.

One lesson I've learned from the 2.11->2.20 proxy story is that it's better to hold back on a feature I'm unsure of than to remove it later. Perhaps, since you're already using the patch, you feel like holding back is the same as removal. If so, please step back and look at the bigger picture.

I disagree with the premise of an explanation being required, much less one that you get to judge as valid or invalid. I'm the one who would be committing to support this feature's code for the long haul -- not you. Your assumption that this ticket is a fait accompli seems... greedy.

Both you and gunzip have given lots of help on transmissionbt.com, so none of this is intended to generate ill will or argument. There's been too much of that lately. Posts that walk down that path will be cheerfully deleted. :)

comment:120 Changed 7 years ago by jordan

(besides, we all know how much jidanni would miss getting his daily updates on this ticket ;)

comment:121 Changed 7 years ago by x190

Sometimes these tickets tend to get a bit off-topic so here is the description quote just to re-focus.

"12/06/07 15:43:15 eisa01 said: It's possible to store the extra data in a separate transmission.dat file or something. That's what µTorrent does, works flawlessly. I guess you could put this in the Application Support folder on macs too, which makes it even better for the user. µTorrent stores it in the folder of the files being downloaded.

The idea is to, as overly discussed in #514, not actually create the files we didn't want in the first place. Instead of creating the deselected files we could keep these pieces in a cache of some kind. ..."

As valid today as it was over 3 years ago.

ijuxda has kindly provided a patch for this problem. See comment:106.

Last edited 7 years ago by x190 (previous) (diff)

comment:122 follow-up: Changed 7 years ago by jordan

thinking-out-loud dept: actually, Transmission has more tools at its disposal now than it did when this ticket was started. Could we get away with throwing away the partial pieces once they pass the checksum test?

  • we could modify libtransmission's memory cache to keep partial pieces in memory, so that we can do a checksum test without saving them to disk
  • libtransmission now marks timestamps per-piece, rather than per-file
  • peer-msgs would need to not advertise that piece in our bitfield, since we couldn't serve it, but that's not the end of the world.
  • peer-mgr would need to know not to re-download that piece, but that's doable too.

I haven't thought this through yet; it may not be easy to do cleanly, or it may be a bad idea (how many incomplete pieces do we want to bloat the memory cache up with?)... but I thought I'd throw it out there for brainstorming, especially since jidanni hasn't been pinged by this ticket in a month ;)

comment:123 Changed 7 years ago by x190

Have been using the patch continuously for 2 months now and it works perfectly on macosx. If you're worried about where to stash the 'Pieces' folder on embedded systems, couldn't that be handled by an ifdef or similar?

Anyway, I'm encouraged to see that work is again being done on this important ticket. Personally, I never want to have to deal with unwanted .parts again!

comment:124 in reply to: ↑ 122 Changed 7 years ago by Waldorf

Replying to jordan:

  • peer-msgs would need to not advertise that piece in our bitfield, since we couldn't serve it, but that's not the end of the world.

Actually, this may very well be the end of the world! If implemented in enough clients, this may drain a swarm from those corner pieces. On a low-seeded torrent, with a few users doing selective seeding, you'll get lots of those dreaded "99%" complaints.
Also, if we later do decide to continue to download the rest of the torrent, probably selectively, it's kind of a waste that we threw away those pieces.

Personal view: If I download selectively, I end up downloading the rest anyway, the next week/month or so. Even for those few .part files I still get, I don't mind cleaning them manually. Disk space has become cheap over the last 3 years. A few .part doesn't hurt.
No matter what implementation, I can already smell the sneaking pieces that do linger in some support folder, that do build up and aren't as easy to get rid of as a few .part files.

But use-cases differ. This is just mine.

comment:125 Changed 7 years ago by ahihi

  • Cc violetmaze@… added

comment:126 Changed 7 years ago by bb

  • Cc bruno.bierbaumer@… added

comment:127 Changed 7 years ago by diamondsw

I hate bumping an old bug, but having had my startup drive fill up overnight due to exactly this issue, I'm rather interested. Now that the world is moving to SSD's, storage space is at a premium again.

Since it seems the patch has been updated (several times), tested, etc - why hasn't it made it into mainline Transmission?

comment:128 Changed 7 years ago by paradisewaits

I LOVED transmission and wouldn't have ever considered uTorrent. However, this bug is a DEALBREAKER for me. I have about 30 gigs to play with for torrents right now (til I get an external HD). I had an 8 gb file added for less than 16 mb of data (piece size), when selecting 1/30 files from as 200 gb torrent. This is unacceptable. I understand the limitations of a sparse filesystem. A solution needs to be implemented. This ticket is years old and many solutions have been proposed. What gives? I still have affinity for Transmission but will be using uTorrent until this gets handled.

comment:129 Changed 6 years ago by iband

This ticket was opened 4 years ago!!! And still no solutions (Transmission for Mac). I have 128Gb SSD and every Gb matters. I don't want to waste it for that 'parts'. Is it so difficult to implement? I like Transmission, but I often face this problem. I guess I would have to look for some other torrent app if the problem won't be solved.

comment:130 Changed 6 years ago by x190

piece_temp.7.patch has been updated for v2.50 and is currently being tested. Hopefully, this important feature will be adopted by the developers.

EDIT: Sorry, some nasty bugs have appeared in this particular update candidate.

Last edited 6 years ago by x190 (previous) (diff)

comment:131 Changed 6 years ago by porg

I go along with the previous discussion participants!
Hopefully this large issue will be considered for solving soon.

comment:132 Changed 6 years ago by bondiblueos9

  • Cc bondiblueos9@… added

comment:133 Changed 6 years ago by cfpp2p

piece_temp.7.patch IS currently being updated and the bugs are SLOWLY being remedied. For anyone wanting to begin testing now please PM me on the forums. A preliminary less buggy patch for testing will be available in approximately three weeks. Coders and testers willing to help would be appreciated; so as to continuously keep this patch up to date.

Changed 6 years ago by cfpp2p

patch for r13255 v250+

comment:134 Changed 6 years ago by cfpp2p

pieceTMP_8.patch tests OK. Way sooner than the three week estimate. Temporary pieces are stored in the default config directory. I'll update so that temporary piece directory can be set from settings.json sometime soon.

patch for libtransmission

Last edited 6 years ago by cfpp2p (previous) (diff)

Changed 6 years ago by cfpp2p

pieceTMP_8-v242.patch for MAC users without necessary SDK to build anything post v2.42

comment:135 Changed 6 years ago by cfpp2p

pieceTMP_8.patch is still the correct for r13255 v2.50+

Changed 6 years ago by cfpp2p

allows changing 'pieces' directory

comment:136 Changed 6 years ago by cfpp2p

new patch, working very well.

peiceTMP_9.patch allows the 'peices' temporary directory to be set in the configuration file settings.json

( https://trac.transmissionbt.com/wiki/EditConfigFiles )

a new key has been added: piece-temp-dir

example:

"piece-temp-dir": "/share/hdd/data/16gb/_trs-pctmp",

the default path is the config directory and a "pieces" sub-directory Changing this setting will only affect new torrents. Existing torrents will continue to use the same directory that was set when they were created. see tr_sessionGetConfigDir() and tr_getDefaultPieceSubDir()

patch for r13255 v2.50+

patch for libtransmission

comment:137 Changed 6 years ago by livings124

Awesome patch! What's the benefit of making that directory user-configurable?

comment:138 Changed 6 years ago by cfpp2p

The user configurable directory allows for users on embedded systems to change the pieces directory so that the temporary pieces are saved NOT to a very small drive that is often the default for the config directory.

comment:139 follow-up: Changed 6 years ago by bb

I made a precompiled version of r13255 with peiceTMP_9.patch applied for Mac OS X. http://dl.dropbox.com/u/5428365/Transmission_r13255_patch9.zip

comment:140 in reply to: ↑ 139 Changed 6 years ago by x190

Replying to bb:

I made a precompiled version of r13255 with peiceTMP_9.patch applied for Mac OS X. http://dl.dropbox.com/u/5428365/Transmission_r13255_patch9.zip

A big thank you to you, sir! :-) Now testing...

Changed 6 years ago by cfpp2p

fixes minor compile issue (MAC and possibly others) thanks! to bb

comment:141 Changed 6 years ago by cfpp2p

bb 's precompiled already has peiceTMP_9b.patch incorporated, so no need to worry

comment:142 Changed 6 years ago by bb

I am also testing this patch on my MyBook? World white light (ipkg: cs05q1armel). If anyone else needs this ipk just drop me a message :)

Changed 6 years ago by cfpp2p

fix minor verification issues

comment:143 Changed 6 years ago by cfpp2p

peiceTMP_9c.patch for r13259 (OK for r13255 also)

fixes minor verification issues

v2.50+

patch for libtransmission

NOTE: piece_temp.7.patch and previous contain the verification issue, so if you are still using piece_temp.7.patch you should upgrade.

Last edited 6 years ago by cfpp2p (previous) (diff)

comment:144 Changed 6 years ago by bb

Here another precompiled version for Mac OS X http://dl.dropbox.com/u/5428365/Transmission_r13259_patch9c.zip

comment:145 Changed 6 years ago by jordan

  • Milestone changed from Sometime to 2.60

Changed 6 years ago by cfpp2p

100% done but skipped files fixed

comment:146 Changed 6 years ago by cfpp2p

pieceTMP_10 allows 100% done but skipped files to be completed by toggling the

file to download. Data is copied from the temporary piece files.

comment:147 Changed 6 years ago by bb

Here another precompiled version for Mac OS X (r13261) http://dl.dropbox.com/u/5428365/Transmission_r13261_patch10.zip

Changed 6 years ago by cfpp2p

fixes verify of 100% done skipped moved to downloads

comment:148 Changed 6 years ago by cfpp2p

pieceTMP_rc1.patch fixes verify, .part and incomplete download directory issues.

100% done but skipped files toggled to download, now verify and behave properly.

comment:149 Changed 6 years ago by bb

Here another precompiled version for Mac OS X http://dl.dropbox.com/u/5428365/Transmission_r13261_patchrc1.zip

Changed 6 years ago by cfpp2p

fixes rare problem with multiple files in a single piece

comment:150 Changed 6 years ago by cfpp2p

MAC users will have to use the web-client to toggle 100% done but skipped files until the MAC client is updated to mirror the behavior of the current v2.50 web-client.

comment:152 Changed 6 years ago by bb

Hi, patch rc1a works fine for me on OS X 10.7 :)

comment:153 Changed 6 years ago by phils

  • Cc transmissionbt@… added

comment:154 Changed 6 years ago by livings124

  • Milestone changed from 2.60 to 2.70

comment:157 in reply to: ↑ 156 ; follow-up: Changed 6 years ago by x190

Replying to bb:

Another build for r13347: https://dl.dropbox.com/u/5428365/Transmission_r13347_patchrc1a.zip

bb: This one has a broken "Preferences Window" (blank).

SL 10.6.8

Last edited 6 years ago by x190 (previous) (diff)

comment:158 in reply to: ↑ 157 ; follow-up: Changed 6 years ago by bb

Replying to x190:

Replying to bb:

Another build for r13347: https://dl.dropbox.com/u/5428365/Transmission_r13347_patchrc1a.zip

bb: This one has a broken "Preferences Window" (blank).

SL 10.6.8

Hey x190: Is the official nightly build also broken? Could you please post a screenshot?

comment:159 in reply to: ↑ 158 ; follow-up: Changed 6 years ago by x190

Replying to bb:

Replying to x190:

Replying to bb:

Another build for r13347: https://dl.dropbox.com/u/5428365/Transmission_r13347_patchrc1a.zip

bb: This one has a broken "Preferences Window" (blank).

SL 10.6.8

Hey x190: Is the official nightly build also broken? Could you please post a screenshot?

Aye, aye, Sir! It is broken on r13350 (latest) as well. Will open a new ticket. It's just a blank window with normal width and title bar but only about one inch in depth and blank. Not your fault, but you might want to remove this build (r13347) from this ticket until it is fixed for SL 10.6.8. Your r13346 build IS okay but NOT the official build!! How strange. I see why---in the "About Window" what you call r13346 is actually 2.50 +(13261) so that should be taken down as well because we need at least 2.52 because of #4894. Thanks! #4942 has been opened for this issue.

Last edited 6 years ago by x190 (previous) (diff)

comment:160 in reply to: ↑ 159 Changed 6 years ago by bb

Replying to x190:

Replying to bb:

Replying to x190:

Replying to bb:

Another build for r13347: https://dl.dropbox.com/u/5428365/Transmission_r13347_patchrc1a.zip

bb: This one has a broken "Preferences Window" (blank).

SL 10.6.8

Hey x190: Is the official nightly build also broken? Could you please post a screenshot?

Aye, aye, Sir! It is broken on r13350 (latest) as well. Will open a new ticket. It's just a blank window with normal width and title bar but only about one inch in depth and blank. Not your fault, but you might want to remove this build (r13347) from this ticket until it is fixed for SL 10.6.8. Your r13346 build IS okay but NOT the official build!! How strange. I see why---in the "About Window" what you call r13346 is actually 2.50 +(13261) so that should be taken down as well because we need at least 2.52 because of #4894. Thanks!

Hi, I removed the old and faulty builds (except r13347 it works on OS X 10.7). I will build a new version if the blank window bug is fixed.

comment:161 follow-up: Changed 6 years ago by x190

This is a patch to r13354 to allow toggling of completed dnd files in the Mac Client Inspector to address comment:150. This patch is meant to be used in conjunction with pieceTMP_rc1a.patch. Needs testing on OS X 10.7.

bb: Could you please test this patch? Also, comment:160 has been addressed for 10.6.8, so current builds should now work properly.

Last edited 6 years ago by x190 (previous) (diff)

comment:162 in reply to: ↑ 161 ; follow-up: Changed 6 years ago by bb

Replying to x190:

This is a patch to r13354 to allow toggling of completed dnd files in the Mac Client Inspector to address comment:150. This patch is meant to be used in conjunction with pieceTMP_rc1a.patch. Needs testing on OS X 10.7.

bb: Could you please test this patch? Also, comment:160 has been addressed for 10.6.8, so current builds should now work properly.

x190: Thanks for the patch :). I will test it (and provide a new build) after my final exams.

comment:163 in reply to: ↑ 162 ; follow-up: Changed 6 years ago by bb

Replying to bb:

Replying to x190:

This is a patch to r13354 to allow toggling of completed dnd files in the Mac Client Inspector to address comment:150. This patch is meant to be used in conjunction with pieceTMP_rc1a.patch. Needs testing on OS X 10.7.

bb: Could you please test this patch? Also, comment:160 has been addressed for 10.6.8, so current builds should now work properly.

x190: Thanks for the patch :). I will test it (and provide a new build) after my final exams.

Hi, there is a new build: https://dl.dropbox.com/u/5428365/Transmission_r13354_toggle.zip

comment:164 in reply to: ↑ 163 Changed 6 years ago by x190

Replying to bb:

Hi, there is a new build: https://dl.dropbox.com/u/5428365/Transmission_r13354_toggle.zip

Thanks, my good man. May you get an A+ on your exams!

comment:165 Changed 6 years ago by bb

X190: the patch seems to work flawless on OS X 10.7

comment:166 Changed 6 years ago by chris-

  • Cc chris@… added

comment:167 Changed 6 years ago by livings124

  • Milestone changed from 2.70 to Sometime

comment:168 Changed 6 years ago by diamondsw

"Sometime", eh? And here I was hopeful that with several round of successful patches we might finally see something done about this.

comment:169 Changed 6 years ago by livings124

It missed the cutoff for 2.70. Hopefully soon.

Changed 6 years ago by cfpp2p

patch for r13456

comment:170 Changed 6 years ago by cfpp2p

patch rc1b for r13456 plus minor bug fix for cache flush when toggling files under unusual circumstances ( thanks x190! :) ). Could someone please compile this for MAC and also https://trac.transmissionbt.com/raw-attachment/ticket/532/toggleInspectorCompletedFiles.patch ( SEE: comment:161 ) and post the finished MAC build here.

Lets ~hope~ we don't miss another cutoff... ?????

Patch is working great and has been under continuous testing for months and months.

comment:171 Changed 6 years ago by bb

I have been testing this patch for months (OS X 10.7). It's working flawless. Here is another build for OS X: https://dl.dropbox.com/u/5428365/Transmission_r13456.zip

comment:172 Changed 6 years ago by sergio

Can we get this back on track for 2.80?

comment:173 Changed 6 years ago by bb

Hi, I was testing the r13456 and there wasn't any issue.

I am now testing 2.73+ so here a new build : https://dl.dropbox.com/u/5428365/Transmission_r13618.zip

comment:174 Changed 6 years ago by x190

Thanks bb. Your contribution is much appreciated! You too, cfpp2p. :-) And now we have yet another happy forum user of your build.

https://forum.transmissionbt.com/viewtopic.php?f=4&t=14078&p=62512#p62511

Guys, why not take the plunge on this one?

comment:175 Changed 5 years ago by joojoo

  • Cc groundzerox15@… added

comment:176 follow-up: Changed 5 years ago by bb

Hi, here is another build :) https://dl.dropbox.com/u/5428365/Transmission_r13660.zip

I had absolutely no problem with the last version for months!

comment:177 Changed 5 years ago by jordan

Could someone attach an up-to-date patch for this?

comment:178 Changed 5 years ago by porg

@bb: Your r13660 works for freshly started torrents but not for already running torrents. In already partially downloaded torrents, in which you select another file for downloading, the neighboring overlapping files are created instead of the according piece-files.

@cfpp2p already told me in a private discussion, that his patch r13456 acts the same way. Not possible within transmission, only workarounds like torrent server on same machine or LAN to re-download quickly and thereby convert into the new format.

So it seems to be not possible that a new version with pieces-patch-functionality offers the pieces-functionality for previously started torrents. Sad.

The noticable improvement over r13456 is, that Transmission remains much more responsive when pausing active torrents. In these situations r13456 caused the "sandball to spin quite many cycles" on Mac OS X 10.6.8.

comment:179 Changed 5 years ago by jordan

The lack of a beachball may also be related to ticket #5168 which first landed in r13651.

Changed 5 years ago by x190

comment:180 Changed 5 years ago by x190

Fixed typo.

Changed 5 years ago by cfpp2p

patch for 2.75 tag

Changed 5 years ago by cfpp2p

patch for 2.75 branch

comment:181 Changed 5 years ago by cfpp2p

Collect unwanted, but received, blocks in a separate file

patches for svn 2.75 tag and branch

comment:182 follow-up: Changed 5 years ago by jordan

cfpp2p, I meant could you attach a patch for trunk, rather than 2.75?

This doesn't belong in a 2.7x bugfix release; it should be considered for a future feature release instead, and libtransmission's clocked a lot of commits in the time since 2.75 was released.

Last edited 5 years ago by jordan (previous) (diff)

comment:183 in reply to: ↑ 182 Changed 5 years ago by cfpp2p

Replying to jordan:

cfpp2p, I meant could you attach a patch for trunk, rather than 2.75?

Yes I could sometime.

There has been an incredible amount of work put into this patch. A small dedicated team has both maintained and de-bugged the patch to continually keep #532 finely tuned and up to date. #532 has been switched back and forth from milestone 2.xx to sometime for no apparent reason.

We have posted several pre-compiled versions for Mac ( most recent 2.75 (13660) ). Also patches for several versions up to and including 2.75 have been maintained and posted for any interested parties. All of these have been well tested, stable and highly functional.

There have been major trunk changes involving quark, variant, etc. Updating the patch for these changes will involve some precise analysis. Obtaining a patch of the previously exhibited quality will require expertise.

If I were assured a definitive milestone and once any possible initial issues with quark, variant, etc. are addressed it will take me a couple weeks to update. Or if anybody else with the desire to update the patch for #532 I would be happy to review.

#532 functionality would be a very fine enhancement for any transmission user.

comment:184 follow-up: Changed 5 years ago by jordan

#532 functionality would be a very fine enhancement for any transmission user.

How does this benefit anyone not trapped on HFS+?

comment:185 in reply to: ↑ 184 ; follow-up: Changed 5 years ago by diamondsw

Replying to jordan:

#532 functionality would be a very fine enhancement for any transmission user.

How does this benefit anyone not trapped on HFS+?

So Mac users aren't important? Are Windows 8 (and future) users unimportant as well?

"Q) What semantics or features of NTFS are no longer supported on ReFS? The NTFS features we have chosen to not support in ReFS are: named streams, object IDs, short names, compression, file level encryption (EFS), user data transactions, sparse, hard-links, extended attributes, and quotas." http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx

comment:186 follow-up: Changed 5 years ago by x190

Seriously, is there any difference between filesystems when '2 = Full' is set? And all filesystems benefit from the removal of up to thousands of unwanted part files cluttering up their Download directory.

With #532 in place, #4262 can be completed.

Last edited 5 years ago by x190 (previous) (diff)

comment:187 in reply to: ↑ 185 Changed 5 years ago by jordan

Replying to diamondsw:

Replying to jordan:

#532 functionality would be a very fine enhancement for any transmission user.

How does this benefit anyone not trapped on HFS+?

So Mac users aren't important?

That's not what I said. The claim was this ticket is a fine enhancement for any transmission user; I'm asking for that claim to be backed up.

comment:188 in reply to: ↑ 186 ; follow-up: Changed 5 years ago by jordan

Replying to x190:

Seriously, is there any difference between filesystems when '2 = Full' is set?

There absolutely is. See the #ifdefs in preallocate_file_full().

And all filesystems benefit from the removal of up to thousands of unwanted part files cluttering up their Download directory.

I hate the idea of dropping this stuff in the .config directory. That will cause ongoing support questions for users whose systems grind to a halt after their config directories fill up their main partition.

cfpp2p's patch has a configurable option for that, but if the past is any indication users will only find the switch after their drives fill up and come to the forums anyway. And having a configuration option for that adds complexity in bug triage.

And when they change that configuration option, their drives will still be full because the patch doesn't address moving files when the pieces directory changes. And Transmission won't know where to find the the old piece fragments, so will it throw a 'missing data' error too?

With #532 in place, #4262 can be completed.

  • What's the graceful way of handling errors if you're moving files between the config dir and download dir and one or both of the partitions is full?
  • How do you cleanly report that to the user? Not just for the straightforward cases of the Mac/GTK+ clients, but also to all the pre-existing 3rd party remote controls out there?
  • I don't see how tr_torrentGetFileMTime() finds the correct time for a piece if, say, there are multiple files in the piece and also a part of the piece in the temp dir. Or for bonus fun, if both ends of the piece are in the temp dir, but there's one or more small, wanted file in the middle of the piece.
  • How does this feature complicate the disk cache? How will this play out when we finally add asynchronous disk IO?
  • What happens if we're moving a file between partitions in asynchronous IO, and downloading s.t. part of the file is in the disk cache, when x190 hits the 'delete file' button from the GUI?
  • What happens if the user changes the piece dir location when the old location is nonempty? Right now the patch loses all the previous pieces. Should we move them? What happens if we try to move them but the new partition can't hold all the pieces? How do we show *that* to the 3rd party remote controls?
  • How do we even test all these edge cases?

IMO a successful patch for #532 would avoid configuration options, have few conditionals in the code, have no 'clever' code sections, and add minimal complexity to pre-existing features. libtransmission is already bloated.

I asked for an updated patch so that I can run it and test out my concerns. Telling me that I need to promise a milestone before the patch will be updated is unpersuasive.

comment:189 Changed 5 years ago by cfpp2p

OK, I'm in error. When I wrote "#532 functionality would be a very fine enhancement for any transmission user." this was a mistake. What I should have written is:

#532 functionality would be a very fine enhancement for many transmission users.

As I don't really think that every single last transmission user would benefit from #532 functionality. Nor do I think that on any given feature or enhancement a benefit will be recognized by every transmission user. Some features of course benefit everyone in the respect that transmission is an excellent p2p client to begin with.

Replying to jordan:

I asked for an updated patch so that I can run it and test out my concerns. Telling me that I need to promise a milestone before the patch will be updated is unpersuasive.

Regarding the above I also wrote:

"Or if anybody else with the desire to update the patch for #532 I would be happy to review."

Nobody owns the patch and it has a long history.

. .

What happens if the user changes the piece dir location when the old location is nonempty? Right now the patch loses all the previous pieces. Should we move them?

That's incorrect. The patch does not lose any previous pieces, as I describe below.

And when they change that configuration option, their drives will still be full because the patch doesn't address moving files when the pieces directory changes. And Transmission won't know where to find the the old piece fragments, so will it throw a 'missing data' error too?

The #532 patch does address when the pieces directory changes. The location that the temporary pieces were saved in originally is saved in the resume file. This avoids the 'missing data' problem. This has been well tested. Drive full issues are not addressed.

Last edited 5 years ago by cfpp2p (previous) (diff)

comment:190 follow-up: Changed 5 years ago by user967

I do not want this feature, if there is to be no setting to opt out of it.

It is not uncommon that I wish to examine some file header before download has completed. Sometimes I will even deselect the file in question, and prioritize the one before it, in order to expediently retrieve the header only. This would be very inconvenient if all fragments were jammed together in one file.

comment:191 in reply to: ↑ 190 Changed 5 years ago by cfpp2p

Replying to user967:

... This would be very inconvenient if all fragments were jammed together in one file.

Not to say that you would want this feature but the patch as it stands does not jam all the temporary pieces into one file. Instead each temporary piece is a single piece sized file with its name based on the piece number. It is possible to view the contents of these pieces individually which I do so occasionally to view a file's header. But in no way is this at all convenient in that the file's first piece name is based on the piece number.

comment:192 follow-ups: Changed 5 years ago by user967

Ok, thanks for the clarification. I could probably live with that if I had to, as I would typically only be interested in some very recently modified piece.

I had not read the implementation, was just responding to the 'transmission.dat' example in the description.

I wonder though, why this is seriously considered at all? I am aware that suggestions such as 'rename torrent basename', 'rename/relocate/delete individual files within torrent' have been declined for reasons to the effect of 'transmission is not a file manager', and it is not clear to me why this is any different.

If you have .part file naming enabled, this could easily be accomplished by a script on completion that hardlinks files to an 'output' directory, excluding the .part files. For gui users (if not possible already), perhaps the ability to copy files from the torrent properties window (sorted by completion) would work.

I simply do not agree that just because some file has been deselected (or just incomplete at the time?) that its partial data should be hidden in some obscure place. The existing features of Transmssion + your OS, make it possible to identify/filter out incomplete files if they are truly of no interest.

The devs have repeatedly declined ideas that have no value other than making it marginally easier to use your Transmission download directory as your 'media store' directory or whatever, and I personally hope that this is abandoned for similar reasons. The .part option is more than enough for any use of this that I am understanding.

comment:193 in reply to: ↑ 192 Changed 5 years ago by cbhl

Replying to user967:

I wonder though, why this is seriously considered at all? I am aware that suggestions such as 'rename torrent basename', 'rename/relocate/delete individual files within torrent' have been declined for reasons to the effect of 'transmission is not a file manager', and it is not clear to me why this is any different.

The feature, as described, would make it possible to download a portion of a torrent that is larger than the available free space when the underlying filesystem doesn't support sparse files. For example, if a 4 GB torrent was comprised of four 1 GB files, and you only had 1.5 GB free, you may not be able to complete this partial download today if your filesystem doesn't support sparse files, because the first block of the file may be the last block of the previous 1GB file (and storing it would require writing a 1 GB file that is almost all zeros, except for the last block).

This may matter for people who run transmission on space-constrained devices, such as older computers, some VPSes, and routers.

Of course, simply switching to a filesystem that supports sparse files would be a workaround for these users, but that usually requires reformatting (and may not even be possible on exotic devices / operating systems).

comment:194 in reply to: ↑ 192 ; follow-up: Changed 5 years ago by jordan

Replying to user967:

I wonder though, why this is seriously considered at all?

Because a couple of people very active in the Transmission community seem to really want this feature. I don't want to write that sweat equity off.

I'm neutral on this feature and my gut feeling is that most people won't use it.

So my ideal outcome would be for a patch to fall in my lap that implements this and gives me no headaches wrt new code complexity, new forum questions, new bug reports, etc. Easy, right? ;) That's why I was arguing against keeping partials in the config partition, adding configuration options, and so on.

comment:195 in reply to: ↑ 194 ; follow-up: Changed 5 years ago by x190

Replying to jordan:

Replying to user967:

So my ideal outcome would be for a patch to fall in my lap that implements this and gives me no headaches

This one? (comment:63) :-)

Changed 2 years ago by jordan

  • Attachment piece_temp.2.patch​ added

For those unsure of this ticket's purpose and utility, I suggest reading the first few dozen comments.

comment:196 in reply to: ↑ 188 Changed 5 years ago by x190

Replying to jordan:

Replying to x190:

Seriously, is there any difference between filesystems when '2 = Full' is set?

There absolutely is. See the #ifdefs in preallocate_file_full().

Full != full or SYS_DARWIN is best b/c of F_ALLOCATECONTIG?

And all filesystems benefit from the removal of up to thousands of unwanted part files cluttering up their Download directory.

True.

I hate the idea of dropping this stuff in the .config directory. That will cause ongoing support questions for users whose systems grind to a halt after their config directories fill up their main partition.

Piece size vs. File size! And configurable to boot! Default to whatever works for you.

  • What's the graceful way of handling errors if you're moving files between the config dir and download dir and one or both of the partitions is full?

Same way as setLocation?

/* try to move the files.
         * FIXME: there are still all kinds of nasty cases, like what
         * if the target directory runs out of space halfway through... */

  • I don't see how tr_torrentGetFileMTime() finds the correct time for a piece if, say, there are multiple files in the piece and also a part of the piece in the temp dir. Or for bonus fun, if both ends of the piece are in the temp dir, but there's one or more small, wanted file in the middle of the piece.

All of this has been addressed. There is no data loss.

comment:197 in reply to: ↑ 195 Changed 5 years ago by jordan

Replying to x190:

So my ideal outcome would be for a patch to fall in my lap that implements this and gives me no headaches

This one? (comment:63) :-)

Changed 2 years ago by jordan

  • Attachment piece_temp.2.patch​ added

No. As I said at the time, that was a thinking-out-loud patch that left many points unaddressed. Its purpose was to drive the ticket towards simpler code; that purpose failed.

I hate the idea of dropping this stuff in the .config directory. That will cause ongoing support questions for users whose systems grind to a halt after their config directories fill up their main partition.

Piece size vs. File size! And configurable to boot! Default to whatever works for you.

Completely misses the point of what I said above about support tickets and code complexity.

I don't see how tr_torrentGetFileMTime() finds the correct time for a piece if, say, there are multiple files in the piece and also a part of the piece in the temp dir. Or for bonus fun, if both ends of the piece are in the temp dir, but there's one or more small, wanted file in the middle of the piece.

All of this has been addressed. There is no data loss.

It doesn't seem to be addressed in the patch. How does tr_torrentGetFileMTime() find the correct time for a piece when both ends of it are in the temp dir?

comment:198 Changed 5 years ago by x190

Jordan, if you or livings were to demonstrate bonafide interest in having this feature implemented, then I'm sure we could all work together to address any real issues that remain for you. Patches and pre-built apps are available for testing. The demand for, and utility of, this feature has been well-documented both here and on the forum.

If none of that works for you, then I think you should move on to other project tickets that you find more important.

comment:199 in reply to: ↑ 176 Changed 5 years ago by cfpp2p

Changed 7 weeks ago by bb

Hi, here is another build :) https://dl.dropbox.com/u/5428365/Transmission_r13660.zip

I had absolutely no problem with the last version for months!

Thanks to bb: :) here is another build based on the 2.7x branch with recent backports. TransmissionMac27x-pcTMPr13960

Last edited 5 years ago by cfpp2p (previous) (diff)

comment:200 Changed 5 years ago by LinusU

  • Cc linus@… added

comment:201 Changed 5 years ago by LinusU

This bug is killing my disk space and causing me a lot of pain on my 128 GiB SSD. Can someone who has the rights to include patches please provide a list of exactly what the patch needs to do to get merged? Then I (will try, haven't done any programming against Transmission before) or someone else here can get the patch done.

I am sure that we will have a few people who will be willing to test aswell. Please I really need this and this bug is about 6 years old, and we have had multiple working patches submitted.

I would prefer that it stored the pieces in Application Support but anything other except a hidden file would be good.

comment:202 Changed 5 years ago by diamondsw

This is nearly the oldest open ticket. Of the other two older tickets, one should be closed (#8 - UPnP/NAT-PMP was implemented ages ago), and one has had a single "bump" in the last three years (#327). This one on the other hand has significant interest, numerous updated patches, yet we get theoretical concerns while the very-real-affecting-us-for-years issues are completely ignored. Perfect has certainly been the enemy of good here.

Just close the thing as "wontfix" if you have no intention of implementing it.

comment:203 Changed 5 years ago by porg

I repeat again that this feature would be very welcome, especially in the day and age of SSD, Fusion-Drive, etc and would welcome a decision now, instead if delaying this issue further 6 years: Either bring the patch into mainline or wontfix.

comment:204 Changed 5 years ago by cfpp2p

Thanks so much to bb: :)

We now have another Mac build with the most recent bug and crash fixes. Based on 2.7x branch.

https://dl.dropboxusercontent.com/u/5428365/Transmission_r14075.zip

mirror: TransmissionMac27x-pcTMPr14075.zip

comment:205 Changed 5 years ago by pathetic_loser

I repeat again that this feature would be very welcome, especially in the day and age of SSD, Fusion-Drive, etc

These deal with sparse files very well on their own. If your file system cant handle sparse files, maybe it's a good idea to use some modern filesystem which can do it, after all :P.

comment:206 Changed 5 years ago by xnoreq

Instead of using the config/pieces dir, why not place the unwanted files in an .unwanted dir inside the torrent's directory? Or if inside the torrent directory is too problematic then ../.unwanted.<torrent name> or something like that?

comment:207 Changed 5 years ago by porg

torrent-dir-x/.unwanted/ ~ unwanted stuff separated, but only visually, not in terms of storage location (still contained in invisible sub dir!) + related stuff together, easy migration – grows torrent-dir in size, which the suggestion wanted to avoid (i.e. if torrents reside on dedicate limited capacity disk, and pieces on other dedicated disk, and pieces can be purged, if torrent inactive one day) – does not work for single file torrents, as they have no directory as parental container

torrent-dir-x/../.unwanted/ + torrent-dir-x doesn't grow + would work with single file torrents – still the storage capacity issue

pieces/torrent-dir-x/ + separated, gives storage and management advantages + central purge location, either per torrent or all torrents – torrent bound to machine, not easily migratable (one needs to move both torrent dir/files and its pieces dir), but hey this use case is less likely than the benefits it brings (storage capacity separation)

comment:208 Changed 5 years ago by cfpp2p

../.unwanted.<torrent name> or something like that?

Beginning with: comment 136

and the 2.7x patch you are not confined to using the config/pieces dir you can do similar to ../.unwanted.torrent name

i.e. testdot.torrent ..\_trs-pctmp\test-dot.8738521f4647d585\

comment:209 Changed 5 years ago by xnoreq

On the torrent-dir-x/.unwanted/ comments:

– grows torrent-dir in size, which the suggestion wanted to avoid (i.e. if torrents reside on dedicate limited capacity disk, and pieces on other dedicated disk, and pieces can be purged, if torrent inactive one day)

I thought the problem is that blocks spreading across file borders cause possibly unwanted files to be allocated with their full size on file systems without sparse files? Of course the torrent-dir will grow in size, but should only grow by the unwanted block's size. The .unwanted dir still can be purged.

– does not work for single file torrents, as they have no directory as parental container

Single file torrents have a single file and therefore no unwanted pieces.

comment:210 Changed 5 years ago by xnoreq

you can do similar to ../.unwanted.torrent name

Not if the path has to be configured in an absolute manner, which seems to be the case atm.

Example: Torrent A is being downloaded to /media/A/torrent-A and torrent B is downloaded to /media/B/torrent-B. Unwanted stuff by default would be somewhere in a config dir in $HOME, or if changed either on /media/A or /media/B.

I think that storing unwanted stuff in an .unwanted directoy in the torrent-dir is the best solution. For example qBittorrent (uses libtorrent rasterbar) does it that way.

comment:211 Changed 5 years ago by porg

Single file torrents have a single file and therefore no unwanted pieces.

Of course! Did not think that through!

comment:212 follow-up: Changed 5 years ago by neuen

I bore with this bug quite long before I had time to look through this trac and find this wonderful discussion. Excuse me if I duplicate anyone's comment - I haven't read all of them thoroughly.

  1. Parts of pieces downloaded for wanted files that belong to unwanted files must NOT BE STORED AS UNWANTED FILE. Even if sparse files are supported.

There are LOCAL FILESYSTEMS which DO NOT SUPPORT SPARSE FILES. One can say most modern filesystems have such support and I agree to some extent but... let's be honest - torrents are often downloaded to some handmade server or nas or even soho router with external hdd attached which may have old os, old software, are io, cpu (or whatever resource except network) bound, or (even worse?) administered by a non-professional - there may be old filesystem under the hood of downloads folder.

NETWORK FILESYSTEMS DO NOT SUPPORT SPARSE FILES. If I want to transfer downloaded torrent via nfs or samba or sshfs or... btsync maybe? dropbox? whatever - none of them support sparse files and I'm stuck with either transferring unwanted files filled with mostly zeroes or once again filtering out unwanted files.

Even if sparse files are supported, RANDOM READ IS QUITE A BURDEN FOR IO OF HDD. Yes, we have ssds nowadays but, again, one will likely have hdds not ssds under his downloads storage of several terabytes because ssds are still quite expensive.

And one DOES NOT WANT to see FILES he EXPLICITLY UNCHECKED.

  1. They must be STORED NEAR WANTED FILES in torrent folder.

Storing them anywhere else will "disconnect" them from the torrent. If you transfer downloaded files to another computer to seed and/or reinstall os, you must remember to check that location otherwise after rehashing your wanted files will be incomplete.

Storing them anywhere else will make you provide enough space for that folder while torrent folder already has enough space by design.

  1. They must be STORED AS SINGLE FILE PER TORRENT. File may be hidden.

Storing them as file per unwanted file (or even file per piece) will lead to lots of small files which aren't even reusable as is if we decide to download unwanted files after all - these small files will contain pieces of unwanted files in wrong order and/or without enough space to insert remaining content.

And every such small file consumes inode. Etc.

  1. IMHO, the thoughts expressed above should not be considered in terms of to be or not to be (aka "there are users which don't want this, we won't implement this") but rather MANDATORY OR OPTIONAL (aka "there are users who optimized their system for current implementation, let's allow them to live happily as well").

comment:213 in reply to: ↑ 212 ; follow-up: Changed 5 years ago by xnoreq

Replying to neuen:

  1. Parts of pieces downloaded for wanted files that belong to unwanted files must NOT BE STORED AS UNWANTED FILE. Even if sparse files are supported.

I see no problem with storing unwanted files in a special unwanted folder if the filesystem supports sparse files. (also see reuse below)

Even if sparse files are supported, RANDOM READ IS QUITE A BURDEN FOR IO OF HDD. Yes, we have ssds nowadays but, again, one will likely have hdds not ssds under his downloads storage of several terabytes because ssds are still quite expensive.

"Random" read already happens when other peers request random pieces from random files. That's what an I/O cache is for. No special way of storing files/pieces/blocks/whatever will change that.

  1. They must be STORED NEAR WANTED FILES in torrent folder.

They should. I agree, hence my previous suggestion of a (hidden) .unwanted folder inside the torrent download directory.

  1. They must be STORED AS SINGLE FILE PER TORRENT. File may be hidden.

Storing them as file per unwanted file (or even file per piece) will lead to lots of small files which aren't even reusable as is if we decide to download unwanted files after all - these small files will contain pieces of unwanted files in wrong order and/or without enough space to insert remaining content.

They would be perfectly reusable when the filesystem supports sparse files. If not, they could still be reusable as-is if the unwanted files are small enough to be completely contained within a piece. Without sparse file support, the pieces should be stored in special piece files. I don't think that a file per piece is problematic. Worst case is two pieces (beginning and end) per file, when the unwanted file is big enough and between two wanted files, right?

comment:214 Changed 5 years ago by cfpp2p

lots of small files which aren't even reusable as is if we decide to download unwanted files after all

The current patch doesn't waste anything. If one ore more non-selected files are toggled to download. The appropriate data is removed from the temporary piece and placed into the now to download file(s) with no waste at all. The temporary piece is deleted automatically when all useable data has been used.

Last edited 5 years ago by cfpp2p (previous) (diff)

comment:215 in reply to: ↑ 213 Changed 5 years ago by neuen

Replying to xnoreq:

Replying to neuen:

  1. Parts of pieces downloaded for wanted files that belong to unwanted files must NOT BE STORED AS UNWANTED FILE. Even if sparse files are supported.

I see no problem with storing unwanted files in a special unwanted folder if the filesystem supports sparse files. (also see reuse below)

  1. Network filesystems do not support sparse files (see above).
  2. You are no longer able to move/copy whole torrent folder without being aware whether all of three - source filesystem, transfer mechanism and destination filesystem - support sparse files.

Replying to xnoreq:

Replying to neuen:

Even if sparse files are supported, RANDOM READ IS QUITE A BURDEN FOR IO OF HDD. Yes, we have ssds nowadays but, again, one will likely have hdds not ssds under his downloads storage of several terabytes because ssds are still quite expensive.

"Random" read already happens when other peers request random pieces from random files. That's what an I/O cache is for. No special way of storing files/pieces/blocks/whatever will change that.

I'm not talking about seeding but rather about actual torrent usage which in most cases is in sequential manner. I'm sorry to remind of but this is the main end-user goal of the torrent client - let user download and consume content. Seeding, rating etc are vital but still side goals.

  1. They must be STORED AS SINGLE FILE PER TORRENT. File may be hidden.

Storing them as file per unwanted file (or even file per piece) will lead to lots of small files which aren't even reusable as is if we decide to download unwanted files after all - these small files will contain pieces of unwanted files in wrong order and/or without enough space to insert remaining content.

They would be perfectly reusable when the filesystem supports sparse files. If not, they could still be reusable as-is if the unwanted files are small enough to be completely contained within a piece. Without sparse file support, the pieces should be stored in special piece files. I don't think that a file per piece is problematic. Worst case is two pieces (beginning and end) per file, when the unwanted file is big enough and between two wanted files, right?

Single unwanted pieces file per torrent in torrent folder may be non-mandatory for usage, it's just mandatory to exist. Sparse files, unwanted folder etc might exists as a usage option as well but they are non-mandatory to exist. Why? Because first way is the only option that will work quite optimally everywhere.

comment:216 Changed 4 years ago by b.pwned

  • Cc alex.bock@… added

comment:217 Changed 3 years ago by mike.dld

Closed #5953 as duplicate of this ticket.

Last edited 3 years ago by mike.dld (previous) (diff)

comment:218 follow-up: Changed 3 years ago by gborri

I've read quite full thread for this patch and the only reason i've found to fix this problem is related to disk space. I come up with this problem because of the integration with other software. Keep in mind that if you use transmission with sickbeard/sickrage or other and want to download a single file from torrent, having not expected file with temp extension break the entire process. With this limitation the integration with other sw is very difficult.

thanks Giovanni

comment:219 in reply to: ↑ 218 Changed 3 years ago by cfpp2p

Replying to gborri:

... With this limitation the integration with other sw is very difficult.

Until this is resolved in the current trunk you can try my experimental version from here: https://github.com/cfpp2p/transmission

Be aware that it's based on the 2.77+ branch. It is kept up to date and is very well tested. As far as I can read from your posts this patched version will resolve the issues you described here and at #5953. I'd recommend that you compile as the Mac daemon https://forum.transmissionbt.com/viewtopic.php?f=4&t=15167

I'm in the process of seeing if I can find someone to compile that most recent version as native for Mac. If you can't do this yourself then you can try the one at comment:204 . But actually... I'd recommend compiling the daemon if you can.

comment:220 follow-up: Changed 3 years ago by gborri

Thanks cfpp2p for your answer.

But the point is not having a custom version for myself. I'm trying to change the behavior of sickrage in order to download single episode for tvshow for which exists a multiepisode torrent.

this problem is a show stopper for making this change.

I have choose transmission because of the good mention that i have read on internet, but probably I have to change client if this issue is not resolved. this change probably make people use different client in order to have this feature.

I hope your patch will be merged in mainstream line.

Thanks Giovanni

comment:221 in reply to: ↑ 220 Changed 3 years ago by cfpp2p

Replying to gborri:

...But the point is not having a custom version for myself. I'm trying to change the behavior of sickrage in order to download single episode for tvshow for which exists a multiepisode torrent....

Thanks to our old friend bb (aka Bruno) there's now for everyone compiled Mac build based on cfpp2p experimental fork 2.7x branch (All commits through Jul 3, 2015).

Transmission-MAC-v276-t532-072015.zip

comment:222 follow-ups: Changed 3 years ago by mike.dld

Attaching the patch rebased against current trunk (r14572), for @jordan and anyone else interested.

Issues encountered while rebasing:

  • TR_CACHED_FILE_INIT in fileset_construct() has index_type and used_at positions mixed up, fixed by changing the order
  • tr_torrentGetFileMTime() passes uninitialized it (index type) to tr_fdFileGetCachedMTime(), fixed by passing TR_FD_INDEX_FILE instead (is this correct?)
  • static tr_torrentFileCompleted() used in setFileDND() while not yet declared or defined, fixed by adding a prototype declaration before its first use

All unit tests passed. No extensive manual testing performed.

I wouldn't merge this patch in its current state IMHO. I still see .part files in download directory (and no .dat files in /pieces/ subdirectory) if I'm unchecking the files short after they started downloading and not right away (think of magnet links for one example). Continuing to store data in .part files if those already exist in this case is not what I'd expect (or do I not understand the purpose of this ticket?). I'd expect UI to ask me whether I'd like to remove the [partially downloaded] file(s) from disk once unchecked, and if I agree, to create at most 2 .dat files (head and tail blocks) for each deselected file (having .part file; if .part files actually contain those head and tail blocks or course) in /pieces/ subdirectory and remove the .part files.

Changed 3 years ago by mike.dld

comment:223 follow-up: Changed 3 years ago by mike.dld

FYI after proof-reading the patch I now see potential for memory leaks and/or crashes in regards to fpbuf and lpbuf handling in setFileDND().

comment:224 in reply to: ↑ 222 ; follow-up: Changed 3 years ago by x190

Replying to mike.dld:

Attaching the patch rebased against current trunk (r14572), for @jordan and anyone else interested.

I still see .part files in download directory (and no .dat files in /pieces/ subdirectory) if I'm unchecking the files short after they started downloading and not right away (think of magnet links for one example). Continuing to store data in .part files if those already exist in this case is not what I'd expect (or do I not understand the purpose of this ticket?). I'd expect UI to ask me whether I'd like to remove the [partially downloaded] file(s) from disk once unchecked, and if I agree, to create at most 2 .dat files (head and tail blocks) for each deselected file (having .part file; if .part files actually contain those head and tail blocks or course) in /pieces/ subdirectory and remove the .part files.

That's why we also need some form of #4808 and #4262 as well, however, we cannot move forward without dedicated skilled developers with a passion for improving Transmission.

comment:225 in reply to: ↑ 224 Changed 3 years ago by mike.dld

Replying to x190:

... we cannot move forward without dedicated skilled developers with a passion for improving Transmission.

You keep saying things like that as if you have a plan. If you do, please just make it happen. If you don't, could you please refrain from further comments in this key, at least on Trac? Thanks.

comment:226 in reply to: ↑ 223 ; follow-up: Changed 3 years ago by cfpp2p

Replying to mike.dld:

  • tr_torrentGetFileMTime() passes uninitialized it (index type) to tr_fdFileGetCachedMTime(), fixed by passing TR_FD_INDEX_FILE instead (is this correct?)

I haven't looked at this enhancement's code in almost three years now since it has been functioning perfect for me. I have been using the patch extensively with a highly modified 2.7x libtransmission and daemon branch since that time. If I look at the code I have for that section I see
it = TR_FD_INDEX_FILE;

All unit tests passed. No extensive manual testing performed.

The patch against 2.7x was capaciously tested by myself and x190.

I wouldn't merge this patch in its current state IMHO. I still see .part files in download directory (and no .dat files in /pieces/ subdirectory) if I'm unchecking the files short after they started downloading and not right away (think of magnet links for one example). Continuing to store data in .part files if those already exist in this case is not what I'd expect (or do I not understand the purpose of this ticket?). I'd expect UI to ask me whether I'd like to remove the [partially downloaded] file(s) from disk once unchecked, and if I agree, to create at most 2 .dat files (head and tail blocks) for each deselected file (having .part file; if .part files actually contain those head and tail blocks or course) in /pieces/ subdirectory and remove the .part files.

I use #4808 for magnet links or for regular torrents I don't start them until unwanted files have been deselected. At most 2 .dat files is correctly achieved. The original poster stated "not actually create the files we didn't want in the first place". That is the purpose of this ticket as I see it. The idea of removing individual files that are at first selected and then later deselected is covered by separate ticket #4262.

Replying to mike.dld:

FYI after proof-reading the patch I now see potential for memory leaks and/or crashes in regards to fpbuf and lpbuf handling in setFileDND().

In torrent.c and immediately after:

  const bool rwfpmovept = fpmovept;
  const bool rwlpmovept = lpmovept;

add

    fpoffset = 0;
    fpoverlap = 0;
    fpbuf = NULL;
    lpoffset = 0;
    lpoverlap = 0;
    lpbuf = NULL;

I see the above in code I'm using, but as I remember things, this fixed compiler warnings as the logic of the code prevented any actual use of uninitialized so I don't know what you mean by crashes and memory leaks. The 2.7x patched code has been in use by a small but not insignificant user base without any such reported issues.

comment:227 in reply to: ↑ 222 Changed 3 years ago by cfpp2p

Replying to mike.dld:

Issues encountered while rebasing:

By the way, nice job rebasing the patch.

comment:228 in reply to: ↑ 226 ; follow-up: Changed 3 years ago by mike.dld

Replying to cfpp2p:

... I see the above in code I'm using, but as I remember things, this fixed compiler warnings as the logic of the code prevented any actual use of uninitialized so I don't know what you mean by crashes and memory leaks. The 2.7x patched code has been in use by a small but not insignificant user base without any such reported issues.

I was thinking about inconsistent handling of fpbuf + fpmovept and lpbuf + lpmovept. Consider these two pieces in setFileDND ():

  if (...)
    {
      ...
      if (...)
        {
          ...
          fpbuf = tr_malloc0 (...);
          if (...)
            {
              tr_free (fpbuf);
              fpmovept = false;
            }
          ...
        }
      else
        {
          fpmovept = false;
        }
    }

and

  if (...)
    {
      ...
      if (...)
        {
          ...
          lpbuf = tr_malloc0 (...);
          if (...)
            {
              tr_free (lpbuf);
              lpmovept = false;
              if (fpmovept)
                tr_free (fpbuf);
            }
          ...
        }
      else
        {
          lpmovept = false;
          if (fpmovept)
            tr_free (fpbuf);
        }
    }

First one sets fpmovept to false if fpbuf is freed or never allocated (so, if fpbuf is invalid). Second one does the same for lpmovept + lpbuf, but also frees fpbuf in some cases (makes it invalid) without setting fpmovept to false.

Later in same function there're some conditional statements based on fpmovept and lpmovept without checks for fpbuf and lpbuf validity. This disconnection of values with absense of asserts and lack of invalid pointers zeroing looks confusing and fragile/error-prone to me, also in regards to potential future changes.

comment:229 in reply to: ↑ 228 Changed 3 years ago by cfpp2p

Replying to mike.dld:

I was thinking about inconsistent handling of fpbuf + fpmovept and lpbuf + lpmovept.

I understand that the two of us have a philosophically different approach and programming style. My non-modular and inline style apparently appears confusing and fragile/error-prone to you. This may be because of not seeing the additional connections of fpbuf related with state change of rwlpmovept to lpmovept, and similarly lpbuf related with state change of rwfpmovept to fpmovept. There is no disconnection of values or inconsistent handling thereof. fpmovept is not solely tied to fpbuf or lpmovept to lpbuf, it's not that simple.

As I previously mentioned it has been nearly three years since I have looked at this patch. The reasons for the logic used was based on purposeful analysis and exhaustive testing by x190 and myself. It puts my memory to the test but I seem to remember that some of this logic was for obscure and fringe cases derived.

In comment:194 Jordan states "...gives me no headaches wrt new code complexity..." and at comment:197 "...drive the ticket towards simpler code; that purpose failed..." The patch as it stands now and has for nearly three years, simply works, does not crash, does not create memory leaks, and does not create inconsistencies. I'm not saying that there is not another way to accomplish the task with simpler code and less complexity.

I would think that the rebasing of the patch to 2.84+ was a lot of work and for that should be appreciated.

comment:230 Changed 3 years ago by diamondsw

Opened: 8 years ago.

I think that says everything that needs to be said.

If the developers don't like the style of the patch, then rewrite it and be done. If you like it, then accept it. If you don't like it and don't have any plans to write it in a way you like, mark it as WONTFIX and be done.

comment:231 Changed 3 years ago by sevenskull

I just got a torrent with more then 2000 files, and i selected about 70 files. But it downloaded 2000 files! I know it must download parts from others files, but that messed up all my files, if i want to organize it, i have to delete more than 1000 files! This is a old problem and a important one, utorrent does that since years.

comment:232 Changed 2 years ago by fetzu

Will this patch ever be implemented?

comment:233 follow-up: Changed 2 years ago by fairshare

I think the easiest solution would be to just store those unwanted files in ~/.cache/transmission/

comment:234 in reply to: ↑ 233 Changed 2 years ago by cfpp2p

Replying to fairshare:

I think the easiest solution would be to just store those unwanted files in ~/.cache/transmission/

With the patch just set that in the configuration file settings.json

"piece-temp-dir": "~/.cache/transmission/",


The user configurable directory allows for users on embedded systems to change the pieces directory so that the temporary pieces are saved NOT to a very small drive that is often the default for the config directory.

There is no "easiest solution". The unwanted files are not just created. The patch is necessary for that.

Discussion has been furthered here https://forum.transmissionbt.com/viewtopic.php?f=2&t=13363#p74519

Last edited 18 months ago by cfpp2p (previous) (diff)
Note: See TracTickets for help on using tickets.