Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#3983 closed Enhancement (invalid)

RFE(Suggestion): store 'resume' files in binary not bencoding - lower I/O

Reported by: Astara Owned by:
Priority: Normal Milestone: None Set
Component: Transmission Version: 2.13
Severity: Normal Keywords:


Resume files are an 'internal' communication pathway, inside of /transmission/ -- that is, they are only created by /transmission/, and they are only read by /transmission/ (supported by the fact that when run under a separate user, the file permission permit read/write only by that user).. The fact that the communication pathway happens through an external /file/ is a necessary way to make the storage of that information persistent.

If /transmission/ was a program native to windows, under some design(s), it might have stored such information in an OS-wide /registry/ (Hierarchical DB) as used in Windows that is also persistent. The information could have been stored 'anywhere', that the persistency trait is supported.

But just as we don't store information in /transmission's/ internal variables in 'bencode' (because it would be inefficient and is not the native format used by the computer), there is really little need to do so in it's persistent, internal storage format. The issues of efficiency have been raised regarding the resume information. The the routines to read and write the resume files are not interpretive, only the data would need to be stored, not the names of the fields. But by storing space and I/O could probably be cut in half, with less processing overhead to convert to and from a non-native format during each I/O cycle.

Change History (4)

comment:1 Changed 11 years ago by jordan

Using a binary format would probably be a faster, but using bencoded data has its benefits. One important advantage is that we can add new keys over time for new features, but still fall back to parsing the old keys when necessary. That way we can change the data we save in the .resume file while still being able to handle old .resume files. We could do that in a binary file too, but the benc objects are a very natural way to handle this in the code without having to reinvent the wheel.

Lastly, was this filed as a bug by accident? This seems more like an enhancement request to me.

comment:2 Changed 11 years ago by Astara

(Enhancement = yes, bug=no)

Binary doesn't need a wheel (i.e. no need to reinvent the wheel (encoding system)). You use native computer binary, for the computer, nothing could be more natural. Adding any encoding system would take more cpu cycles and memory. The simpler code in talking 'native' would reduce memory usage as well as cpu usage.

Putting a version indicator in as the first word would be a good idea then you could do as you say -- since anytime you upgrade the format, you'd have to have the smarts in the .resume read/write routine to handle different 'versions' of the file. With a version indicator, you could have 1 conditional to test for the new part, whereas with a changing format, you'd have to read the bencoded data, datum by datum, to decipher which format you were using. You'd read a version, a fix-length segment depending on version, a len for the variable part then the variable part in 1 more read. Four high level read/write operations vs. the 'many' required now (say high level, as I don't know how much they are buffered).

In converting, a general format converter could also be written to go from the binary->benc (and back) if you really wanted to have a benc file to diddle with...

Last edited 11 years ago by Astara (previous) (diff)

comment:3 Changed 11 years ago by jordan

  • Resolution set to invalid
  • Status changed from new to closed

I appreciate the suggestion, but the current system doesn't seem particularly broken to me. By having the data already in benc, there's no need to write additional code to go from a new format to benc (and back), so that's less code to worry about.

benc/json files are read into memory in a single read() call in libtransmission/utils.c's tr_loadFile().

Maybe this is something that ijuxda would be interested in coding up in one of his experimental branches, so that its performance can be compared to the current system. But overall I think the current system "ain't broke." :)

comment:4 Changed 11 years ago by livings124

  • Type changed from Bug to Enhancement
Note: See TracTickets for help on using tickets.