Opened 9 years ago

Closed 8 years ago

#4918 closed Enhancement (incomplete)

Enable direct reads to prevent disk cache pollution

Reported by: jammi Owned by: gvdl
Priority: Normal Milestone: None Set
Component: Transmission Version: 2.51
Severity: Normal Keywords:
Cc:

Description

As discussed on IRC, it would be nice to perform non-cached reads. The rest of the system wouldn't get as i/o starved, when the disk cache contains less items that get a cache misses. Some further action after that could be some nice torrent-specific statistics and in-app disk caching to better serve the usage patterns of Transmission.

Change History (5)

comment:1 Changed 9 years ago by gvdl

  • Owner set to gvdl
  • Status changed from new to assigned

I was on the discussion. Although I won't commit that this should make the performance increase in all circumstances the disabling the filesystem caching for media files is generally a good idea. Everything depends on how good the internal seeding cache is. In any case the use of FNOCACHE and the Linux(posix_fadvise:POSIX_FADV_DONTNEED)/*BSD (fadvise:FADV_DONTNEED) is probably worth while. They also seem to be available through glibc

See also Application buffer caching paper.

It should be optional though, controlled by a config setting.

Last edited 9 years ago by gvdl (previous) (diff)

comment:2 Changed 8 years ago by jordan

This ticket needs more information.

  • In which circumstances is this advisable?
  • Could the people proposing this please do some cursory benchmarks with & without the posix_fadvise() hints to see what kind of performance change we're looking at?
  • Suggesting a config setting here feels like you're not sure whether or not this feature is a net positive. If we know the use cases for direct reads, is a config setting warranted? Adding an option for esoteric options like this will just generate lots of user questions and confused forum threads.

comment:3 Changed 8 years ago by x190

Here's a quote from Apple's 'FileSystemProgrammingGuide?'. The System Has its Own File Caching Mechanism Disk caching can be a good way to accelerate access to file data, but its use is not appropriate in every situation. Caching increases the memory footprint of your app and if used inappropriately can be more expensive than simply reloading data from the disk. Caching is most appropriate for files you plan to access multiple times. If you have files you only intend to use once, you should either disable the caches or map the file into memory. Disabling File-System Caching When reading data that you are certain you won’t need again soon, such as streaming a large multimedia file, tell the file system not to add that data to the file-system caches. By default, the system maintains a buffer cache with the data most recently read from disk. This disk cache is most effective when it contains frequently used data. If you leave file caching enabled while streaming a large multimedia file, you can quickly fill up the disk cache with data you won’t use again. Even worse is that this process is likely to push other data out of the cache that might have benefited from being there. Apps can call the BSD fcntl function with the F_NOCACHE flag to enable or disable caching for a file. For more information about this function, see fcntl.

comment:4 Changed 8 years ago by jordan

x190: we already use F_NOCACHE in tr_set_file_for_single_pass(). I was asking when would this be advisable in Transmission but isn't already being used.

comment:5 Changed 8 years ago by jordan

  • Resolution set to incomplete
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.