Opened 10 years ago

Closed 10 years ago

#4341 closed Bug (fixed)

Unable to save resume file. Too many open files.

Reported by: hank Owned by: jordan
Priority: Normal Milestone: 2.33
Component: libtransmission Version: 2.32
Severity: Critical Keywords:
Cc:

Description

Application pauses all transfers. I have 44 transfers running on mac 10.5.8 and the newest version of transmission. I get Unable to save resume file. Too many open files. or Too many open files. as the error message. Transmission is totally dead for me.

Change History (15)

comment:1 Changed 10 years ago by x190

Forum posters have reported this issue as well.

https://forum.transmissionbt.com/viewtopic.php?f=2&t=11898

comment:2 Changed 10 years ago by jordan

In the preferences dialog, how many peers per-torrent and overall are you allowing?

comment:3 Changed 10 years ago by jordan

  • Component changed from Transmission to libtransmission
  • Owner set to jordan

r12541 is a possible fix for this.

Hank, could you test with a nightly build from https://build.transmissionbt.com/job/trunk-mac/ and see if that solves the problem for you?

comment:4 Changed 10 years ago by jordan

hank: ping

comment:5 Changed 10 years ago by x190

A question, if I may:

fd limit.c: const int FILE_CACHE_SIZE = 32

My current count of T's open files is 81, not counting peer connections and only running 3 torrents. Hopefully, not an issue? I probably don't understand the concept but could you tell me what is included in that 1024 (FD_SETSIZE) limit? Just peer connections or all open files (which could be a huge number)?

Here's a couple of system file items that I'm posting just so that you can confirm for yourself that you have got it right (might be helpful?).

/*
 * The following macro defines the number of fd_masks in an fd_set:
 */

#ifndef FD_SETSIZE
#   ifdef OPEN_MAX
#	define FD_SETSIZE OPEN_MAX
#   else
#	define FD_SETSIZE 256
#   endif
#endif
#ifdef FD_SETSIZE
#define	__DARWIN_FD_SETSIZE	FD_SETSIZE
#else /* !FD_SETSIZE */
#define	__DARWIN_FD_SETSIZE	1024
#endif /* FD_SETSIZE */

Last edited 10 years ago by x190 (previous) (diff)

comment:6 follow-up: Changed 10 years ago by jordan

It would mostly involve peer socket connections and locally-open files.

I don't understand what you mean about "so that you can confirm for yourself that you have got it right (might be helpful?)". The value of FD_SETSIZE is system-dependent, so Transmission can't make assumptions about the actual value being 256 or 1024 or anything else.

comment:7 in reply to: ↑ 6 Changed 10 years ago by x190

Replying to jordan:

It would mostly involve peer socket connections and locally-open files.

Activity Monitor on the Mac allows the user to inspect a process and list all "open" files for that process. A wc -l in Terminal can then be used to give a count on the saved output. Looking at this output shows every selected file of every torrent listed as "open" (I am running only 3 torrents atm) in addition to the expected system and T stuff like blocklists. As you know, some torrents contain over 1000 files and multiply that by hundreds or thousands of torrents and you can easily get over 100,000 files. I assume the majority of these files must be handled outside of that measly 1024 (or whatever) figure? In the forum, you made the following statement---"Well, first things first... I'd like to get tester feedback about whether or not the new setrlimit code solves the "too many open files" error?". Would not the answer to that query depend on exactly what is included in FD_SETSIZE, which from my testing is 1024 on a Mac using the current code?

I'm attempting to gain a basic understanding of the issue in order to try to help forum users in a more knowledgeable manner.

Last edited 10 years ago by x190 (previous) (diff)

comment:8 Changed 10 years ago by jordan

Ah, gotcha.

The key idea to understand is that in Unix (and therefore in OS X), nearly everything is a file. So if we open a local file, that uses up a file descriptor, and if we open a socket to a peer, that uses up a file descriptor too.

FILE_CACHE_SIZE is the number of "local data" files that we'll leave open for reading and writing. Even if a torrent has thousands of files, we'll never have more than FILE_CACHE_SIZE of them open at once.

Most of the file descriptors will probably be used up by sockets, rather than by open local files.

Does that make sense?

comment:9 Changed 10 years ago by x190

Thanks, that confirms my suspicions although I'm still a little confused about whether a fudge factor is built in or not, and also whether "local data files" includes system and support files needed by Transmission or are those files loaded into memory at start-up and then not considered "open" in the file descriptor sense? My reference to "fudge factor" was due to the fact that in the following comment you appear to be allowing all file descriptors (FD_SETSIZE) to be used by sockets (peer connections) leaving none for FILE_CACHE_SIZE. https://trac.transmissionbt.com/ticket/4164#comment:20

To completely change your line of thought, I need to ask whether the "too many open files" issue being addressed by this ticket, might actually be a side effect of the corrupted "Resume" folder/file issue which has appeared frequently in the forum recently. Please don't ask me to technically justify that suspicion, but the "Resume" issue reporters have listed a wide range of symptoms, so I felt this might be one more.

comment:10 follow-up: Changed 10 years ago by jordan

My guess is that system and support files would be part of the overall count too, but I don't know that for certain on OS X.

comment:11 in reply to: ↑ 10 Changed 10 years ago by x190

Replying to jordan:

My guess is that system and support files would be part of the overall count too, but I don't know that for certain on OS X.

I guess I'll have to leave it at that. :lol:

Is this what you wanted to hear? https://forum.transmissionbt.com/viewtopic.php?f=2&t=11898&start=15#p55424

comment:12 Changed 10 years ago by jordan

  • Milestone changed from None Set to 2.33
  • Resolution set to fixed
  • Severity changed from Blocker to Critical
  • Status changed from new to closed

comment:13 Changed 10 years ago by jordan

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:14 Changed 10 years ago by jordan

  • Status changed from reopened to new

comment:15 Changed 10 years ago by jordan

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.