Set this higher to prevent errors during high load
Overview
Operating systems (Linux and macOS included) have settings which limit the number of files and processes that are allowed to be open. This limit protects the system from being overrun. But its default is usually set too low, when machines had way less power. Thus a “gotcha” that is only apparent when “too many files open” crashes appear only under load (as in during a stress test or production spike).
man bash, the Bash manual says the ulimit command (common among Linux flavors) provides “control over the resources available to the shell and to processes started by it”.
-
Obtain the current limit of file descriptors
ulimit -n
An example output: “256” or “10032”.
PROTIP: On MacOS, the maximum number that can be specified is 12288.
-
Obtain the current limit of processes
ulimit -u
An example output: “1418”.
On macOS
-
Obtain the current limit:
launchctl limit maxfiles
The response output should have numbers like this:
maxfiles 65536 200000
The first number is the “soft” limit and the second number is the “hard” limit.
Configuration changes are necessary if lower numbers are displayed, such as:
maxfiles 256 unlimited
-
If the soft limit is too low (such as 256), set the current session to:
sudo launchctl limit maxfiles 65536 200000
Some set it to 1048576 (over a million).
Since sudo is needed, you are prompted for a password.
PROTIP: Because this would go back to defaults on reboot, add this command in your ~/.bash_profile
Alternately, install Facebook’s Watchman utility which watches and adjusts automatically.
PROTIP: Take both a full/complete backup to ensure fall-back. Also run a benchmark performance measurement before and after changing the configuration to detect issues.
-
Over several versions, Apple has been changing the way system-wide open file limits can be set upon restart.
Yosemite and older
NOTE: On older MacOSX Yosemite, to make the settings permanent, increase the limits, edit (or create) file /etc/launchd.conf to contain:
limit maxfiles 65536 200000
Sierra and newer versions
Newer versions of macOS do not reference the file due to security considerations.
On newer macOS Sierra and High Sierra, Dejan Kitic and this found that two plist files need to be added.
-
Copy in folder /Library/LaunchDaemons/ plist files from a GitHub repository:
https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxfiles.plist at 524288
https://github.com/wilsonmar/mac-setup/blob/master/configs/limit.maxproc.plist at 2048
-
Invoke the files:
sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist sudo launchctl load -w /Library/LaunchDaemons/limit.maxproc.plist
The files’ ownership needs to be changed to “root:wheel”.
Their permissions need to be “-rw-r–r–”, set by sudo chmod 644.
-
So how do you turn csrutil off? Google says “sudo csrutil disable…”. But not so easy. That can only be done in Recovery Mode. So, reboot, hold command + R to enter Recovery Mode, once there open terminal and do csrutil disable… Finally a breakthrough…disabled.
-
Now you can adjust the process limit on Mac OS X Yosemite and El Capitan versions:
sudo ulimit -n 65536 200000
Since sudo is needed, you are prompted for a password.
-
To increase the inotify watchers max limit, edit (or create) file /etc/sysctl.conf (inherited from BSD) to contain:
kern.maxfiles=49152 kern.maxfilesperproc=24576
or
kern.maxfiles=200000 kern.maxfilesperproc=200000
Alternately, run interactive commands:
sudo sysctl -w kern.maxfiles=5242880
The response:
kern.maxfiles: 49152 -> 5242880
sudo sysctl -w kern.maxfilesperproc=524288
The response:
kern.maxfilesperproc: 24576 -> 524288
-
Restart the system for the new limits to take effect.
-
After restarting, verify the new limits by running:
On RedHat and CentOS
-
To enable PAM-based user limits so that non-root users, such as the riak user, may specify a higher value for maximum open files, edit (or create) file /etc/security/limits.conf to contain:
* soft nofile 65536 * hard nofile 200000
If the file already exists, append the above to the bottom of the file.
-
To activate limits, edit (or create) file /etc/pam.d/login to contain:
session required pam_limits.so
- Restart the machine.
-
Verify by “ulimit -a” again.
ulimit -n ulimit -Hn # Hard limit ulimit -Sn # Soft limit
If the response is “10032”