Solution
Docker Executor
Docker executors do not allow privileged access, and thus, cannot be used to update the number of open files. In such cases, it is recommended to use the machine executor.
Machine Executor
With kernel-level privileges, users can update the number of open files to a custom value using commands such as ulimit.
Here are the steps to increase the limits at a system-wide level:
sudo sed -i 's/#DefaultLimitNOFILE=1024:524288/DefaultLimitNOFILE=131072:524288/g' /etc/systemd/system.conf sudo sed -i 's/#DefaultLimitNOFILE=/DefaultLimitNOFILE=131072:524288/g' /etc/systemd/user.conf sudo systemctl daemon-reexec echo fs.file-max=131072 | sudo tee -a /etc/sysctl.conf sudo su - circleci -c 'ulimit -Sn' -s '/bin/bash'
Additionally, changes can be made to the 01-openfiles.conf Ubuntu configuration file located in the /etc/security/limits.d/ directory. This configuration file is shipped with the Ubuntu OS and is not anticipated to change unless deprecated by Canonical.
Conclusion
While the ulimit approach is commonly recommended, making changes to the 01-openfiles.conf within the /etc/security directory is also a safe alternative.