This time, we do troubleshooting inotify issue: too many open files. the story begun when there were many users having difficulties of accessing VM’s console on proxmox hypervisor. when users clicked the console menu, it doesnt appeared as expected, only gives a blank pages with error message like the picture above. Some of can access smoothly, but others failed.
By observing this behaviour, it seems like there an issue with resource limitation where when it reached the threshold level more users cannot be served properly.
when i check the error log, i got message
Unable to create new inotify object: Too many open files at /usr/share/perl5/PVE/INotify.pm line 389.
TASK ERROR: Failed to run vncproxy
and when i check the line 389 on the file, i got the picture below.
so i run “sysctl -a –pattern inotify” to find out which parametes related to inotify and found below:
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 8192
I then increases each original value above by 4 folds and save the entries to /etc/sysctl.conf file in order to make it permanent.
the result, access to container and console is now better 🙂
note: this inotify issue also happens when you have lots of virtual machines or containers. the setting above could be a relieve. 🙂