Friday, March 11, 2011

Shared Memory and Semaphore Limits in Red Hat ES

I ran into a few issues recently, where I ran out of semaphore arrays when implementing a new web application. The application is a single-tier setup with all 34 databases on the same server which can get pretty busy at times and needs to have the kernel tweaked a little bit to allow for a large amount of concurrent processes.

To check the shared memory and semaphore limits in RHEL:

ipcs -al

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 256 *** <- What I needed to update
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 32
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

Apparently the maximum number of arrays was too low (default on RHEL was 128 I believe).

To change THIS specific value, we need to update the running kernel parameters like so:

echo "250 32000 32 256" > /proc/sys/kernel/sem

**Note this will NOT MAKE THE KERNEL CHANGES PERMANENT

Obviously the format is:

[per array] [system wide] [per semop] [max arrays]

To make the changes permanent, we need to update /etc/sysctl.conf with the following:

kernel.sem = 250 32000 32 256

Another good change would be to update the max queues at the same time:

kernel.msgmni = 128

We could run 'sysctl -p' to update the running kernel with the permanent settings.

Double check with 'sysctl -a'!