Shared Memory and Semaphore Limits in Red Hat ES

I ran into a few issues recently, where I ran out of semaphore arrays when implementing a new web application. The application is a single-tier setup with all 34 databases on the same server which can get pretty busy at times and needs to have the kernel tweaked a little bit to allow for a large amount of concurrent processes.

To check the shared memory and semaphore limits in RHEL:

ipcs -al

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 32768
max total shared memory (kbytes) = 8388608
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 256 *** <- What I needed to update
max semaphores per array = 250
max semaphores system wide = 32000
max ops per semop call = 32
semaphore max value = 32767

------ Messages: Limits --------
max queues system wide = 32
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

Apparently the maximum number of arrays was too low (default on RHEL was 128 I believe).

To change THIS specific value, we need to update the running kernel parameters like so:

echo "250 32000 32 256" > /proc/sys/kernel/sem

**Note this will NOT MAKE THE KERNEL CHANGES PERMANENT

Obviously the format is:

[per array] [system wide] [per semop] [max arrays]

To make the changes permanent, we need to update /etc/sysctl.conf with the following:

kernel.sem = 250 32000 32 256

Another good change would be to update the max queues at the same time:

kernel.msgmni = 128

We could run 'sysctl -p' to update the running kernel with the permanent settings.

Double check with 'sysctl -a'!

SSH Auto-Login - No password!

There comes a time in every Linux Admin's career when we need to set up an automated SSH login to a remote server. Whether it be backups, replication or just plain convenience, this is something that can save lots of time throughout the life of a server. Keep in mind that you should have a seperate user for each process and try not to use the 'root' account. Of course, if you haven't changed the default sshd config file then root can't log in directly anyways...

Steps that we'll cover:

1. Set up .ssh under the /home/user/ folder.
2. Create a DSA keypair (private/public).
3. Copy the public key from one server to the other.
4. Append the public key info to the authorized_keys file.
5. Update twitter account with link to this article.

[TO BE DONE ON BOTH SERVERS]

First, make sure your user has the correct home directory structure set up:

mkdir -p /home/user/.ssh
cd /home/user/.ssh

Next, we'll generate a dsa (as opposed to rsa) keypair:

ssh-keygen -t dsa

You should now see these files in your .ssh directory:

id_dsa - Private key (DO NOT DISTRIBUTE THIS FILE!)
id_dsa.pub - Public Key (This is the one to distribute)

Now we'll ssh from SERVER1 to SERVER2:

ssh user@SERVER2
cd /home/user/.ssh

Now use scp to get the public key from SERVER1:

scp user@SERVER1:/home/user/.ssh/id_dsa.pub ./SERVER1.pub
(this will grab the id_dsa.pub from SERVER1 and rename it in the current directory)


Append the public key to the authorized_keys file:
cat SERVER1.pub >> authorized_keys

Perform the same steps on the other server. If you only need a one-way connection, then you should only set up one server with the public key. A basic rule of security in any operating system is: If you don't need it, don't enable it!

I should also mention that OpenSSH was created by uber-genius Theo DeRaadt (as part of the OpenBSD O/S project) who lives in Alberta, Canada. He was born in South America, but we won't hold that against him...

OpenSSH is THE secure remote access standard on (AFAIK) all Linux/BSD/Unix operating systems. I believe it to be the most important utility ever developed...yeah, it's that important!

Windows vs. Linux vs. OSX - Which is the best desktop?

It's hard to have an un-biased opinion on the age-old Windows vs. Linux vs. OSX argument (sorry uber geeks, no MINIX this time) as many people spend more time in one environment than that other. If you read ANY other comparisons, just keep that in mind. Also, the version of Linux that I'm using as a comparison is: LinuxMint v.7 - Gnome Edition (based on Ubuntu, which is based on Debian).

Windows - NT, 98, 98SE, 2000, XP, Vista, Windows 7
Cost: Desktop $200-$320 (Win 7)
Stereotype: "Normal" people (usually in bad mood due to spyware problems)

Pros:

• Compatibility (every app ever made can run on Windows)
• Ease of use (very good UI)
• Support & Purchase availability
• Regular updates
• Games (nearly the only major gaming O/S)
  
• Hardware independant (runs on anything, generally)
• Hardware support (requires drivers)

Cons:

• Price
• Viruses, malware, spyware (due to architecture of O/S)
• Stability (in some cases)
• Drivers
• Security (although regular updates help)
• The semi-annual reinstallation
• Too many versions

Who uses it:

• Pretty much everyone you know
• Businesses (due to popularity of the Office suite and application compatibility)
• Gamers

What else?

Let's get it straight. Windows is THE desktop standard and has been for as long as personal computers have been in your house and occupying your time. Nearly every application ever made runs on Windows and it will likely be this way for the forseeable future. But the news isn't all good. In many cases Windows requires a full reinstallation every 6 months due to viruses, trojans, malware, spyware, adware, browser hijacking, phishing...the list goes on and on. I understand that the latest versions are more secure, but still...I shouldn't have to 'be careful' when checking my email or checking out Facebook. If you've ever had to call Microsoft after losing your installation CD, you know it's almost 'not worth the hassle'...

Keep in mind that Windows' security vulnerabilities are mostly due to it's massive usage (IMO), but still...Windows is a very good operating system and there's a reason that it's #1 on the desktop (for now).

OSX - Panther, Tiger, Leopard, Snow Leopard
Cost: $600+ (hardware inclusive - basic model)
Stereotype: "Artsy" people dressed in black, wearing french berets, drinking non-fat lattes, stealing coffee shop wi-fi (due to money spent on lattes and expensive laptops)

Pros:

• Graphics and special effects are fantastic
• Stable (as it only runs on specific hardware)
  
• Upgrade price ($29 Snow Leopard upgrade? amazing!)
• Good Backup/Restore (Timevault) support
  
• Very easy to use
• Secure - nearly no viruses or spyware
• UI - very easy to learn and use
• Media support is top-notch (video/music/picture integration, the whole 'iLife' thing)
  

Cons:

• Only runs on Apple hardware (which is more expensive)
• Windows application compatibility (although this is getting much better)
• Very limited in what you can control
  

Who uses it:

• Graphics designers
• Web developers
• Publishers
• Rich people
• Video/Music producers

What else?

The early Mac experience was not one to be desired. It sucked to be quite honest. The only people in their right mind that would subject themselves to such torture were newspaper/magazine editors and graphic designers.

Now, cue the savior, Mac OSX. This "new" version took the best parts of the FreeBSD and NetBSD (a Unix variant) and incorporated it into the heart of the O/S, "nextstep", making it extremely secure and stable. The graphics and special effects were easily two years ahead of the competition, making it instantly 'cool' among young professionals and everyone who was pissed off at Microsoft (after having to reinstall twice a year and losing their resumes and photos from aunt Mae's second wedding). OSX is great for managing and manipulating music, video and photos from nearly any source. It's also quite easy to learn for those who are making the switch from Windows. Automatic backup (including revision control) is quite appealing to professionals who now choose OSX as their primary platform. Plus, "...holy shit those new macs are cool, eh!?"

Linux - Ubuntu, Fedora, Suse, RedHat, Gentoo, plus 8000000 other distributions
Cost: $0 ..."but only if your time is worthless"
Stereotype: Super geeks living in the dark basement of their parent's house, compiling kernels, drinking strong coffee, wearing birkenstocks and stained T-shirts with witty sayings on them

Pros:

• Free
• Secure
• Stable
• Open source
• Free applications
• New versions are interoperable with Windows and OSX
• Easily available
• Easy to install and use
• Graphics and UI are generally very good
  

Cons:

• Too many versions
• Upgrades usually 'break shit'
• Too many choices

Who uses it:

• Advanced users (hackers)
• Networking and security professionals
• Nearly every web site administrator in the world runs Apache on Linux
• IT guys
• You do! (cell phones, smart phones, routers, hubs, embedded devices)

What else?

10 years ago, Linux as a desktop was a big disappointment. There were way too many versions of the O/S, even BEFORE choosing a window manager. You had to be a genius to use it. It was extremely unstable and a huge pain in the ass. As an IT professional, I remember spending 6 HOURS editing my xorg.conf, just to get the right display resolution. Extremely frustrating to say the least...and whenever I updated the system EVERYTHING stopped working. Broken packages (if I could even figure out HOW to install it) and "dependancy hell" killed my Saturday nights. And support?? Like some message board is going to solve all my problems...gimme a break...

Apparently, 10 years is a long time.

Fast-forward to 2009 - Linux is now big business. Redhat, Ubuntu and Suse all have paid desktop support and are becoming the 'other guy' that gets mentioned whenever the quarterly budget review happens. Installing software used to be a headache (and a personal peeve of mine) but now, nearly every distribution has a graphical package manager. Although most of the well-known applications have different names than their Windows equivalent (MS Office - Openoffice, Outlook Express - Evolution, Internet Explorer - Firefox) each one is free and works just as well. Even the special effects manager "compiz fusion" has a 1-click installation and configuration and (IMO) looks better than any version of Windows. In addition, I have not once had to use the command line to do anything. Even wireless. Honestly though, it takes me at least 3 hours to build a Windows workstation (I'm talking drivers, updates and programs) and I have to be in the room the whole time. With the newest version of Linux (Mint) it's less than half of that and everything was working, even my webcam. Not one additional driver needed.

Windows vs. OSX

• OSX experience is "cooler" but Windows as some nice effects as well (transparencies)
  
• OSX is more stable
• OSX is more secure
  
• Windows has more available applications (especially games)
• Windows is less expensive
• Windows can run on my current hardware
  
• Windows has too many versions
  

Verdict: OSX at home, Windows at work

Linux vs. Windows

• Linux and associated applications are free in cost
• Linux has too many different versions
• Windows has far more applications
• Linux supports more devices upon installation
• Windows may have more complete hardware support (functionality), but you need to find the drivers
• Linux has superior security and stability
• Linux Compiz fusion graphics are better than Windows
• Windows transparencies are better than Linux

Verdict: Linux - no viruses and all my stuff works without drivers? I'll pay free bucks for that!

Linux vs. OSX

• OSX is more stable (because Apple gets to choose the hardware)
• Linux runs on my current hardware
• OSX is much easier to upgrade (plus defined releases)
• OSX is far more expensive
• Linux is free
• Linux has more applications and you don't have to pay for them
• OSX has smoother graphics but Linux is very close
  
• OSX and Linux are equally secure
• OSX feels very restrictive compared to Linux
• Linux and OSX have many similarities
  

Verdict: OSX if you're rich, Linux if you're smart

Windows vs. Linux vs. OSX

Overall verdict?

We've seen a few of the main differences between the three dominant desktop operating system players. I suppose that the only thing that's been determined is that there is more than enough room for each one to improve. There's no such thing as 'the perfect desktop'...just yet...

To sum it up, I'd like to offer the following advice:

• If you want to be cool, get OSX
• If you want to work, get Windows
• If you want to learn, get Linux.

Seriously though, moving forward into the next decade, we're going to see the fine lines between these three operating systems disappear. The next big step will be having OSX, Windows and Linux work seamlessly in one hetergeneous environment (we're actually starting to see it right now). Each one will become more like the others and in 5 years' time, you won't be able to tell the difference between them.


------------------------------
A little background on myself:

I grew up with computers (my parents owned a computer store). Ever since the days of the 486DX (yes, with the 'turbo' button of course...it makes it 'go faster'!) and modern operating systems in general, I've been completely in-love with the technology. This was my focus throughout high school, college and subsequent 10 years in the IT industry.

Notes:

• Compared Linux version is Linux Mint v. 7 - Gnome edition
  
• When I refer to Linux, I mean "The Linux kernel and GNU toolset" in general
• Nobody reads footnotes anymore
  

Various Linux Commands - When you Need 'em!

From time to time I'll run into a request like "OMG, you HAVE to fix the mail queue RIGHT NOW!! We can't get our report and it's stopping us from submitting our POs in time for tomorrow's delivery!". Being the good (and prepared) Sysadmin that I am, I'm always ready to jump in and help out...My problem is that I don't always write down important commands where I should! (They're scattered all over the place, mostly in the form of post-it notes with poor handwriting and coffee stains)

Yes yes, I know it shouldn't take me 5 mins to search google to figure out how to re-run a sendmail queue, but these things slip your mind sometimes...especially when you have 6 people, including a manager and a director, looking over your shoulder waiting for the mail to 'start flowing'!

This is just a quick and dirty list of the commands that are specific to the things I deal with on a day-to-day basis. I won't go into much detail, so if you'd like more information: stop being so lazy and google it yourself!

Sendmail

'mailq' - List locally-queued mail
'mailq -Ac' - List sent, but undelivered, mail
'/usr/sbin/sendmail -v -q -d3.30' - Show current load in relation to sendmail.cf settings
'/usr/sbin/sendmail -qR /path/to/queue' - Force a mail queue run immediately
'tail /var/log/maillog' - Show output of mail log

Samba

'smbpasswd -a ' - Add user to samba
'smbpasswd -d/-e ' - Disable/enable samba user
'smbpasswd -n ' - Null password for user

Printing

'lpstat -t' - Show every printer, including queue
'/usr/bin/enable or disable' - Enable or disable printer, use the ABSOLUTE path
'lp -i -H restart' - Restart print job
'lpmove ' - Move a stalled job to different printer
'lprm -P ' - Remove print job
'lpadmin -P -E -v socket://ip.addr.of.prtr:9100' - Add and enable a new printer

Generic

'netstat -an' - Show all listening ports
'lsof -Pnl +M -i4' - Show files/daemons listening on specific ports/protocols
'w | wc' - List # of currently logged-on users
'finger ' - Not what you think...it's a legit command...I swear!
'chage -l ' - Show password aging for a user

I'll be updating this post with things that I think may be helpful to others...Personally, I didn't even know about the sendmail command for showing the settings vs. the current load...I found it on a sendmail development project blog from about 8 years ago...you never know!

When to NOT recommend Linux desktop solutions!

My mom always told me: "..Andrew, there's a time and a place for everything..". Usually she said this when I was doing something that I definitely was NOT supposed to be doing!...like playing soccer in the kitchen with the dog while she was trying to make dinner...

I think that we can all agree that Linux is great, right? Well, there are always going to be situations where Linux is simply not the answer. For example: My parents had a workstation that had gone belly-up and were in need of a replacement. Great! Linux to the rescue!

...or so I thought...

The problem was, my dad ONLY knows Windows XP. The ONLY thing he knows how to do is double-click on Internet Exploder, select his drf.com (horse racing site) and look at the track entries for that day, then watch the race via streaming video.

Simple enough task no? Well, I went about installing the latest stable version of Ubuntu and everything was going fine. The problem was: It was different. Not really the usage of the O/S or the browser...but the 'little things'. Icons were different. Programs had different names. The windows looked different. It was, as he put it, just "weird" and he didn't "know it".

After 5 minutes he gave up. I was devastated.

Immediately I dove into technical manuals and man pages thinking that I could find the solution in there, as I often do...The problem was, there WAS no technical answer.

The answer was simply: "Linux was not the answer"

This story reminded me of some other things to take into consideration BEFORE recommending a Linux-based desktop solution:

• Can you provide the additional support the client will need?

• Are there business procedures written specifically for the operating system?

• Are they open-minded?

• Is the $$$ savings on licensing enough to justify losses in time when learning new technologies?

• Do they even have time to learn a new O/S? How much time will it take to retrain?

• Is the nature of their business DEPENDENT on their computing platform?

Obviously this is only scratching the surface and there are myriads of things to consider when investing in newer, better technologies such as Linux. Hopefully this will get everyone thinking the next time a client "...needs a new desktop...quick!".