Bumps along the way of deploying puppet

In my new environment we have about 100 servers of various flavors... predominately CentOS and Solaris with several RedHat servers and a couple of Windows and Debian boxes. The configurations, versions and patch releases are all over the place. Some of these boxes are quite old (cough) Fedora 5 (cough) (cough) solaris 9 (cough).

My first goal is simply to get puppet onto all of these servers. Of the ~100 servers I need to manage, about 30 of them are dev/qa/test boxes. I now have puppet installed on all of them. There were a few bumps along the way.

Impediments

1. The right repository--I'm sure for the yum guru's out there, this will seem trivial, but it was a problem for me. A repository I was initially using had an older version of puppet (which I did not realize immediately). It wasn't until one of the boxes I was installing puppet on already had a repository configured with a new version of puppet did I realize I had a problem. And it wasn't until I tried connecting it to the puppet-server that I realized I had a problem because I got this somewhat unhelpful error: Error 400 on SERVER: No support for http method POST

Thanks to http://bitcube.co.uk/content/puppet-errors-explained for the explanation.

So, I updated the puppet-master and I fixed the repository I was using and now I'm getting the latest and greatest.

2. Yum dependencies--Occasionally I ran into dependency issues when running yum install. It wasn't terribly clear to me why I got these errors, but generally, it happened when there was a longer list of dependencies. I was able to work around this typically, by simply doing a yum install of one of the dependent packages first, and then trying the yum install puppet again and it worked.

3. Old OSes without the required packages--In some cases I could not work around the the dependencies because the OS version was so old--Fedora 8, 7 and 5. These OSes were looking for libselinux-util which wasn't made available until Fedora 10! Note to self: put these systems on the top of the list to retire.

4. puppetmaster directory details: Also worth mentioning, it took me some time to sort out which directories and where they need to be located on the puppetmaster. I'm not sure if this is a poor documentation problem, or a user problem, but it took some trial and error to get it right.

I needed to have:

/etc/puppet/manifests/site.pp
/etc/puppet/modules

and as an example under /etc/puppet/modules I needed:

/etc/puppet/modules/sudo/manifests/init.pp

Resources

1. AWESOME very helpful and engaged channel: puppet IRC. IRC server irc.freenode.net, room: #puppet
2. List of common puppet errors with pointers to fix: http://bitcube.co.uk/content/puppet-errors-explained
3. Of course the puppet docs, particularly for installing puppet on solaris: http://projects.puppetlabs.com/projects/1/wiki/Puppet_Solaris
4. RPM search: http://rpm.pbone.net/

(updated to clean-up layout, edit fonts, etc)

Implementing DevOps

I've started at a new company. They are a very large company with a medium sized web presence operating several on-line brands for a niche audience.

Generally they are a functioning company with a well established environment that is running well enough, but is ready for an overhaul. They have a mid-to-long term project to consolidate different content management systems into a unified content management system that allows for sharing of content between brands. This larger project provides an opening to perform a major face lift on the internal operations. WooHoo!

Currently releases are largely un-automated, time consuming, take place during off-hours, require quite a few people on-line to do the actual work and testing or just to be on-hand in the event something blows up. There seems to be plenty of room to implement a DevOps methodology for releases, particularly automation and measurement.