I've always liked the phrase "configuration drift." It sounds like a wind blew through the racks in the data center just so, gently jumbling that one file down in /opt/etc that's the last place you look when a service goes down.
Realistically, that drift probably came from human error: a junior admin making a novice mistake, or someone solving a problem with a small but critical change. Either way – mysterious cloud creatures, or everyday people – things change on the systems we maintain, and it can take a lot of time to make them right again.
As a sysadmin, you spend a lot of time trying to get everything just right. You shouldn't have to spend all your time trying to keep everything just right. With just a little automation, you won't have to.
When we ran our 2013 State of DevOps survey, we heard from thousands of sysadmins who reported time and time again that one of the keys to high performance – more agility, fewer failures, faster recoveries, more code shipped – was simple automation.
By starting small and getting good at automating one discrete task, you can establish a foundation for bigger automation projects. And the more you automate all the little things, the easier it is to think about the big projects you're passionate about, and create more value for your organization.
Automation shouldn't have to be a slog, either. Rather than hacking up brittle shell scripts loaded down with conditional logic and more lines of apologetic comments than actual code, it's a good idea to pick tools that are purpose-built for the job, and that provide a way for you to build on your first small successes to create larger, more ambitious automation projects.
Starting Small: Automating Your SSH Configuration
Kelsey Hightower, operations manager at Puppet Labs, recorded a quick demo of how to automate a common pain point with Puppet Enterprise. The video shows how you can take a known good ssh configuration file, write just a few lines of Puppet code, then automate the deployment of the configuration to nodes across your infrastructure. Kelsey's demo also shows how the new configuration is constantly enforced, so even if configuration drifts, the puppet master will ensure it's reverted to the correct state right away.
Start With Something That Already Works
To get started with a small automation project, you should start with a familiar service in a known good state. If you're not trying to learn a new service or tool at the same time you're trying to automate or enforce its configuration , there will be fewer things to get wrong and less to worry about.
Kelsey's demo starts off with a working ssh configuration that needs a simple, common tweak: Following the PCI recommendation that users log in to a remote system using their own accounts instead of the root user's.
One thing you'll notice in the demo is that when Kelsey copies to his puppet master the configuration file he's going to use, he appends the name of the operating system to the end of the filename.
That last step takes advantage of Facter, a library from Puppet Labs that allows agents to send operating system and configuration information to their puppet master. If you're copying a file from a non-Debian system, you can find out how to name the file by logging into that system and running this command:
facter operatingsystem
Write a Little Bit of Puppet Code
Next, you need to create a Puppet manifest. One of the benefits of starting with a service you know, and using a tool that's purpose-built for IT automation, is that even if the particular arrangements of curly braces and commas aren't familiar, the underlying concerns they address ought to be. Let's look at the manifest Kelsey wrote for his demo:
# === Authors # Author Name # # === Copyright # Copyright 2013 Puppet Labs # class ssh { file { '/etc/ssh/sshd_config': ensure => present, owner => 'root', group => 'root', mode => '0644', source => "puppet:///modules/ssh/sshd_config.${operatingsystem}", notify => Service['ssh'] } service { 'ssh': ensure => 'running', enable => 'true', } } Looking at the "file" stanza in the gist, we can see:
- We want to manage the file /etc/ssh/sshd_config
- We want the file to be owned by the root user and group
- We want the file to be readable by everyone, but writable only by its owner
There are a few things very specific to Puppet's configuration language there, too:
- The "source" parameter describes the file we want to use to configure the agent, and it uses the Facter operatingsystem fact to do that
- The "notify" parameter ensures that if there are any changes to the file we're managing, the ssh service will be restarted.
Looking at the "service" stanza, we can see two parameters:
- "ensure => 'running' makes sure the ssh service is running at all times (so if it's stopped at some point, the puppet master will make sure it restarts)
- "enable => 'true' makes sure the service is started at boot (so you don't have to wait until the first puppet run for the service to start)
Try It Out
Another advantage of starting small with your first automation project is that it's much easier to make sure everything went according to plan. In this case, Kelsey's demo concerns itself with a single line in one file. When he does his first puppet run on the agent's node, it's pretty easy to pick out what happened: The node checks in, reports on the status of the file, and the puppet master corrects the contents to reflect that root logins should not be permitted.
Once you've successfully automated this fairly small concern, Puppet Enterprise ensures that you don't have to worry about configuration drift again: If the file changes for whatever reason (sunspots and junior admins included), the correct configuration will be enforced the next time the puppet agent checks in with its puppet master.
With the Puppet Enterprise console's node management view, you can also quickly review the state of all the nodes in your infrastructure. When a change is detected that requires the puppet master to take action and correct the situation, the node management report will call that change out:
By clicking the link for the changed node, you can take a look at a much more detailed report that shows exactly which files changed, and how.
So you can follow along with Kelsey's simple, quick exercise in automation, apply it to all the nodes in your infrastructure, then rest easy knowing a small but vital concern will be taken care of in the future.
Watch the Demo, Try It Out for Yourself
Once you've watched the video , you can download Puppet Enterprise and try it out on 10 nodes for free. That's plenty to experiment with that one small pain point you'd like to automate.
No comments:
Post a Comment