Download Building Hosts with Puppet: Managing Configuration with Modules and Definitions and more Exams Logic in PDF only on Docsity!
C H A P T E R 2
Building Hosts with Puppet
In Chapter 1 we installed and configured Puppet, created our first module, and applied that module and its configuration via the Puppet agent to a host. In this chapter, we’re going to extend this process to build some more complete modules and hosts with Puppet for a hypothetical company, Example.com Pty Ltd. Each host’s functionality we build will introduce new Puppet concepts and ideas. Example.com Pty Ltd has four hosts we’re going to manage with Puppet: a Web server, a database server, a mail server and our Puppet master server located in a flat network. You can see that network in Figure 2-1.
Figure 2-1. The Example.com Pty Ltd Network
Like many organizations, though, Example.com is not a very homogenous environment and each host uses a different operating system, as follows:
- mail.example.com – (Red Hat Enterprise Linux 5)
- db.example.com – (Solaris 10)
- web.example.com – (Ubuntu 10.04)
- puppet.example.com – (Ubuntu 10.04)
To solve this problem, we’ll begin by working through how we use Puppet in a multiple operating system environment. Be sure you’ve installed the base operating system on these hosts as described in Chapter 1, because we’ll perform some basic configuration on the hosts. We’ll start with configuring SSH for each host, then we’ll install and configure some role-specific applications for the hosts as follows:
- Postfix (mail.example.com)
- MySQL (db.example.com )
- Apache and a website (web.example.com)
- Manage the Puppet master with Puppet (puppet.example.com )
As we configure each host, we’ll introduce some of the different features and functions available in Puppet. By the end of the chapter you’ll have a firm grasp of the basics. In subsequent chapters, we’ll build on this knowledge and introduce some of Puppet’s more advanced features.
Getting Started
Before proceeding, we must have the proper setup, so we need to install the Puppet master and agent and then create node definitions for each of our hosts.
Note As we mentioned in Chapter 1, the Puppet software is called the “agent.” Puppet calls the definition of the
host itself a “node.”
Installing Puppet
First, we need to install the Puppet master and agent. We’re going to install the Puppet master on puppet.example.com and the Puppet agent on all our hosts, including puppet.example.com. We’re installing the agent on the Puppet master because we’re going to use Puppet to manage itself! We then need to connect, create and sign certificates for each host. To do this, you should follow the installation instructions for the relevant operating system from Chapter 1 on each of the four hosts. For example, for installation on the Red Hat Enterprise Linux host, use the instructions in the Installing on Red Hat Enterprise Linux and Fedora section. You can then move on to configuring the nodes (aka hosts).
Tip If you use a provisioning tool like Kickstart or Preseed, you can also include Puppet installation and signing
as part of your build process. You can see an example of how to do that at
http://projects.puppetlabs.com/projects/1/wiki/Bootstrapping_With_Puppet.
Specifying a Default Node
You can also specify a special node called default. This is, as you’d imagine, a default node. If no other node definition exists, then the contents of this node are applied to the host.
node default { include defaultclass }
Node Inheritance Structure
Lastly, Puppet has a simple node inheritance structure. You can use this to have one node inherit the contents of another node. Only one node can be inherited at a time. So, for example, we might want the node web host to inherit the contents of a node called base.
node base { include sudo, mailx }
node 'web.example.com' inherits base { … }
Here we’ve defined the base node to include the modules sudo and mailx and then specified that the web node inherits the contents of this node. This means the web node would include sudo and mailx in addition to any classes included in its own node definition. Inheritance is cumulative and you can specify an inheritance structure like so:
node base { … }
node webserver inherits base { … }
node 'web.example.com' inherits webserver { … }
Here the webserver node inherits the contents of the base node, and then in turn the web.example.com node cumulatively inherits the contents of both nodes.
Caution When starting out with Puppet it is common to structure the assignment of classes to nodes using
inheritance and a base node. This structure allows classes common to every node to be placed in the base node.
This organization structure may pose a problem in the future as the number of nodes and the scale of puppet
increases and base classes need to be added or removed from only a subset of all nodes. In order to avoid future
refactoring, avoid using node inheritance in preference of a flat node classification tree. A good alternative to the
base node and class inheritance is to employ conditional statements, which we’ll introduce later in this chapter, to
determine which classes a node should and should not receive instead of relying on node inheritance.
Variable Scoping
The concept of node inheritance is a good place to talk about an important and sometimes tricky concept in Puppet: variable scoping. Let’s imagine we’ve decided to configure some variables in our nodes, for example:
node base { $location = "dc1" … $location = "dc2" }
In most programming languages, the $location variable would start out with a value of "dc1" and then, when it was next assigned, it would change to a value of "dc2". In Puppet, these same two statements cause an error:
err: Cannot reassign variable location at /etc/puppet/manifests/node.pp:
Why is this? Puppet is declarative and hence dynamically scoped. Allowing variable reassignment would have to rely on order in the file to determine the value of the variable and order does not matter in a declarative language. The principal outcome of this is that you cannot redefine a variable inside the same scope it was defined in, like our node. Let’s take another example, of a class this time instead of a node:
class ssh_sudo { $package = "openssh" package { $package: ensure => installed }
$package = "sudo" package { $package: ensure => installed } }
You can see that we’ve tried to define the $package variable twice. If we were to try to compile and apply this configuration, the Puppet agent would return the following error:
err: Cannot reassign variable package at /etc/puppet/modules/ssh/manifests/init.pp:
node 'web.example.com' { include apache }
node 'web2.example.com' inherits 'web.example.com' { $apacheversion = "2.0.42" }
Here we’ve created a class called apache and a package resource for the apache2 package. We’ve also created a variable called $apacheversion and used that as the value of the ensure attribute of the package resource. This tells Puppet that we want to install version 2.0.33 of Apache. We’ve then included our apache class in a node, web.example.com. But we’ve also decided to create another node, web2.example.com, which inherits the contents of the web.example.com node. In this case, however, we’ve decided to install a different Apache version and therefore we specified a new value for the $apacheversion variable. But instead of using this new value, Puppet will continue to install the 2.0.33 version of Apache because the $apacheversion variable is maintained in its original scope of the web.example.com node and the new variable value is ignored. There is a work-around for this issue that you can see here:
class apache { $apacheversion = "2.0.33" package { "apache2": ensure => $apacheversion, } }
class base { include apache }
node 'web.example.com' { $apacheversion = "2.0.42" include base }
Instead of defining a base node we’ve defined a base class that includes the apache class. When we created our node, we specified the $apacheversion we want and then included the base class, ensuring we’re in the right scope. We could put other like items in our base class and specify any required variables.
Note You can learn more about variable scoping, workarounds and related issues at
http://projects.puppetlabs.com/projects/puppet/wiki/Frequently_Asked_Questions#Common+Misconce
ptions.
With Puppet installed and node definitions in place, we can now move on to creating our modules for the various hosts. But first, let’s do a quick refresher on modules in general.
Making (More) Magic With Modules
In Chapter 1, we learned about modules: self-contained collections of resources, classes, files that can be served, and templates for configuration files. We’re going to use several modules to define the various facets of each host’s configuration. For example, we will have a module for managing Apache on our Web server and another for managing Postfix on our mail server. Recall that modules are structured collections of Puppet manifests. By default Puppet will search the module path, which is by default /etc/puppet/modules/ and /var/lib/puppet/modules, for modules and load them. These paths are controlled by the modulepath configuration option. This means we don’t need to import any of these files into Puppet – it all happens automatically. It’s very important that modules are structured properly. For example, our sudo module contains the following:
sudo/ sudo/manifests sudo/manifests/init.pp sudo/files sudo/templates
Inside our init.pp we create a class with the name of our module:
class sudo { configuration… }
Lastly, we also discovered we can apply a module, like the sudo module we created in Chapter 1, to a node by using the include function like so:
node 'puppet.example.com' { include sudo }
The included function adds the resources contained in a class or module, for example adding all the resources contained in the sudo module here to the node puppet.example.com. Let’s now see how to manage the contents of our modules using version control tools as we recommended in Chapter 1.
Note You don’t have to always create your own modules. The Puppet Forge at http://forge.puppetlabs.com
contains a large collection of pre-existing modules that you can either use immediately or modify to suit your
environment. This can make getting started with Puppet extremely simple and fast.
Version Controlling Your Modules
Because modules present self-contained collections of configuration, we also want to appropriately manage the contents of these modules, allowing us to perform change control. To manage your content, we recommend that you use a Version Control System or VCS.
This tells us that when we commit that Git will add the contents to the repository and create a revision based on that state. Now let’s commit our revision to the repository.
$ git commit –a –m "This is our initial commit"
The –m option specifies a commit message that allows us to document the revision we’re about to commit. It’s useful to be verbose here and explain what you have changed and why, so it’s easier to find out what’s in each revision and make it easier to find an appropriate point to return to if required. If you need more space for your commit message you can omit the –m option and Git will open your default editor and allow you to type a more comprehensive message. The changes are now committed to the repository and we can use the git log command to see our recent commit.
$ git log
We can see some information here about our commit. First, Git uses SHA1 hashes to track revisions; Subversion, for example, uses numeric numbers – 1, 2, 3, etc. Each commit has a unique hash assigned to it. We will also see some details about who created the commit and our commit message telling us what the commit is all about. Every time you add a new module or file you will need to add it to Git using the git add command and then commit it to store it in the repository. I recommend you add and commit changes regularly to ensure you have sufficiently granular revisions to allow you to easily roll back to an earlier state.
Tip If you’re interested in Git, we strongly recommend Scott Chacon’s excellent book Pro Git – also published by
Apress. The book is available in both dead tree form and online at http://progit.org/book/. Scott is also one of
the lead developers of the Git hosting site, GitHub – http://www.github.com, where you can find a number of
Puppet related modules.
Our simple sudo module is a good introduction to Puppet, but it only showcased a small number of Puppet’s capabilities. It’s now time to expand our Puppet knowledge and develop some new more advanced modules, starting with one to manage SSH on our hosts. We’ll then create a module to manage Postfix on mail.example.com, one to manage MySQL on our Solaris host, db.example.com, another to manage Apache and web sites, and finally one to manage Puppet with Puppet itself. We’ll also introduce you to some best practices for structuring, writing and managing modules and configuration.
Creating a module to Manage SSH
We know that we first need to create an appropriate module structure. We’re going to do this under the /etc/puppet/modules directory on our Puppet master.
$ cd /etc/puppet/modules $ mkdir –p ssh/{manifests,templates,files} $ touch ssh/manifests/init.pp
Next, we create some classes inside the init.pp file and some initial resources, as shown in Listing 2-2.
Listing 2-2. The ssh module
class ssh::install { package { "openssh": ensure => present, } }
class ssh::config { file { "/etc/ssh/sshd_config": ensure = > present, owner => 'root', group => 'root', mode => 0600, source => "puppet:///modules/ssh/sshd_config", require => Class["ssh::install"], notify => Class["ssh::service"], } } class ssh::service { service { "sshd": ensure => running, hasstatus => true, hasrestart => true, enable => true, require => Class["ssh::config"], } }
class ssh { include ssh::install, ssh::config, ssh::service }
We’ve created three classes: ssh, ssh::install, ssh::config, and ssh::service. As we mentioned earlier, modules can be made up multiple classes. We use the :: namespace syntax as a way to create structure and organization in our modules. The ssh prefix tells Puppet that each class belongs in the ssh module, and the class name is suffixed.
Note We’d also want to create a sshd_config file in the ssh/files/ directory so that our
File["/etc/ssh/sshd_config"] resource can serve out that file. The easiest way to do this is to copy an existing
functional sshd_config file and use that. Later we’ll show you how to create template files that allow you to
configure per-host configuration in your files. Without this file Puppet will report an error for this resource.
The ssh::install Class
Now that we’ve created our structure, let’s look at the classes and resources we’ve created. Let’s start with the ssh::install class containing the Package["openssh"] resource, which installs the OpenSSH package. It looks simple enough, but we’ve already hit a stumbling block – we want to manage SSH on all of Example.com’s hosts, and across these platforms the OpenSSH package has different names:
- Red Hat: openssh-server
- Ubuntu: openssh-server
- Solaris: openssh
How are we going to ensure Puppet installs the correctly-named package for each platform? The answer lies with Facter, Puppet’s system inventory tool. During each Puppet run, Facter queries data about the host and sends it to the Puppet master. This data includes the operating system of the host, which is made available in our Puppet manifests as a variable called $operatingsystem. We can now use this variable to select the appropriate package name for each platform. Let’s rewrite our Package["openssh"] resource:
package { "ssh": name => $operatingsystem? /(Red Hat|CentOS|Fedora|Ubuntu|Debian)/ => "openssh-server", Solaris => "openssh", }, ensure => installed, }
You can see we’ve changed the title of our resource to ssh and specified a new attribute called name. As we explained in Chapter 1, each resource is made up of a type, title and a series of attributes. Each resource’s attributes includes its “name variable,” or ”namevar,” and the value of this attribute is used to determine the name of the resource. For example, the Package and Service resources use the name attribute as their namevar while the File type uses the path attribute as its namevar. Most of the time we wouldn’t specify the namevar, as it is synonymous with the title, for example in this resource:
file { "/etc/passwd": … }
We don’t need to specify the namevar because the value will be taken from the title, "/etc/passwd". But often we’re referring to resources in many places and we might want a simple alias, so we can give the resource a title and specify its namevar this way:
file { "passwd": path => "/etc/passwd", … }
We can now refer to this resource as File["passwd"] as an aliased short-hand.
Note You should also read about the alias metaparameter, which provides a similar capability, at
http://docs.puppetlabs.com/references/latest/metaparameter.html#alias.
In our current example, the name of the package we’re managing varies on different hosts. Therefore, we want to specify a generic name for the resource and a platform-selected value for the actual package to be installed. You can see that inside this new name attribute we’ve specified the value of the attribute as $operatingsystem followed by a conditional syntax that Puppet calls a “selector.” To construct a selector, we specify the a variable containing the value we want to select on as the value of our attribute, here $operatingsystem, and follow this with a question mark (?). We then list on new lines a series of selections, for example if the value of $operatingsystem is Solaris, then the value of the name attribute will be set to openssh, and so on. Notice that we can specify multiple values in the form of simple regular expressions, like /(Solaris|Ubuntu|Debian)/.
Note Selector matching is case-insensitive. You can also see some other examples of regular expressions in
selectors at http://docs.puppetlabs.com/guides/language_tutorial.html#selectors.
We can also specify a value called default.
default => "ssh",
This value is used if no other listed selection matches. If we don’t specify a default value and no selection matches then the name attribute would be set to a nil value. As can you imagine, this requirement to select the appropriate value for a particular platform happens a lot. This means we could end up scattering a lot of very similar conditional statements across our Puppet code. That’s pretty messy; a best practice we recommend is to make this look a lot neater and more elegant by moving all your conditional checks to a separate class. We usually call that class module::params, so in our current case it would be named ssh::params. Like before, we’re going to store that class in a separate file. Let’s create that file:
$ touch ssh/manifests/params.pp
We can see that class in Listing 2-3.
Listing 2-3. The ssh::params class
class ssh::params { case $operatingsystem { Solaris: { $ssh_package_name = 'openssh' } /(Ubuntu|Debian)/: { $ssh_package_name = 'openssh-server'
We’re going to come back to the ssh::params class and add more variables as we discover other elements of our OpenSSH configuration that are unique to particular platforms, but for the moment how does including this new class change our Package["ssh"] resource?
package { $ssh::params::ssh_package_name: ensure => installed, }
You can see our namespacing is useful for other things, here using variables from other classes. We can refer to a variable in another class by prefixing the variable name with the class it’s contained in, here ssh::params. In this case, rather than our messy conditional, the package name to be installed will use the value of the $ssh::params::ssh_package_name parameter. Our resource is now much neater, simpler and easier to read.
Tip So how do we refer to namespaced resources? Just like other resources,
Package[$ssh::params::ssh_package_name].
The ssh::config Class
Now let’s move onto our next class, ssh::config, which we can see in Listing 2-4.
Listing 2-4. The ssh::config class
class ssh::config { file { "/etc/ssh/sshd_config": ensure = > present, owner => 'root', group => 'root', mode => 0440, source => "puppet:///modules/ssh/sshd_config", require => Class["ssh::install"], notify => Class["ssh::service"], } }
We know that the location of the sshd_config files will vary across different operating systems. Therefore, we’re going to have to add another conditional for the name and location of that file. Let’s go back to our ssh::params class from Example 2-3 and add a new variable:
class ssh::params { case $operatingsystem { Solaris { $ssh_package_name = 'openssh' $ssh_service_config = '/etc/ssh/sshd_config' } … }
We add the $ssh_service_config variable to each of the cases in our conditional and then update our file resource in the ssh::config class:
file { $ssh::params::ssh_service_config: ensure = > present, … }
Again, we have no need for a messy conditional in the resource, we can simply reference the $ssh::params::ssh_service_config variable. We can also see that the file resource contains two metaparameters, require and notify. These metaparameters both specify relationships between resources and classes. You’ll notice here that both metaparameters reference classes rather than individual resources. They tell Puppet that it should create a relationship between this file resource and every resource in the referenced classes.
Tip It is a best practice to establish relationships with an entire class, rather than with a resource contained
within another class, because this allows the internal structure of the class to change without refactoring the
resource declarations related to the class.
For example, the require metaparameter tells Puppet that all the resources in the specified class must be processed prior to the current resource. In our example, the OpenSSH package must be installed before Puppet tries to manage the service’s configuration file. The notify metaparameter creates a notification relationship. If the current resource (the service’s configuration file) is changed, then Puppet should notify all the resources contained in the ssh::service class. In our current case, a “notification” will cause the service resources in the ssh::service class restart, ensuring that if we change a configuration file that the service will be restarted and running with the correct, updated configuration.
Tip In Puppet 2.6.0, a shorthand method called “chaining” was introduced for specifying metaparameter
relationships, such as require and notify. You can read about chaining at
http://docs.puppetlabs.com/guides/language_tutorial.html#chaining_resources.
So why specify the whole ssh::service class rather than just the Service["sshd"] resource? This is another piece of simple best practice that allows us to simplify maintaining our classes and the relationships between them. Imagine that, instead of a single package, we had twenty packages. If we didn’t require the class then we’d need to specify each individual package in our require statement, like this:
require => [ Package["package1"], Package["package2"], Package["package3"] ],
$ssh_service_config = '/etc/ssh/sshd_config' $ssh_service_name = 'sshd' } … }
Let’s also look at our Service[$ssh::params::ssh_service_name] resource (at the start of this section), as this is the first service we’ve seen managed. You’ll notice two important attributes, ensure and enable, which specify the state and status of the resource respectively. The state of the resource specifies whether the service is running or stopped. The status of the resource specifies whether it is to be started at boot, for example as controlled by the chkconfig or enable-rc.d commands. Puppet understands how to manage a variety of service frameworks, like SMF and init scripts, and can start, stop and restart services. It does this by attempting to identify the service framework your platform uses and executing the appropriate commands. For example, on Red Hat it might execute:
$ service sshd restart
If Puppet can’t recognize your service framework, it will revert to simple parsing of the process table for processes with the same name as the service it’s trying to manage. This obviously isn’t ideal, so it helps to tell Puppet a bit more about your services to ensure it manages them appropriately. The hasstatus and hasrestart attributes we specified in the ssh::service class is one of the ways we tell Puppet useful things about our services. If we specify hasstatus as true, then Puppet knows that our service framework supports status commands of some kind. For example, on Red Hat it knows it can execute the following:
$ service sshd status
This enables it to determine accurately whether the service is started or stopped. The same principle applies to the hasrestart attribute, which specifies that the service has a restart command. Now we can see Puppet managing a full service, if we include our new ssh module in our Puppet nodes, as shown in Listing 2-5.
Listing 2-5. Adding the ssh Module
class base { include sudo, ssh }
node 'puppet.example.com' { include base }
node 'web.example.com' { include base }
node 'db.example.com' { include base }
node 'mail.example.com' { include base }
Here we’ve created a class called base, in which we’re going to place the modules that will be base or generic to all our nodes. Thus far, these are our sudo and ssh modules. We then include this class in each node statement.
Note We talked earlier about node inheritance and some of its scoping issues. As we explained there, using a
class instead of node inheritance helps avoids these issues. You can read about it at
http://projects.puppetlabs.com/projects/puppet/wiki/Frequently_Asked_Questions#Common+Misconce
ptions.
With a basic SSH module in place, and we can now manage the SSH daemon and its configuration.
Creating a Module to Manage Postfix
Let’s now create a module to manage Postfix on mail.example.com. We start with a similar structure to our SSH module. In this case, we know which platform we’re going to install our mail server on so we don’t need to include any conditional logic. However, if we had multiple mail servers on different platforms, it would be easy to adjust our module using the example we’ve just shown to cater for disparate operations systems.
postfix postfix/files/master.cf postifx/manifests/init.pp postfix/manifests/install.pp postfix/manifests/config.pp postfix/manifests/service.pp postfix/templates/main.cf.erb
The postfix::install class
We also have some similar resources present in our Postfix module that we saw in our SSH module, for example in the postfix::install class we install two packages, postfix and mailx:
class postfix::install { package { [ "postfix", "mailx" ]: ensure => present, } }
Note that we’ve used an array to specify both packages in a single resource statement this is a useful shortcut that allows you specify multiple items in a single resource.
