Setting up Hiera on Puppet Server⌗

It’s been a while due to, well, life. As promised, this blog will conclude the Configuring Puppet Server series by setting up Hiera.

If you need to get caught up, please see Part 1 and Part 2

What is Hiera?⌗

Hiera is a key/value lookup tool for configuration data. There are 3 principal versions of Hiera used in modern Puppet:

  • Hiera 3
  • Hiera 4
  • Hiera 5

If you’d like more information about the difference between the three versions, you can find it in the Hiera 5 Docs. For the purposes of our blog post, we’ll be using Hiera 5 as Hiera 3 will be depracated soon and Hiera 4 was a beta for Hiera 5.

Installing Hiera⌗

If using the Puppet Agent All-In-One Package provided by Puppet, Inc then Hiera 3 and Hiera 5 are both installed as part of the installation, no additional steps needed.

Configuring Hiera⌗

In Hiera 3, you had a single, global configuration file found at /etc/puppetlabs/puppet/hiera.yaml

In Hiera 5, there are now three potential configuration locations:

  • /etc/puppetlabs/puppet/hiera.yaml – purely for backwards compatibility and migrating from Hiera 3 to Hiera 5+
  • $ENVIRONMENT_PATH/hiera.yaml – Environment Level Data – will override Module level data and defaults set in the code.
  • $MODULE_ROOT/hiera.yaml – Module Level Data – Primarily used to set sane defaults. Designed as a replacement for params.pp.

This tutorial only covers the initial setup of a Puppetserver and the Production Environment, so the Hiera 3 Global yaml and the Module data yaml will not be covered here.

Since the Evnironment Level Data is configured per environment and lives in the Environment directory, it will need to go into the puppet-control repository if you are using r10k. We will not be covering Module level data in this post. That will come in a later post dedicated to Hiera.

The first thing we need to do is configure the hiera.yaml for our r10k environment. The hiera.yaml file simply exists in the root of your puppet-control repo.

Create the hiera.yaml file in the root of your puppet-control repo that we’ve been using in these previous posts. It will look similar to this:

---
version: 5 # Required in Hiera 5

defaults: # Used for any hierarchy level that omits the below keys.
  datadir: hieradata # The path, relative to the hiera.yaml"s directory, to use for lookups
  data_hash: yaml_data # Use the YAML backend if no other backend is specified.

hierarchy:
  - name: "Per-node data" # Human readable name
    path: "nodes/%{trusted.certname}.yaml" # File path, relative to datadir. File extension is required!

  - name: "Common Data"
    path: "common.yaml"

The above is a simple hiera.yaml configuration file. Note the comments in the code example for descriptions about each key.

The above config is telling Hiera the following facts about its lookup:

  • Use Hiera version 5
  • The default data directory is <environment_root>/hieradata
  • The default data backend is YAML
  • There are 2 Hierarchies – Per-node data and Common data
  • Per-node data is a hierarchy with a single path:
    • Lookup data by node name in path – <environment_root>/hieradata/nodes/<certname>.yaml
  • Common data is also a hierarchy with a single path:
    • Lookup common data in path – <environment_root>/hieradata/common.yaml
  • Lookup data in each hierarchy starting from top to bottom.

That’s it, Hiera is now configured. Granted this won’t be much use without hieradata.

Adding hieradata⌗

Now we need to add some data to hiera. We can add this directly to the puppet-control repository, or create a separate repository to contain this data. For the sake of brevity, we will be adding the hieradata directly to the puppet-control repository.

First thing we want to do is add a hieradata directory to the root of the puppet-control to match the datadir value in the example above.

Next we need to create a nodes directory in the hieradata directory to match our “Per-node data” hierarchy path.

Ok, now before we can create any hieradata, we need to add some lookups to the code. So, open your site.pp file in the puppet-control repo and edit it to look like this:

node "puppetserver.local" {
  class { "::puppetdb::master::config":
    puppetdb_server => "puppetdb.local",
  }

  class { "::r10k":
    sources => {
      "r10k" => {
        "remote"  => lookup("r10k::remote"),
        "basedir" => lookup("r10k::basedir", "/etc/puppetlabs/code/environments"),
      },
    },
  }

  class { "::r10k::webhook":
    user  => "root",
    group => "root",
  }

  class { "::r10k::webhook::config":
    use_mcollective  => false,
    public_key_path  => "/etc/pki/tls/certs/puppetserver.local.public.pem",
    private_key_path => "/etc/pki/tls/private/puppetserver.local.pem",
  }
}

node "puppetdb.local" {
  class { "::puppetdb":
    ssl_listen_address   => "puppetdb.local",
    open_ssl_listen_port => true,
    postgres_version     => "9.6",
  }
}

node default {}

We’ve replaced the r10k remote and basedir attributes with the Puppet lookup() function. This function simply looks up data based on the key name you pass it. So in this case, the code is telling puppet to do the following:

  • Get the value of remote by looking up the r10k::remote key in Hiera and fail if nothing is found.
  • Get the value of basedir by looking up the r10k::basedir key in Hiera, but if nothing is found, set it to /etc/puppetlabs/code/environments.

Of course, the lookup on remote is going to fail because we don’t have any data in Hiera yet. So, let’s create that data.

create a new file in hieradata/nodes/ called puppetserver.local.yaml and place the following content in there:

---
r10k::remote: "https://github.com/user/puppet-control"

Where the Github URL is the puppet-control repo created for the previous posts.

Now run puppet agent -t on the “puppetserver.local” VM. You should see a puppet run, but there shouldn’t be any changes as we didn’t actually change any data within the compiled puppet catalog.

Let’s do that now for fun. Open the site.pp file back up one last time and add the following code to it in the puppetdb.local node block:

...

node "puppetdb.local" {
  class { "::puppetdb":
    ssl_listen_address   => "puppetdb.local",
    open_ssl_listen_port => true,
    postgres_version     => "9.6",
  }

  file { "/tmp/test":
    ensure  => file,
    user    => lookup("file::user"),
    group   => lookup("file::group"),
    mode    => "0644",
    content => lookup("file::content"),
  }
}

...

Now create the common.yaml file inside the hieradata/ directory with the following content:

---
file::user: root
file::group: root
file::content: "Hello world!"

Ok, now run puppet agent -t on the “puppetdb.local” VM. The newly compiled catalog should create a new file at called test in the /tmp directory on the system that is owned by root and grouped into root. The file should contain Hello world! as its only content.

That’s it. That’s all there is to Hiera.

Conclusion⌗

Hiera is an incredibly powerful tool for abstracting configuration and infrastructure data away from your Puppet code. This is incredibly useful when creating Modules, Roles, and/or Profiles that are designed to be reusable and dynamic.

Hiera comes with four built-in backends – YAML, JSON, HOCON, and EYAML – but you can extend that with your own custom backends written in Puppet or Ruby if necessary.

Hiera and the Lookup function have a wide array of options and uses. See the links below to learn more and feel free to reach out via E-Mail, Slack, or IRC with any questions you may have.

Resources⌗

Contact⌗