ReleaseEngineering/PuppetAgain: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
Line 101: Line 101:
* [[ReleaseEngineering/PuppetAgain/HowTo/Set up a user environment]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Set up a user environment]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Add new secrets]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Add new secrets]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Bootstrap a Puppetmaster]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Change secrets]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Change secrets]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Build RPMs]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Build RPMs]]
Line 116: Line 117:
== The Goals ==
== The Goals ==


* PuppetAgain should be usable as a whole for folks outside of Mozilla, Inc. who want to build similar systems
* PuppetAgain should be usable as a whole for folks outside of Mozilla, Inc. who want to build similar systems (see "Organizations" below)
* Client images should proceed automatically from base image install to a fully-operational state.  While refimages may be employed, this is done only as an optimization.
* Client images should proceed automatically from base image install to a fully-operational state.  While refimages may be employed, this is done only as an optimization.
* We do not keep distinct reference images.  Reference images are used only as an optimization to avoid pounding the puppet servers when installing dozens of new hosts.  When a new refimage snapshot needs to be made, a fresh machine is rebuilt from scratch, snapshotted, and then returned to service.
* We do not keep distinct reference images.  Reference images are used only as an optimization to avoid pounding the puppet servers when installing dozens of new hosts.  When a new refimage snapshot needs to be made, a fresh machine is rebuilt from scratch, snapshotted, and then returned to service.
Line 123: Line 124:
* Documentation (here) is a part of the patch.
* Documentation (here) is a part of the patch.
See [[ReleaseEngineering/PuppetAgain/HowTo/Hack on PuppetAgain]] for more detail
See [[ReleaseEngineering/PuppetAgain/HowTo/Hack on PuppetAgain]] for more detail
== Organizations ==
Each distinct instance of puppetagain is referred to as an organization, and tagged with a short identifier (e.g., "moco" for the mozilla releng instance, or "seamonkey" for seamonkey).  Within an organization, configuration and secrets are shared, and everything runs from the same set of manifests.  Configuration and secrets can differ between organizations.


== Puppetmasters ==
== Puppetmasters ==
Line 131: Line 136:
* [[ReleaseEngineering/PuppetAgain/Puppetmasters]]
* [[ReleaseEngineering/PuppetAgain/Puppetmasters]]
* [[ReleaseEngineering/PuppetAgain/Puppetization Process]]
* [[ReleaseEngineering/PuppetAgain/Puppetization Process]]
* [[ReleaseEngineering/PuppetAgain/Base Images]]
* [[ReleaseEngineering/PuppetAgain/Certificate Chaining]]
* [[ReleaseEngineering/PuppetAgain/Certificate Chaining]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Bootstrap a Puppetmaster]]
* [[ReleaseEngineering/PuppetAgain/HowTo/Bootstrap a Puppetmaster]]

Revision as of 19:35, 26 April 2013

PuppetAgain is an implementation of configuration management with puppet for Mozilla release engineering. It is intended to make machine management easier and more flexible for release engineering, while also making that management transparent enough that it can be reliably duplicated by users outside of the company.

Documentation - Manifests & Modules

The Puppet manifests themselves are documented here. Any new modules should be added to the proper list below.

Stages

Stages need to be defined globally in Puppet manifests, and this is done in manifests/stages.pp. The following stages are available, aside from 'main', the default stage.

  • network - This stage should handle any network related configurations for some specific cases (like AWS)
  • packagesetup - This stage should handle any preliminaries required for package installations, so that subsequent package installations do not need to require them explicitly.
  • users - This stage creates user accounts; while this is normally automatically required, the requirement doesn't work with the temporary 'darwinuser' type.

Nodes

manifests/nodes.pp defines all of the nodes the puppet masters recognize. Note that all nodes are defined for all masters.

In anticipation of using an external node classifier (ENC), node definitions should only include classes - do not define any resources within nodes. In general, the included classes should be in the toplevel module.

Host-specific values are specified as node-scope variables, as these are easier to represent in an ENC. Such variables (including some Puppet gotchas) are described in node-scope variables.

extlookup/

We store csv files pertaining to the Config module here (under manifests/extlookup/). Expect to find the local secrets details and possibly a symlink to a local-config in this dir.

Modules

All substantial configuration is done with puppet modules, in the modules directory. Each should have its own page describing both how to use the module, and how the module works, below:

Infrastructure

These modules are part of the puppet system itself, and provide support to other modules as needed

Action

These modules actually get stuff done.

Utility

These modules are more generic, and probably useful outside of PuppetAgain.

Third-Party

These are modules taken from elsewhere. When adding, remember to verify license compatibility and ensure proper credit.

Custom Plugins

We have a single custom fact defined:

  • $puppetizing - 'true' if being run from puppetize.sh, empty otherwise

Custom functions, types, and providers are documented in the modules that implement them. Most are in shared.

How To

System Description

This section describes how PuppetAgain is built at Mozilla. External implementations may not have all of these bells and whistles. This link contains details oriented toward Mozilla IT and ops folks.

The Goals

  • PuppetAgain should be usable as a whole for folks outside of Mozilla, Inc. who want to build similar systems (see "Organizations" below)
  • Client images should proceed automatically from base image install to a fully-operational state. While refimages may be employed, this is done only as an optimization.
  • We do not keep distinct reference images. Reference images are used only as an optimization to avoid pounding the puppet servers when installing dozens of new hosts. When a new refimage snapshot needs to be made, a fresh machine is rebuilt from scratch, snapshotted, and then returned to service.
  • OS does not imply role. Roles are defined in node declarations, by including toplevel::* classes.
  • Include all necessary dependencies. Debugging dependency errors when building a new reference system is no fun.
  • Documentation (here) is a part of the patch.

See ReleaseEngineering/PuppetAgain/HowTo/Hack on PuppetAgain for more detail

Organizations

Each distinct instance of puppetagain is referred to as an organization, and tagged with a short identifier (e.g., "moco" for the mozilla releng instance, or "seamonkey" for seamonkey). Within an organization, configuration and secrets are shared, and everything runs from the same set of manifests. Configuration and secrets can differ between organizations.

Puppetmasters

PuppetAgain masters are managed by PuppetAgain. Each organization can have 1 or more masters, arranged in a cluster. There is one "distinguished master" in the cluster. This master is distinguished only for purposes of simplifying synchronization -- the cluster will continue to operate indefinitely without the distinguished master, although master-master communication (secrets and CRLs) will not work.

See the following for more details, noting that most of this is not required for an external PuppetAgain implementation.

Puppet Versions

The releng puppet infrastructure will strive to keep up to date with the most recent stable versions released by Puppet Labs.

Base Images and Puppetizing

The base images for this infrastructure are barely-modified OS installs. They have just enough installed that they can connect to a puppet server, get certificates, and puppetize on boot.

Note that, while most of PuppetAgain is intended to be easily replicated, the deployment system is probably not easily replicated, and is best left out of any external implementations.

Data

Puppet deals with a lot of big files - packages, mostly. We don't want these in hg! They are instead managed as data. This means several big file trees available at http://repos/$treename and, from puppet, at puppet://$treename. See ReleaseEngineering/PuppetAgain/Data for details on what's available, how it is implemented, and some how-tos.

This data is available outside of Mozilla via HTTP and rsync at http://puppetagain.pub.build.mozilla.org/data and rsync://puppetagain.pub.build.mozilla.org/data.

Packages

See ReleaseEngineering/PuppetAgain/Packages for information about proper handling of packages in PuppetAgain.

Manifests

Manifests are at http://hg.mozilla.org/build/puppet.

History

Releng once used a puppet infrastructure based on Puppet-0.24.8, and manifests at http://hg.mozilla.org/build/puppet-manifests/. This had a few weaknesses:

  • lots of assumptions and fragile dependencies based on bugs in 0.24.8
  • very few modules - mostly manifest files, organized per slave type, rather than per service/purpose
  • many references to external files which are not as available as the repo itself
  • puppet manifests assume some manual ref-image steps; external exact reproduction is extremely difficult

Dustin started work on a new puppet deployment - chronicled at User:Djmitche/New Releng Puppet Infrastructure. That's this puppet.

Retrieved from "https://dev.wikimo.nonprod.webservices.mozgcp.net/index.php?title=ReleaseEngineering/PuppetAgain&oldid=649881"