Login | Register
My pages Projects Community openCollabNet

autoinst
Wiki: UserGuide

Edit this page | Links to this page | Page information | Attachments | Refresh page

 

Autoinst User Guide

Welcome to the Autoinst User Guide. Autoinst is a tool for version controlled, templated, host configuration. This guide describes how to use Autoinst to manage configurations and templates, explains Autoinst concepts, how to use the command line interfaces and the API methods.

Autoinst Concepts

Autoinst is designed to help its users manage host configurations, with the following features:

  • Version controlled repository of configuration information
  • Ability to reconstruct host configurations at any point in time
  • Ability to write a single template to manage multiple similar configuration files

Why another configuration system?

Autoinst is limited in scope by design. It is there to solve the following problems:

  • Host configurations might be centrally managed, but do not have a revision system.
  • It is not easy to share tested configurations between groups.
  • Configuration might involve manual processes and tools.

Autoinst does not attempt to solve the following problems:

  • Binary packaging. There are already well-defined, well-tested systems such as RPM, yum, and dpkg that can package binaries and manage their installation (and dependencies) on hosts.
  • File distribution and host management. There are already well-tested systems such as cfengine, Puppet, and slack that are able to manage the distribution of files onto hosts.

Instead, Autoinst is meant to complement these existing solutions, by providing a version-controlled configuration framework. For example, someone may manage a system using the following tools:

  • YUM/RPM to manage system packages
  • cfengine to distribute and apply changes to hosts
  • Autoinst to version control configuration files, and publish tagged configs to cfengine for distribution.

By implementing such a solution, the entire system stack could be reconstructed from scratch from a tag in Autoinst, if Autoinst is used to version control Kickstart/Anaconda, YUM configs, cfengine policies, and even application configs.

Autoinst Resources

Autoinst has some fundamental concepts that are used to manage configurations. Configurations are managed using 'resources', which are one of the following:

  • Hosts - a server (or OS instance) that is managed in Autoinst.
  • Groups - a group of hosts, organized under a name.
  • Actions - a managed configuration template. Represents a single configuration file on a system.
  • Profiles - a set of configuration files, organized under a name.
  • Stylesheets - XSL templates used to generate configuration files. Stylesheets take Action templates and are transformed using Stylesheets to produce the final product.

Hosts, Groups, Actions, and Profiles may all have Properties, which are settings specific to that resource. A Property is simply a key-value pair representing a setting.

Hosts

Hosts are used to represent managed servers, or managed OS instances if virtualization is being used. Hosts have a hostname and a set of optional properties. For example:

<?xml version="1.0" encoding="UTF-8"?>

<!-- an example of what a host document looks like in the repository -->
<host version="3.0.0" name="web1.east.foo.bar.com"> 

  <!-- properties specific to this host -->
  <properties>
    <property name="ipAddress">192.168.0.4</property>
    <property name="macAddress">00:00:1C:B5:F0:D3</property>
    <property name="kickstart.partitionTable">
<![CDATA[
part / --fstype ext3 --size=1 --grow
part /boot --fstype ext3 --size=512
part swap --size=1024
part raid.01 --size=512 --ondisk=sda
part raid.02 --size=512 --ondisk=sdb
part raid.03 --size=512 --ondisk=sdc
raid /export/data --level=5 --device=md0 raid.01 raid.02 raid.03
]]>
</property>
  </properties>
</host>

Groups

Autoinst manages hosts by assigning them to one or more groups. Group names represent a set of host resources that are managed under an arbitrary arrangement.

Groups also have a list of profiles, which represent a set of configurations to apply to member hosts of the group. The section on profiles will explain this in more detail.

Like hosts, groups can also have a set of optional properties. For example:

<?xml version="1.0" encoding="UTF-8"?>

<!-- an example group representing our application called "foo".
     groups can be arbitrarily named. -->
<group version="3.0.0" name="application.foo">

  <!-- a list of hosts in this group -->
  <hosts>
   <host name="web1.east.foo.bar.com"/>
  </hosts>

  <!-- a list of profiles this group includes -->
  <profiles>
   <profile name="webserver"/>
  </profiles>

  <!-- properties specific to this group -->
  <properties>
   <property name="foo.masterserver">master1.west.foo.bar.com</property>
  </properties>
</group>

Group Naming Conventions

If you have a lot of groups, it makes sense to create and follow a consistent convention for group names. For example, one could arrange groups in this fashion:

<type>.<name>.<optional-subname>

Which could result in groups such as the following:

  • application.foo
  • datacenter.east.rack11
  • datacenter.west.rack12
  • sysadmin.cfengine.master

Of course, it is completely up to you how to you want to organize your group namespace. Usually groups represent a set of common functions (e.g. an application role), or to represent something to do with the environment - datacenter location or cluster identifier, for example.

Subgroups

Groups can also have subgroups, which means that "meta-groups" can be created that have no members, but simply have other groups as members.

Actions

Actions represent configuration files. They have a template section that represents the configuration file. They also reference a XSL file that is used to transform the template into its final result.

<?xml version="1.0" encoding="UTF-8"?>

<action version="3.0.0" name="apache.httpd.conf" 
        filename="/etc/httpd/conf/httpd.conf"
        user="nobody" group="nobody" mode="0644"
        stylesheet="default.xsl" method="text" encoding="UTF-8">

  <!-- the default properties for this action
       usually overridden by the profile, group, or host config -->
  <properties>
    <property name="apache.httpd.server.type">standalone</property>
    <property name="apache.httpd.server.name">localhost</property>
    <property name="apache.httpd.port">80</property>
    <property name="apache.httpd.server.admin">webmaster@localhost</property>
    <property name="apache.httpd.server.root">/etc/httpd</property>
    <property name="apache.httpd.document.root">/var/www/html</property>
    <property name="apache.httpd.error.log">logs/error_log</property>
    <property name="apache.httpd.log.level">warn</property>
    <property name="apache.httpd.pid.file">logs/httpd.pid</property>
    <property name="apache.httpd.custom.log">logs/access_log</property>    
  </properties>


<!-- a template from which a configuration file is generated, after merging 
     in properties and applying a stylesheet.  Normally these might be 
     wrapped in CDATA tags, but for the sake of readability we omit 
     it here.  -->
<template>

# My Apache webserver configuration

ServerType <property name="apache.httpd.server.type"/>
ServerName <property name="apache.httpd.server.name"/>
Port <property name="apache.httpd.port"/>
ServerAdmin <property name="apache.httpd.server.admin"/>
ServerRoot <property name="apache.httpd.server.root"/>
DocumentRoot "<property name="apache.httpd.document.root"/>"
ErrorLog "<property name="apache.httpd.error.log"/>"
LogLevel <property name="apache.httpd.log.level"/> 
PidFile <property name="apache.httpd.pid.file"/>
CustomLog <property name="apache.httpd.custom.log"/>

</template>

</action>

Profiles

Profiles represent a set of configuration files to apply to a host. Like hosts and groups, they can also have properties. They don't represent the actual configuration files (actions do), but rather a set of configuration files.

For example, if we wanted to create a 'webserver' profile that would manage Apache configurations on various hosts, we could create the following:

<?xml version="1.0" encoding="UTF-8"?>

<profile version="3.0.0" name="webserver">

  <!-- the actions that are associated with this profile -->
  <actions>
   <!-- override both the output file location and the stylesheet 
        this will let you customize the output location, the stylesheet,
        the output method and the output encoding -->
   <action name="apache.httpd.conf"/>
  </actions>

  <!-- properties specific to this profile -->
  <properties>
   <property name="apache.httpd.custom.log">/export/data/logs/foo_log</property>
  </properties>
</profile>

Stylesheets

Stylesheets are XSL templates that are used to transform the action's template into the final result. Usually, the default stylesheet is enough. It replaces <property/> XML tags with the merged property values as a result of template processing.

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                version="1.0">

<xsl:template match="/">
<xsl:apply-templates select="/action/template"/>
</xsl:template>

<!-- this substitutes template properties with the ones that are generated -->
<xsl:template match="property"><xsl:variable name="name" select="@name"/><xsl:value-of select="/action/properties/property[@name=$name]"/></xsl:template>

<xsl:output method="text"/>

</xsl:stylesheet>

Property Precedence

When templates are processed, properties are evaluated in the following order: * Action properties (the defaults) * Profile properties * Group properties * Host properties

What this means is that the properties defined in the action can be overwritten by profile properties, and those properties can be overwritten by group properties, and those properties can be overwritten by host properties.

Revisions

Every historical state in the repository has a unique number that is generated by the version control system. Editing, adding, or changing anything in the repository increments the revision number by one.

Using Autoinst, you can access any revision from the repository using the '-r' or '-rev' option. For example:

ai -rev 1234 list hosts

would fetch the hosts from revision number 1234.

Tags

Tags are a symbol representing a revision in the repository. They are usually used for deployments, for example, if you tag the repository with the string 'tag-51', you can always deploy configurations from that tag and will consistently get the revision at that tag.

Development work and changes can continue on the trunk, but that will not disturb the tag. So you can continue to use that tag for deployments while others make changes to the trunk. Also it is very convenient for rolling back changes on hosts -- you could easily rollback from 'tag-51' to 'tag-50', assuming you tagged the repository back when you deployed 'tag-50' to the hosts.

Branches

Branches are used to manage changes in parallel. There is a default branch called the TRUNK where changes normally are placed. Creating a branch spawns a new line of development from the trunk.

Branches are typically used when there are long-living changes that need to be experimented with or otherwise keep separate from the rest of the system. So if someone was testing a big configuration change to the system that would take a month to develop, we don't want their work to disrupt our daily work on the trunk. So you could create a new branch for this developer, where the month-long changes would go.

Like the trunk, you can tag a branch in order to create a symbolic revision that can be used for deployments.

Command Line Interface

Autoinst includes three command line utilities for viewing, creating, and modifying resources in the repository.

ai

The ai utility takes zero or more options and a set of commands. This command needs to have HTTP access to the REST API in order to function.

ai options

Following are the options available to the ai tool.

  • -a [action], --action=[action]: specifies the action to be used with the 'get' or 'edit' command.
  • -b [branch], --branch=[branch]: specifies the branch to use when fetching or editing resources.
  • -c [config file], --config=[config file]: specifies the configuration file location. Currently unused.
  • -d [url], --dist=[url]: specify the base URL to the Autoinst distribution server.
  • -f [filename], --file=[filename]: specify an input filename, used for creating or editing resources. Use '-' to accept input from stdin.
  • -g [group], --group=[group]: specifies the group name to get or edit.
  • -m [message], --message=[message]: log the specified message in version control.
  • -o [host], --host=[host]: specifies the hostname to get, edit, or create.
  • -p [profile], --profile=[profile]: specifies the profile name to get, edit, or create.
  • -P [password], --password=[password]: specifies the password to use for authentication.
  • -r [revision], --rev=[revision]: specifies the revision number to use for fetching or editing resources. By default this is set to the HEAD.
  • -s [stylesheet], --stylesheet=[stylesheet]: specifies the stylesheet to get, edit, or create.
  • -t [tag], --tag=[tag]: specifies the tag to use when fetching resources.
  • -U [username], --username=[username]: specifies the username to use for authentication.
  • -x, --xml: return the command output as XML instead of human-readable.

ai commands

ai commands are used to manage resources in the repository.

branch

branch BRANCHNAME

Creates a branch. This copies the contents of the trunk to a new branch specified by branchname.

clone

clone TAG1 TAG2

The clone command copies a tag, from tag1 to tag2.

edit

edit

The edit command is used to create or edit a new resource.

get

get

The get command is used to fetch resources from the repository.

list

list TYPE

The list command is used to list resources from the repository of type TYPE.

remove

remove

The remove command is used to remove resources from the repository.

run

run

The run command is used to generate a configuration file based on the group, host, profile, and action.

tag

tag TAGNAME

The tag command is used to tag the repository with a symbolic name for the current revision.

aiadmin

The aiadmin command is used for administrative tasks, such as creating the original repository. This command is run on a machine with filesystem access to the repository, as it does not go through the REST API.

aiadmin options

Following are the options available to the aiadmin tool.

  • -a [action], --action=[action]: specifies the action to be used with the 'get' or 'edit' command.
  • -b [branch], --branch=[branch]: specifies the branch to use when fetching or editing resources.
  • -c [config file], --config=[config file]: specifies the configuration file location.
  • -d [url], --dist=[url]: specify the base URL to the Autoinst distribution server.
  • -f [filename], --file=[filename]: specify an input filename, used for creating or editing resources. Use '-' to accept input from stdin.
  • -g [group], --group=[group]: specifies the group name to get or edit.
  • -m [message], --message=[message]: log the specified message in version control.
  • -o [host], --host=[host]: specifies the hostname to get, edit, or create.
  • -p [profile], --profile=[profile]: specifies the profile name to get, edit, or create.
  • -P [password], --password=[password]: specifies the password to use for authentication.
  • -r [revision], --rev=[revision]: specifies the revision number to use for fetching or editing resources. By default this is set to the HEAD.
  • -s [stylesheet], --stylesheet=[stylesheet]: specifies the stylesheet to get, edit, or create.
  • -t [tag], --tag=[tag]: specifies the tag to use when fetching resources.
  • -U [username], --username=[username]: specifies the username to use for authentication.
  • -x, --xml: return the command output as XML instead of human-readable.

aiadmin commands

cat

cat PATH

This command echoes a resource from the repository.

copy

copy PATH1 PATH2

This command copies a resource from PATH1 to PATH2. This is essentially the same as a 'svn copy' command, when using the SVN repository implementation.

create

create

This command creates an version control repository and various base artifacts needed to use an autoinst repository.

ls

ls PATH

This command lists resources from the repository.

remove

remove PATH

This command removes a resource from the repository. This is essentially the same as a 'svn remove' command, when using the SVN repository implementation.

aiupdate

The aiupdate command is used to manage hosts that do not have the full Autoinst stack installed. aiupdate only requires Python and not the Autoinst modules. aiupdate has no commands, because its only command is to update the host is it being run on.

aiupdate options

Following are the options available to the aiupdate tool.

  • -b [branch], --branch=[branch]: specifies the branch to use when updating this host.
  • -d [url], --dist=[url]: specify the base URL to the Autoinst distribution server.
  • -H [host], --host=[host]: specifies the hostname of this machine.
  • -P [password], --password=[password]: specifies the password to use for authentication.
  • -r [revision], --rev=[revision]: specifies the revision number to use for updating this host.
  • -t [tag], --tag=[tag]: specifies the tag to use when updating this host.
  • -v, --verbose: turn on verbose output
  • -y, --yes: answer yes to all questions

Environment Variables

For convenience there are environment variables that can be defined. These environment variables can be used for any of the Autoinst command line utilities.

  • AUTOINST_DIST: specifies the Autoinst distribution server's base URL
  • AUTOINST_EDITOR: specifies the editor to use for 'edit' commands
  • AUTOINST_USERNAME: set the username to use for authentication
  • AUTOINST_PASSWORD: set the password to use for authentication

UserGuide (last edited 2009-03-29 00:42:46 -0700 by ?mlum)