Introduction
If you're using the multi-project support that CruiseControl offers, you'll often find that most of your projects in your config-file are very similar:
- they all use the same modificationset-tags, but of course different repositories;
- they all use the same attributes for the htmlemail except for the url and logdir, etc. (update: since CruiseControl 2.2.1, you can use plugin defaults for this: check the docs)
- within a project, you repeat the name of the project a lot since you're using it for all your directories.
If you have a lot of projects, than maintaining the config.xml can become a tedious job. To ease the task of config-maintenance, many people choose to use some form of generation for their config.xml. There are various ways to do this, depending on your needs. This page is meant to document some of the techniques that can be used and give some examples of how you would use them.
Usage scenarios
There are various reasons for generating your config-file. Avoiding redundancy and cost of maintenance is the most common one, but there are other scenario's too.
A couple of ideas of why and how you could do this:
- extracting the XML that's literally the same for some or every project so it can easily be reused
- using tokens or properties replacing for example the project-name
- define templates for some or all of the projects in your organization and fill in specifics for each of them
- generating a config.xml through a web-application, where developers can add their own project by filling out a form
- generating multiple CruiseControl-projects for each 'real-life'-project, like a daily incremental build and a full nightly build
Keep in mind that the current version of CruiseControl (2.2.1) and older versions need to be restarted if you add a new project to the config-file. Changes to existing projects will be picked up automatically at the next run.
Techniques
Currently, the following techniques are documented:
XML Entities
A commonly used technique for taking the redundancy out of your config.xml is to use XML Entities. This is not used to actually generate a new file, but to include fragments of XML in your config.
This example is taken from a posting by Klaus Rennecke on the CruiseControl User-list.
Create a config-entities.ent file with
SGML declarations like:
<!-- the URL where to fetch the CVS history file -->
<!ENTITY historyurl
'http:
<!-- the CVS history plugin -->
<!ENTITY cvshistory
'<plugin name="cvshistory" classname="net.sf.fraglets.cca.CVSHistory"/>'>
<!-- the build bootstrapper -->
<!ENTITY buildbootstrapper
'<antbootstrapper buildfile="build-bootstrapper.xml"/>'>
<!-- the location of the artifacts directory -->
<!ENTITY artifactsdir
'/Some/place/on/the/build/machine/or/cluster/where/there/is/enough/space'>
And then use them in config.xml. Note that I replaced project specific
text with the ellipsis (...) in this example:
<?xml version="1.0"?>
<!DOCTYPE cruisecontrol [
<!ENTITY % config-entities SYSTEM "config-entities.ent">
%config-entities;
]>
.
.
.
<project name="..." buildafterfailed="false">
&antbootstrapper;
&cvshistory;
<bootstrappers>
<currentbuildstatusbootstrapper file="logs/.../buildstatus.txt"/>
&buildbootstrapper;
</bootstrappers>
<modificationset requiremodification="true" quietperiod="1200">
<cvshistory historyurl="&historyurl;">
<module name="..."/>
<module name="..."/>
<module name="..."/>
. . .
</cvshistory>
</modificationset>
<schedule interval="600">
<ant antscript="runant.bat" uselogger="true" buildfile="build-....xml"/>
</schedule>
<log>
<merge dir="work/.../src/.../logs"/>
<merge dir="work/.../src/.../logs"/>
</log>
<publishers>
<currentbuildstatuspublisher file="logs/.../buildstatus.txt"/>
<artifactspublisher dir="work/.../zip" dest="&artifactsdir;/..."/>
<htmlemail mailhost="&mailhost;"
returnaddress="&returnaddress;"
buildresultsurl="&buildresults;/..."
skipusers="false"
reportsuccess=""
spamwhilebroken="false"
defaultsuffix=""
css="&css;"
xsldir="&xsldir;"
logdir="logs/...">
&emailmap;
</htmlemail>
</publishers>
<dateformat format="dd.MM.yyyy HH:mm:ss"/>
</project>
In this second example the entities are defined in the same file. You can see the drawback that you can't place eg. the htmlemailpublisher in an entity because the projectname is different.
<?xml version="1.0"?>
<!DOCTYPE cruisecontrol [
<!ENTITY svnplugins
'<plugin name="svn" classname="net.sourceforge.cruisecontrol.sourcecontrols.SVN"/> <plugin name="svnbootstrapper"
classname="net.sourceforge.cruisecontrol.bootstrappers.SVNBootstrapper"/>'>
<!ENTITY currentbuildbootstrapper
'<currentbuildstatusbootstrapper file="\\Server\BuildLogs\currentbuild.txt"/>'>
<!ENTITY currentbuildpublisher
'<currentbuildstatuspublisher file="\\Server\BuildLogs\currentbuild.txt"/>'>
<!ENTITY mailaddress
'gerd.zanker@web.de'>
]>
<cruisecontrol>
<project name="PROJECT_ONE" buildafterfailed="false">
&svnplugins;
<modificationset requiremodification="true" quietperiod="1200">
<svn LocalWorkingCopy="..\PROJECT_ONE"/>
</modificationset>
<bootstrappers>
<svnbootstrapper file="build.xml" localWorkingCopy="..\PROJECT_ONE"/>
¤tbuildbootstrapper;
</bootstrappers>
<schedule interval="100">
<ant antscript="vant.bat" buildfile="build.xml" target="checkout" multiple="1" antWorkingDir="..\PROJECT_ONE" />
</schedule>
<log dir="\\Server\BuildLogs\PROJECT_ONE" encoding="ISO-8859-1">
<merge file="../PROJECT_ONE/TestData/PROJECT_ONETestLog.xml" />
</log>
<publishers>
¤tbuildpublisher;
<htmlemail mailhost="mailserver"
returnaddress="&mailaddress;"
buildresultsurl="http://Server:8080/cruisecontrol/buildresults/PROJECT_ONE"
skipusers="true"
reportsuccess="true"
spamwhilebroken="false"
css="CC2/reporting/jsp/css/cruisecontrol.css"
xsldir="CC2/reporting/jsp/xsl"
logdir="\\Server\BuildLogs\PROJECT_ONE">
<always address="&mailaddress;"/>
</htmlemail>
</publishers>
</project>
<project name="PROJECT_TWO" buildafterfailed="false">
&svnplugins;
<modificationset requiremodification="true" quietperiod="1200">
<svn LocalWorkingCopy="..\PROJECT_TWO"/>
</modificationset>
<bootstrappers>
<svnbootstrapper file="build.xml" localWorkingCopy="..\PROJECT_TWO"/>
¤tbuildbootstrapper;
</bootstrappers>
<schedule interval="200">
<ant antscript="vant.bat" buildfile="build.xml" target="all" multiple="1" antWorkingDir="..\PROJECT_TWO" />
</schedule>
<log dir="\\Server\BuildLogs\PROJECT_TWO" encoding="ISO-8859-1">
<merge file="../PROJECT_TWO/TestData/PROJECT_TWOTestLog.xml" />
</log>
<publishers>
¤tbuildpublisher;
<htmlemail mailhost="mailserver"
returnaddress="&mailaddress;"
buildresultsurl="http://Server:8080/cruisecontrol/buildresults/PROJECT_TWO"
skipusers="true"
reportsuccess="true"
spamwhilebroken="false"
css="CC2/reporting/jsp/css/cruisecontrol.css"
xsldir="CC2/reporting/jsp/xsl"
logdir="\\Server\BuildLogs\PROJECT_TWO">
<always address="&mailaddress;"/>
</htmlemail>
</publishers>
</project>
</cruisecontrol>
Ant
Ant has some built-in tasks that can assist in generating a config.xml from a template.
The two key components you will probably want to use will be replace-filters (which are built in with Ant) and foreach loops (which require AntContrib). The replace-filters will allow you to substitute values for tokens that you place in your template. The foreach loop will allow you to iterate through multiple projects, and easily add or remove projects.
Here is an example which makes use of four files. Two are centralized - these actually build the config.xml file: an ant script (buildConfig.xml) and a properties file (buildConfig.properties). The other two files exist in every project and can be customized for each project: a template (cruiseconfig.template) and a properties file for the template (cruiseconfig.properties). Alternatively, you could centralize the template too, and only place the cruiseconfig.properties files in each project. The advantage of having multiple copies of the template is that each project can modify the template for itself. Perhaps the best solution would be to use a centralized template which would be used if a project-specific template were not found.
When you run ant on buildConfig.xml, it creates a config.xml file, then iterates through all the projects that are listed in the buildConfig.properties file. For each project, it makes a copy of the cruiseconfig.template file, replacing each token with its value, as specified in the cruiseconfig.properties file. It then appends that file to the end of the config.xml file. It finishes by putting the closing tag on the config.xml file.
This example assumes that you have a CruiseControl work directory and that each project has an identical directory structure, at least for storing the cruiseconfig files. We use clearcase, so the directory structures for the projects have names that may be strange to non-clearcase users. You can ignore the things like "viewroot" and "myvob" - they are just normal directories as far as this example is concerned.
For detailed information on filtersets and the foreach task, refer to the Ant documentation.
buildConfig.properties
Stored in your cruisecontrol work directory (e.g., C:/cruise/work)
# project.list is a list of the paths to the projects that you will iterate through.
project.list=C:/clearcase/viewroot/project_x,\
C:/clearcase/viewroot/project_y,\
C:/clearcase/viewroot/project_z
# cruiseconfig.file.path is the path in each project where
# the cruiseconfig.template and cruiseconfig.properties files are stored.
# E.G., for project_x, those files would be in:
# C:/clearcase/viewroot/project_x/myvob/build
cruiseconfig.file.path=myvob/build
# these are just the start and end tags for the config.xml file
config.start=<cruisecontrol>
config.end=</cruisecontrol>
buildConfig.xml
Stored in your cruisecontrol work directory (e.g., C:/cruise/work)
<project basedir="." name="buildConfig" default="build-loop">
<target name="init">
<property name="config" value="config.xml" />
<property file="buildConfig.properties"/>
<taskdef resource="net/sf/antcontrib/antcontrib.properties">
<classpath>
<pathelement location="antjars/ant-contrib-0.3.jar"/>
</classpath>
</taskdef>
</target>
<target name="build-loop" depends="init" if="project.list">
<concat destfile="${config}">
${config.start}
</concat>
<foreach
target="build-single"
list="${project.list}"
param="current.project.path"
inheritall="true"
inheritrefs="false" />
<concat destfile="${config}" append="true">
${config.end}
</concat>
</target>
<target name="build-single">
<copy
file="${current.project.path}/${cruiseconfig.file.path}/cruiseconfig.template"
tofile="${current.project.path}/${cruiseconfig.file.path}/cruiseconfig.result"
overwrite="true">
<filterset begintoken="~~" endtoken="~~">
<filtersfile file="${current.project.path}/${cruiseconfig.file.path}/cruiseconfig.properties"/>
<filter token="template.project.viewroot" value="${current.project.path}" />
</filterset>
</copy>
<concat destfile="${config}" append="true">
<filelist dir="${current.project.path}/${cruiseconfig.file.path}" files="cruiseconfig.result" />
</concat>
</target>
</project>
cruiseconfig.properties
Every project has its own copy of this, stored in the location defined in the centralized buildConfig.properties file. Eg, Project X might have its copy stored in: C:/clearcase/viewroot/project_x/myvob/build.
template.project.name=MYPROJECT
# the branch is for clearcase modificationset
template.project.branch=cruisecontrol_development
template.project.buildfile=myvob/build/build.xml
template.modificationset.quietperiod=5
template.schedule.interval=30
template.project.buildmgr=clack@deweycheatemandhowe.com
template.project.emails=<map alias="hugh" address="hldewey@deweycheatemandhowe.com" />\
<map alias="mike" address="measter@deweycheatemandhowe.com" />\
<map alias="stella" address="spdiaz@deweycheatemandhowe.com" />
cruiseconfig.template
Every project has its own copy of this, stored in the location defined in the centralized buildConfig.properties file. Eg, Project X might have its copy stored in: C:/clearcase/viewroot/project_x/myvob/build.
Obviously, this is just an example and you'll need to completely rewrite it for yourself. It should give you a good idea on how to use the tokens, however. Notice that the values for tokens can be simple words, or, as in the case of template.project.emails, they can be more complicated.
<project name="~~template.project.name~~" buildafterfailed="false">
<plugin name="htmlemailpublisher" classname="net.sourceforge.cruisecontrol.publishers.HTMLEmailPublisher" />
<plugin name="artifactspublisher" classname="net.sourceforge.cruisecontrol.publishers.ArtifactsPublisher"/>
<bootstrappers>
<currentbuildstatusbootstrapper file="logs/~~template.project.name~~/buildstatus.txt"/>
</bootstrappers>
<modificationset quietperiod="~~template.modificationset.quietperiod~~">
<clearcase branch="brtype:~~template.project.branch~~"
viewPath="~~template.project.viewroot~~"/>
</modificationset>
<schedule interval="~~template.schedule.interval~~">
<ant
antscript="C:\ant\ant.bat"
buildfile="~~template.project.viewroot~~/~~template.project.buildfile~~"
target="ccupdate build"
uselogger="true"
usedebug="false" />
</schedule>
<log dir="logs/~~template.project.name~~">
<merge file="~~template.project.viewroot~~/myvob/reports/pmd_report.xml"/>
<merge file="~~template.project.viewroot~~/myvob/reports/cpd_report.xml"/>
<merge file="~~template.project.viewroot~~/myvob/reports/javancss_report.xml"/>
<merge file="~~template.project.viewroot~~/myvob/reports/jdepend_report.xml"/>
</log>
<publishers>
<currentbuildstatuspublisher file="logs/~~template.project.name~~/buildstatus.txt"/>
<artifactspublisher
dest="artifacts/~~template.project.name~~"
dir="~~template.project.viewroot~~\myvob\build\ears" />
<htmlemailpublisher
mailhost="mailhost.mycompany.com"
returnaddress="~~template.project.buildmgr~~"
logdir="logs/~~template.project.name~~"
xsldir="C:\tools\cruise\cruisecontrol-2.1.5\reporting\jsp\xsl"
css="C:\tools\cruise\cruisecontrol-2.1.5\reporting\jsp\css\cruisecontrol.css" >
<failure address="~~template.project.buildmgr~~" />
~~template.project.emails~~
</htmlemailpublisher>
</publishers>
</project>
XSLT
Here is an example how XSLT can be used to create config.xml out of a template. You can use e.g. the xslt Ant task to take the template.xml and transform it according to the ReplaceProjectName.xsl file into the resulting config.xml which can be used for CruiseControl.
template.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!-- XSLT file which repleases ${projectname} with either ProjectA or Project2 -->
<?xml-stylesheet type="text/xsl" href="ReplaceProjectName.xsl"?>
<!DOCTYPE cruisecontrol [
<!-- the currentbuild bootstrapper -->
<!ENTITY currentbuildbootstrapper
'<bootstrappers>
<currentbuildstatusbootstrapper file="\\Server\BuildLogs\${projectname}\currentbuild.txt"/>
</bootstrappers>'>
<!-- log -->
<!ENTITY log
'<log dir="logs"><merge dir="test-results/${projectname}"/></log>'>
<!-- schedule -->
<!ENTITY schedule
'<schedule interval="30" >
<ant buildfile="build${projectname}.xml" target="cleanbuild" multiple="5" />
</schedule>'>
<!-- publishers -->
<!ENTITY publishers
'<publishers>
<currentbuildstatuspublisher file="${projectname}.txt" />
<email mailhost="REPLACE WITH MAILHOST"
returnaddress="REPLACE WITH RETURN EMAIL ADDRESS"
defaultsuffix=".com"
buildresultsurl="http:>
<always address="REPLACE WITH BUILD MASTER EMAIL ADDRESS" />
<failure address="REPLACE WITH FAILURE EMAIL ADDRESS" />
</email>
</publishers>'>
]>
<cruisecontrol>
<project name="ProjectA">
¤tbuildbootstrapper;
<modificationset quietperiod="30" >
<cvs cvsroot="REPLACE WITH CVSROOT FOR PROJECT A" />
</modificationset>
&schedule;
&log;
&publishers;
</project>
<project name="ProjectTWO">
¤tbuildbootstrapper;
<modificationset quietperiod="30" >
<cvs cvsroot="REPLACE WITH CVSROOT FOR PROJECT TWO" />
</modificationset>
&schedule;
&log;
&publishers;
</project>
</cruisecontrol>
ReplaceProjectName.xsl (XSLT file):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xsl:stylesheet [
<!ENTITY projectplaceholder '${projectname}'>
]>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<!--
Copy every node with every attribute, but replace the projectplaceholder string
in an attribute value with the current project name (name attribute of the project node)
-->
<xsl:template match="*">
<xsl:copy>
<xsl:for-each select="@*">
<xsl:choose>
<xsl:when test="not(contains(.,'&projectplaceholder;'))">
<xsl:copy><xsl:value-of select="."/></xsl:copy>
</xsl:when>
<xsl:otherwise>
<xsl:attribute name="{name()}">
<xsl:value-of select="substring-before(.,'&projectplaceholder;')"/>
<xsl:value-of select="ancestor-or-self::project/@name"/>
<xsl:value-of select="substring-after(.,'&projectplaceholder;')"/>
</xsl:attribute>
</xsl:otherwise>
</xsl:choose>
</xsl:for-each>
<xsl:apply-templates select="node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Config.xml (result of the template transformation)
<?xml version="1.0" encoding="utf-8"?>
<cruisecontrol>
<project name="ProjectA">
<bootstrappers>
<currentbuildstatusbootstrapper file="\\Server\BuildLogs\ProjectA\currentbuild.txt"/>
</bootstrappers>
<modificationset quietperiod="30">
<cvs cvsroot="REPLACE WITH CVSROOT FOR PROJECT A"/>
</modificationset>
<schedule interval="30">
<ant buildfile="buildProjectA.xml" target="cleanbuild" multiple="5"/>
</schedule>
<log dir="logs">
<merge dir="test-results/ProjectA"/>
</log>
<publishers>
<currentbuildstatuspublisher file="ProjectA.txt"/>
<email mailhost="REPLACE WITH MAILHOST"
returnaddress="REPLACE WITH RETURN EMAIL ADDRESS"
defaultsuffix=".com"
buildresultsurl="http://localhost:8080/cruisecontrol/buildresults/ProjectA">
<always address="REPLACE WITH BUILD MASTER EMAIL ADDRESS"/>
<failure address="REPLACE WITH FAILURE EMAIL ADDRESS"/>
</email>
</publishers>
</project>
<project name="ProjectTWO">
<bootstrappers>
<currentbuildstatusbootstrapper file="\\Server\BuildLogs\ProjectTWO\currentbuild.txt"/>
</bootstrappers>
<modificationset quietperiod="30">
<cvs cvsroot="REPLACE WITH CVSROOT FOR PROJECT TWO"/>
</modificationset>
<schedule interval="30">
<ant buildfile="buildProjectTWO.xml" target="cleanbuild" multiple="5"/>
</schedule>
<log dir="logs">
<merge dir="test-results/ProjectTWO"/>
</log>
<publishers>
<currentbuildstatuspublisher file="ProjectTWO.txt"/>
<email mailhost="REPLACE WITH MAILHOST"
returnaddress="REPLACE WITH RETURN EMAIL ADDRESS"
defaultsuffix=".com"
buildresultsurl="http://localhost:8080/cruisecontrol/buildresults/ProjectTWO">
<always address="REPLACE WITH BUILD MASTER EMAIL ADDRESS"/>
<failure address="REPLACE WITH FAILURE EMAIL ADDRESS"/>
</email>
</publishers>
</project>
</cruisecontrol>
Velocity-templates
If you need a solution that's more flexible than possible with straightforward XML-inclusion or token replacement, using a templating engine is often a good way of achieving this. Velocity is a Java template engine from the Apache Jakarta project. It allows you to write templates and merge these templates with content provided through a so-called Context-object from Java-code. The template language used in Velocity, VTL, allows for simple property-substition but also defines control statements and the possibility to perform callbacks on the Java objects in your Context.
This features are demonstrated in a small example I wrote. The ConfigGenerator-class needs two jars for Velocity and JDom. Notice that jdom.jar is already included with CruiseControl.
This is how it works:
the ConfigGenerator expects to find a template called config.vm. It also needs the path to an XML-file as a commandline-argument. This XML-file is parsed to a JDom Document and made available to the template.
The config.vm included expects an attribute called template for every xml-element named <project>. It then uses the template referred to by this attribute to merge a project; thus, you can easily use different templates for certain projects. You can define your own attributes and subelements for projects and use them in the template of your choice.
This is all generic, so no changes to the Java source code would be needed to change templates and the like.
Maven
Maven has a plugin that will generate your CruiseControl configuration file based on data in your Project Object Model. You need to set the home of CC in a property called maven.cruisecontrol.home or pass it in. Then call
maven cruisecontrol -Dmaven.cruisecontrol.home=c:\cruisecontrol
The resulting cruisecontrol.xml file will at least save you significant typing! Additionally, it will use the <developer> information to match from CVS id's to email addresses.
However, what is much more impressive is to leverage the Multiproject plugin to generate your configuration files for a series of projects! Just enter
maven multiproject:goal -Dgoal=cruisecontrol -Dmaven.cruisecontrol.home=c:\cruisecontrol
and it will generate cruisecontrol.xml files for all the sub projects!
see http://maven.apache.org/reference/plugins/cruisecontrol/ for more information
