Dashboard > CruiseControl > ... > Understanding the alternatives to CruiseControl > CI Feature Matrix
CI Feature Matrix Log In | Sign Up   View a printable version of the current page.

Added by Jeffrey Fredrick , last edited by Adam Collard on May 26, 2009  (view change) show comment
Labels: 
(None)

Note:: This is starting as a straight copy of the document at codehaus since that word is that it will be archived (see comments on the page at codehaus).

There are many Continuous Integration systems available. This page is an attempt to keep an unbiased comparison of as many as possible of them. The goals are:

  • Make it easier to choose an appropriate CI tool for your project.
  • "Healthy competition aid" for the people involved in the development of these various CI systems.

Note: If you want a new feature in one of these, please use the respective project's issue trackers. Don't add feature requests here.

Legend:
The feature exists
The feature exists, but is buggy or not well tested
The feature is planned for the near future
The feature doesn't exist

Project info CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Project info
Project origin ThoughtWorks ThoughtWorks ThoughtWorks ThoughtWorks Jay Flowers Timpani Software Mozilla Project Brian Warner Urbancode Urbancode Atlassian
PMEase PMEase Apache Gump Apache CSH Consult OpenMake Software
 OpenMake Software
 Viewtier Systems John Keiser Zutubi JetBrains java.net VSoft Technologies Hericus Software Project origin
Open Source
Source provided to customers
+OpenAPI 		Open Source
Implementation language Java C# Ruby Java C# C# Perl Python Java Java Java Java Java Java Java C# Java, C++, C, C#, JSP
Java, C++, C, C#, JSP
Java Perl Java Java Java C# Java Implementation language
Free Free edition available
Free 2-user version
Free for Open Source Projects
Free for Open Source Projects and Community
Free for Open Source Projects
Free for Open Source Projects
Free for Small Teams and Open Source Projects
Free edition available

Free Community Edition 		Free
Issue tracker JIRA JIRA Lighthouse Mingle Google ? Bugzilla Trac private JIRA JIRA
Supports Javaforge JIRA JIRA JIRA JIRA Official support Bugzilla Jira JetBrains Tracker java.net private Issue tracker
Online demo here here here here here here Working implementation here Requestor
Five Minute Intro 		Building Open Source projects
 here Use demo/domo ? ? here Eval download Online Demo
 Online Demo
 Working implementation
Online Demo here Live Demo Live Demo Online demo
Number of active developers 5 4
5+ ? 1+ 5 ? 5+ ?
-
4+
? 3 ? 4 1 5-10 5-10 ? ? 2 5~7 5-10 ? ? Number of active developers
SCM support CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed SCM support
AccuRev
plugin 		? ?
plugin 		AccuRev
AlienBrain ? ? ? ? ? ? ? ? ? AlienBrain
Bazaar ? ? ? ? ? ? ? ? Bazaar
BitKeeper ? ? ? ? ? ? ? ? ?
plugin 		BitKeeper
ClearCase
plugin 		?
plugin 		ClearCase 
CA Harvest ? ? ? ? ? ? ? ? ? CA Harvest
CM Synergy   ? ? ? ? ? ? ? ? ? CM Synergy
CVS CVS
Dimensions ? ? ? ? ? ? ?
? ? Dimensions
"File system SCM" ? "File system SCM"
git ? ?
plugin 
? ? ? ? ? ? ?
plugin 		git
HTTP file ? ?

plugin
(via process trigger)
HTTP file
Mercurial ? ? ?
plugin 		? ? ? ? ? ? ? ?
plugin 		Mercurial
MKS ? ? MKS 
Perforce (p4) ?
plugin 		Perforce (p4)
PVCS ? ?
plugin 		PVCS
SourceGear Vault ? ?

SourceGear Vault
StarTeam ?
plugin 		StarTeam 
Subversion Subversion 
Surround ? ?

Surround 
Team Foundation Server
(via plugin) 		? ? ? ? ? ? ? ? ? ?
plugin 		Team Foundation Server
VSS ?
plugin 		VSS
VSS Journal ? ? ? VSS Journal
SCM related support CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed SCM related support
SCM filtering ? ? ? SCM filtering
Multi-SCM ?
(SVN only) 		about ? ? Multi-SCM
Can create new SCM repositories ? ? ?
via FinalBuilder 		Can create new SCM repositories
Footprints the built binaries back to SCM managed source versions
? 
? 
? 
? 
? 
? 
? 

 
Uses built in artifact repo instead.  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Footprints the built binaries back to SCM managed source versions
Audits source files used in binaries but not stored in a SCM repository
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? ? 
? 
? 
? 
? 
? 
? 
? 
Audits source files used in binaries but not stored in a SCM repository
Build management CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Build management
Parallel builds (ability to build several projects simultaneously) ? Parallel builds (ability to build several projects simultaneously)
Distributed builds
via contrib 		 
 
?
Distributes the workload of the pre and post build activities across multiple agents
Distributes the workload of the pre and post build activities across multiple agents 		Distributed builds
Runs compilers and linkers with Build Avoidance (incremental builds)
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Runs compilers and linkers with Build Avoidance (incremental builds)
Accelerates calls to the compilers and linkers through multi-threaded processing
? 
? 
? 
? 
? 
? 
? 

 
  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Accelerates calls to the compilers and linkers through multi-threaded processing
Agents' code auto-updated ? ? ? ?
? ? ? ? ? ? ? Agents' code auto-updated
Performs source code dependency scanning, impact analysis and reporting
? 
? 
? ? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
Performs source code dependency scanning, impact analysis and reporting
Auto-gens build.xml based on IDE project files.
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? 
? ? 
? 
? 
? 
Auto-gens build.xml based on IDE project files.
Manually force builds
Email 		?
Not needed 		Manually force builds
X-platform and X-language compile management
? 
? 
? 
? 
? 
? 
? 

  		? 
? 
? 
? 
? 
? 
? 
? 
? 
? ? 
? 
? 
? 
X-platform and X-language compile management
SCM trigged builds
(if supported by SCM) 		  ? SCM trigged builds
SCM poll based builds
Tinderbox2 		?
SCM poll based builds
Temporal build scheduling   ? ? Temporal build scheduling
Builds promotion ?   ? about ?
plugin 		Builds promotion
Interproject Dependencies ? ?   ? ? ? ? ? ? ? ? Interproject Dependencies
Builds deletion ?   ? ? ? ?
automatic
Builds deletion
Reproduce history builds ?   ? ?
Reproduce history builds
Proactive (can prevent build breakages) ?   ? ? ? ? ? ?
Proactive (can prevent build breakages)
Detect new failing tests while build ? ?   ? ? ?
(via report) 		? ? ? ? ? ? ?
? Detect new failing tests while build
Notify when first test in build fails ? ?   ? ? ? ? ? ? ? ? ? ? ? ?
Notify when first test in build fails
Security CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Security
User authentication  
For management 		?
For management 		User authentication
User authorization schemes   ? ?
Database
configurable
User authorization schemes
LDAP Integration   AD
? ? ? ? LDAP Integration
Kerberos ?   ? ? ? ? ? ? ? ? ? ? ? ? ? Kerebos
Single Sign On ?   ? ? ?
via Crowd 
? ? ? ? ? ? ? ? ? Single Sign On
Custom JAAS ?   ? ? ?
via Crowd 
? ? ? ? ?

? ? ? ? Custom JAAS
Publishing CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Publishing
Confluence   ?
plugin 		? ?

Confluence
Email Email
Run executable   ? Run executable
FTP  
plugin 		?
plugin 		FTP
IRC   ? ?
plugin 		IRC
Jabber   ? ?
plugin
Jabber
Lotus Sametime  
via XMPP 		?
plugin
Lotus Sametime
NetSend   ?

NetSend 
ProjectStart   ? ?
ProjectStart 
RSS   ? RSS
SCP   ? ? ?
plugin
SCP 
Windows System Tray
plugin 
?
plugin
Windows System Tray
Formatted Logging
XML
XML 		 
(In Relational Db)
Xml, Html 		?
? Formatted Logging
Yahoo Messenger   ? ? ?


Yahoo Messenger
MSN Messenger ? ?   ? ? ? MSN Messenger
X10  
via XMPP 		? ?


X10 
Web interface CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Web interface
View changesets Bonsai ?
 

 
Bonsai
View changesets
Add new projects  
"Tree" only 		?
 

 

"Tree" only 		Add new projects
Clone projects   ? ?
 

 

Clone projects
Delete projects   ?
 

 
Delete projects
Modify projects   ?
 

 

Modify projects
Kill builds   ?
 

 
Kill builds
Pause builds   ?
 

 
Pause builds
Access to build artifacts   ?
 

 

Access to build artifacts
Browse CI's working copy ? ?
 

 

Browse CI's working copy
Delete CI's working copy   ? ?
 

 

Delete CI's working copy
Search in builds   ? ?
 

 

Search in builds
Historic graphs   ?

Historic graphs
Self-updating web page
?


Ajax 		Self-updating web page
|
Multi project support  

Multi project support
Multi project view   ?

Multi project view
Add/remove agent machines (for distributed builds) ? ?   ? ? ? ? ? ?


Add/remove agent machines (for distributed builds)
Directly supported build tools CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Directly supported build tools
Shell / command script  
Via command line
?


via FinalBuilder
Shell / command script
Ant  
Via command line

via FinalBuilder
Via command line Ant
Groovy   Via command line
Via command line


plugin 		Via command line Groovy
OpenMake Meister

         
Via command line 		                      OpenMake Meister
Maven   Via command line
Via command line


Via command line Maven
Maven2   Via command line
Via command line

Via command line Maven2
Make   Via command line
Via command line 		? ?


Via command line Make
MsBuild ?   Via command line ? ?
Via command line 		? ? ? ? ?

?
plugin
via FinalBuilder 		Via command line MsBuild
NAnt  
Via command line 		?


plugin
via FinalBuilder 		Via command line NAnt
Rake (Ruby) ? ? ? ? ?
Via command line
Via command line 		? ? ? ? ?

? ?
plugin
plugin 		Via command line Rake (Ruby)
Visual Studio ('devenv')   Via command line
Via command line 		? ?


via FinalBuilder 		Via command line Visual Studio ('devenv')
FinalBuilder   Via command line
Via command line 		Via command line



Via command line FinalBuilder
Issue & PM integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory
Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister
OpenMake Mojo
Parabuild Tinderbox3 Pulse TeamCity Hudson FinalBuilder Server
Zed Issue & PM integrations
Bugzilla   ?  
link2issues
plugin
Bugzilla
ClearQuest ? ?   ? ? ? ? ? ? ? ? ?

? ? ?
ClearQuest
Confluence   ?


Confluence 
JIRA   ?  


plugin
JIRA
Mingle  
Mingle
QualityCenter
?
?
  ? ?
?
?

  		?
?
?
?
?
?

?
?
?
?
?
?
QualityCenter
Rally
?
  ? ?
?

 
?
?
?
?
?
?


?
?
?
?
?
Rally
Rubyforge.org   ? ? ? ? ?


Rubyforge.org
Scarab   ? ? ?


Scarab
Sourceforge.net   ? ? ? ? ?


Sourceforge.net
Trac   ? ? ? ?


plugin
Trac
VersionOne

?
  ? ?
?
?

 
?
?
? ?
?
?


?
?
?
?
?
?
VersionOne
Test Tool integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Test Tool integrations
Agitar ?
  ?
?
?
?
?
?
?
?
?


?
?
?
?
?
?
? Agitar
CppUnit result rendering ? ?   ? ? ? ? ? ? ?

? ?
plugin 		? ? CppUnit result rendering
JUnit result rendering   ?


? JUnit result rendering
NUnit result rendering   ?


plugin
? NUnit result rendering
QualityCenter test rendering ?
?
  ?
?
?

  		?
?
?
?
?
?


?
?
?
?
?
?
? QualityCenter test rendering
PHPUnit result rendering
? ?   ? ? ? ? ? ? ? ?

? ? ? ? ? ? PHPUnit result rendering
PMD result rendering
? ?   ? ? ?
  publishes html
plugin 		? ? ? ? ?
? ? ?
plugin 		? ? PMD result rendering
Clover result rendering ? ?   ? ? ? ? ? ? ?

? ?
about
plugin
? Clover result rendering
Selenium result rendering ? ?   ? ? ? ? ? ? ? ?

? ?
about
? Selenium result rendering
SilkCentral
?
?
  ?
?
?

  		?
?
?
?
?
?


?
?
?
?

plugin 		?
? SilkCentral
IDE Integrations CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed IDE Integrations
Eclipse Plugin  
Atlassian Eclipse Connector 		?


about
Eclipse Plugin
VS 2005 Plugin ? ?   ? ? ? ? ? ? ? ?

? ? ?
VS 2005 Plugin
IntelliJ Plugin   ?
Atlassian IntelliJ Connector 		?


about
IntelliJ Plugin
Source Inspection integration CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Source Inspection integration
Bonsai ? ?   ? ? ? ? ?

?
Bonsai
ViewVC   ? ?


ViewCVS
Fisheye   ?



Fisheye
Remote Management API CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Remote Management API
REST ? ?   ? ?
? 		? ? ? ? ? ?

? ? ? ?
details 		? REST
SOAP  
Asynchronous Messaging 		? ?


SOAP
XML-RPC   n/a ?


XML-RPC
JMX N/A   n/a ?


plugin
JMX
Hessian ?   n/a ?


Hessian
Client Library ? ?   .Net Remoting
? ? about ? ? ? ? ? ?

? ? ?
OpenAPI
Client Library
Installation and Configuration CruiseControl CruiseControl.NET CruiseControl.rb Cruise CI Factory Drumbeat CI Tinderbox & Tinderbox2 BuildBot Anthill Professional Anthill Bamboo Luntbuild professional LuntBuild Gump Continuum Sin OpenMake Meister OpenMake Mojo Parabuild Tinderbox3 Pulse TeamCity (EAP) Hudson FinalBuilder Server Zed Installation and Configuration
Windows installer   ?


Windows installer
Self contained distribution (except SCM clients) ?


Self contained distribution (except SCM clients)
Additional dependencies JRE, SCM client .NET Runtime, IIS, SCM client   SCM client .NET Runtime, IIS, SCM client, depends on the packages you choose
.NET Runtime, SQL Server (MSDE Included in distribution) Perl, CVS Python, Twisted JRE, SCM client JDK, Servlet container, SCM client JRE
jdk, SCM client jdk,tomcat,SCM client ? JRE, SCM client Subversion, .NET runtime/IIS or Mono/XSP JRE, Perl, SCM Client
JRE, Perl, SCM Client