How long can CruiseControl run unattended before requiring some kind of maintenance? Think of things like memory leaks or seemingly random hanging.
How easy is it to add new projects? Modify existing ones?
How easy is it to deal with a lot of projects?