•
•
•
•
Jim Weirich wrote in to say that he has it on good authority that the KeySpan USA-19HS will properly drive a Firecracker. The Keyspan adapter comes with Mac OS X drivers that speak the handshake signals the Firecracker listens for. Huzzah! That means you can remotely control your lava lamps from your Mac.
Enjoy!
Your software is being automatically built and tested on a schedule. It even sends you an email when the code doesn’t compile or pass its tests. You’re certainly ahead of most projects, but email is just so 90s. Even if you could manage to find those build failure emails amidst all that spam, you’re reading yesterday’s news. Indeed, you may already be ignoring the status of the scheduled build.
The Monitoring chapter of the book offers alternative, in-your-face, worth-getting-up-for-in-the-morning techniques for monitoring scheduled builds. The most popular technique came by way of a story contributed by Alberto Savoia. He describes how his project uses red and green lava lamps to radiate the status of their scheduled build. Better yet, those lamps are controlled using X10 devices such as those used to turn on your household lamps so that you don’t arrive home to a dark house.
Well, as you might imagine, I could hardly wait to build my very own build-monitoring lava lamp kit. And as bonus material for readers of the book, I’ve crafted a bit o’ software that integrates with CruiseControl. So now you too can enjoy red and green bubbles on your project!
To get started, you need some automation gear. Think of these gadgets as this year’s essential project accessories:
This kit includes:
Cost: $39.99
(Props go to the folks at x10.com for supporting this project by supplying me with a complimentary kit. It all fits in a wee box, so I can carry it from project to project.)
With that kit, you can control two lava lamps — one plugged into the transceiver module and the other plugged into the lamp module. You can optionally purchase another appliance module if you want to control two appliances. For example, you might want your build process to turn on a coffee pot when the build fails and then kick start your margarita machine when the build is fixed.
I used the Hot Rock Lite F/X (yellow earth/blue liquid and red earth/purple liquid). Note for legal purposes that these lamps (shown in pictures below) are not LAVA(R) brand motion lamps, but those will work just as well.
Cost: $9.99 each at Target or Walmart
It’s an open source Java library that includes the CruiseControl plug-in, an API to make your wildest X10 dreams come true, detailed instructions, and an ever-so-useful collection of tests.
Way down deep, the library uses the Java Communications API to send bits out over the serial port and into the Firecracker Computer Interface. (Linux users will need the RXTX implementation). Michel Dalal’s Java X10 CM17A API library, an implementation of the FireCracker (CM17A) Communications Specification, is used to send out the correct 1s and 0s in response to human-friendly commands. Many thanks to him for doing all the low-level bit twiddling and sharing the goodies with us!
Cost: Free to readers of Pragmatic Project Automation
With that hardware in hand, you’re ready to start the assembly process. The Firecracker Automation System includes instructions written for your average home electronics consumer, so your average computer/network geek should have no trouble. I’ll spare you all the gory details and instead run through a quick visual tutorial of my setup.
Start by plugging the Firecracker Computer Interface into a serial port of your scheduled build machine:
This little gem sends a wireless signal from the computer to the transceiver module. Notice that you don’t lose the serial port. You can plug another serial device into the back of Firecracker Computer Interface.
Next, plug the transceiver module into any wall outlet within your electrical wiring system. Then turn on the lamp you want the build process to light up when the build fails (the red lamp) and plug it into the transceiver module:
See that antennae on the transceiver? The transceiver picks up the RF signal sent by the Firecracker Computer Interface connected to the computer, converts it into an X10 signal, and broadcasts the X10 signal across the electrical wiring system.
Every X10 module is uniquely identified by a house code (A-P) and a unit code (1-16). By default, the transceiver is configured to listen on "A1". So when the Firecracker Computer Interface sends a signal that tells module "A1" to turn on, the device that’s connected to the transceiver—the red lamp—is turned on.
Next, plug the lamp module into a wall outlet and set its house and unit code to "A2". Then turn on the lamp you want the build process to light up when the build passes (the green lamp) and plug it into the lamp module, like so:
When the Firecracker Computer Interface sends a signal instructing the "A2" module to turn on, the transceiver picks up the signal and broadcasts it out over the electrical wires. The lamp module hears the signal and turns on the green lamp.
That’s it for assembly!
At this point it’s a good idea to make sure you can turn these lamps on and off at will using the free software (separate download) that emulates the Palm Pad Remote Control. It sends signals through the Firecracker Computer Interface, so it’s a good sanity check that you have everything hooked up correctly.
OK, now you want to hook the lava lamps up to your scheduled build process running in CruiseControl. That’s where the Pragmatic Automation X10 software comes in.
The README file describes how to install and test the software in detail. The final step is to register the CruiseControl plug-in that effectively wires up the lamps to indicate the status of each CruiseControl build cycle. Just to demonstrate how easy that is, here’s the XML you need to add to CruiseControl’s config.xml file:
<plugin name="x10publisher" classname="com.pragauto.X10Publisher"/>
<publishers>
<x10publisher port="COM1" passDeviceCode="A2" failDeviceCode="A1" />
</publishers>
Edit the attributes of the <x10publisher> element as necessary for your serial port and device codes. A complete config.xml file is included in the project as a reference.
Once you’ve fired up CruiseControl and a build succeeds, you’ll see something like this on the console:
BUILD SUCCESSFUL Total time: 10 seconds [cc]Jul-08 16:56:53 Project - Project dms: merging accumulated log files [cc]Jul-08 16:56:53 Project - Project dms: publishing build results [cc]Jul-08 16:56:53 X10Publisher - Turning pass device (A2) on; fail device (A1) off ... [cc]Jul-08 16:56:56 Project - Project dms: idle [cc]Jul-08 16:56:56 Project - Project dms: next build in 1 minutes
At that point, the green lava lamp should turn on. Bask in that pleasant glow for a moment. When the lamp gets warmed up, you’ll get entertained by happy, green bubbles:
And then at some point somebody might check in code that causes the build to fail. (Hey, it happens to even the best programmers once in a while.) Here’s the early warning sign that the scheduled build is in trouble:
Eek! Notice that the ambient light in the red lamp goes on immediately. It will take a while for the red lava to heat up and start to boil. In the meantime, there’s work to be done. When you check the CruiseControl log, you’ll see something like this:
BUILD FAILED file:C:/work/dms/builds/checkout/dms/build.xml:77: Tests failed! Check test reports. Total time: 5 seconds [cc]Jul-08 16:58:20 Project - Project dms: merging accumulated log files [cc]Jul-08 16:58:20 Project - Project dms: publishing build results [cc]Jul-08 16:58:20 X10Publisher - Turning pass device (A2) off; fail device (A1) on... [cc]Jul-08 16:58:23 Project - Project dms: idle [cc]Jul-08 16:58:23 Project - Project dms: next build in 1 minutes
If it takes too long to fix the build, the red lamp will start to get angry:
The object of the game is to keep the green lamp glowing. :-)
You may have noticed that when the transceiver module turns on and off it makes a fairly loud snapping sound. That sound is caused by the mechanical relay inside. (The lamp module doesn’t make that sound, I suspect because it’s a low-current device that doesn’t use a mechanical relay.)
This audible feedback turns out to be quite handy if your team is working in close quarters. The "snap" could be yet another sound in your project soundscape. By plugging the red lamp into the transceiver module (A1), you’ll hear that snapping sound just before the red lamp turns on. So if you’re heads down (without headphones), the sound will alert you to a build failure. Indeed, you’ll hear the build break!
These lamps get insanely hot. If you don’t let them cool down for at least five minutes before touching the glass, you’ll find out just how hot. (Yes, even the green one is hot.)
The instructions for my lamps note that they perform best if operated for less than ten hours at a time. Your local fire marshal would certainly agree. But who will remember to turn off the lamps at the end of every programming day? Well, that’s a job for the ol’ computer. Just write a shutdown program that uses the Pragmatic Automation X10 software to turn off the lamps. For example:
new X10Device("COM1", "A1").off();
new X10Device("COM1", "A2").off();
Then schedule an at or cron job on the scheduled build machine that runs the shutdown program at some hour of every evening, and another program that turns the lamps on every morning.
The Firecracker Computer Interface plugs into a serial port. Modern day Macintosh computers don’t have a serial port. Sadly, none of the USB-to-serial converters I tried worked with the Firecracker Computer Interface. (I’d love to hear otherwise.)
You can purchase X10 kits that plug into USB, but I chose to use the Firecracker kit because it’s relatively cheap and it’s what is described in the book. Thankfully, by writing and testing most of this software against a mock device, I was able to do the majority of the development on my PowerBook. :-)
It’s relatively easy and inexpensive to make build monitoring a spectator sport. Put the lava lamps in a highly-visible area of your project and use them as visual (and audible) feedback devices, and to show off your programming prowess, of course. And that’s just the beginning. You can use any appliance to monitor anything that’s valuable to your team.
Are you using feedback devices to spice up your project? If so, please share your story.
Enjoy!
Suppose you’ve just launched a new web application. Being a savvy automator, you decide to set up monitors for significant system events like, say, new users signing up. Now it’s always good to pick a physical device that gives visual or audible feedback that in some way matches the logical event you’re monitoring. So what kind of feedback goes along with a new user signing up?
Ian Bishrey couldn’t exactly wire up a human, so he went with the next best thing: a screeching Wowwee Alive! Monkey.
You know you want one of these on your project, and Ian has instructions and code for how he hacked the remote controller and wrote a Java-based serial interface (with tests) to send commands to the animatronic monkeys.
In the email he sent, Ian concluded by stating matter of factly "Using the remote means we can control an army of chimps simultaneously." I had to read that sentence a couple times, and each time the mental images got progressively more frightening. Added to which, Ian mentioned in passing that they also use the monkeys to give feedback on "low NLP linguistic classification confidence score events". So to summarize, we have a web 2.0 application that’s taking on new users and in some way processing natural languages, all being monitored by a potential army of monkeys.
You know, when I challenged readers of the automation book to find creative ways to monitor their software projects, I never imagined it could come to this…
You know you want to extract useful information from your project in an automated way, but which tools should you use and how much configuration will they take?
John Brugge has taken a number of popular open-source tools and knitted them together for you. He’s calling it Glean, and his goal is to help your project get quick and easy feedback by simply dropping Glean into your build system. It’s analogous to Maven plugins, but aimed at projects using Ant, and less ambitious.
Currently Glean includes scripts to drive the following tools:
The download includes basic documentation on how to set up Glean to run against a source tree, as well as sample configurations for a handful of open-source projects.
John packaged up Glean to scratch an itch for his clients, and he’s offering it back to the community as a token of appreciation for all the open-source tools. Glean is distributed under an MIT license.
Enjoy!
Chumby looks to be a fun way to monitor automation in and around projects. It’s a portable Internet-connected device that displays visual widgets. Just "trust the Chumby" and it autodiscovers your wifi network.
But what makes Chumby special is the open and hackable design. Want a custom widget? Make one. Got some hardware you’d like to integrate? Plug it into the chumbilical.
This is going to be fun to watch. Please send pictures of Chumby at work on your software project!
Bill Bumgarner expertly steps you through how to hook an RSS feed into a local Subversion server.
Ethan wrote in to give us a heads-up on his new-and-improved CVS logging tool dubbed cvslog2web.
cvslog2web is similar to the CVS commit hooking and log info to RSS techniques we’ve seen before, but offers it in one convenient package and adds a few more goodies. In addition to an Atom syndication format, you also get recent history as HTML, permalinks to individual log entry pages, and some careful thought on compacting/summarizing the log entries.
Ganesan Janarthanam got inspired by the idea of providing continuous integration feedback using a lava lamp and created the CruiseControl Dashboard Yahoo Widget. It lets you monitor multiple projects with an animated lava lamp.
I’ve been remiss to have delayed mentioning SwitchTower until now. It’s an open-source utility that makes one-step deployment to one or more machines both possible and painless.
Say, for example, you need to roll out a new release of your killer app to two web servers, four application servers, and a database server. You could manually log in to each of those machines and type a list of commands, or just let SwitchTower do it for you:
rake deploy
That single incantation causes SwitchTower to execute the commands listed in your deployment recipe in parallel (and atomically) on multiple machines. (It uses ssh to communicate with the remote machines.) It’s really that easy.
But then you notice that the application you just deployed has a bug. After giving your manager fifty push-ups, you’re thankful you can press the "undo" button:
rake rollback
That command rolls back the application to the last deployed version on all deployed machines. Yes, it’s really that easy.
I’ve been starting to use SwitchTower on all my Rails projects, and demo’ing it in our Pragmatic Rails Studio. As a result, SwitchTower has truly spoiled me. It’s an automator’s dream, and I can’t imagine going back to manual deployment gyrations.
That brings us to the point of this elevator pitch: Can you deploy and roll back software releases this easily and consistently on your project? If not, I’d recommend you give the SwitchTower manual a read, especially if you aren’t building Rails applications. That’s right, even if you’ve never heard of Ruby or Rails you owe it to yourself to check out SwitchTower because its concepts aren’t specific to Ruby or Rails. And in fact it seems possible to use SwitchTower as the deployment vehicle for apps of any size and color by customizing a core set of tasks. I’d love to hear of efforts to adapt SwitchTower to J2EE projects, for example.
Every software project can benefit from a deployment process that takes less time and money without sacrificing quality. SwitchTower is an archetype of automated deployment. Check it out before rolling your own solution.
Andrew shared the following build frequency chart:
We see a flurry of good builds, then a six-day dry spell where almost no builds take place. Did somebody accidentally unplug the continuous build machine? Were all the developers at a conference?
Perhaps there’s a better explanation when we remember that a build is only triggered when code is changed. In this case, no changes added up to good news for Andrew’s team:
Congrats!
CruiseControl 2.3.1 was released today with an enviable display of nips and tucks.
Get it here.
I’m glad to hear that someone else peeked under the CruiseControl metrics tab for a glimpse at their project’s build frequency. Here’s an interesting (colorful!) one from a reader who shall remain anonymous:
Notice how different it is from the first team's chart. For starters, the broken build frequency seems high, but at least they’re periodically getting back to a healthy state. I’d suggest they could do a bit more local testing before checking in code, or perhaps just get a bit more sleep. The person who submitted the chart included the following commentary:
What else do you see when you look at this chart?
In response to the source code activity charts, Robert Watkins kindly sent in this example chart displayed free of charge in the CruiseControl metrics tab.
It shows the frequency of builds which, of course, are triggered by commits to the source code repository. Looks like a slick way to also monitor trends in scheduled build breakage. What does your build frequency chart look like?
Take a peek at these example charts of source code repository activity. What do they imply? I have no idea, but I find them fascinating.
I wonder what the charts would look like on a project where the team builds things in small chunks (where small means "measured in minutes") and checks in code directly after each thing is built and tested. Or would that kind of predictable development rhythm just make the data less interesting? At any rate, it would be interesting to use a bit o’ automation to keep a running chart from start to end of a project.
Oh, and please be careful not to read too much into these charts. I shudder at the thought of some pointy-haired manager using them as input to an employee bonus formula.
The dedicated team of folks who write code to keep your code continuously integrated have obviously been busy this summer. CruiseControl 2.3 was released yesterday with a goodly amount of new features and plugins (release notes).
Get it here.
Martin Fowler has written an excellent article describing the strengths of using Rake as a build language, and how it compares to our old friends make and Ant.
Having used Rake extensively in recent months on a Rails project, I can’t underscore enough how handy it is to have a full-blown language at your fingertips when crafting a build system. I got a taste of this power when I experimented with using Groovy to script Ant. I didn’t get a chance to push those boundaries before moving on to Rake, but it’s the same concept: build systems have much to gain from the use of an internal domain-specific language.
A must read!
SD Times just posted a new article concerning lengthy builds siphoning energy off agile processes.
I think we’ll continue to see the tools and processes of old getting in the way of the rapid feedback required to move quickly and confidently. It’s an opportunity to revisit the value of those tools and processes, and adjust accordingly.
I didn’t realize how much automation we have in and around our Rails project until the other day when our shared development server took a vacation and never came back. It’s like that backup strategy that only gets tested when the hard drive fails. Except in this case it was pretty much a yawner. Once we decided to go to a new machine, we were up and running in no time. There were no panic-stricken fire drills. For all intents and purposes, it was business as usual.
Part of the credit for no downtime goes to the fact that we all work untethered on our local machines. (This turns out to be super handy on long airplane rides.) This forces a discipline of keeping things relatively self contained. But we do hook up to the mother ship often (or right after those plane rides) to synchronize our work in a shared Subversion repository. The contents of that repository represent life. On demand, we can recreate anything in the repository—any version of a file from the time the project started up to the present. Everything else is out of our control.
The key to repeatability lies in how much stuff you put in version control. It’s best just to go long. On our project, James meticulously made sure that our Rails application was as self contained as possible. Using only the Subversion repository as a source, our entire application and its supporting infrastructure can be built from scratch. We regularly put that to the test on our local machines, and the development server is just another machine.
It’s relatively easy to automate small things, but it’s especially rewarding when those recipes are applied to automate the big things, as well. Indeed, for large projects, no single recipe will do. But in his latest blog, James discusses some strategies that you might be able to pick up and apply to your project.
Inspired by the recent Dashboard widget, Dmitri Maximovich wrote a Firefox plugin to monitor builds in CruiseControl. He’s looking for your feedback to help make it even better…
The Dashboard in Tiger is an ideal way to monitor automation at work with just a glance. I’ve been anxiously awaiting word of attractive widgets that monitor builds and running programs.
Jason Sypher and Mark Husson came through with style today by unveiling the CruiseControl Widget. Check out the screenshots. Yum!
Last update:
Fri May 16 20:25:18 2008
Alex Bakman
Andrew Wheeler
Ben Pryor
Better SCM
Brad Appleton
Brian Harry
Buck Hodges
Damon Poole
David Anderson
Eric Sink
Frank Schophuizen
Ian Olsen
James Manning
Joel from Canada
Kevin Parker
Longacre
Martin Woodward
Mike Clark
Mike Mason
MS Team Foundation
Plastic SCM
Rob Caron
Robert Cowham
Slava Imeshev
Version Control Blog
vincem