Your Test Suite Sucks… do it again

A pretty smart dude I worked with once told this story of a team he was on.  They spent a few days, maybe even a week, deploying their app.  It was painful, obviously.  They’re response: “That sucked.  Let’s do it again tomorrow.”

Most people try to avoid hard things or at least delay them until they have a good reason, but some things, important things… we should do those more often.  We need to practice the hard things.  They in turn become less risky and less scary bookends to our prideful efforts.

Testing is one of those things.

A few months back there was a thread going on the Professional Scrum Developer Trainers list about topics to include in the future.  Specifically this was in regards to the .Net flavor of the course, because we were getting updates about what was coming down the pipe in the tooling.  My excitement in that course has always focused on testing and with Jason Zander announcing continuous testing for all the major test runners in Visual Studio vNext that was my suggestion.

Adam Cogan, a smart dude in his own right, followed up asking me to describe why I thought that was such a big deal.  I sent a rather long reply that I found useful today while describing the testing pyramid to coworker Steve Gentile.  I thought I’d provide that reply here and get everyone’s thoughts:

We give passing credence to the idea of fast tests that run often versus slow tests that we expand to include before checkin/commit or during automated build (gated).  Unfortunately, the later becomes a catch all for the oops, just got lazy scenario. 

I remember when I first got the automated testing bug.  I wrote some tests (calling them unit tests) that integrated with a 3rd party tool, gave great coverage and awesome feedback.  We knew within 5-10 minutes if a new 3rd party version could be released.  Giant leap.  The next step, though, was hidden.  I would code for 20, 30, 90 minutes or more and then run my tests while I grabbed a cappuccino.

My current passer-by, rock your world tip/tool is NCrunch for .Net or Guard for everyone else (we need this).  Autotest.Net was the first try from James Avery, but it didn’t have any traction and was unstable at best.  Most teams have tests, but they aren’t using them effectively.   (Ok, Nuget and the no-commit pattern is a very close second!)

I run a group call Cincy Clean Coders.  We do katas, often as a whole group.  The first Randori kata we did was eye opening for the audience.  Someone started refactoring to solve a red test and another person threw a fit.  “What are you doing?!  You’re red!  You can’t refactor when your red.”  The driver didn’t intend to be refactoring, they were just solving the current red test in their mind.  They didn’t realize they were mixing many steps into one.  There is a mental burden to doing the simplest thing, running the tests, and only then refactoring.

Teams make a great jump when they automate their important tests.  Nightly feedback from QTP, Test Pro, or Coded UI is a big step.  Consciously mixing unit, integration and acceptance tests is amazing.  Our current tools mandate the start of the feedback loop be a conscious decision.  “So all those changes, wonder if things are still working?  Let’s find out.”  We don’t say it, but that’s how we act.

When your tests run every time you pause (NCrunch) or save a file (Guard) you come to expect feedback NOW.  You skip even CTRL+R,Y until your “finished”, only to find you just did 3,5, 10 refactorings.  Which one caused the problem?  Or you write a test that calls a web service, taking .65 seconds.  Pretty soon “fast” tests take 30-60 seconds (or more!) and your CI build takes 20 minutes.

Continuous testing is (IMO) the next step in shortening the feedback loop.  You start organizing your tests more effectively (or at all).  In good environments, I even use this as a chance to commit.  I now have the luxury of completely ignoring the path I took to my current source situation.  Get into a tizzy, reset/undo checkout.  Committing 10-20 times in an hour isn’t uncommon here.  Neither is branching half a dozen times.

In the PSD I can demo and talk to the importance of running tests often and having confidence, but a blaring red failure on the screen for that last edit really hits home.

I want your test suite to be an invaluable tool.  Continuous testing is one of those pit of success kind of things.  We need more of those.  But when signals pop up solve the real problem.  Don’t delay the pain.

Naming Distributed Teams

I try to remind everyone as often as possible that the most complex part of solving problems is people.  So many little things impact people every day.

Today I came across this tip from Johanna Rothman for distributed teams:

Call the teams by their feature names, not by their geographic location. It’s too easy to call the teams by "Headquarters" or "Cincinnati" or "Paris" or "Bangalore", and any name other than "Headquarters" implies a hierarchy among the teams.

Recognizing and breaking these subtle habits is hard, but the sign of an organization more concerned with results and teamwork than title and seniority.


Are these teams distributed or co-located?  Each has the skills needed to deliver a feature for our project/product that meets the definition of done.  Does this scale more or less than resource pools?

Cone of Uncertainty Graphic with Events in Time

I’ve always received great feedback when describing the cone of uncertainty to groups on a whiteboard using events in time rather than simply an abstract timeline.  Today I put a graphic together that I can easily share or refer to.  Thought I’d post it for others as well.  Feel free to use it and let me know what you think. 

Disclaimer: All events are provided as examples which can and may occur as you work toward solving a problem.  I’m not advising you to include these explicitly in your process.


Good Books for the Softer Side of Agile

I was a member of the Agile Engineering panel at the Path to Agility in Columbus, OH yesterday.  It was a great event (if not overcrowded) with some amazing names. 

The panel started quickly down the dreaded TDD path.  You know the one where we proselytize about how everyone’s doing it, you should too, and the world will seamless fall into harmony once you get it.  It’s true, but how many times can you hear it from a talking head.  Good grief.  Go find a pair.

Not too long in a savior came from the back of the room and asked the panel what resources where available that discussed the soft skills of successful agile teams.  Barry jumped on this, because, well, he wrote a book about it!  (I have copy – thanks Improving!).  This monopolized the remainder of the session which was fantastic IMO.  From executive buy in to nasty team members and apairia it was great.

In the end I recall the following books being mentioned for further reading.  I dread buying a Kindle, Nook, etc. because I’ll simply flush my wallet down the toilet when lists like this come up:

    Individuals and Interactions (Barry’s Book)

    Managing Humans

    Passionate Programmer

    Coaching Agile Teams

Go grab a few and share them around the office.  Have some hug sessions and maybe pick up a beer for your pair.

Cincy Clean Coders – Code You Can Show Your Friends

What would a Monday following an Agile conference be without the announcement of a new group focused on quality practices!


I’m happy to announce that Cincy Clean Coders is really going to happen.  Details can be found at, but suffice it to say the inaugural meeting will be held April 7th at 6pm in the Max Train facilities in Mason, Ohio.

Mark Haskamp was strong armed brave enough to offer to go first with me.  I’ll update the site with the topics for the first meeting soon.

Big Thanks to Ernie Stormann and Parag Joshi who helped push this through.  And a big kudos to the rest of Twitter for being excited about a topic like this.  It says a lot about where our industry wants to be.

Help! You might notice I’m not a designer.  If you want to help, we could use a site design that doesn’t compare to notepad in a browser.  Nothing special, just, you know, some other than black on white text.