Your Test Suite Sucks… do it again

A pretty smart dude I worked with once told this story of a team he was on.  They spent a few days, maybe even a week, deploying their app.  It was painful, obviously.  They’re response: “That sucked.  Let’s do it again tomorrow.”

Most people try to avoid hard things or at least delay them until they have a good reason, but some things, important things… we should do those more often.  We need to practice the hard things.  They in turn become less risky and less scary bookends to our prideful efforts.

Testing is one of those things.

A few months back there was a thread going on the Professional Scrum Developer Trainers list about topics to include in the future.  Specifically this was in regards to the .Net flavor of the course, because we were getting updates about what was coming down the pipe in the tooling.  My excitement in that course has always focused on testing and with Jason Zander announcing continuous testing for all the major test runners in Visual Studio vNext that was my suggestion.

Adam Cogan, a smart dude in his own right, followed up asking me to describe why I thought that was such a big deal.  I sent a rather long reply that I found useful today while describing the testing pyramid to coworker Steve Gentile.  I thought I’d provide that reply here and get everyone’s thoughts:

We give passing credence to the idea of fast tests that run often versus slow tests that we expand to include before checkin/commit or during automated build (gated).  Unfortunately, the later becomes a catch all for the oops, just got lazy scenario. 

I remember when I first got the automated testing bug.  I wrote some tests (calling them unit tests) that integrated with a 3rd party tool, gave great coverage and awesome feedback.  We knew within 5-10 minutes if a new 3rd party version could be released.  Giant leap.  The next step, though, was hidden.  I would code for 20, 30, 90 minutes or more and then run my tests while I grabbed a cappuccino.

My current passer-by, rock your world tip/tool is NCrunch for .Net or Guard for everyone else (we need this).  Autotest.Net was the first try from James Avery, but it didn’t have any traction and was unstable at best.  Most teams have tests, but they aren’t using them effectively.   (Ok, Nuget and the no-commit pattern is a very close second!)

I run a group call Cincy Clean Coders.  We do katas, often as a whole group.  The first Randori kata we did was eye opening for the audience.  Someone started refactoring to solve a red test and another person threw a fit.  “What are you doing?!  You’re red!  You can’t refactor when your red.”  The driver didn’t intend to be refactoring, they were just solving the current red test in their mind.  They didn’t realize they were mixing many steps into one.  There is a mental burden to doing the simplest thing, running the tests, and only then refactoring.

Teams make a great jump when they automate their important tests.  Nightly feedback from QTP, Test Pro, or Coded UI is a big step.  Consciously mixing unit, integration and acceptance tests is amazing.  Our current tools mandate the start of the feedback loop be a conscious decision.  “So all those changes, wonder if things are still working?  Let’s find out.”  We don’t say it, but that’s how we act.

When your tests run every time you pause (NCrunch) or save a file (Guard) you come to expect feedback NOW.  You skip even CTRL+R,Y until your “finished”, only to find you just did 3,5, 10 refactorings.  Which one caused the problem?  Or you write a test that calls a web service, taking .65 seconds.  Pretty soon “fast” tests take 30-60 seconds (or more!) and your CI build takes 20 minutes.

Continuous testing is (IMO) the next step in shortening the feedback loop.  You start organizing your tests more effectively (or at all).  In good environments, I even use this as a chance to commit.  I now have the luxury of completely ignoring the path I took to my current source situation.  Get into a tizzy, reset/undo checkout.  Committing 10-20 times in an hour isn’t uncommon here.  Neither is branching half a dozen times.

In the PSD I can demo and talk to the importance of running tests often and having confidence, but a blaring red failure on the screen for that last edit really hits home.

I want your test suite to be an invaluable tool.  Continuous testing is one of those pit of success kind of things.  We need more of those.  But when signals pop up solve the real problem.  Don’t delay the pain.

Determine Framework Version & Profile in a #NuGet Install.ps1

[Updated: @davidfowl suggested using the FrameworkName rather than parsing myself.  Thanks!]

I’m working on the Caliburn.Micro Nuget package and one of the things that we need is to add a default Bootstrapper, ShellView, and ShellViewModel to your project.  One awesome thing Rob Eisenberg has done is provide WPF, Silverlight and WP7 versions of Caliburn.Micro.  Unfortunately, the platforms are different enough that these three pieces we would like to add are different.

I say this is unfortunate only because Nuget doesn’t support framework and profile specific content through conventions.  That’s only available for binaries.  It does, however, provide hooks into the install process via PowerShell. 

Maybe you see where I’m headed…

The hard part for me was figuring out the framework version of the project targeted.  In the end, here’s how you can pull the TargetFrameworkMoniker in your install.ps1:

param($rootPath, $toolsPath, $package, $project)

  $moniker = $project.Properties.Item("TargetFrameworkMoniker").Value

  $framework= new-object System.Runtime.Versioning.FrameworkName($moniker)

  write-host "Identifier: " $framework.Identifier 
  write-host "Version: " $framework.Version
  write-host "Profile: " $framework.Profile

# WPF4 will be NETFramework,Version=v4.0,Profile=Client
# SL4 will be Silverlight,Version=v4.0
# WP7 will be Silverlight,Version=v4.0,Profile=WindowsPhone

Happy packaging!

Cincinnati Professional Scrum Developer .Net Course (Feb 21st)

[Correction: This is the first for the new year.  This certainly is not the first ever.]

I’m happy to announce that our first Professional Scrum Developer course of 2011 is scheduled for February 21st.  This is a course which practices 5 days of craftsmanship, collaboration, and sustainable pace via the Scrum framework.  Practice is the key to success and there is no better way to build a successful Agile team than to work together, continuously with a coach available to guide you through unfamiliar terrain.

Every team has it’s own happy path to a successful, self-organizing, self-managing Scrum team and it is not an easy one to navigate.  During these 5 days we will experience brown-field product development using Modern Engineering Practices such as TDD, Continuous Integration, Refactoring, and more.  Our case study is an ASP.Net MVC application.

Register for the course with who graciously provides the facility and infrastructure., founded by Ken Schwaber to improve the profession of software development so that developers love their work and our customers love working with developers, describes the Professional Scrum Developer course in detail.  You’ll also find the course syllabus here.

Look forward to seeing you there.

Greatest Overeager Design Decision: Assignment

In a getting back to basics moment of mine, I started through the TDD Problems.  I found myself on the Console Interaction problem and decided to take a step back from after refactoring.  This is what I saw:

var shape = _console.AskForShape();

var rectange = GetRectange();


ReSharper was hinting at it, but it should have screamed.  Why do I need to hang on to shape?  I can imagine my mindset at the time.  I was worried about the inevitable decision I would have to make about 20 tests down the road.  That little assignment was the beginning of my overeager, premature design.

Question your code.

Where did Alt.Net go?

These last few weeks have seen a flurry of emotions from the .Net community.  It appears that a few product releases and a presumptive priority change for IronRuby have caused the world to come crashing down and passengers of the .Net ship to jump overboard.  I’m glad I don’t work for these emotionally unstable individuals, because if we had any debate (I’m aggressive and loud) I’d be concerned they would run from the room crying or worse yet, quit (seems more likely from the rhetoric I hear).  If this is what the Alt.Net community is made of then OSS in the world of .Net has absolutely no chance and the outside OSS world should fear their new recruits.

During the late ‘90s and early 2000s immense business systems where built on the powerhouse platform known as Microsoft Office.  Having worked, well, at all, I’ve seen first hand the ugly, but quite impressive apps built by accountants and marketiers (mark it down, that’s my word) generating great revenues or savings for their companies.  I have replied to requests to “make that an enterprise web app” with “that will fail miserably, keep using it.”  The apps are a disaster waiting to happen themselves, but that’s why we have enterprise backup systems.  It’s just not worth the IT Department’s time to rebuild Excel when it is perfectly suited for their purpose.

In come WebMatrix, LightSaber, I mean LightSwitch, and all the other “low-end” tools.  These are obviously a trend that show Microsoft is abandoning the “high-end” developer (are we really that pompous that we consider ourselves “high-end”, get over yourself you geek.  I bet they make more cash per feature than you).  Or could it be that they are making up for the fact that the last great act of feature development shown by Microsoft to the “low-end”, revenue generating, feature shipping developer is the Ribbon Bar.  I guess you could add SharePoint in there, but I won’t bate you. 

Since 2002, we’ve seen 4 iterations of Silverlight, 4 versions of the Framework, WPF, WCF, the rise of PnP, 5 versions of Visual Studio, Azure, Hyper-V, Team Foundation Server (like it or not), and a dearth of other tools in the same time that Excel has received… drum role… the Ribbon.  Yes there are nicer icons to represent values and you can make prettier tables or charts, but seriously they still use VBA!  In that same time period we’ve seen 2, count them, 2 releases of  the platform on which many businesses survive.

So yes, Microsoft has thrown the Excel and Access productivity worker (not lowend developer) a bone.  They’ve made it easier for you to say, “Sorry, I don’t do Master-Details forms.  Here’s WebMatrix, call me when you want to ship.”  Get over it, start an OSS project or, if you liked it so much, help the IronRuby team build that platform up.  Whatever you do, stop whining and do something.