devXero's blog

a blog about agile, development, and automation

For internal builds is it acceptable to have some failing tests?

Posted by Mike Longin on December 19, 2008

So the typical answer to this question is no.   Usually with some harshly worded questions such as:

  • Why would you have a test in there, if you dont mind if failing?
  • Why would you even consider releasing a build with failing code?

But let me add some context.  When you have 20+ teams using the build and the test that fails is specific to one team AND the code is for a more “minor” feature.  Does that change the equation?  Or how about it affects 5 teams, but not the other 25.  Is that ok?

I am curious what people think.  

To me its worth starting to think of multiple tiers.  Tier one tests must pass every time.  Tier 2 tests are tests that affect 10% of the teams and while an internal build can be deployed, these teams must code freeze and fix the issue.  Tier 3 tests are limited to an individual team and that team would need to freeze and fix but everyone else can continue.  I am sure someone will tell me I am wrong, but to me its worth at least thinking about.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: