devXero's blog

a blog about agile, development, and automation

Test steps vs Test execution time

Posted by Mike Longin on September 30, 2009

This has been bothering me for a while and wanted to bring it up.  When writing tests I have already mentioned the concept of 200 test steps.  By limiting yourself to only 200 steps, you will find tests easier to write, maintain, and run.  However that really does not take into account test execution time.  I am sure we could easily find an example of a test that is only 10 steps in length but that takes 20 minutes.  I believe optimum length is less than 10 minutes, but preferably 5.

I really do not have a reason for that time, other than gut feeling but the math seems to work in my favor.  If a test takes 5 minutes to run, and 5 minutes to fix a bug.  It means at minimum you are going to need 15 minutes to fix any bug (5 to verify it, 5 to fix, and 5 to verify the fix.)  Once we start getting over that 15 minute threshold I believe we are really setting ourselves up for failure.

UPDATE:   So just spent over an hour working on a test, starting to think that 5 minutes may even be too long…

Does anyone else have tips or tricks, or experience with this?

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: