devXero's blog

a blog about agile, development, and automation

Ways to improve the reliability and efficiency of SWAT/UI tests (Part 1 – Sleeps are evil)

Posted by Mike Longin on October 4, 2009

This is the start of a series of posts I am going to be writing on how to improve both the reliability and efficiency of SWAT tests.  Any ideas I present here can be applied to any UI testing tool (SWAT, Selenium, WatiN, QTP, etc…).  The most important concept to take away from these posts will be that tests can be made both reliable and efficient with just a few simple techniques.

For the first post we will be looking at the Sleep command (http://ulti-swat.wikispaces.com/SwatCommandsSleep).  Almost all UI testing frameworks contain some version of this command.  This command is also sometimes known as a “Wait” Command.

Lets start out with why this command is bad:

  • The command will slow down a test suite
  • No matter what, the time you set for the sleep is always too slow or to long.  Either it will be too long, and your test will just sit there waiting for nothing, or it will be too short and the test will move on before it should.

Assuming you are still reading this, and agree that sleeps are bad, you are probably asking what you can do to avoid sleep commands and make your test both stronger and more reliable.

In my experience I have found there are 3 typical reasons why a sleep command is most commonly used (there are others but I feel at least for the moment that these are edge case scenarios):

  • An asynchronous event is happening on the page that takes longer then SWAT’s default timeout
  • An event on the page is waiting for a SQL event (such as a save to finish)
  • A browser takes longer to load (and set its title bar) then SWATS default timeout

With SWAT, 3 commands are provided to deal with these situations

Here is some more on these commands

AssertElementExistsWithTimeout is meant to allow users to avoid sleeps when they are doing a check on the page.  Many times either a page is a slow loading page, or an asynchronous action is happening on the page, and testers will drop a sleep on the page to attempt to determine when the action has completed.  The standard AssertElementExists command has an adjustable timeout (configured in the config file) but is usually set for just 5-10 seconds.  Because of this people often drop a sleep command to wait for the page.  Instead a tester can use the AssertElementExistsWithTimeout command and set a timeout in milliseconds to tell SWAT to keep polling and checking if this element exists yet.  No matter how long you set the timeout for, as soon as the element is found, SWAT move on.  In my own experience I usually set the timeout from 60000 (1 minute) to 360000 (6 minutes) with the comfort and knowledge that as soon as the criteria is met, the test will move on, but not before it is ready.

AssertDBRecordExistsWithTimeout is meant for any time the page needs to wait for a DB event to finish.  The timeout is again in milliseconds.  I typically use this any time I am waiting for a SQL transaction such as a save to finish.

AssertBrowserExists is meant for times when a popup takes so long to load, that the default timeout for AttachToWindow times out.  This will allow SWAT to keep on waiting for a longer time.

In the next post I will examine some other best practices for increasing test efficiency and reliability.

UPDATE: Got some great feedback already. Sleeps do have their uses. However the important part to take away from this post is that they are easily over used or used incorrectly.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: