devXero's blog

a blog about agile, development, and automation

Posts Tagged ‘Automation’

SWAT 3.0 Released

Posted by Mike Longin on January 19, 2011

We are very excited to announce the release of SWAT 3.0.  The biggest announcements are the new support of Chrome on the Windows OS and Safari on OSX.  In addition we now support calling SWAT from the command line independent of any test runner tool.   Finally we have made a roughly 30% improvement on the speed of SWAT running against the IE browser.  As usual we have also fixed a number of bugs and smaller issues that are detailed in the complete release notes.
New Features and Updates
  1. New Browsers Supported
    1. Chrome
      1. New custom built plugin for Chrome allows SWAT to easily control the Chrome Browser
      2. http://ulti-swat.wikispaces.com/Setting+up+Chrome
    2. Safari
      1. Custom app for OSX now allows SWAT to stream commands to an OSX based system.
        1. http://ulti-swat.wikispaces.com/Setting+Up+Mac
        2. http://ulti-swat.wikispaces.com/Setting+up+Safari
  2. SWAT Console
    1. Tests can now be run independent of any test runner.
    2. As of now this feature is for SWAT only and does not support any custom fitnesse fixtures.
    3. http://ulti-swat.wikispaces.com/SWAT+Console+User+Information
  3. New Commands
    1. UI
      1. AssertJSDialog
        1. Added an optional timeout to allow users to select how long the command will search for a dialog box
      2. AssertTopWindow
        1. Allows users to verify if a window is the top most window on the screen
        2. http://ulti-swat.wikispaces.com/AssertTopWindow
    2. Database
      1. RestoreAllTables
        1. Users can now restore all of the tables they have saved at once instead of one at a time
  4. Defects Resolved
    1. CORE – AttachToBrowser always finds the about:Blank window
    2. CORE – DB Connect now automatically closes the old connection when it creates new ones
    3. CORE – FF – AssertElementExists will find an element when the identifier is empty
    4. CORE – FF – PressKeys does not ignore CancelEvent in Javascript
    5. CORE – FF – Special characters above the standard 128 return correctly when
    6. CORE – FF – The AttachToWindow failure due to “OK” being added to title has been fixed
    7. CORE – IE – attachToModalDialog is now setting the currentIEBrowser
    8. CORE – IE – Browser screenshots work correctly in IE9
    9. CORE – IE – KillAllOpenBrowsers is now respecting AbandonTest
    10. CORE – IE – OnBeforeUnload JSDialogs in IE9 are compatible with SWAT
    11. CORE – IE – PDF pages open automatically in IE9
    12. CORE – Screenshots don’t throw a “Not attached to window” exception.
    13. CORE – SQL checks are passing in the editor and not in Fitnesse
    14. CORE – SWAT waits for all frames to load when waiting for the browser
    15. EDITOR – Editor is respecting the -c attribute for included files
    16. EDITOR – Values of 0 can now be saved in the Delay between commands option

Posted in Uncategorized | Tagged: , , , , , | Leave a Comment »

Running multiple versions of Firefox (with SWAT)

Posted by Mike Longin on January 26, 2010

Adam Goucher just did a great post on how to run multiple versions of Firefox with Selenium and I though I would steal some of his blog and just remove all of that crazy Selenium stuff and substitute in SWAT. First let me give credit where credit is due:

http://adam.goucher.ca/?p=1370&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+AdamGoucher+(Adam+Goucher)

To summarize:

There are times you may need to run multiple versions of Firefox. For instance maybe you want to be testing the newest beta while still keeping around the files that the majority of your users are using. Here is how to do so.

  • Download as many releases as you want / care about
  • Rather than install it into Applications like you normally would, install it into a folder in Applications that relates to the version.
  • Start each one and uncheck the Preference that would have the browser check for updates

Once you have downloaded the versions of Firefox you would like to test against you just need to configure SWAT to use the one you want to test with.  This can be done via the Editor.  Click on the “Settings” menu item and then the “Other Settings” tab.  Inside you will find a “Firefox root directory” text box.  Place the exe of the Firefox version you would like to use for testing, and then just run your firefox tests.  This will allow you to test against any version of Firefox.

Posted in Uncategorized | Tagged: , , , | Leave a Comment »

Ways to improve the reliability and efficiency of SWAT/UI tests (Part 4 – Safely Using InnerHTML)

Posted by Mike Longin on October 28, 2009

This will be the last of my posts on this specific topic for now, though I am sure many more posts will be made along the same lines in the not to distant future.  For this one I want to look at safely using InnerHTML.  Many people do not realize how vastly different browser can treat HTML.  Specifically HTML created dynamically as part of a web 2.0 application.  Below you will find some generalized examples of html in 3 different browsers.

–IE 7

<option value=fl selected>Florida</option>

Read the rest of this entry »

Posted in Uncategorized | Tagged: , , , | Leave a Comment »

Internal Presentation on Reliabilty and Efficiency of UI tests

Posted by Mike Longin on October 28, 2009

I did a presentation internally at Ultimate Software on improving the Reliability and Efficiency of tests and I thought it would be worth posting on here as well.  I will also be making a few more posts to cover all the topics I outlined in the presentation.

Improving SWAT\UI Test Reliability and Efficiency

Posted in Uncategorized | Tagged: , , , , | Leave a Comment »

A shortfall of UI automation and a creative way to deal with it

Posted by Mike Longin on October 28, 2009

One of my biggest flaws with automation in my eyes is that when you run most UI automation (excluding bitmap comparisons that are more trouble then they are worth) the UI can change but the test will not catch it.  This is actually a very common scenario and usually when people discuss solutions they go back to bitmap comparisons.  However I have heard (but not confirmed) that Google has a better way.

What Google has done is to run their UI automation on very public monitors.  Then when people are walking by, hopefully if they see a defect they will go let the responsible team know and it will be rectified.  I think this is an absolutely great idea and one I hope to implement some time in the near future.

Posted in Uncategorized | Tagged: , , , , | Leave a Comment »

Automation is validation

Posted by Mike Longin on October 28, 2009

At a presentation I did earlier this week one of my coworkers made the comment

Automation is for validation, testing requires hands, eyes, and a brain

I have heard similar comments before but I thought that this was worth repeating.  When you create and run automation, you are validating against what you already know.  The script (whether its above or below the UI) was based on some known state and values.  For the most part the automation can only tell you when something expected did NOT happen.  The majority of scripts will not tell you when something unexpected did happen.

For example if I have a login page that takes a user name and password.   When I fill in the user name and password, and press a button I log in.  Now if someone were to add an additional step to my login process, any automation I have would obviously catch that.  However if someone added something that did NOT affect login to the login page then most likely my scripts would NOT catch that.  These things could include changes to the UI, popups, new text, or possibly something more malignant.

This is where the basic difference between automation and testing come from.  When you test you not only are working with base data and expected results, but you have the ability to comprehend the unexpected. What extensive automation really does is allow testers more time to look into the unexplored areas of the system.  More time can be spent on exploratory testing since your regression suite is handling all of the expected tests and scenarios.  By spending more time doing this you are liable to find all of those edge case scenarios that your customers are guaranteed to find.

So in sum it all up, automation is a hugely important part of the development process, but to help identify that what is working hasn’t broke not to “test”. When it comes to testing, it takes someone (or maybe one day something) that can identify the unexpected and can appropriately identify what to do about it.

Posted in Uncategorized | Tagged: , , , , , | Leave a Comment »

Ways to improve the reliability and efficiency of SWAT/UI tests (Part 3 – The truth behing AssertElementDoesNotExist)

Posted by Mike Longin on October 7, 2009

Were now up to part 3, and I am really thinking about ditching the numbers, I think this list is going to get very long very…

But on to part 3 and this will be a short one.  AssertElementDoesNotExist was one of the first commands created for SWAT.  Unfortunately many people do not know some of the dirty and dark secrets of this command.

Read the rest of this entry »

Posted in Uncategorized | Tagged: , , , , | Leave a Comment »

Ways to improve the reliability and efficiency of SWAT/UI tests (Part 2 – Reliably checking select boxes)

Posted by Mike Longin on October 6, 2009

This post is written more for a SWAT user since it has to do specifically with how SWAT users check the values of SELECT boxes.  However the ideas here could be applied to other UI testing tools as well.

For this post we will be using the example Select Box below:

<SELECT id="mySelectBox">
  <OPTION value="A">Alpha></OPTION>
  <OPTION value="b">Beta</OPTION>
</SELECT>

Read the rest of this entry »

Posted in Uncategorized | Tagged: , , , , , | Leave a Comment »

Ways to improve the reliability and efficiency of SWAT/UI tests (Part 1 – Sleeps are evil)

Posted by Mike Longin on October 4, 2009

This is the start of a series of posts I am going to be writing on how to improve both the reliability and efficiency of SWAT tests.  Any ideas I present here can be applied to any UI testing tool (SWAT, Selenium, WatiN, QTP, etc…).  The most important concept to take away from these posts will be that tests can be made both reliable and efficient with just a few simple techniques.

For the first post we will be looking at the Sleep command (http://ulti-swat.wikispaces.com/SwatCommandsSleep).  Almost all UI testing frameworks contain some version of this command.  This command is also sometimes known as a “Wait” Command.

Lets start out with why this command is bad:

  • The command will slow down a test suite
  • No matter what, the time you set for the sleep is always too slow or to long.  Either it will be too long, and your test will just sit there waiting for nothing, or it will be too short and the test will move on before it should.

Assuming you are still reading this, and agree that sleeps are bad, you are probably asking what you can do to avoid sleep commands and make your test both stronger and more reliable.

Read the rest of this entry »

Posted in Uncategorized | Tagged: , , , , , , , , | Leave a Comment »

HTML Part 2 (Containers)

Posted by Mike Longin on October 4, 2009

For part 2 we are going to look at a few different container types and tables.

First some container types you may experience:

  • DIV
  • SPAN
  • P

Each of these are common containers you may come across.  They each have some common similar attributes:

  • InnerHTML – This is the area between the open and close tag.
    • Example: <DIV>This is the innerHTML </div>
  • InnerHTML is ALL of the HTML between the tags, including pieces of other tags
    • Example:<DIV><SPAN>This is some text</SPAN></DIV>
      • The innerHTML of the div is “<SPAN>This is some text</SPAN>”
      • The innerHTML of the span is “This is some text”
  • Id
  • Name
  • Class

Read the rest of this entry »

Posted in Uncategorized | Tagged: , , , , , , | Leave a Comment »