Slides

An Introduction to UI testing

1. New Browsers Supported
   1. Chrome
      1. New custom built plugin for Chrome allows SWAT to easily control the Chrome Browser
      2. http://ulti-swat.wikispaces.com/Setting+up+Chrome
   2. Safari
      1. Custom app for OSX now allows SWAT to stream commands to an OSX based system.
         1. http://ulti-swat.wikispaces.com/Setting+Up+Mac
         2. http://ulti-swat.wikispaces.com/Setting+up+Safari
2. SWAT Console
   1. Tests can now be run independent of any test runner.
   2. As of now this feature is for SWAT only and does not support any custom fitnesse fixtures.
   3. http://ulti-swat.wikispaces.com/SWAT+Console+User+Information
3. New Commands
   1. UI
      1. AssertJSDialog
         1. Added an optional timeout to allow users to select how long the command will search for a dialog box
      2. AssertTopWindow
         1. Allows users to verify if a window is the top most window on the screen
         2. http://ulti-swat.wikispaces.com/AssertTopWindow
   2. Database
      1. RestoreAllTables
         1. Users can now restore all of the tables they have saved at once instead of one at a time
4. Defects Resolved
   1. CORE – AttachToBrowser always finds the about:Blank window
   2. CORE – DB Connect now automatically closes the old connection when it creates new ones
   3. CORE – FF – AssertElementExists will find an element when the identifier is empty
   4. CORE – FF – PressKeys does not ignore CancelEvent in Javascript
   5. CORE – FF – Special characters above the standard 128 return correctly when
   6. CORE – FF – The AttachToWindow failure due to “OK” being added to title has been fixed
   7. CORE – IE – attachToModalDialog is now setting the currentIEBrowser
   8. CORE – IE – Browser screenshots work correctly in IE9
   9. CORE – IE – KillAllOpenBrowsers is now respecting AbandonTest
   10. CORE – IE – OnBeforeUnload JSDialogs in IE9 are compatible with SWAT
   11. CORE – IE – PDF pages open automatically in IE9
   12. CORE – Screenshots don’t throw a “Not attached to window” exception.
   13. CORE – SQL checks are passing in the editor and not in Fitnesse
   14. CORE – SWAT waits for all frames to load when waiting for the browser
   15. EDITOR – Editor is respecting the -c attribute for included files
   16. EDITOR – Values of 0 can now be saved in the Delay between commands option

http://www.androidguys.com/2011/01/01/android-sms-bug-exist/

In it the author comments “But the burden of proof is on those claiming a bug exists.”  Which to me leads to an interesting question.

Does the burden of proof lie on those claiming a bug exists or on the company to prove a bug does NOT exist?

To me the burden lies with the company not the user.  To a user the bug exists, even if the company has not seen it.  As Josh Rynne (@jrynne) pointed out “good business practice would suggest that the company should prove that the bug does not exist…. otherwise you’re basically saying you don’t believe your customers by forcing them to prove it”.  Case in point is the antenna gate scandal where Apple placed the onus on the consumer and not on the company.  It cost quite a bit in bad publicity when Apple had to retract its denials and accept responsibility.

However I believe we can take this idea one step further.  While discussing this with Chris Taylor (@agiletester) his comments were “In reality, in good design, there is no such thing as user error.  There may be dumb users, but you are not going to make money saying it.”  A truly good design should not allow users to make user errors.  Relating back to this article, if the issue really was user error in how they were switching between conversations, the design should have helped users avoid input mistakes in that sort of scenario.

So to answer my own question, the burden of proof for any bug truly lies on the creator not the user.  It is only the users responsibility to alert the company to the issue and the company has to determine if the bug is in the working of the application or the use.

Where I work we have a very flat management structure.  What this means to us is that their are very few steps between an employee and a director.  In fact virtually all members of a team report directly to a director.  Currently there are about 200 members of the staff and 2 directors (though we are scaling up to 4).  As part of this, we have created what I am calling a cloud based management team.  What this means is that while people have a direct supervisor, they tend to go to any supervisor to discuss any issue.  If director A is busy, people go see B and vice versa.  This tends to allow people the ability to talk to people that they are most comfortable with and also allows more access to our directors since they are so interchangeable.  Overall I have found this to be an incredibly simple, yet effective way to manage a large group of people without a large amount of overhead.  It is definitely not a perfect solution (which I believe to be one of the reasons we are scaling up), however it has been working well for us.  I believe that as we scale up it will be more like adding nodes to the cloud and not forcing people to deal with one specific manager.

http://sourceforge.net/projects/ultifit/.

The primary purpose for the development of UltiFit was to create a tool for
“push button regression” of your Fitnesse-based testing inventory, achieving
overnight (for multiple platforms) what might otherwise require an entire team
working all week (for a single platform). When we started using Fitnesse
combined with our UI testing tool (SWAT) we found that tests would fail and
cause a systematic collapse of all of the tests after it. Additionally the
standard fitnesse suite interface did not allow us to run tests from multiple
suites at once. With that in mind we went about creating a more robust test
runner. Initially UltiFit allowed the execution of a single test at a time.
Since then we have added grid functionality which allows a cluster of machines
to share responsibilities for running Fitnesse tests. Internally we use UltiFit
to run our UI automation tool (SWAT) and running just one team’s tests could
take as long as 12 hours. Once the grid functionality was introduced we were
able to drop that time to under 4 hours. Since we always have the option to add
more servers to the grid, we can shave even more time if we choose.

Some of the additional benefits of UltiFit include:

• Hands free automated execution/management of a group of individual Fitnesse/SWAT tests
• Recovering from any issues during execution
• Killing tests that are stuck for some reason
• Organizing/reporting the results and system information
• Archiving history of test results making it easy to find exactly when a test failed along with complete details of what failed
• Detailed timing information captured for each test and for total group of tests
• Easily re-running failed tests (only)
• Command line interface enables scheduled execution
• Grid based execution can dramatically improve regression time (Both XP and Vista supported)
• Loosely coupled, easily configurable design:
• • Involves a single .exe for regular or grid-based deployment
  • Tool is easily configured for any new Fitnesse test inventory

In short, UltiFit allows us to execute and manage our tests while we sleep.
Hopefully at least 1 or 2 others may find this useful as well. If you have any
questions or are interested in helping out with development please let me know
or visit us at http://sourceforge.net/projects/ultifit/support.

Request.Browser.IsMobileDevice

However while this works great for the iPhone this did not work at all for Android based devices. After a large amount of searching I found the MobileESP open source project (http://code.google.com/p/mobileesp/) for determining if a browser is from a mobile device. Initially I used thier entire code base. However I felt that solution was too bulky and I ended up just finding the code for locating an Android based device and using that instead.

Request.UserAgent.ToLower().IndexOf("android") != -1

Even if you do not need their entire code base, the project is a great repository for determining how to identify any mobile based browser

Meta Tag

This tag is needed to help autofit the page and to prevent zooming.  (If you ever wondered why some sites on the iPhone do not zoom, this is the answer)


<meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;" />


CSS

To create my app I copied the layout here

I used essentially the same css as available on the above site.  The most important note I did find in my research however, is not to use fixed widths, and instead to use percentages.

Icon

For websites designed for mobile browsing, it is important to have a custom bookmark icon.  This way if someone saves the bookmark to thier home screen it stands out.  The two site below cover how to do this

http://eisabainyo.net/weblog/2009/06/12/making-a-website-iphone-friendly-using-css/

http://stackoverflow.com/questions/1951381/configuring-android-web-applications

In short you need to have these two tags

<link rel="apple-touch-icon" href="../../apple-touch-icon.png" />
<link rel="apple-touch-precomposed" href="../../apple-touch-icon.png" />

Inspirations

A great site for mobile app inspirations is http://cssiphone.com/.  I attempted to make my site look like these (didn’t really succeed however.)

http://ulti-swat.wikispaces.com/SwatIntroDemos

Specifically

An Introduction to UI testing using SWAT

Applying modern software development techniques to automating the web UI

Please let me know any feedback or questions you might have.