The holy grail of a automation and why it isn’t as holy as I thought
Posted by Mike Longin on August 24, 2009
I have spent the past year dreaming of my (and quite a few others) holy grail of the automation world. In my mind this has always been a large grid (or cloud) based system that can run all of your functional tests at once. In more realistic terms it would at least turn an 12 hours suite into an 40-80 minute run. I have talked about this numerous times, and felt that Ultimate was really going to get there with the recent aquisition of lab manager. Jason Huggins (http://twitter.com/jhuggins) said something at AA-FTT that really made me stop in my tracks.
(Loosely remembered) Even if we can run our full suite in a much shorter time span, the programmers still wont be able to do that BEFORE every checkin
Him saying that really just blew me away because I really had not thought about that before.
Today I had the opportunity to hear J.B. Rainesbergers speak about the inherent flaws in integration testing and how we need to move to micro tests that only test one object at a time. To do this requires extensive mocking, which I have not always been a fan of after having to debug\modify others code like this and feeling the pain of doing so. However the past two days of comments really make me realize just how long these integration tests take to run, and the absolute need to have as few as possible.
I am hoping that over the next few days I can attend more sessions such as these 2 and see where my thought path lines up on what my personal holy grail of automation should be and maybe some steps to lead me down that path.