Brett McGinnis

Automation: The Invisible Feature

What is Automation?

Well to be technical, only for a moment, it is some code that runs another set of code. To be more specific, it is a set of code that acts like a person would by tapping, typing, and swiping in our theoretical app. More often than not automation is never seen by the end user (that’s you and I downloading things from the App Store). So if it’s not in the app I downloaded, then what is it? What is it used for? Why do I need it? Well, historically speaking, automation is taking something people don’t like doing, or something that is super repetitive, and making machines do it for us. I always think of those crazy robot arms that help build cars nowadays.

car manufacturing robots

Now that that’s out of the way, how does automation help build software? Well to discuss this I am going to need an imaginary product, a tester, and a computer. Let’s call this product “FakeBlock”, the tester “Bob”, and the computer “Jenkins”. Now in order to release FakeBlock, Bob wants to test it and make sure it’s working well before releasing it, meaning before users can download it from the App Store. In order to test, I have tasked Bob to write test cases to make sure this product works well before releasing. For this example, I am only going to talk about one particular test case named “Logging In”. The description of the test case is to make sure Bob, and end users, can log in to their FakeBlocks. Just to be REALLY clear here is the test case:


Test Case: Logging In

  1. Launch FakeBlock
  2. Enter in Bob’s email address
  3. Enter in Bob’s password
  4. Tap Log In

Expected Result: Bob sees his FakeBlock feed.


This is a clear, concise, and meaningful test case. From this simple test case Bob already knows some things about the app. The app built properly, didn’t crash, and Bob was able to successfully log in. So how long did it take for Bob to write, execute, and report on this test case? Let’s say an hour: a bit long in my opinion, but this is an example so take my word for it.


Now let’s talk about the same test case but this time Bob automated the test. The description of the test case is the same but the steps would probably look closer to something like this:


Test Case: Logging In

  1. Launch FakeBlock
  2. Wait for Log In Page
  3. Find the Email Field
  4. Enter in Bob’s email address
  5. Find the Password Field
  6. Enter in Bob’s password
  7. Find the Log In Button
  8. Tap Log In

Expected Result: Bob sees his FakeBlock feed.


Phew, that took a little bit of effort and some of those steps are a little more difficult than they might let on. For example, how does my test code know that I am logged in? There are a couple ways to do that. For now let’s assume Bob has some foolproof way to check if he actually logged in and then pass or fail the test appropriately. Let’s say the effort to do all this work was one day.


At this point it becomes obvious that automating the test case takes a lot more time to write and run than its human run counter part. You may also have some questions like if automating something takes significantly more time to do for even a simple test case like this, why bother at all? Isn’t it more cost effective to just always do manual testing? And the answer is it depends. If Bob only wanted to know the result of the test case exactly one time and had no intentions of running the test case ever again, then I think the manual test case makes more sense. It’s cheaper (since it takes up less of Bob’s time), documented, and Bob got the result he needed to release.


Now consider, what if Bob wanted to run this test case more than one time? What if Bob was testing a product that was going to take weeks, months, or even years to build? If Bob leaves the test case as a manual (human run) test case, then for every build/feature/bug Bob would have to rerun that hour long test case every time to get the result. This is where automation really starts to shine. Jenkins (remember him?) is the computer that can run all of Bob’s automated tests. Jenkins can also check to see if any changes have happened (Continuous Integration) and automatically run the tests. Then if something explodes Jenkins will email Bob. To help clear things up here is a chart (charts are good).

Quality Effort Over Time

As this not completely made up chart indicates, money spent increases over time for the manual test case and the automated test case has a large initial investment. To answer the original question, “Isn’t it more cost effective to just always do manual testing?” clearly after day 8 of testing it becomes increasingly less cost effective to test this test case manually.


Awesome, obviously automation is the best thing ever invented since sliced bread, and Bob should always do automation, right? That answer is not as straightforward as it seems; automation is only one of the tools available to QA. Automation is really good at objective results, such as is text displayed, is a button tappable, or did the app crash? It’s not as useful at more subjective tests like does the app look “good”, is the app hard to use, or is the app slow? Often times both manual and automated testing is the best “secret sauce” to a good test plan. Let Jenkins do what he is good at (repetitive yes/no results) and let Bob do what he is good at (representing the end user).


So why do I call automation the invisible feature? The reason is that automation is treated like a feature: Bob estimates the effort, builds the automation during his sprint, and verifies everything is working properly. Except this feature is never included in the final build (the end user never sees the automation). The user does however see the benefits of automation, such as a well rounded stable app, an app that works well on all phones, and an overall enjoyable user experience. Automation has even more use cases, like internationalization testing, fragmentation testing (testing on multiple types of devices), and load testing (acting like 100,000 or more users). Those topics I will save for next time, but I am always open to talking about them over a cup of coffee (you can treat me of course ^_^).


Image courtesy of Samuel Zeller for Unsplash.

Brett McGinnis

Brett McGinnis is the Director of Quality Assurance and is responsible for releasing products with the high quality our clients have come to expect. He applies his extensive background in applied computational and mathematical sciences to automate the systems which L4 Digital uses to test and review digital products and services prior to delivery.

Share this:

More Posts

Want Alerts When We Post New Stuff?
L4 Digital. All rights reserved. All wrongs reserved. © 2008-2017