Some checklists for Automation Testing

By analyzing the current test case bucket in this manner you can quickly determine which test cases can be automated immediately and the priority for automating them that will give the best return on investment. Use the following list of questions to determine if you want to go for the automation testing for your application:

1. The expected results of a single function are usually very predictable.
2. The steps in the test are very repeatable and usually short.
3. API testing consists of using supported programming languages to drive product functions. Many times these tests are based upon the unit tests generated by development.
4. Command line testing uses higher level scripting languages such as Perl, REXX, DOS Batch Files, Shell Scripts, etc.
5. These automated tests usually execute very rapidly.
6. Do any of the current test cases drive function through a programmable interface?
7. Are any of these interfaces frozen or at least stabilized?
8. Which test cases are executed most often?
9. Which test cases take the most time or effort to execute?
10. Which test cases have expected results that are known or at least predictable?
11. Which test cases seem to fail most often because of product defects?
12. Which test cases have to updated or rewritten most often.

Some other factors to consider:

Prerequisites and Dependencies

1. Test plan in place that includes resources and time to automate test case execution.
2. Test cases written and validated via manual test execution.
3. Might require development to put into the product some sort of testability "hooks" or to expose some API's or command line interface.
4. Develop a Test Execution Automation Strategy.
5. Design, develop, implement and deploy an automated test execution system.
6. Maintain the test execution automation system from release to release.
7. Continually expand automated test execution into more advanced areas of testing.

Skills Required

1. Basic programming skill in the language(s) of choice.
2. Knowledge of the tool/harness used to drive the automated suite of tests.

Cost of Implementation

1. Education and development of required scripting/programming skill.
2. Cost of test tool/harness license and maintenance license if applicable.
3. Education and development of test tool/harness usage.
4. Effort to automate the execution of desired test cases.
5. Effort to run automated system and analyze the results.
6. Cost to maintain the automated system.

Type of Automation Testing:

1. Scenario based tests driven through a GUI and/or Web Interfaces.
2. Error or negative path testing.
3. Self-Checking tests, ie those that verify the final results against the expected results.
4. Load/Stress/Performance testing.
5. System monitoring testing, such as memory leakage detection, resource usage monitoring, I/O, core dumps, etc.