Obstacles of moving from manual to automation testing
- Too busy running manual tests
- Put automation off until the “next release.”
- Company culture is not open to change
- Company is not sure about the investment
- Not enough knowledge or experience to implement
- Not sure how or where to start
Steps to go from manual to automation testing
Step 1: Build a good test case foundation
- Why create a good test case foundation?
Good manual test cases are blueprints for what you will and will not be able to automate. Explore automation framework options - Aim for goodness, not perfection
- Test cases will evolve and change as you dig in deeper and learn more
- Pick a tool to store and organize your manual test case. This can be as simple as you need
Common test case storage options:
- Option 1: Documentation only examples
– Excel sheet (Microsoft, google docs)
– Wiki page - Option 2: Software tool examples
– Testlink – open source – https://testinglink.org/
– Testrail – server and cloud offerings – https://www.gurock.com/testrail/
Manual test case organization considerations:
- The option should include tracking which tests can/should be automated and what has been automated.
- Are you planning on storing test results and/or test plans with your manual test cases?
- Create a wish list (need to have versus nice to have)
Tips: you can start simple and then import your list into a more complex tool if needed or wanted
Test case management tool wish list example:
| Must-Have | Nice to Have |
| – Hosted off-site (cloud)
– Test plan management- The ability to export test cases, results, and plans – Versioning for test cases |
– Variety of export format options (for test summaries)
– Integrates with the defect tracking system (e.g., Assembla, Jira) – Integrates with Slack – The ability to execute automated tests from the tools |
Step 2: Explore automation framework options
- By framework, I mean what your tests are coded in that will run the actual tests. For example, the framework is what will actually send the test to the Selenium Server.
- What if you don’t know what framework are available?
If you don’t know, ask resources around you including: Developing team members, QA folks in professional organizations, QA/ Tech mailing lists or Forums, Slack groups, Twitter experts - Narrow down your list to about three options and create a matrix that compares each option. This could include building your own.
Framework options and examples:
- If you are using Selenium, you have some great options: Java, Robot, Python, Ruby, Cucumber (and many more). This can be hosted or self-managed.
- Many Selenium hosting providers give you “wizards” that will help you set up the framework environments that will work with the solution.
Step 3: Explore infrastructure options
- What hardware is needed?
– Servers: physical, self-managed virtual machines, cloud hosted
– How will the hardware be installed? maintained?
– Backup strategy - What software is needed?
– Buy versus Build versus Hosted
– Versioning strategy for the manual and automated test cases - Will the automated test cases exist in the same repository as the project code?
Step 4: Make the first choice:
- After you have proved to yourself that your approach will work, time to get the team to buy in on your solution
- What is the quickest way to get to Proof of Concept (POC)?
– Develop a good plan and get moving
– The plan should include a REAL DATE for a POC
– Many times you can take advantage of trial membership for some testing products and tools. If you need the trial extended to reach out to the company and just ask. - Have at least one backup plan in case the first choice does not work out.
Step 5: Start simple
- What is the simplest way to implement your framework approach using your actual product?
Selenium example: Can you use Selenium IDE scripts to confirm assumptions about product test-ability?
Step 6: Confirm your choice works
- Go beyond the simple checks you have performed
- Confirm that your choice will work in your specific environment
- Many software providers will allow you to sign up for trial accounts so you can take a closer look.
Tips: there is no harm in asking for an extension on a trial account if you needed more time for analysis.
Step 7: POC for team buy-in
- Proof of Concept (POC) is a chance to share your results, recommendations, and excitement with the large team.
- Best to show a live demo if possible
- Have dates and requirements for implementation
- Helpful to have metrics or other supporting detail to exemplify why automation
Step 8: Implement
My automation implementation:
- Git: QA repository containing test cases, test queues, and vagrant/AWS configuration
- Jenkins: takes QA git repository and deploys on a Linux system based o config settings
- Linux system: launches vagrant/AWS system and executes Robot scripts.
- Testing bot: robot scripts are executed against the Selenium server(s) and the internal vagrant/AWS system.
- Slack: posts test results from the Linux system to the team. This is broken up between pass and failed test results with emojis.
Note: This video is originated from SeleniumConf UK