How to create integration or regression web UI tests on a complex application? How difficult is it to automate all the expected results/behaviors?
Not everything can be tested automatically, that’s for sure, especially if we are talking about complex application with multiple different databases RDBMS and No SQL, background tasks and complex file repositories containing different file types. As you can see there is a lot that should be taken in mind and the answer is not a simple one…
In order to understand the concept let’s think of a movie that you can pause, take a look on a specific frame and see if the actor will actually make it to the next scene or even rewind it in order to review a missing dialogue. The concept is more or less the same. Having in mind that the application is working in a deterministic way and using the same test data every time you can rerun the tests like rewinding the movie, “by pressing this button will I be navigated on the next page? Let’s run the next test case…” and so on.
Now let’s see how something like this can work in a maintainable way. What we need is to setup all the available scenarios on a data level, so that they can be projected on the application’s UI. In order to accomplish that, all repositories (DB & file) should be backed up. Every test run should take place after the re-initialization (restore) of the backup data. The concept is that we can create and store the data of every test case, that we would like to test and keep them on backups. Think of it as pausing the application by keeping it as it is and restoring it when the automated testing is about to run. Every test case has depending information that is isolated and is not modified by another test case. In that way even the most complex scenarios could be replicated with a minimum effort and with less complexity as possible.
But how to add new automated tests and build upon the existing information?
Build upon the existing data for every new test case can be tricky, but if there are information maps and by that I mean mapping all the data with test cases using name conventions and also keep actual maps similar to the mind mapping graphs that for example will help defining the affected data after a test case refactoring. Track changes on data by adding comments and version in a similar way as you would do on source control. Also keep the data in one place like a zip file in order to apply versioning in a simple way.
In order to succeed, you will definitely need to create a backup – restore tool that could handle any available type of data repository. Before running the tests, a restore action should run in order to ensure the required data integrity by restoring them every time.
It is also worth to mentioning that background tasks do not have a place in such a scenario. The basic concept is that the application is deterministic and we cannot not allow background processes to manipulating the data that are being restored and are intended to test specific scenarios.