Model-driven automated testing
Model-driven automated testing (Model-Based Testing, hereinafter we will abbreviate as MBT) is a software testing method that expresses the behavior of the system as one or more models, and then automatically generates and executes test cases from the models. The core idea of this approach is to shift the focus of the testing process from manually writing test cases to creating and maintaining models that describe the behavior of the system.
Advantages
Compared with ordinary tests, MBT-based tests have the following advantages:
| Benefits | Description |
|---|---|
| Coverage | A model is an abstract representation of system behavior, which can help testers better understand and analyze system functions and performance. Thereby improving test coverage. |
| Test Efficiency | By automatically generating test cases from the model, it reduces the workload of manually writing test cases and improves test efficiency. |
| Maintainability | Model-driven test cases are easy to maintain, because when the system changes, you only need to update the model instead of manually modifying a large number of tests Example. |
| Reusability | Models can be reused in multiple test projects, reducing duplication of work and improving test quality. |
operation steps
The MBT-based test mainly has the following steps:
-
Create a model:
First, a model that describes the behavior of the system needs to be constructed. This model is usually represented graphically, such as a state machine, Petri net, or flow chart. Vertices in the model represent states of the system, and edges represent transitions between states. -
Generate test cases:
By analyzing the model, test cases can be automatically generated. Test case generation algorithms can be selected according to different objectives (such as coverage, path length, etc.). Commonly used algorithms include random testing, covering all paths, covering all edges, etc. -
Execute the test:
Test the real system with automatically generated test cases. This step usually requires a test execution engine that can map operations in the model to operations in the actual system. For example, use Selenium WebDriver for web application testing. -
Validation results:
Compare the behavior of the actual system with the expected behavior of the model to determine whether the system meets the requirements. If problems are found, the model can be updated and test cases regenerated.
Let’s take a look at the application of the AltWalker library for model-based automated testing in actual business.
What is AltWalker?
AltWalker is a graphical model-based automated testing framework for writing, executing and managing model-based tests. It is mainly used for testing complex systems such as web applications, mobile applications, etc. It supports running test model cases written in .NET/C# and Python3.
Install Altwalker
Enter the following command at the command line to install AltWalker:
pip install altwalker
Check if the correct version is installed
$ altwalker --version AltWalker, version 0.3.1
Small test
Create a test project
Create a new folder for our test project. In this folder, execute the following command to create MBT:
altwalker init -l python test-project
After executing this command, a directory test-project containing model template code will be generated in the current directory. The directory structure is as follows:
# tree
test-project/
├── models/
│ ├── default.json
└── tests/
├── __init__.py
└── test.py
Run Test
You only need to execute the following command to run the model defined in default.json and cover the link of the model. When the link is executed, it will run the corresponding test.py Code in:
$ cd test-project $ altwalker online tests -m models/default.json "random(edge_coverage(100))"
Operation effect
Write your own graph model
The above is just a single link Demo, let’s write our own model together.
For the model editor, we can choose to use it online, or use the plug-in of vscode, of course, we can also choose to build one ourselves.
Online Model Editor
We can use the online model editor to write the model.
VScode extension
You can search for AltWalker Model Visualizer on the extension
, you can also visit the link to download the extension yourself to edit and view the model.
Local deployment
You can refer to: GitHub – altwalker/model-editor: A web based editor and visualizer for models written using the GraphWalker JSON format.
Model writing including login, product selection, payment, and logout
Click to view the corresponding model
model effect
1. Verification model
Save this model to the newly created project default.json, and then execute the following command to check whether there is any problem with the model:
altwalker check -m models/default.json "random(never)"
2. Verify whether the test code matches the model
altwalker check -m models/default.json "random(never)"
Because we haven’t written the test code yet, the following error will appear:
You can see that the suggested code is given in the above error report, and we copy it into test.py.
class PayModel:
def state_pay(self):
pass
def action_init(self):
pass
def state_select_product(self):
pass
def state_logout(self):
pass
...
3. Run test
Execute the following command on the command line to run the test:
altwalker online tests -m models/default.json "random(edge_coverage(100))" --report-xml-file report.xml
This will run all test cases defined in the default.json and test.py files.
4. View test report
After the test execution is complete, AltWalker will generate a test report named report.xml in JUnit format. We can use any test reporting tool that supports the JUnit format to view this report.
There are other ways to provide test reports, you can view the official documents.
A more complex example
If we have a mall that needs verification, including login, user personal center, product home page, product detail page, payment interface, order interface, etc., we want to automate such a website, and there may be the following scenarios:
| Scenario |
|---|
| Login interface -> Do: enter password to log in -> user home page -> Do : View product A -> A product details page -> Do: Click the second product button -> Product C details page -> Do: Click payment -> Payment interface -> Do: Cancel payment -> Order countdown interface -> Do :Close the browser -> Close the browser |
| Login interface -> Do: Enter the password to log in -> User home page -> Do: View product A -> A product details page- > Do: Click the second product button -> Product C details page -> Do: Click payment -> Payment interface -> Do: View order -> History order interface -> Do: Close browser -> Browser close |
| Login interface -> Do: Enter password to log in -> User home page -> Do: View product A -> A product details page -> Do: Click the second product button -> Product C details page -> Do: click payment -> payment interface -> Do: view order -> history order interface -> Do: view order details -> order details interface -> Do: close browser -> browser close |
| Login interface -> Do: Enter password to log in -> User homepage -> Do: View product A -> A product details page -> Do: Click the second product button- > Product C details page -> Do: Click User Center -> User Center Interface -> Do: View Order -> History Order Interface -> Do: Close Browser -> Browser Close |
| Login interface -> Do: Enter password to log in -> User homepage -> Do: View product A -> A product details page -> Do: Click the second product button -> Product C details page -> Do: Click User Center -> User Center Interface -> Do: View Orders -> Historical Order Interface -> Do: View Order Details -> Order Details Interface -> Do: Close Browser -> Browser Close |
| Login interface -> Do: Enter password to log in -> User home page -> Do: View product A -> A product details page -> Do: Click similar product B -> Product B details page -> Do :Click to buy -> payment interface -> Do: cancel payment -> order countdown interface -> Do: close browser -> browser close |
| login interface -> Do: Enter password to log in -> user home page -> Do: view product A -> A product details page -> Do: click similar product B -> product B details page -> Do: click buy -> payment interface -> Do: view order -> History order interface -> Do: Close browser -> Browser close |
| Login interface -> Do: Enter password to log in -> User home page -> Do: View products A -> Product details page of A -> Do: Click on similar product B -> Product B details page -> Do: Click to buy -> Payment interface -> Do: View order -> Historical order interface -> Do: View order details – > Order details interface-> Do: Close browser-> Browser close |
| … |
Use MBT icons to show (arrows for actions)
Everyone should be able to find that the pictures drawn by using altwalker can very intuitively show the operations that can be performed between each page, and it is easy to expand, which is very consistent with what we said when we were doing page automation The POM (Page Object Model).
This is where the most important value of altwalker lies: Suitable for testing complex systems, such as web applications, mobile applications, etc.
Click to view the corresponding model
For a complete method of using POM, you can view the official examples: GitHub – altwalker/altwalker-examples: A growing list of examples that use AltWalker.
Use in combination with existing framework
If we only want to use the link organization capability in the model, we can also use the following code to generate the corresponding link according to the written model, and then organize the use case scenarios, which can be used for automatic use case generation.
graph_data = {
"name": "Default Models",
"models": [
{
"name": "DefaultModel",
"generator": "random(never)",
"startElementId": "v0",
"vertices": [
{
"id": "v0",
"name": "Login Screen"
...
]
}
model = graph_data['models'][0]
edges = model['edges']
vertices = model['vertices']
start_vertex_id = model['startElementId']
# build graph
graph = {}
for edge in edges:
source = edge['sourceVertexId']
target = edge['targetVertexId']
if source not in graph:
graph[source] = []
graph[source].append((target, edge['name']))
# Mapping of vertex IDs to vertex names
vertex_name_map = {vertex['id']: vertex['name'] for vertex in vertices}
# Depth-first search
def dfs(_graph, start_vertex, _path, _visited):
_visited. add(start_vertex)
_path.append(vertex_name_map[start_vertex])
if start_vertex not in_graph:
print(" -> ". join(_path))
else:
for neighbor, action in _graph[start_vertex]:
if neighbor not in _visited:
_path.append(f"Do:{action}")
dfs(_graph, neighbor, _path, _visited)
_path. pop()
_path. pop()
_visited. remove(start_vertex)
# Print all possible links and their operations, preceded by "Do"
visited = set()
path = []
dfs(graph, start_vertex_id, path, visited)
Summary
Through the above steps, we learned how to use AltWalker for model-driven automated testing. AltWalker is a powerful testing framework that can help us write, execute and manage test cases more efficiently.
Of course, model-based testing also has some limitations, such as the accuracy and completeness of the model have a greater impact on test results, and additional costs may be required for model construction and maintenance. Therefore, in practical applications, it is necessary to decide whether to adopt the model-based testing method according to the specific needs of the project. Hope this article helps you!
Finally: The complete software testing video tutorial below has been organized and uploaded, and friends who need it can get it by themselves 【Guarantee 100% free】
Software Testing Interview Document
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Ali, Tencent, and Byte, and some Byte bosses have given authoritative answers. Finish this set The interview materials believe that everyone can find a satisfactory job.
