Performance testing uses automated testing tools to simulate a variety of normal, peak, and abnormal load conditions to test various performance indicators of the system.
My definition of performance testing clearly describes the methods and processes of performance testing:
Performance testing aims at system performance indicators, establishes performance testing models, formulates performance testing plans, develops monitoring strategies, executes performance scenarios under scenario conditions, analyzes and determines performance bottlenecks and tunes them, and finally obtains performance results to evaluate system performance indicators. Whether it meets the established value.
I combined it with my own practical experience and reorganized the common performance testing implementation process in the project.
0. Early preparation
Before officially starting performance testing, complete at least the following preparations:
1. System basic function verification
Performance testing is generally a testing activity after the software system has basically completed development or deployment. It requires that the object under test basically meets the functional requirements and has a certain degree of stability. It is necessary to ensure that at least one functional coverage test of the system is conducted before performance testing.
2. Preliminary business and architecture analysis
In order to establish an intuitive understanding and analysis of system performance, the more important and commonly used business scenario modules of the system should be analyzed in a targeted manner to prepare for the subsequent test plan design.
3. Form a performance team
Establish a performance testing team based on needs, which should at least include performance test design and execution personnel, development and operation and maintenance personnel.
1. Performance requirement analysis
Just like functional testing, we need to spend a certain amount of energy to figure out what the performance requirements are. In this step, we need to communicate with customers, project managers, products, developers and other related personnel, collect various data, analyze the system under test, confirm the performance test goals, determine the performance scenarios that need to be covered, and finally transform them For measurable performance indicators and monitoring strategies.
1.1 Analysis of performance testing purpose
I was once responsible for the performance testing of a new system before it went online. The customer’s performance requirements for this system were very clear: the interfaces on key business processes can reach 100TPS, the response time is within 200ms, and when all interfaces reach 100TPS Able to operate stably for 24 hours.
Obviously, I was lucky this time, but it’s hard to be lucky all the time. Many times when we make the decision to perform performance testing, the purpose is usually not so clear, and we need to continue to ask and dig out the purpose of performance testing. Testing purposes usually cannot escape the scope listed below:
- Test the maximum capacity of the system and feel confident after going online
- Comparing the new version with the old version, the performance does not decrease
- Verify that the system can operate stably under a certain load pressure
- System performance is tuned to an acceptable range
You can also use Application Area to more concisely summarize the purpose of performance testing:
- Capability verification: To verify the performance capabilities of the system in a given environment, it means only performing performance verification against the given indicators.
- Planning capabilities: Test the maximum capacity of the system, verify the system’s performance scalability, identify key points for system capability expansion, and give suggestions for improving its performance scalability.
- Performance tuning: Conduct comprehensive performance testing for a given system and improve system performance at the same time.
1.2 Business Model
In the preliminary preparation, we already have a basic understanding of the various businesses involved in the system. Generally speaking, performance testing will not be conducted on all functions of the system. It is essential to select key and sufficiently representative business functions and determine and establish the corresponding business model. There are differences in the logic, business volume, and system resources consumed by each business. For example, in an e-commerce system, under different promotion forms, users use different functions in completely different proportions, and the pressure on the system is naturally also different. Therefore, the business model directly affects our judgment of the system’s processing capabilities. If the selection of business and proportion in the business model is very different from production, it will directly lead to the test results having no reference value.
How to select representative businesses in the system? The rules generally followed are to select businesses with high business volume, frequently used by users, risky businesses, and businesses with a growth trend in the future. The online system can analyze the interface requests and proportions during the peak time period through the system logs, and then sort out the business types and business proportions based on the system business logic, so as to simulate the production business scenario more realistically; in different time periods When there are large differences in business types and business proportions, multiple representative business models need to be selected. For new systems that are not yet online, business types and business proportions can be determined based on the judgment of the business team and user research.
The general business model is as follows. Some businesses with a ratio of 0 are not listed in the table:
| Business name | Test scenario business ratio |
|---|---|
| Business 1 | 2.56% |
| Business 2 | 20.34% |
| Business 5 | 40.25% |
| Business 7 | 36.85% |
1.3 Performance Goals
For different performance testing application fields, the performance goals will also be different. The following is a similar description of the performance goals in each field:
- Proficiency verification area: The application can handle 200 concurrent users’ access to user A with a maximum response time of 1 second; there are 400 users at the peak time, and the allowed response time is extended to 2 seconds;
- Planning capability area: The daily business throughput of business A of the system will reach 4,000 in the next 3 months. Find out the performance bottleneck of the system and give suggestions to support this business volume;
- Performance tuning area: Through performance tuning testing, the response time of business A and business B of this system was improved to 1 second under the condition of 200 concurrent users.
In these performance target descriptions, performance test indicators such as response time, number of concurrent users, and throughput of specific system services are clearly defined. Of course, the definition of system resources at this time should also be added, such as: This application Able to handle 200 concurrent users’ access to user A with a maximum response time of 1 second. At this time, the server’s CPU usage does not exceed 70% and the memory usage does not exceed 80%. There are 400 users at the peak time, and the allowed response time is extended to 2 seconds, at this time, the server’s CPU usage does not exceed 85%, and the memory usage does not exceed 90%.
Now I have also found a lot of test friends and created a communication group to share technology, sharing a lot of technical documents and video tutorials we collected. If you don’t want to experience the feeling of not being able to find resources when studying on your own, having no one to answer your questions, and persisting for a few days before giving up. You can join us to communicate. And there are many technical experts who have made certain achievements in automation, performance, security, test development, etc. Share their experience, and also share many live lectures and technical salons You can learn for free! Focus on it! Open source! ! ! QQ group number: 110685036 [password: csdn999]
In the article “Performance Test Indicators”, the commonly used business indicators and resource indicators in performance testing have been introduced in detail. When actually determining the performance test goals, you can select appropriate indicators from them and determine the specific indicators of each indicator according to business needs. After determining the value, you can finally get the following table:
2. Performance test design and preparation
2.1 Performance testing environment
Generally speaking, performance testing requires an independent test environment with the same configuration as the production environment. However, due to higher costs, equal ratio construction (1/2, 1/4, etc.) is usually used. However, due to the limitations of resource application, budget, and numerous environment construction steps, it may not be possible to have an independent test environment. At this time, you can consider using existing test environments or production environments for testing, and identify in advance the challenges faced by using these environments. risk. At the same time, you need to consider whether the business involving third-party services needs to use Mock instead. If not, you need to communicate with the person in charge of the third-party service in advance to schedule.
In a performance testing project I participated in, we finally adopted this environment strategy: To C business-related services use an independent, 1/2 ratio built performance testing environment, and To B business is in a functional testing environment conduct. In this way, although we finally have an environment for testing, the reference significance of the final test results is obviously reduced, and because the functional testing environment is also used for testing new functions, our performance testing and functional testing have serious conflicts. When the stress test started, the system couldn’t bear the large number of requests and crashed. In order not to affect the progress of the functional test, the execution of the performance test basically had to wait until the functional test executor got off work.
After the performance test environment plan is determined, the operation and maintenance colleagues can start deploying the performance environment. After the performance environment deployment is completed, the availability of the environment must be verified first.
2.2 Performance scenario design
Scene comes from the English scenario, which originally means: the actions of characters in film and television plots that take place in a specific time and space.
A more authentic description of the “scenario” in the performance scenario is: under the established environment (including dynamic expansion and other strategies), established data (including data changes during scenario execution), established execution strategy, and established monitoring, Execute performance scripts, observe changes in performance status parameters at each level of the system, and determine in real time whether the analysis scenario meets expectations.
Performance scenarios will not exceed the following categories:
During the actual test execution, we need to benchmark the businesses involved one by one to find out the maximum TPS and corresponding response time of each business in the system; next is the capacity performance scenario, based on the proportion of each business in the performance target , constantly increasing the pressure. During the pressurization process, you need to pay attention to always maintaining the business ratio of each business, and finally get the maximum TPS and response time of the mixed business. Of course, if there are multiple scenarios with different business ratios in your project, It also needs to be tested again; then there is the stability performance scenario, which requires the business and operation and maintenance departments to jointly give an indicator: the system must run stably for a week and support 20 million business volumes. Based on this, we can calculate the stability Duration in the performance scenario: 20 million? Based on the capacity TPS obtained in the capacity performance scenario (the sum of all business TPS); the last is the abnormal performance scenario, in which case 50% of the maximum TPS in the capacity performance scenario can be used. Do it, and then perform abnormal operations on the business server, database server, etc.
2.3 Performance test data
In performance testing, the data we want to pay attention to mainly include basic data and parameterized data.
- Basic laying data
In a production environment, the system will definitely not always be a newly deployed empty system. There will be a certain amount of data already stored in databases, file CDNs, Redis caches, etc. Therefore, before performance testing begins, we need to simulate the actual data volume of business operations in real scenarios.
2. Parameterized data
During the execution of performance scenarios, we will use two types of data: unique data and reusable data. The amount of parameterized data depends on the scene run time and TPS.
For unique data, such as data required for deletion operations, it cannot be used again after being deleted. The calculation method for the required amount of data is very simple. For example, in a scenario that runs for 20 minutes, if the TPS is 100, it will require at least 20min * 60s * 100 = 120,000 pieces of data.
For reusable data, such as querying a single record, repeatedly using the same parameter (ID) for query is obviously inconsistent with the real scenario. There is no fixed limit on the number of items in this scenario, and it can only be judged based on the actual business. Of course, you can also calculate it according to the data volume calculation method of unique data, and then use it as needed in actual execution.
In addition to the amount of data that can satisfy the operation of performance scenarios, it must also meet the data distribution in the production environment. For example, in accounting applications, the number of accounts users record every day is usually between 5 and 20. The amount of data in the thousands or even tens of thousands is definitely unreasonable.
So how to put the designed parameterized data into the system? Parameterized data can be divided into two categories: the first category is already existing in the background database, and the second category is that the data will be inserted into the database only after the script is executed successfully;
For the first type of data, taking user login as an example, the login interface needs to compare the parameters with the data in the background database. This requires that the data must first exist in the database, and the pressure tool can also know the specific data values. In actual projects, multiple database tables are usually involved, and the relationships between the tables must be sorted out first.
For the second type of data, taking the note-taking function of an accounting application as an example, you only need to parameterize the corresponding data in the pressure tool.
2.4 Monitoring design
In performance testing, monitoring is a very important link, and monitoring is the prerequisite for performance analysis.
First, we need to analyze the system architecture. After knowing the components and services used in the architecture, we can then find corresponding monitoring methods and methods in a targeted manner. Specifically, when designing, you can use the idea of from global to directional, list the counters to be used, and form an evidence chain for analysis.
As for what monitoring tool to use to obtain the values of these counters, it is recommended to use the idea of Prometheus + Exporter. The specific plan still needs to be further confirmed with operation and maintenance colleagues, who will implement the deployment of monitoring tools.
Performance monitoring is a big topic. To do it well still requires a lot of effort to learn and practice. This article can only be used as an introduction, and everyone still needs to study it carefully on their own.
2.5 Performance test script
First of all, we must determine the performance testing tools, such as Jmeter, Gatling, Locust, etc. There is already a lot of useful information on the Internet about the comparison and specific usage tutorials of these tools. This article will not introduce the use of specific tools.
After selecting the tool, you can start writing the corresponding script and debugging it. I will publish a separate article as an example for this part.
There is another step that needs to be emphasized here. When executing the performance script, you need an execution machine. The specific number of machines needed needs to be judged according to your own actual situation. The execution machine usually needs to be applied for in advance, and the one with higher configuration is better.
After the performance test design is completed, you can start writing the “Performance Test Plan” document.
3. Performance test execution
With the previous design and preparation work in place, we can finally start executing the performance scenario.
There is not much to say in this part. You just need to pay attention to whether the business indicators and monitored resource indicators during the execution process are recorded.
4. Performance test result analysis and tuning
After each performance scenario is executed, the corresponding results must be sorted and output from the pressure tools and monitoring tools, usually in the form of tables and charts. Then based on these results, it is judged whether the test passed or not. If the performance is not up to standard, it is necessary to further analyze the root cause of the performance. However, performance problems are usually related to each other and influence each other. The phenomenon seen on the surface is probably not the fundamental problem, but caused by another problem. This requires that the evidence chain when monitoring and collecting data must be complete, which can help us find the truth step by step.
Tuning may be a long process that requires us to keep trying, be patient and confident.
The points that affect performance and the tuning methods are as follows, for reference only:
- Database: Check index settings and check whether SQL statements can be optimized
- Business interface: whether the process is redundant; whether the returned data is redundant; whether the data processing logic can be optimized
5. Performance test report
Finally, the performance test report can be output, clarifying the performance test goals, test environment, data construction rules, performance results, risks, etc.
Finally, I would like to thank everyone who has read my article carefully. Looking at the increase in fans and attention, there is always some courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly!
Software testing interview document
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Python entry skill treeHomepageOverview 387715 people are learning the system