Layered automated testing
5 to 10 years ago, the automated testing we came into contact with was more focused on automated testing of the UI layer. Mercury’s WinRunner/QTP was a typical representative of the commercial automated testing products of that era. At that time, everyone simply wanted to use a Automated tools replace human clicks, and commercialization or privatization frameworks become popular.
Layered automated testing advocates that automated testing is required at different stages (levels) of the product. In “Google Software Testing”, 70% of Google’s investment is unit testing (small testing), 20% is interface/integration testing (medium testing), and 10% is automated testing of the UI layer (large testing). It is the familiar pyramid model. The higher you go up, the more difficult it is to implement automation, and the lower the return on investment. (It needs to be emphasized that automated testing at the UI layer, as the test closest to user operations, still has its meaning and scenarios. ).
The significance of interface testing
Interface testing is to verify the interaction between two or more module applications (usually in the form of interfaces). The focus of testing is to check the exchange, transmission and control management process of data, including the number of times of processing.
The core strategy of interface testing is to ensure the accuracy and stability of the system as the core, and use continuous integration as a means to improve testing efficiency, enhance user experience, and reduce product development costs.
Interface testing should protect the writing of code, enhance the confidence of developers and testers, expose hidden bugs in advance, allow developers to fix bugs as soon as possible, and allow business testers to be more comfortable during testing. , to minimize the number of underlying BUGs, to make the product development process more agile, to shorten the product development cycle, and finally, after the product is launched, to make users use it more smoothly and to make users feel that the product and service have zero defects.
Different from unit testing, interface testing is essentially a black box test, so it is very suitable for full-time test engineers to participate and cover.
Now I have also found a lot of test friends and created a communication group to share technology, sharing a lot of technical documents and video tutorials we collected. If you don’t want to experience the feeling of not being able to find resources when studying on your own, having no one to answer your questions, and persisting for a few days before giving up. You can join us to communicate. And there are many technical experts who have made certain achievements in automation, performance, security, test development, etc. Share their experience, and also share many live lectures and technical salons You can learn for free! Focus on it! Open source! ! ! QQ group number: 110685036 [password: csdn999]
Interface testing framework selection
- Currently, the most common method for selecting an interface testing framework is to use UI-based interface testing frameworks such as JMeter, soapUI, postman, and robotframework.
The advantage is that business testers do not need to write test code or seldom write test code, and the entry barrier is low. In the past few years, many companies have developed similar testing frameworks, with front-end and back-end, and are maintained by full-time test developers. Business testers only You need to know how to operate without participating in specific coding.
This method seems very high-end, but the actual problem is that the main work during the execution process becomes the maintenance of the test framework, which relies heavily on the design and development capabilities of full-time test developers. Every time a new interface protocol is added (such as Dubbo, Hessian or internally customized protocols), it is necessary to add support to the framework; more fatally, once the core test developers flow, it will easily cause the collapse of the entire interface testing system; in addition, it will also affect the growth of the skills of business testers. It’s not fair. I have interviewed too many engineers who can only use the XXX testing framework of a large company but have no understanding of the specific implementation.
It has been predicted in “The Way of Google Software Testing” that confidential and privatized basic testing facilities cannot obtain the imagined benefits. This approach means that it is expensive and slow, and it is difficult even between different projects within the company. Achieve reuse. The future testing infrastructure must be based on shared code and open source frameworks, and test developers need to make more use of and contribute to open source projects. I recently re-read this book that almost changed the software testing industry, and found that the predictions in it are so accurate. Of course, it can also be believed that the entire domestic industry is evolving in the same way as Google.
- Use Java interface frameworks such as Junit and Testng to directly write test code for testing, and at the same time establish basic libraries or methods to abstract some repetitive work.
Somewhat similar to unit testing, this method is easy to expand and flexible to implement. As a programmer, you can use code to implement flexible scene organization and functions. It only requires a little secondary development, but it requires test engineers to have a certain coding foundation.?
This method was still relatively difficult to implement in the past few years because there were very few test engineers who knew Java code in the market. However, today, when more and more development capabilities are required for test engineers, it is no longer as difficult as imagined. Difficulty, coding ability in languages such as Java/Python has also become a basic requirement when recruiting for our team.
In addition, the reason why Java is used here instead of other languages is mainly because the team’s technical reserve is Java. It has a rich open source test library, and the products of general Internet companies are basically developed using the Java framework, and the development team’s technology It is very necessary to keep the stack consistent.
GTest interface testing framework
Many companies have made various interface frameworks, all of which are designed and developed based on their own company’s business foundation. Based on our own business characteristics, we have also encapsulated our own interface testing framework gtest-framework, which is gradually being used in developers’ unit testing.
What gtest-framework does:
- Pre-data preparation and automatic cleaning.
- Implementation and encapsulation of common interface protocols.
- Dependency injection configuration support.
- Such as the integration of various common processing methods such as files, pictures, xml, characters, etc.
- Extensions of assertion methods, etc.
Key practices for interface testing
1. Data preparation
Data preparation for interface testing generally refers to data preparation for the database, and sometimes also includes data preparation for files and caches. The specific implementation can be considered from the following aspects:
(1) Prepare test data in a hard-coded way. When writing test code, insert whatever data is used. In order to avoid data duplication, many people are accustomed to using random characters or random numbers (this method may cause instability in test cases, so try to avoid it).
(2) Test data can be prepared directly by calling other APIs. This is more useful when testing the top-level services. For example, when testing the purchase of goods, you need to prepare the product data to be purchased and the user data of the purchased goods. At this time, , you can directly call the API to generate products and the API to generate users to directly generate test data. This method is simple to implement, but the premise is that the corresponding API needs to be available and the API function is correct.
(3) Use Excel or XML to prepare test data. This method of preparing test data is mainly aimed at the preparation of object data. For example, a piece of product data can correspond to a piece of data in Excel, because generally development uses pojo mapping, and when preparing When testing data, the setting of these pojo object properties is often repetitive and requires a lot of work. Preparing it in Excel or XML can reduce the need to repeatedly prepare this data in the code.
Generally, we use two methods: 2/3. Among them, 3 is a method that mainly uses the features of Dbunit, Spring-test, Unitils and other test frameworks to add custom annotations through secondary development, and can easily import files in Excel or XML format. And automatically roll back the data after the test is completed.
/**
* @ClassName: TestJdbcDataSet
* @Description: Use custom TestDataSet annotation to prepare test data, recommended.
* @author Cay.Jiang
* @date July 10, 2017 9:10:29 AM
*
*/
public class TestJdbcDataSet extends BaseCase{
Map<String, Object> args = new HashMap<String, Object>();
@Test
@TestDataSet(locations={"/tmp/domaininfo.xls"},dsNames={"mysqlDataSource"})
public void test01_mysql(){
args.put("selfdomain", "baidupc2");
List<Map<String, Object>> result=JdbcUtil.queryData(mysqlJdbcTemplate, "domaininfo", args);
System.out.println(result);
assertEquals(" Partner access name ",result.get(0).get("remark"));
}
}
For /tmp/domaininfo.xls in the above code, see: domaininfo.xls, where the Excel format uses Sheet as the table name, the first line defines the field name, and the remaining lines are the corresponding data.
Multiple data sets:
@TestDataDataSet(locations={“Data1.xls”,”Data2.xls”},dsNames={“dsNameA”,”dsNameB”}), the data of Data1.xls will be inserted into the database pointed by dsNameA, and the data of Data2.xls The data will be inserted into the database pointed to by dsNameB
2. Assertion
Common assertion methods include Assert and Hamcrest that come with JUnit. The assertion method that comes with JUnit has very limited functions and can only meet the most basic needs. Hamcrest is relatively feature-rich, but the library has not been updated for many years. And Hamcrest, like JUnit’s own assertion method, has a fatal flaw, that is, when there are multiple assertions in a case, if one of the assertions fails, the assertions after it will not be executed. Here I recommend a new assertion artifact AssertJ to everyone.
AseertJ: Known as streaming assertion. What is streaming? A common assertion statement can only assert one checkpoint on the actual value. AseertJ supports one assertion statement on multiple checkpoints on the actual value at the same time. This makes the assertion statement more concise and suitable for reading. AseertJ also supports executing all assertions at once and then collecting all failed assertions and feeding them back together. Of course, AseertJ also has many other features. You can refer to the official documentation to explore them slowly. The following are examples to illustrate the advantages of AseertJ:
public class TestCase extends BaseCase{
UserProfileBO user = new UserProfileBO();
@Before
public void init(){
user.setAddress(" Hangzhou ");
user.setMobile("1386800000");
user.setUserName("Test Account");
}
/*
* JUnit built-in assertions
*
* 1. After one of the assertions fails, all subsequent assertions will not be executed.
* 2. Fewer assertion methods are supported
*
*/
@Test
public void testAssertJUnit(){
Assert.assertEquals(" address ",user.getAddress()," Ningbo ");
Assert.assertEquals("Mobile phone",user.getMobile(),"13868000000");
Assert.assertEquals("Mobile phone",user.getUserName(),"Test");
Assert.assertNotNull(user.getMobile());
Assert.assertTrue(user.getMobile().startsWith("138"));
Assert.assertTrue(user.getMobile().length() == 11);
}
/*
* Hamcres assertion
*
* 1. After one of the assertions fails, all subsequent assertions will not be executed.
* 2. The supported assertion methods are rich, but they have not been updated for many years.
*
*/
@Test
public void testHamcrestMatchers() {
MatcherAssert.assertThat(user.getAddress(), equalTo(" Ningbo "));
MatcherAssert.assertThat(user.getMobile(), equalTo("13868000000"));
MatcherAssert.assertThat(user.getUserName(), equalTo(" test "));
MatcherAssert.assertThat(user.getMobile(), allOf(is(nullValue()),startsWith("136")));
}
/*
* AssertJ assertion
*
* 1. Supports unified feedback of failed assertions after all assertions are executed.
* 2. Supports a wide range of assertion methods.
* 3. Supports streaming assertions for easy reading.
*
*/
@Test
public void testAssertJ(){
// Assertion collection, after all assertions are executed, failed assertions will be uniformly fed back.
SoftAssertions.assertSoftly(softly -> {
softly.assertThat(user.getAddress().equals(" Ningbo "));
softly.assertThat(user.getMobile().equals("13868000000"));
softly.assertThat(user.getUserName().equals(" test "));
});
// Streaming assertion
Assertions.assertThat(user.getMobile())
.isNotNull()
.startsWith("136")
.hasSize(11);
}
}
3.jenkins integrated interface test
(1) Set the warehouse address and identity information of the test code
(2) Set maven running parameters
If you want to execute some interface use cases, you can execute the specified case through -Dtest=XXX (test class name). Multiple class names are separated by commas “,”
(3) Set up the Job execution mechanism. The figure below shows regular execution every day.
4.pipeline parameter injection
As mentioned before when writing the pipeline, our interface test code needs to support external parameter injection, such as the test service address. Different branch codes may be deployed on different test servers, and need to be driven by parameterization in the pipeline. Different servers are used for interface testing.
Here we can use maven’s -D (Properties attribute) to achieve this, for example:
(1)dubbo uses the properties configuration file, but the specific parameters are packaged and replaced using ${key} placeholders.
(2) Specify the parameter values in the corresponding configuration file in the maven pom file (the parameter values specified here will be overwritten by the parameter values passed through maven -D)
Note here: You also need to start the filter filter of resources
(3) Use the maven command line to set property values
And parameterize the value to support pipeline parameter passing
5.Pipeline code implementation
stage('Interface automated testing') {
steps{
echo "starting interfaceTest......"
script {
// To ensure that jetty startup is completed, a judgment is added to ensure that the jetty server is accessible before executing the interface layer test.
timeout(5) {
waitUntil {
try {
// Ensure that the jetty service port is started successfully
sh "nc -z ${serverIP} ${jettyPort}"
//sh "wget -q http://${serverIP}:${jettyPort} -O /dev/null"
return true
} catch (exception) {
return false
}
}
}
// Pass the parameters IP and Port into the job of the interface test. You need to ensure that the job parameters of the interface test can be injected.
build job: ITEST_JOBNAME, parameters: [string(name: "dubbourl", value: "${serverIP}:${params.dubboPort}")]
}
}
}
Test code specifications (for reference only)
- Test project naming convention
Interface test:
Generally, an independent test project is required. The naming rule of the test project is: “test-” + the name of the project being tested, such as test-kano.
unit test:
There is no need to rebuild an independent test project, just put it in the same project as the development code.
- Test directory definition specification
The test code is uniformly placed under “src/test/java” of the test project.
Test configuration files are uniformly placed under “src/test/resources”.
- Package name definition specification
The same as the package name in the project being tested.
- Test class naming convention
The naming rules for test classes are: start with Test and end with the name of the object it is to test, for example
Test + the business being tested, Test + the interface being tested, Test + the class being tested
Another way is to end with Test and start with the name of the object it is testing, e.g.
Tested business + Test, tested interface + Test, tested class + Test
Depending on personal habits, in order to facilitate case positioning, the current testing team generally uses the first method.
- Test case naming convention
The naming rule of test cases is: test + test case operation _ conditional state, uniformly use lowerCamelCase style, and must follow camel case.
The word convention is the same as the test class naming
- Common constraints on interface test code
(1) Data cleaning and structure
- @BeforeClass @Before does data preparation and other related operations: before loading the test class, you need to load the scenario data common to all test cases, and at the same time, load special test data when running a single test case.
- @AfterClass @After does test data cleaning and other related operations: clean up the use case site after executing relevant tests.
(2) Assertion
- Don’t make unnecessary assertions
In test mode, it is sometimes tempting to overuse assertions. This practice makes maintenance more difficult and should be avoided at all costs. Explicitly test only the features indicated by the test method names, since keeping test code to a minimum is an important goal for generic code. - Use explicit assertions
You should always use assertEquals(a, b) in preference to assertTrue(a == b) because the former will give more meaningful test failure information. This rule is particularly important when the input values are not known in advance. - The order of parameters of assertions must be appropriate
(3) Test cases remain independent
- Make sure the test code is independent of the project code.
- In order to ensure that the test is stable, reliable and easy to maintain, test cases must not depend on each other, nor can they depend on the order of execution.
(4) Test code should consider error handling
- If the previous code fails to execute, subsequent statements will cause the code to crash and the remaining tests will not be executed. Be prepared for test failures at all times to avoid a single failed test item interrupting the execution of the entire test suite.
- Don’t write your own catch code block, that is, only if the test fails, there should be no catch situations.
Conclusion
Interface testing is a very complex system that is difficult to explain clearly in one article. Other practices include interface test coverage statistics, standard protocol page testing platform (for products or testers without coding capabilities), and maven project skeleton. The establishment of standardized testing projects, the specific implementation of interface protocols such as dubbo/hessian/restful/webservice, etc. are not explained in detail due to space limitations. Looking forward to discussing it with everyone.
Finally, I would like to thank everyone who has read my article carefully. Looking at the increase in fans and attention, there is always some courtesy. Although it is not a very valuable thing, if you can use it, you can take it directly!
Software testing interview documents
We must study to find a high-paying job. The following interview questions are from the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
