python+requests+pytest+allure automation framework

1. Core library

1. requests request request
2. openpyxl excel file operation
3. loggin log
4. smtplib sends email
5. configparser
6. unittest.mock mock service

2. Directory structure

• base
• utils
• testDatas
• conf
• testCases
• testReport
• logs
• other

2.1base

• base_path.py stores the absolute path to prevent errors when executing dos commands or Jenkins.
• base_requests.py encapsulates requests, selects different methods to execute scripts according to method, and handles request exceptions at the same time

2.1.1 base_path.py

import os

#Project root path
_root_path = os.path.split(os.path.split(os.path.realpath(__file__))[0])[0]

# Report path
report_path = os.path.join(_root_path, 'testReport', 'report.html')

# Log path
log_path = os.path.join(_root_path, 'logs/')

#Configuration file path
conf_path = os.path.join(_root_path, 'conf', 'auto_test.conf')

# Test data path
testdatas_path = os.path.join(_root_path, 'testDatas')

# allure related configuration
_result_path = os.path.join(_root_path, 'testReport', 'result')
_allure_html_path = os.path.join(_root_path, 'testReport', 'allure_html')
allure_command = 'allure generate {} -o {} --clean'.format(_result_path, _allure_html_path)

2.1.2 base_requests.py

import json
import allure
import urllib3
import requests
import warnings
from bs4 import BeautifulSoup
from base.base_path import *
from requests.adapters import HTTPAdapter
from utils.handle_logger import logger
from utils.handle_config import handle_config as hc


class BaseRequests:

    def __init__(self, case, proxies=None, headers=None, cookies=None, timeout=15, max_retries=3):
        '''
        :param case: test case
        :param proxies: The result is displayed in fiddler:
        {"http": "http://127.0.0.1:8888", "https": "https://127.0.0.1:8888"}
        :param headers: request headers
        :param cookies: cookies
        :param timeout: Request default timeout 15s
        :param max_retries: Retry 3 times by default after request times out
        '''
        self.case = case
        self.proxies = proxies
        self.headers = headers
        self.cookies = cookies
        self.timeout = timeout
        self.max_retries = max_retries
        self.base_url = hc.operation_config(conf_path, 'BASEURL', 'base_url')

    def get_response(self):
        '''Get request results'''
        response = self._run_main()
        return response

    def _run_main(self):
        '''send request'''
        method = self.case['method']
        url = self.base_url + self.case['url']
        if self.case['parameter']:
            data = eval(self.case['parameter'])
        else:
            data=None

        s = requests.session()
        s.mount('http://', HTTPAdapter(max_retries=self.max_retries))
        s.mount('https://', HTTPAdapter(max_retries=self.max_retries))
        urllib3.disable_warnings() # Ignore browser authentication (https authentication) warnings
        warnings.simplefilter('ignore', ResourceWarning) # Ignore ResourceWarning warnings

        res=''
        if method.upper() == 'POST':
            try:
                res = s.request(method='post', url=url, data=data, verify=False, proxies=self.proxies, headers=self.headers, cookies=self.cookies, timeout=self.timeout)
            except Exception as e:
                logger.error('POST request error, error message: {0}'.format(e))

        elif method.upper() == 'GET':
            try:
                res = s.request(method='get', url=url, params=data, verify=False, proxies=self.proxies, headers=self.headers, cookies=self.cookies, timeout=self.timeout)
            except Exception as e:
                    logger.error('GET request error, error message: {0}'.format(e))
        else:
            raise ValueError('method is get and post')
        logger.info(f'Request method: {method}, request path: {url}, request parameters: {data}, request headers: {self.headers}, cookies: {self.cookies}')

        # with allure.step('Interface request information:'):
        # allure.attach(f'Request method: {method}, request path: {url}, request parameters: {data}, request headers: {headers}')

        # Extension: Is full contract verification required? How to handle the response when the response results are of different types?
        return res


if __name__ == '__main__':
    # case = {'method': 'get', 'url': '/article/top/json', 'parameter': ''}
    case = {'method': 'post', 'url': '/user/login', 'parameter': '{"username": "xbc", "password": "123456"}'}
    response = BaseRequests(case).get_response()
    print(response.json())

2.2 utils

(Only take the core part)

• handle_excel.py
  – Excel operation, framework requirements, the final read data needs to be saved in the format of list nested dictionary [{}, {}]
  – Other operations
• handle_sendEmail.py
  – python uses the smtp protocol to send emails and pop3 to receive emails.
  – You need to enable the pop3 service function. The password here is the authorization code. Enable the service by yourself.
• handle_logger.py log processing
• handle_config.py
  Configuration file processing, here only the domain name is configurable, you can change the domain name when switching environments
• handle_allure.py
  The report generated by allure needs to be opened by calling the command line. The command is directly encapsulated here.
• handle_cookies.py(omitted)
  Added in git, processing cookiesJar object
• handle_mock.py(omitted)
  Supplemented in git, the framework is not used, but it is also encapsulated into methods
• param_replace(omitted)
  Encapsulate commonly used parameterized operations into classes

2.2.1 handle_excel.py

import os

#Project root path
_root_path = os.path.split(os.path.split(os.path.realpath(__file__))[0])[0]

# Report path
report_path = os.path.join(_root_path, 'testReport', 'report.html')

# Log path
log_path = os.path.join(_root_path, 'logs/')

#Configuration file path
conf_path = os.path.join(_root_path, 'conf', 'auto_test.conf')

# Test data path
testdatas_path = os.path.join(_root_path, 'testDatas')

# allure related configuration
_result_path = os.path.join(_root_path, 'testReport', 'result')
_allure_html_path = os.path.join(_root_path, 'testReport', 'allure_html')
allure_command = 'allure generate {} -o {} --clean'.format(_result_path, _allure_html_path)

2.2.2 handle_sendEmail.py

import smtplib
from utils.handle_logger import logger
from email.mime.text import MIMEText # Specifically send text emails
from email.mime.multipart import MIMEMultipart # Send text, attachments, etc.
from email.mime.application import MIMEApplication # Send attachments

class HandleSendEmail:

    def __init__(self, part_text, attachment_list, password, user_list, subject='interface_autoTestReport', smtp_server='smtp.163.com', from_user='[email protected]', filename='unit_test_report.html'):
        '''
        :param part_text: text
        :param attachment_list: attachment list
        :param password: third-party email server password
        :param user_list: recipient list
        :param subject: subject
        :param smtp_server: Email server
        :param from_user: sender
        :param filename: attachment name
        '''
        self.subject = subject
        self.attachment_list = attachment_list
        self.password = password
        self.user_list = ';'.join(user_list) # Multiple recipients
        self.part_text = part_text
        self.smtp_server = smtp_server
        self.from_user = from_user
        self.filename = filename

    def _part(self):
        '''Build email content'''
        # 1) Construct an email collection:
        msg = MIMEMultipart()
        msg['Subject'] = self.subject
        msg['From'] = self.from_user
        msg['To'] = self.user_list

        # 2) Construct the email body:
        text = MIMEText(self.part_text)
        msg.attach(text) # Add the text to the email body

        # 3) Construct email attachment:
        for item in self.attachment_list:
            with open(item, 'rb + ') as file:
                attachment = MIMEApplication(file.read())
            # Name the attachment:
            attachment.add_header('Content-Disposition', 'attachment', filename=item)
            msg.attach(attachment)

        # 4) Get the complete email content:
        full_text = msg.as_string()
        return full_text

    def send_email(self):
        '''send email'''
        # QQ mailbox must be added with SSL
        if self.smtp_server == 'smtp.qq.com':
            smtp = smtplib.SMTP_SSL(self.smtp_server)
        else:
            smtp = smtplib.SMTP(self.smtp_server)
        # Log in to the server: .login(user=email_address,password=third-party authorization code)
        smtp.login(self.from_user, self.password)
        logger.info('--------Email is being sent--------')
        try:
            logger.info('--------Email sent successfully--------')
            smtp.sendmail(self.from_user, self.user_list, self._part())
        except Exception as e:
            logger.error('Error sending email, error message is: {0}'.format(e))
        else:
            smtp.close() # Close the connection

if __name__ == '__main__':
    from base.base_path import *
    part_text = 'The attachment is an automated test report, the framework uses pytest + allure'
    attachment_list = [report_path]
    password=''
    user_list = ['']
    HandleSendEmail(part_text, attachment_list, password, user_list).send_email()

2.2.3 handle_logger.py

import sys
import logging
from time import strftime
from base.base_path import *

class Logger:

    def __init__(self):
        # Log format
    custom_format = '%(asctime)s %(filename)s [line:%(lineno)d] %(levelname)s: %(message)s'
        # Date format
        date_format = '%a, %d %b %Y %H:%M:%S'

        self._logger = logging.getLogger() # Instantiation
        self.filename = '{0}{1}.log'.format(log_path, strftime("%Y-%m-%d")) # Log file name
        self.formatter = logging.Formatter(fmt=custom_format, datefmt=date_format)
        self._logger.addHandler(self._get_file_handler(self.filename))
        self._logger.addHandler(self._get_console_handler())
        self._logger.setLevel(logging.INFO) #Default level

    def _get_file_handler(self, filename):
        '''Output to log file'''
        filehandler = logging.FileHandler(filename, encoding="utf-8")
        filehandler.setFormatter(self.formatter)
        return filehandler

    def _get_console_handler(self):
        '''Output to console'''
        console_handler = logging.StreamHandler(sys.stdout)
        console_handler.setFormatter(self.formatter)
        return console_handler

    @property
    def logger(self):
        return self._logger

'''
Log level:
critical A serious error that will cause the program to exit
error Error within controllable range
warning warning message
info prompt information
debug Detailed output record when debugging the program
'''
# Example
logger = Logger().logger


if __name__ == '__main__':
    import datetime
    logger.info(u"{}:Start XXX operation".format(datetime.datetime.now()))

2.2.4 handle_config.py

import configparser

# Configuration file class
class HandleConfig:
    def operation_config(self, conf_file, section, option):
        cf = configparser.ConfigParser() # Instantiation
        cf.read(conf_file)
        value = cf.get(section, option) # Positioning
        return value


handle_config = HandleConfig()
if __name__ == '__main__':
    from base.base_path import *
    base_url = handle_config.operation_config(conf_path, 'BASEURL', 'base_url')
    print(base_url)

2.2.5 handle_allure.py

import subprocess
from base.base_path import *

class HandleAllure(object):

    def execute_command(self):
        subprocess.call(allure_command, shell=True)

handle_allure = HandleAllure()

2.3testDatas

The excel test case file must end with .xlsx, and the use case structure is as follows:

2.4conf

Place the configuration file at the end of .conf

2.5 testCases

• conftest.py
  – Fixture function, use case pre- and post-operation
  – Construct test data
  – Other advanced operations
  – Note that the password and user_list in the email need to be replaced with the email address and service password you tested.
• test_wanAndroid.py test case script
  – Parameterization: pytest.mark.parametrize(‘case’,[{},{}])
  – Interface association:
  – Configure associated parameters as global variables
  – Use global variables to replace parameters before use case execution
  – Use the is_run parameter to indicate the parameterized use case, take it out, and assign it to the global variable
  – cookies:
  – Handle cookies in the same way as associated with interfaces
  – steps
  – Collect use cases
  – Execute use cases
  – assertion
  – Construct test report
  – send email

2.5.1 conftest.py

import pytest
from base.base_path import *
from utils.handle_logger import logger
from utils.handle_allure import handle_allure
from utils.handle_sendEmail import HandleSendEmail

'''
1. Construct test data
2. fixture replaces setup,teardown
3. Configure pytest
'''

def pytest_collection_modifyitems(items):
    """
    When the test case collection is completed, the Chinese name and nodeid of the collected item will be displayed on the console.
    """
    for item in items:
        item.name = item.name.encode("utf-8").decode("unicode_escape")
        item._nodeid = item.nodeid.encode("utf-8").decode("unicode_escape")
        # print(item.nodeid)

@pytest.fixture(scope='session', autouse=True)
def send_email():
    logger.info('-----session level, execute wanAndroid test case-----')
    yield
    logger.info('-----session level, wanAndroid use case execution ends, send email:-----')
    """Execute allure command """
    handle_allure.execute_command()
    # send email
    part_text = 'The attachment is an automated test report, the framework uses pytest + allure'
    attachment_list = [report_path]
    password=''
    user_list = ['']
    HandleSendEmail(part_text, attachment_list, password, user_list).send_email()

2.5.2 test_wanAndroid.py

import json
importpytest
import allure
from base.base_requests import BaseRequests
from utils.handle_logger import logger
from utils.handle_excel import HandleExcel
from utils.param_replace import pr
from utils.handle_cookies import get_cookies

handle_excel = HandleExcel()
get_excel_data = HandleExcel().get_excel_data()
ID = ''
COOKIES = {}
PAGE = ''

class TestWanAndroid:

    @pytest.mark.parametrize('case', get_excel_data)
    def test_wanAndroid(self, case):
        global ID
        global COOKIES
        # Parameter substitution
        case['url'] = pr.relevant_parameter(case['url'], '${collect_id}', str(ID))

        if case['is_run'].lower() == 'yes':
            logger.info('------The id of the executed use case is: {0}, and the use case title is: {1}------'.format(case['case_id'], case['title' ]))
            res = BaseRequests(case, cookies=COOKIES).get_response()
            res_json = res.json()

            # Get cookies after login
            if case['case_id'] == 3:
                COOKIES = get_cookies.get_cookies(res)

            if case['is_depend']:
                try:
                    ID = res_json['data']['id']
                    # Write the parameterized data used into excel
                    handle_excel.rewrite_value('id={}'.format(ID), case['case_id'], 'depend_param')
                except Exception as e:
                    logger.error(f'Failed to obtain id, error message is {e}')
                    ID = 0

            # Make allure report
            allure.dynamic.title(case['title'])
            allure.dynamic.description('<font color="red">Request URL:</font>{}<br />'
                                       '<font color="red">Expected value:</font>{}'.format(case['url'], case['excepted']))
            allure.dynamic.feature(case['module'])
            allure.dynamic.story(case['method'])

            result=''
            try:
                assert eval(case['excepted'])['errorCode'] == res_json['errorCode']
                result = 'pass'
            except AssertionError as e:
                logger.error('Assert Error: {0}'.format(e))
                result = 'fail'
                raise e
            finally:
                # Format the actual results and write them into excel
                handle_excel.rewrite_value(json.dumps(res_json, ensure_ascii=False, indent=2, sort_keys=True), case['case_id'], 'actual')
                #Write the use case execution results to excel
                handle_excel.rewrite_value(result, case['case_id'], 'test_result')


    def test_get_articleList(self):
        '''Turn the page, parameterize the page'''
        global PAGE
        pass


    def test_mock_demo(self):
        '''Use mock service to simulate server response'''
        pass


if __name__ == '__main__':
    pytest.main(['-q', 'test_wanAndroid.py'])

2.6 testReport

• Store the html test report and install the plug-in pip install pytest-html
• Store allure test report, plug-in installation pip install allure-pytest

2.7 logs

Store log files

2.8 Other documents

• run.py main run file
• pytest.ini configures pytest’s default behavior, running rules, etc.
• requirements.txt depends on the environment
  – Automatically generate pip freeze
  – Install pip -r install requirements.txt

3. Summary

1. Allure has many interesting operations, and even controls the execution behavior of use cases. If you are interested, you can expand it. You can also read the previous blog.
2. The difficulty in implementing the framework lies in interface dependencies
3. Interface automation should avoid complex interface dependencies. Complex dependencies will only cause uncontrollable testing.
4. Note that frequent operations on excel will consume performance
5. If you are interested, you can integrate this framework in Jenkins
6. The demo interfaces in this article are all adopted from this site. Thanks to the author for providing free interfaces.
   https://www.wanandroid.com/
7. Project git address:…(git is encrypted and will be added later))

Finally, I would like to thank everyone who read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, if you can use it, you can take it directly:

This information should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey. I hope it can also help you!

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Python entry skill treeHomepageOverview 385544 people are learning the system