Foreword
When we use pytest or unittest for interface testing and find a bug, we will provide a detailed description to RD, but often Not enough. In order to reproduce + verify faster, RD usually asks you for curl. Don’t say you don’t have it at this time. As a test and development engineer, this demand is still very high. Easy to solve. At this time, we need to write a function to convert the request into a curl command and that’s the solution. Of course, this article will also introduce the basic knowledge and usage of cURL. cURL is a command line tool for sending and receiving data via URLs. It supports multiple protocols, including HTTP, FTP, SMTP, etc., and offers a wealth of options and features. We will cover cURL installation, sending GET and POST requests, handling responses, and more to help you quickly get started using cURL for network requests.
Introduction to cURL
cURL is an open source command line tool for making network requests. It is a cross-platform tool that can be used on multiple operating systems such as Linux, Windows, and macOS. cURL supports multiple protocols, including HTTP, HTTPS, FTP, SMTP, POP3, and more.
cURL provides a wealth of options and functions, allowing us to flexibly send requests, set request headers, process responses, etc. It also supports advanced functions such as data transfer resumption, file upload and download.
cURL installation
Linux
In most Linux distributions, cURL comes pre-installed. If not, you can install it using a package manager. For example, on Ubuntu you can execute the following command:
sudo apt-get install curl
Windows
On Windows, you can download the precompiled executable file from the cURL official website (curl.se/windows/) and add it to the system path.
macOS
On macOS, cURL is installed by default. If you need an update or a custom installation, you can install it using a package manager such as Homebrew.
Basic usage
Send GET request
Sending a GET request using cURL is very simple. Just execute the following command in the command line:
curl URL
where URL is the address to be requested. For example:
curl https://api.example.com/users
cURL will send a simple GET request and print the response to the console.
Send POST request
To send a POST request, you specify the request method using the -X option and pass the request data using the -d option. For example:
curl -X POST -d 'username=admin & amp;password=123456' https://api.example.com/login
In the above example, we sent a POST request to the login interface with the username and password.
Set request header
Use the -H option to set request headers. For example:
curl -H 'Content-Type: application/json' https://api.example.com/users
In the above example, we set the Content-Type of the request header to application/json.
Handling responses
cURL outputs responses to standard output by default. If you need to save the response to a file, you can use the -o option to specify a filename. For example:
curl -o response.json https://api.example.com/users
In the example above, cURL saves the response to a file named response.json.
Compression request and decompression response
When sending an HTTP request, the server can choose to compress the transmitted data to reduce the amount of data and improve transmission efficiency. Common compression algorithms include gzip and deflate. When using the --compressed option, cURL will include appropriate header information in the request, instructing the server to compress the response, and automatically decompress the response when it is received. This can reduce network transmission time and resource usage.
Example using the --compressed option:
curl --compressed https://api.example.com/data
The above command will send a GET request with compressed request headers to https://api.example.com/data and automatically decompress it after receiving the response.
Note: Not all servers support compression, and there is no guarantee that all responses will be compressed. But on servers that support compression, using the --compressed option can save bandwidth and improve transmission efficiency.
Verification of server certificate
When using cURL to send an HTTPS request, cURL verifies that the server’s certificate is valid and trusted by default. This is to ensure secure communication with the server and prevent man-in-the-middle attacks. However, sometimes the server’s certificate may be invalid or self-signed, causing verification to fail. In this case, you can use the --insecure option to bypass verification of the server certificate.
Example using the --insecure option:
curl --insecure https://api.example.com/data
The above command will send an HTTPS request to https://api.example.com/data but will not verify the validity of the server’s certificate. This allows the connection to continue under certain circumstances, but also poses a security risk because there is no way to ensure that the communication to the server has not been tampered with.
Note that in production environments it is recommended not to use the --insecure option to ensure secure communication and avoid potential security issues. You should consider using the --insecure option only during development and testing, or if you are confident that the connection to the server is secure.
Advanced features
cURL also provides some advanced features that make network requests more flexible and powerful.
- Breakpoint resume download: Use the
-Coption to implement the breakpoint resume download function, which is used to continue downloading interrupted files. - File upload: Use the
-Foption to upload files. For example,curl -F 'file=@path/to/file' https://api.example.com/uploadwill upload the specified file to the server. - Cookie support: Use the
-band-coptions to handle and send cookies. - Timeout setting: Use the
-moption to set the request timeout. - HTTPS support: cURL supports HTTPS by default, with SSL/TLS encryption.
Realize request transfer to crul
Once you understand the basic usage, this function can be easily implemented.
def to_curl(request, compressed: bool = False, verify: bool = False) -> str:
parts = [
('curl', None),
('-X', request.method)
]
?
for k, v in sorted(request.headers.items()):
parts + = [('-H', f'{k}: {v}')]
?
if request.body:
body = request.body
if isinstance(body, bytes):
body = body.decode('utf-8')
parts + = [('-d', body)]
?
if compressed:
parts + = [('--compressed', None)]
?
if verify:
parts + = [('--insecure', None)]
?
parts + = [(None, request.url)]
?
flat_parts = []
for k, v in parts:
if k:
flat_parts.append(quote(k))
if v:
flat_parts.append(quote(v))
return ' '.join(flat_parts)
?
The code is easy to understand. It mainly explains the following two points:
1. When adding request headers to cURL, use the sorted() function to sort the request headers to ensure the order consistency of the request headers.
If the order of HTTP request headers is inconsistent when sent to the server, it may cause server parsing errors or unexpected behavior. Although in most cases the order of request headers will not affect the actual results, to avoid potential problems it is best to maintain a consistent order of request headers.
Sorting the request headers using the sorted() function ensures that the order of the request headers is consistent each time the loop is executed, resulting in the same cURL command. To be safe, especially when it comes to HTTP requests, it is recommended to order request headers to ensure consistency and predictability.
2. The quote function is a method in the shlex module, which is used to safely escape strings when generating command line parameters.
In the command line, some special characters have special meanings, such as spaces, quotation marks, backslashes, etc. If the string contains these special characters, in order to ensure correct parsing of the command line, the string needs to be escaped, that is, the special characters must be processed to prevent them from being misunderstood as part of the command line syntax.
The quote function performs appropriate escape processing on strings according to the rules of different operating systems and command line parsers. It adds a backslash “ before special characters that need to be escaped to ensure normal parsing on the command line.
For example, to pass the string "Hello World" containing spaces as a command line argument, you can use the quote function to escape it to ensure it is correctly Recognized as a single parameter:
from shlex import quote ? arg = "Hello World" quoted_arg = quote(arg) print(quoted_arg)
The output is:
'Hello World'
The quote function returns an escaped string. The string is enclosed in quotation marks to indicate that it is an integral parameter. This way, when concatenating command line parameters, spaces in the string will be correctly recognized and processed as a single parameter when parsing the command line.
Finally
At this point, the function of request forwarding to curl has been implemented, which is convenient for yourself and others. Tips to improve efficiency.
Finally: The following are supporting learning materials. For those who are doing [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse has also accompanied me through the most difficult journey. I hope it can also help you. ! [100% free without any tricks]
Software testing interview applet
A software test question bank that has been used by millions of people! ! ! Who is who knows! ! ! The most comprehensive interview test mini program on the Internet, you can use your mobile phone to answer questions, take the subway, bus, and roll it up!
Covering these interview question sections:
1. Basic theory of software testing, 2. web, app, interface function testing, 3. network, 4. database, 5. linux
6. Web, app, interface automation, 7. Performance testing, 8. Programming basics, 9. HR interview questions, 10. Open test questions, 11. Security testing, 12. Computer basics
