This article mainly explains the usage suggestions of the Python Requests module session and the methods of all cookies in the entire session.
Test Code
Server: The following is a server made with flask, which is used to set cookies and print request headers.
# -*- coding: utf-8 -*-</code><code>from flask import Flask, make_response, request</code>
<code>app = Flask(__name__)</code>
<code>@app.route('/a1')</code><code>def a1():</code><code> print(request.headers)</code><code> rp = make_response ()</code><code> rp.set_cookie('a1', '123')</code><code> return rp</code>
<code>@app.route('/a2')</code><code>def a2():</code><code> print(request.headers)</code><code> rp = make_response ()</code><code> # rp.set_cookie('a2', '234')</code><code> return rp</code>
<code>@app.route('/a3')</code><code>def a3():</code><code> print(request.headers)</code><code> rp = make_response ()</code><code> rp.set_cookie('a3', '345')</code><code> return rp</code>
<code>if __name__ == '__main__':</code><code> app.run(host='0.0.0.0')
(Swipe left and right to view the complete code)
client:
# -*- coding: utf-8 -*-</code>
<code>import requests</code>
<code>url1 = 'http://192.168.2.159:5000/a1'</code><code>url2 = 'http://192.168.2.159:5000/a2'</code><code>url3 = 'http://192.168.2.159:5000/a3'</code>
<code>cookies = requests.utils.cookiejar_from_dict({'test': 'test'})</code><code>print(type(cookies), cookies) # RequestsCookieJar object</code><code>s = requests.session()</code><code>s.cookies = cookies # The cookie set here test=test is attached to all requests</code><code>s.headers = {'h1 ':'h1'} # The request header h1=h1 set here is attached to all requests</code><code>r1 = s.get(url1, cookies={'r1': \ 'r1'},headers={'h2':'h2'}) # Temporarily add cookie r1=r1 and header h2=h2 The next request will not have this cookie and header</code> <code>r2 = s.get(url2)</code><code>requests.utils.add_dict_to_cookiejar(s.cookies, {'xx': 'xx'}) # In the next request, Permanently add xx cookie</code>
<code>r3 = s.get(url3)</code>
<code># r1.cookies is a RequestsCookieJar object, you can use requests.utils.dict_from_cookiejar(r1.cookies) to convert it into a dict</code><code># I found that I can convert it directly with dict, which makes it easier to write Convenient</code><code>print(dict(r1.cookies)) # print the cookies set in the return result of the r1 request</code><code>print(dict(r2.cookies)) # print the return of the r2 request The cookies set in the result</code><code>print(dict(r3.cookies)) # Print the cookies set in the return result of the r3 request</code>
<code>print(dict(s.cookies)) # s.cookies contains all cookies in the entire session request (temporarily added such as r1 above is not included)
(Swipe left and right to view the complete code)
Start the server first, then the client.
Run results
The server prints the result:
192.168.2.159 - - [26/Jun/2019 17:28:00] "GET /a1 HTTP/1.1" 200 -</code><code>Host: 192.168.2.159:5000</code> <code>Accept-Encoding: identity</code><code>H1: h1</code><code>H2: h2</code><code>Cookie: test=test; r1=r1</code> <code>192.168.2.159 - - [26/Jun/2019 17:28:00] "GET /a2 HTTP/1.1" 200 -</code><code>Host: 192.168.2.159:5000</code> <code>Accept-Encoding: identity</code><code>H1: h1</code><code>Cookie: test=test; a1=123</code> <code>192.168.2.159 - - [26/Jun/2019 17:28:00] "GET /a3 HTTP/1.1" 200 -</code><code>Host: 192.168.2.159:5000</code> <code>Accept-Encoding: identity</code><code>H1: h1</code><code>Cookie: test=test; xx=xx; a1=123
(Swipe left and right to view the complete code)
The client prints the result:
<class 'requests.cookies.RequestsCookieJar'> <RequestsCookieJar[<Cookie test=test for />]></code><code>{'a1': '123'}</code><code>{}</code><code>{'a3': '345'}</code><code>{'test': 'test', 'xx ': 'xx', 'a1': '123', 'a3': '345'}
(Swipe left and right to view the complete code)
Summary and usage suggestions
It can be seen from the server-side printing that if we do not set the User-Agent, the request header of the requests module is python-requests/2.21.0, which is not the request header of a normal browser, which is why we must modify the request when we make a crawler head for a reason.
Using requests.session() can help us save all cookies during this session, saving us from getting the cookie of the previous request, updating the cookie, resetting it, and then requesting such operations.
Cookies and headers that will be carried throughout the session set by s.cookies and s.headers.
Cookies and headers set in the form of s.get(url1, cookies={‘r1’: ‘r1’},headers={‘h2′:’h2’}) will not be overwritten The request headers and cookies set in s.cookies and s.headers are only added to this request, and the r1 and h2 here will not be carried in the next request.
requests.utils.add_dict_to_cookiejar(s.cookies, {‘xx’: ‘xx’}) can set a fixed cookie for s: xx, the cookie of this setting is not temporary, and will be carried in subsequent requests.
The result of r1.cookies is a RequestsCookieJar object, which can be converted by dict to get a dict whose content is the cookie set in the r1 request response header. If no new cookie is set for the current request, the dict is followed by an empty dictionary.
The result of s.cookies is the cookie set during the entire session (all requests sent through s), and all the set cookies can be obtained through dict(s.cookies)
It is recommended that we set up the common parts in advance, such as headers, cookies, and proxies, during use.
Recently, I found that if some cookies are set multiple times during the whole process, direct use of dict to force the transfer will fail. The safest way is to use requests.utils.dict_from_cookiejar(s.cookies) to get dictionary-type cookies.
Learning arrangement
As someone who has been here, I also hope that everyone will avoid some detours. If you don’t want to experience the feeling of not being able to find information when learning, no one answering questions, and giving up after a few days of persistence, here I will share with you some automation. The learning resources of the test hope to help you along the way. [100% Free Guaranteed]
How to get the video file:
This document and video material, For friends who want to engage in [software testing], it should be the most comprehensive and complete preparation warehouse. This warehouse has also accompanied me through the most difficult journey. Can help you! All of the above can be shared, and you can collect it yourself by clicking the small card below.