Python Flask uses gevent or grpc.gevent module to implement asynchronous non-blocking

Directory

0. Preface:

1. Introduction to gevent.monkey:

2. Introduction to grpc.gevent:

3. Flask Demo code

4. Pressure test results

5. Stress test report

6 Conclusion

0. Foreword:

Flask itself is not an asynchronous framework, so there will be performance bottlenecks when processing high concurrent requests. However, Flask can improve concurrency performance through integration with other asynchronous frameworks and libraries.

So the following will introduce the two coroutine libraries of gevent and gprc, how to combine them into the Flask project to improve the concurrency performance of the project itself.

In this article, I will write a simple interface code, use the ab (Apache bench) tool to perform request pressure testing, and observe how the two modules improve the performance of the Flask project.

1. gevent.monkey introduction:

gevent is a coroutine-based Python network programming library that makes writing asynchronous code easier and more efficient. In gevent, an efficient coroutine and event loop mechanism are implemented through greenlet and libev.

The monkey module is an important module in the gevent library. It can dynamically replace the blocking I/O operation in the standard library with the non-blocking I/O operation of gevent, so that the original synchronous blocking program becomes asynchronous non-blocking program of. By using the monkey module, we can easily change the program into an asynchronous program based on coroutines without modifying the original code, and improve the concurrent performance of the program.

Specifically, the monkey module can replace the blocking I/O operations in the standard library through the following functions:

  • patch_all(): Used to automatically replace all replaceable blocking I/O operations in the standard library with gevent’s non-blocking I/O operations.
  • patch_socket(): Used to replace the blocking I/O operation in the socket module with the non-blocking I/O operation of gevent.
  • patch_select(): Used to replace the blocking I/O operation in the select module with the non-blocking I/O operation of gevent.
  • patch_time(): Used to replace the blocking sleep operation in the time module with the non-blocking sleep operation of gevent.

It should be noted that when using the monkey module, some side effects may occur, such as modifying the global variables and functions of the standard library, resulting in unpredictable behavior of other modules. Therefore, when using the monkey module, you should read the documentation carefully and perform necessary tests and verifications.

2. grpc.gevent introduction:

grpc.experimental.gevent is a Python module for using gRPC’s asynchronous calls with the gevent coroutine library. It provides a gRPC asynchronous call implementation based on gevent coroutines, and can be easily used with other gevent-based network programming libraries.

When using the grpc.experimental.gevent module, the combination of gevent and gRPC can be realized through the following steps:

  1. Create a gevent coroutine pool to manage all gRPC asynchronous calls.
  2. Use the grpc.experimental.gevent interceptor to convert all gRPC client calls to gevent coroutines.
  3. Run all gRPC asynchronous calls in the gevent coroutine pool.
  4. Handle all gRPC responses in a coroutine.

Using the grpc.experimental.gevent module can effectively improve the concurrent processing capability of gRPC applications, and can make gRPC applications more compatible with other gevent-based network programming libraries. It should be noted that when using the grpc.experimental.gevent module, you should read the documentation carefully and perform necessary tests and verifications to ensure the correctness and stability of the program.

3. Flask Demo code

Sleep for 2 seconds by time.sleep(2) to simulate the blocking situation, and then we start to test the concurrency separately

import time
from flask import Flask

# Method of using gevent
# from gevent import monkey
# monkey. patch_all()

# How to use grpc
# import grpc.experimental.gevent as grpc_gevent
# grpc_gevent.init_gevent()

 
app = Flask(__name__)
 
@app.route('/') #Match routing
def hello():
    # Simulate blocking
    time. sleep(2)
    return "Hello World"
 
if __name__ == '__main__':
    app.run(port=7779, threaded=True)

4. Pressure test results

Command ab -c 100 -n 5000 http://127.0.0.1:7779/ pressure test

Flask + gevent Flask + grpc_gevent
requests per second 49.77 49.83
Test time spent 100.455 100.334

5. Pressure test report

# Flask + grpc_gevent

Server Software: Werkzeug/2.2.3
Server Hostname: 127.0.0.1
Server Port: 7779

Document Path: /
Document Length: 11 bytes

Concurrency Level: 100
Time taken for tests: 100.334 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 920000 bytes
HTML transferred: 55000 bytes
Requests per second: 49.83 [#/sec] (mean)
Time per request: 2006.675 [ms] (mean)
Time per request: 20.067 [ms] (mean, across all concurrent requests)
Transfer rate: 8.95 [Kbytes/sec] received

Connection Times (ms)
              min mean[ + /-sd] median max
Connect: 0 0 0.3 0 3
Processing: 2001 2005 9.3 2004 2101
Waiting: 2001 2005 9.2 2003 2101
Total: 2001 2005 9.5 2004 2102

Percentage of the requests served within a certain time (ms)
  50% 2004
  66% 2004
  75% 2005
  80% 2005
  90% 2006
  95% 2009
  98% 2028
  99% 2067
 100% 2102 (longest request)


# Flask + gevent

Server Software: Werkzeug/2.2.3
Server Hostname: 127.0.0.1
Server Port: 7779

Document Path: /
Document Length: 11 bytes

Concurrency Level: 100
Time taken for tests: 100.455 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 920000 bytes
HTML transferred: 55000 bytes
Requests per second: 49.77 [#/sec] (mean)
Time per request: 2009.092 [ms] (mean)
Time per request: 20.091 [ms] (mean, across all concurrent requests)
Transfer rate: 8.94 [Kbytes/sec] received

Connection Times (ms)
              min mean[ + /-sd] median max
Connect: 0 0 0.3 0 3
Processing: 2001 2006 11.2 2002 2050
Waiting: 2001 2006 11.2 2002 2050
Total: 2001 2006 11.2 2002 2051

Percentage of the requests served within a certain time (ms)
  50% 2002
  66% 2003
  75% 2003
  80% 2003
  90% 2024
  95% 2038
  98% 2046
  99% 2048
 100% 2051 (longest request)

6. Conclusion

Through the pressure test, it is found that grpc_gevent seems to improve the performance of flask better. It may be because I only use sleep to simulate blocking in my code, but the I/O operations in real projects are more complicated, such as database query, file reading and writing, and other operations. , using gevent may be more suitable, because it can convert these I/O operations to non-blocking, thereby improving the concurrent performance of the server.

If your Flask project uses pymilvus, and you use gevent to implement asynchronous non-blocking, you will get stuck when the project starts, stuck in the milvus connection method. At this time, you can replace it with grpc_gevent to solve this problem.