Comprehensive article on the most comprehensive knowledge key points in the history of Python (super detailed version)

Internet knowledge

  • What is HTTPS?

    • The project’s secure HTTP protocol, https requires a cs certificate, data encryption, port 443, safe, the same website will have a higher https seo ranking
      Common response status codes

  • Idempotence and security of http request methods

  • -WSGI

  • RPC

  • CDN

  • SSL (Secure Sockets Layer), and its successor Transport Layer Security (TLS), are a security protocol that provides security and data integrity for network communications.

  • SSH (Secure Shell Protocol) is the abbreviation of Secure Shell and was developed by the Network Working Group of the IETF; SSH is a security protocol based on the application layer. SSH is currently the most reliable protocol designed to provide security for remote login sessions and other network services. Using the SSH protocol can effectively prevent information leakage during remote management. SSH was originally a program on UNIX systems and quickly expanded to other operating platforms. SSH, when used correctly, can close holes in your network. SSH clients are available for a variety of platforms. Almost all UNIX platforms-including HP-UX, Linux, AIX, Solaris, Digital UNIX, Irix, and others-can run SSH.

  • TCP/IP

    • Although it stands to reason that after all four messages have been sent, we can directly enter the CLOSE state, but we must pretend that the network is unreliable and the last ACK may be lost. Therefore, the TIME_WAIT state is used to resend ACK messages that may be lost.
    • Because when the server receives the SYN connection request message from the client, it can directly send the SYN + ACK message. The ACK message is used for response, and the SYN message is used for synchronization. But when closing the connection, when the server receives the FIN message, it is likely that the SOCKET will not be closed immediately, so it can only reply with an ACK message first, telling the client, “I received the FIN message you sent.” Only when all the messages on my server side have been sent can I send the FIN message, so it cannot be sent together. Therefore, a four-step handshake is required.
    • Three-way handshake (SYN/SYN + ACK/ACK)
    • Wave four times (FIN/ACK/FIN/ACK)
    • TCP: connection-oriented/reliable/byte stream based
    • UDP: connectionless/unreliable/message-oriented
    • Three handshakes, four waves
    • Why is there a three-way handshake when connecting, but a four-way handshake when closing?
    • Why does the TIME_WAIT state need to pass 2MSL (maximum segment survival time) before returning to the CLOSE state?

  • XSS/CSRF

  • HttpOnly prohibits js scripts from accessing and operating cookies, which can effectively prevent XSS

Mysql database

  • Index improvement process

Linear structure->Binary search->hash->Binary search tree->Balanced binary tree->Multi-way search tree->Multi-way balanced search tree (B-Tree)
Mysql interview summary basics

  • https://segmentfault.com/a/1190000018371218

  • Mysql interview summary advanced article

    • https://segmentfault.com/a/1190000018380324

  • An in-depth explanation of Mysql

    • http://ningning.today/2017/02/13/database/mysql in simple terms/

  • When clearing the entire table, InnoDB deletes it row by row, while MyISAM deletes and creates the table from scratch.

  • The text/blob data type cannot have a default value, and there is no case conversion during query.

  • When does an index expire?
    Try to avoid using != or <> operators in where clauses, otherwise the engine will give up using the index and perform a full table scan.
    Try to avoid using or in the where clause to connect conditions, otherwise the engine will give up using the index and perform a full table scan. Even if there is a conditional index, it will not be used. This is why you should use or as little as possible.
    If the column type is a string, the data must be quoted in the condition, otherwise the index will not be used
    You should try to avoid performing functional operations on fields in the where clause, which will cause the engine to give up using the index and perform a full table scan.
    For multi-column indexes, the index will not be used if it is not the first part used
    Like fuzzy query starting with %
    Implicit type conversion occurs
    The leftmost prefix principle is not satisfied
    Failure scenario:

    Do not perform functions, arithmetic operations, or other expression operations on the left side of “=” in where clauses, otherwise the system may not use the index correctly
    You should try to avoid expression operations on fields in the where clause, which will cause the engine to give up using the index and perform a full table scan.

    It is not suitable for columns with fewer key values (columns with more repeated data). For example: set enum columns are not suitable (null can be added to the enumeration type (enum), and the default value will automatically filter the space set (set) and enumeration. Similar, but only 64 values can be added)
    If MySQL estimates that using a full table scan is faster than using an index, it does not use the index.

  • What is a clustered index
    B + Tree leaf nodes store data or pointers?
    MyISAM index and data separation, use non-clustered
    InnoDB data files are index files, and primary key indexes are clustered indexes.

Redis command summary

Why so fast?

  • Because Redis is a memory-based operation, the CPU is not the bottleneck of Redis. The bottleneck of Redis is most likely the size of the machine memory or the network bandwidth. Since single-threading is easy to implement and the CPU will not become a bottleneck, it is logical to adopt a single-threaded solution (after all, using multi-threading will cause a lot of trouble!).

  • Memory-based, written in C

  • Use multi-channel I/O multiplexing model, non-blocking IO

  • Use a single thread to reduce switching between threads

  • Data structure is simple

  • Build your own VM mechanism to reduce the time of calling system functions

  • Advantage

    • High performance – Redis can read at a speed of 110,000 times/s and write at a speed of 81,000 times/s.
    • Rich data types
    • Atomic – All operations of Redis are atomic, and Redis also supports atomic execution of several operations after they are fully merged.
    • Rich features – Redis also supports publish/subscribe (publish/subscribe), notifications, key expiration and other features
      What is a redis transaction?
    • A mechanism to package multiple requests and execute multiple commands at once and in sequence
    • Implement transaction functions through multi, exec, watch and other commands
    • Python redis-py pipeline=conn.pipeline(transaction=True)

  • persistence method

    • save (synchronization, which can ensure data consistency)
    • bgsave (asynchronous, used by default when shutdown, no AOF)
    • RDB (snapshot)
    • AOF (append log)

  • How to implement queue

    • push
    • rpop

  • Commonly used data types (Bitmaps, Hyperloglogs, range queries, etc. are not commonly used)

    • skiplist(skip list)
    • intset or hashtable
    • ziplist (continuous memory block, the header of each entry node saves the length information of the preceding and following nodes to implement a two-way linked list function) or double linked list
    • Integer or sds(Simple Dynamic String)
    • String: Counter
    • List: User’s attention, fan list
    • Hash:
    • Set: the user’s followers
    • Zset (ordered set): real-time information rankings

  • Differences from Memcached

    • Memcached can only store string keys
    • Memcached users can only add data to the end of an existing string through APPEND and use this string as a list. But when deleting these elements, Memcached uses a blacklist to hide the elements in the list, thereby avoiding operations such as reading, updating, and deleting elements.
    • Both Redis and Memcached store data in memory and are both in-memory databases. However, Memcached can also be used to cache other things, such as pictures, videos, etc.
    • Virtual memory – Redis can swap some Values that have not been used for a long time to disk when the physical memory is used up.
    • Storage data security – after Memcached hangs up, the data is gone; Redis can save it to disk regularly (persistence)
    • The application scenarios are different: In addition to being used as a NoSQL database, Redis can also be used as a message queue, data stack and data cache; Memcached is suitable for caching SQL statements, data sets, user temporary data, delayed query data and Session, etc.

  • Redis implements distributed lock

    • Use setnx to implement locking, and you can also add a timeout through expire
    • The value of the lock can be a random uuid or a specific name
    • When releasing a lock, use uuid to determine whether it is the lock. If so, execute delete to release the lock.

  • common problem

    • When the number of visits increases sharply, service problems occur (such as slow response time or no response), or non-core services affect the performance of core processes, it is still necessary to ensure that the service is still available, even if the service is compromised. The system can be automatically downgraded based on some key data, or a switch can be configured to achieve manual downgrade.
    • Data expires, update cached data
    • Initialize the project and add some commonly used data to the cache
    • When requesting access to the data, it does not exist in the query cache and does not exist in the database.
    • The cached data expires in a short period of time, and a large number of requests access the database.
    • cache avalanche
    • cache penetration
    • Cache warm-up
    • cache updates
    • Cache downgrade

  • Consistent Hash Algorithm

    • Ensure data consistency when using a cluster
      Implementing a distributed lock based on redis requires a timeout parameter

    • setnx

  • Virtual Memory

  • memory thrashing

Linux

Unix five i/o models

 + project select
 + project poll
 + project epoll
 + When the project concurrency is not high and the number of connections is very active
 + The project is not much improved compared to select
 + Suitable for situations where the number of connections is large but the number of active links is small
 + blockingio
 + non-blocking io
 + Multiplexing io (use selectot to implement io multiplexing under Python)
 + signal driven io
 + Asynchronous io (Gevent/Asyncio implements asynchronous)

A command manual that is easier to use than man
*tldr: A manual with command examples
The difference between kill -9 and -15

*-15: The program stops immediately/stops when the program releases the corresponding resources/the program may still continue to run.
*-9: Due to the uncertainty of -15, use -9 to kill the process immediately. Paging mechanism (memory allocation management scheme that separates logical addresses and physical addresses):

 + The operating system manages memory efficiently and reduces fragmentation
 + The logical address of the program is divided into fixed-size pages
 + Physical addresses are divided into frames of the same size
 + Correspond to logical address and physical address through page table

segmentation mechanism

 + In order to meet some logical requirements of the code
 + Data Sharing/Data Protection/Dynamic Links
 + Continuous memory allocation within each segment, discrete allocation between segments

Check cpu memory usage?
+ top
+ free View available memory and troubleshoot memory leaks

Design pattern

Singleton pattern

Factory pattern

construction pattern

Data structures and algorithms

Python implements various data structures

Quick sort

selection sort

insertion sort

merge sort

Heap sort heapq module

stack

queue

binary search

The above is all the content shared today. If you find it useful, please like and save it!

Python experience sharing

Learning Python well is good whether it is used for employment or as a side job to make money. Moreover, learning Python well can also fit in with future development trends-artificial intelligence, machine learning, deep learning, etc.
The editor is a Python development engineer and has compiled a set of the latest Python system learning tutorials, including basic Python scripts to web development, crawlers, data analysis, data visualization, machine learning, etc. If you also like programming and want to change careers, do a side job or improve work efficiency by learning Python, this [latest complete set of Python learning materials] will definitely be useful to you!

The editor has prepared the following seed materials for friends who are interested in Python!

For 0 basics beginners:

If you are a novice and want to get started with Python quickly, you can consider training!

  • The learning time is relatively short, and the learning content is more comprehensive and focused.
  • You can find a study plan that suits you

Including: Python activation code + installation package, Python web development, Python crawler, Python data analysis, artificial intelligence, machine learning, Python quantitative trading and other learning tutorials. Let you learn Python systematically from scratch!

1. Learning routes in all directions of Python

The Python all-direction route is to organize the commonly used technical points of Python to form a summary of knowledge points in various fields. Its usefulness is that you can find corresponding learning resources according to the above knowledge points to ensure that you learn more comprehensively.

2. Learning software

If a worker wants to do his job well, he must first sharpen his tools. The commonly used development software for learning Python is here, saving everyone a lot of time.

3. Introductory learning video

When we watch videos and learn, we can’t just move our eyes and brain but not our hands. The more scientific learning method is to use them after understanding. At this time, hands-on projects are very suitable.

4. Practical cases

Optical theory is useless. You must learn to follow along and practice it in order to apply what you have learned to practice. At this time, you can learn from some practical cases.

5. Interview materials

We must learn Python to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and Alibaba bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.


The latest complete set of [Python introductory to advanced information & amp; practical source code & amp; installation tools](safe link, click with confidence)

I have uploaded it to CSDN official. If you need it, you can scan the official QR code below to get it for free [100% free guaranteed]

*That’s it for today’s sharing. If you like it and it is helpful to you, please remember to like and follow ~ See you next time!