Get more information
Personal website: Brother Tao talks about Python
When it comes to text processing and searching, regular expressions are a powerful and indispensable tool in Python.
Regular expression is a pattern description language used to search, match and process text. It can quickly and flexibly find, identify and extract the required information in large amounts of text data.
Basic concepts of regular expressions
1. Character matching
Regular expressions are patterns composed of ordinary characters (such as letters, numbers, and symbols) and metacharacters (characters with special meanings).
The simplest regular expressions are patterns containing only ordinary characters that exactly match the corresponding characters in the input text.
For example, the regular expression apple will exactly match the string apple in the input text.
2. Metacharacters
Metacharacters are characters with special meaning in regular expressions. Here are some common metacharacters and their meanings:
.: Matches any character except newline characters.*: Matches zero or more repetitions of the previous character.+: Matches one or more repetitions of the previous character.?: Matches zero or one repetition of the previous character.^: Matches the beginning of the input string.$: Matches the end of the input string.\: Used to escape the next character so that it does not have special meaning.
3. Character class
A character class is an expression that matches a character in a set of characters. Character classes can be defined via [], for example:
[aeiou]: Matches any vowel.[0-9]: Matches any numeric character.
4. Predefined character classes
Regular expressions also provide some predefined character classes for matching common character sets, such as:
\d: matches any numeric character, equivalent to[0-9].\D: matches any non-numeric character, equivalent to[^0-9].\w: Matches any letter, number or underscore character, equivalent to[a-zA-Z0-9_].\W: Matches any non-letter, non-digit, or non-underscore character, equivalent to[^a-zA-Z0-9_].\s: Matches any whitespace character (space, tab, newline, etc.).\S: Matches any non-whitespace character.
Using regular expressions in Python
In Python, the regular expression module re provides a wealth of functions and methods to process regular expressions. The following are some commonly used re module functions and methods:
1. re.match()
The re.match(pattern, string) function is used to match a pattern from the beginning of the string. If the pattern matches, returns a match object; otherwise returns None.
import re
pattern = r'apple'
text = 'apple pie'
match = re.match(pattern, text)
if match:
print("Match found:", match.group())
else:
print("No match")
2. re.search()
The re.search(pattern, string) function is used to search for the first occurrence of a pattern in a string. Start searching anywhere in the string.
import re
pattern = r'apple'
text = 'I have an apple and a banana'
search = re.search(pattern, text)
if search:
print("Match found:", search.group())
else:
print("No match")
3. re.findall()
The re.findall(pattern, string) function is used to find all parts of a string that match a pattern and return them in the form of a list.
import re pattern = r'\d + ' text = 'There are 3 apples and 5 bananas in the basket' matches = re.findall(pattern, text) print(matches) # Output: ['3', '5']
4. re.finditer()
The re.finditer(pattern, string) function is similar to re.findall(), but returns an iterator for accessing matches one by one.
import re
pattern = r'\d + '
text = 'There are 3 apples and 5 bananas in the basket'
matches = re.finditer(pattern, text)
for match in matches:
print("Match found:", match.group())
5. re.sub()
The re.sub(pattern, replacement, string) function is used to search for a pattern in a string and replace it with the specified string.
import re pattern = r'apple' text = 'I have an apple and a banana' replacement = 'orange' new_text = re.sub(pattern, replacement, text) print(new_text) # Output: "I have an orange and a banana"
6. Matching objects and grouping
The match object is an object returned by functions such as re.match(), re.search(), etc., and contains detailed information about the match. The content of the match can be accessed using the methods and properties of the match object.
import re
pattern = r'(\d{2})/(\d{2})/(\d{4})'
date_text = 'Today is 09/30/2023'
match = re.search(pattern, date_text)
if match:
print("Full match:", match.group(0))
print("Day:", match.group(1))
print("Month:", match.group(2))
print("Year:", match.group(3))
Advanced techniques for regular expressions
Regular expressions can be used not only for basic matching and replacement, but also for more complex text processing tasks through some advanced techniques. Here are some common advanced regular expression tips:
1. Use capturing groups
Capturing groups are parenthesized portions of a regular expression that can be used to extract matching substrings.
import re
pattern = r'(\d{2})/(\d{2})/(\d{4})'
date_text = 'Today is 09/30/2023'
match = re.search(pattern, date_text)
if match:
day, month, year = match.groups()
print(f"Date: {<!-- -->year}-{<!-- -->month}-{<!-- -->day}")
2. Non-greedy matching
By default, regular expressions are greedy and will match as many characters as possible. You can add ? after the quantifier to achieve non-greedy matching.
import re pattern = r'<.*?>' text = '<p>Paragraph 1</p> <p>Paragraph 2</p>' matches = re.findall(pattern, text) print(matches) # Output: ['<p>', '</p>', '<p>', '</p>']
3. Logical OR operation
Use the vertical bar | to implement a logical OR operation, which can be used to match any one of multiple patterns.
import re pattern = r'apple|banana' text = 'I have an apple and a banana' matches = re.findall(pattern, text) print(matches) # Output: ['apple', 'banana']
4. Backreferences
Backreferences can refer to captured groups, matching the same text repeatedly in a pattern.
import re pattern = r'(\w + ) \1' text = 'The cat cat jumped over the dog dog' matches = re.findall(pattern, text) print(matches) # Output: ['cat cat', 'dog dog']
Application scenarios of regular expressions
Regular expressions are widely used in text processing. The following are some common application scenarios:
-
Data verification: Used to verify whether the formats of phone numbers, email addresses, ID numbers, etc. are legal.
-
Log analysis: Used to extract specific information from log files, such as IP address, timestamp, etc.
-
Data extraction: Used to extract data from HTML, XML and other documents, such as links and content in web crawlers.
-
Text Search and Replace: Used to search for specific keywords or replace text in text.
-
Data cleaning: Used to clean and normalize data, such as removing redundant spaces, punctuation marks, etc.
-
Tokenization and Tokenization: Used to segment text into words or tokens.
-
Language processing: Used to identify language features in text, such as sentence boundaries, word stemming, etc.
-
Password policy: Used to strengthen password policies, such as checking whether passwords contain specific characters, length, etc.
Summary
Regular expressions are powerful text processing tools in Python that can handle a variety of text data, from simple matching and replacement to complex data extraction and analysis.
Whether you’re working with everyday text data or performing advanced text analysis, regular expressions are an indispensable skill.
Python learning route
Get more information
Personal website: Brother Tao talks about Python
If you want to get more and richer information, you can click on the business card below the article and reply [High-quality information] to get a comprehensive learning information package.
Click on the link card below the article and reply [Quality Information] to directly receive the information gift package.