Regular Expressions Overview
Overview
Use a “string of symbols” to describe data with common attributes
Basic regular list
Extended regular list
Prepare materials
[root@localhost ~]# head -5 /etc/passwd > user [root@localhost ~]# cat user root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
Test ^ $
[root@localhost ~]# grep root user root:x:0:0:root:/root:/bin/bash [root@localhost ~]# grep ^root user root:x:0:0:root:/root:/bin/bash [root@localhost ~]# grep root$ user [root@localhost ~]# grep bash$ user root:x:0:0:root:/root:/bin/bash [root@localhost ~]# vim user [root@localhost ~]# grep ^$ user #^$ represents a blank line [root@localhost ~]# cat user root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin 5 [root@localhost ~]# grep -v ^$ user # -v means the opposite, query except blank lines, spaces also represent characters root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin 5
[ ] Collection
Matches any character in the set. If at least one character is found, the match is successful.
[^ ] Negate the set
grep "[root]" user #Find any character of r, o, t grep "[rot]" user #The effect is the same as above grep "[^rot]" user #Display content other than r or o or t grep "[0123456789]" user #Find all numbers grep "[0-9]" user #The effect is the same as above grep "[^0-9]" user #Display content other than numbers grep "[a-z]" user #Find all lowercase letters grep "[A-Z]" user #Find all uppercase letters grep "[a-Z]" user #Find all letters grep "[^0-9a-Z]" user #Find all symbols
Exercise: Write a script that asks the user for an integer number and gives a prompt if the user does not respond.
#!/bin/bash read -p "Please enter an integer number: " n echo $n | grep "[^0-9]" [$? -ne 0 ] & amp; & amp; echo ok || echo "Must enter a number
. Matches a single character
* Matches the previous character any number of times and cannot be used alone
.* concatenate to represent wildcard characters
[root@localhost ~]# grep "*" user [root@localhost ~]# grep "ro*t" user root:x:0:0:root:/root:/bin/bash [root@localhost ~]# grep ".*" user root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin 5
Test \{n\} \{n,\} \{n,m\} \(\)
grep "ro\{1,2\}t" user #Look for rt, there can be 1~2 o in the middle
grep "ro\{2,6\}t" user #Find rt, there can be 2~6 o's in the middle
grep "ro\{1,\}t" user #Find rt, there can be 1 or more o in the middle
grep "ro\{3\}t" user #Find rt, there must be only 3 o's in the middle
grep "\(0:\)\{2\}" user #Find two consecutive 0s: The function of parentheses is to combine characters into a whole
Extended regular expression
grep "ro\{1,\}t" user #Use basic regular rules to find o appearing 1 or more times
egrep "ro{1,}t" user #Use extended regular rules, the effect is the same as above, more streamlined
egrep "ro + t" user #Use extended regular rules, the effect is the same as above, the most streamlined
grep "roo\{0,1\}t" user #Use basic regular rules to find the second o appearing 0~1 times
egrep "roo{0,1}t" user #Use extended regular rules, the effect is the same as above, more streamlined
egrep "roo?t" user #Use extended regular rules, the effect is the same as above, the most streamlined
egrep "(0:){2}" user #Find two consecutive 0s: The function of parentheses is to combine characters into a whole
egrep "root|bin" user #Find lines with root or bin
egrep "the\b" abc.txt #Find the word the, numbers, letters, and underlines are not allowed on the right
egrep "\bthe\b" abc.txt #No numbers, letters, or underscores are allowed on both sides of the
egrep "\<the\>" abc.txt #The effect is the same as above
Thinking: How to match a large range of numbers? For example, 250-255
Method 1: [root@localhost ~]# grep "25[0-5]" shuzi.txt 253 456 234 34 254 Method Two: [root@localhost ~]# egrep "250|251|252|253|254|255" shuzi.txt 253 456 234 34 254
Basic usage of sed (sed streaming editor, line-by-line processing)
Overview
Non-interactive addition, deletion, modification and query of documents can be performed
Format
Predirectives | sed options conditional directives
sed options conditions directives documentation
Options, instructions, conditions
Option -n masks default output
Option -r supports extended regular expressions
Option -i Modify source file
Command p output d delete s replace
Condition line number regular expression
Line number case
head -5 /etc/passwd > user #Prepare materials sed -n 'p' user #Output all lines sed -n '1p' user #Output line 1 sed -n '2p' user #Output line 2 sed -n '3p' user #Output line 3 sed -n '2,4p' user #Output lines 2~4 sed -n '2p;4p' user #Output lines 2 and 4 sed -n '3, + 1p' user #Output the 3rd line and the following 1 line sed -n '1~2p' /etc/passwd #Output odd lines sed -n '2~2p' /etc/passwd #Output even lines
Use regular expressions as conditions
Use regular expressions as conditions sed -n '/^root/p' user #Output lines starting with root sed -n '/root/p' user #Output lines containing root sed -nr '/^root|^bin/p' user #Output lines starting with root or lines starting with bin. | is an extended regular expression and requires the r option. Special usage sed -n '1!p' user #Output except the content of line 1, ! is the inversion sed -n '$p' user #Output the last line sed -n '=' user #Output the line number. If it is $=, it is the line number of the last line
Sed command’s s replacement basic function (s/old content/new content/options)
[root@svr5 ~]# vim shu.txt #New material 2017 2011 2018 2017 2017 2024 2017 2017 2017 sed 's/2017/6666/' shu.txt #Replace the first 2017 in all lines with 6666 sed 's/2017/6666/2' shu.txt #Replace the second 2017 in all lines with 6666 sed '1s/2017/6666/' shu.txt #Replace the first 2017 in line 1 with 6666 sed '3s/2017/6666/3' shu.txt #Replace the third 2017 in line 3 with 6666 sed 's/2017/6666/g' shu.txt #Replace all 2017 in all lines sed '/2024/s/2017/6666/g' shu.txt #Find the line containing 2024 and replace all 2017 inside with 6666
Thinking: What if you want to replace /bin/bash with /sbin/sh?
sed comprehensive script application
Fulfill the following requirements:
- Find the local account name using bash as login shell
- List the shadow password records for these accounts
- Save each line of “Account Name –> Password Record” to the file
#!/bin/bash
u=$(sed -n '/bash$/s/:.*//p' /etc/passwd) #Find the line ending with bash in the passwd document, and then delete the colon and the content after the colon in the line , p here means only displaying the rows where s was successfully replaced, and finally assigning it to u
for i in $u #Give those bash account names to the for loop
do
pass=$(grep $i /etc/shadow) #Use each account name to find the corresponding information in shadow
pass=${pass#*:} #Pinch your head and delete from left to right to the first colon
pass=${pass%%:*} #Remove the tail from right to left to the last colon. After the above steps, pass is the final password.
echo "$i --> $pass" #Shout according to the format. If you want to save it to a file, use append redirection.
done
Regular expression supplement
\w matches numbers, letters, and underscores
egrep "roo\w" user #Find the string that must be followed by roo with numbers, letters or underscores
\s matches spaces and tab keys
egrep "roo\s" user #Find the string that is followed by roo and is followed by a space or the space typed by the tab key, if not No output
\d matches numbers, equivalent to [0-9]
egrep "(25[0-5]\.|2[0-4][0-9]\.|1?[0-9]?[0-9]\.){ 3}(25[0-5]|2[0-4][0-9]|1?[0-9]?[0-9])" #match ip address
grep -P "(25[0-5]\.|2[0-4]\d\.|1?\d?\d\.){3}(25[0- 5]|2[0-4]\d|1?\d?\d)" #match ip address
Exercise
Match IP address
sed supplement
Append to line a and add c to line i to replace the entire line.
sed 'a 666' user #Append 666 below all lines sed '1a 666' user #Append 666 below line 1 sed '/^bin/a 666' user #Append 666 below the line starting with bin sed 'i 666' user #Add 666 above all lines sed '5i 666' user #Add 666 above line 5 sed '$i 666' user #Add 666 above the last line sed 'c 666' user #Replace all lines with 666 sed '1c 666' user #Replace the first line with 666
( ) retains functional testing, retaining means copying
cat abc.txt #Prepare materials first 100 laowang 98 gangge 59 laoniu sed -r 's/([0-9] + )(\s + )([a-z] + )/\3\2\1/' abc.txt #Use the replace function to change the text column ,The parentheses here are equivalent to retaining (copying), \1 is equivalent to pasting the content in the first parentheses
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Linux skill treeLinux practical commandssed command 10264 people are learning the system