[Comprehensive explanation of Linux commands] 051. Field delimiters and process control in the Linux Awk scripting language

Article directory

Set field delimiter
- flow control statement
- - conditional judgment statement
  - loop statement
  - - while statement
    - for loop
    - do loop
  - Other statements
- Array application
- - Array definition
  - Read the value of an array
  - Array related functions
  - Use of two-dimensional and multi-dimensional arrays
Learn `python` from scratch

Set field delimiter

The default field delimiter is a space, you can use -F “delimiter” to explicitly specify a delimiter:

awk -F: '{ print $NF }' /etc/passwd
# or
awk 'BEGIN{ FS=":" } { print $NF }' /etc/passwd

In the BEGIN statement block, you can use OFS = “delimiter” to set the delimiter of the output field.

Flow control statement

In the while, do-while and for statements of Linux awk, break and continue statements are allowed to be used to control the process direction, and statements such as exit are also allowed to be used to exit. break interrupts the currently executing loop and jumps outside the loop to execute the next statement. if is a process selection usage. In awk, flow control statements, grammatical structures, and c language types. With these statements, many shell programs can actually be handed over to awk, and the performance is very fast. The following is the usage of each statement.

Conditional judgment statement

if(expression)
  Statement 1
else
  Statement 2

Statement 1 in the format can be multiple statements. In order to facilitate judgment and reading, it is best to enclose multiple statements with {}. The awk branch structure allows nesting, and its format is:

if(expression)
  {<!-- -->Statement 1}
else if(expression)
  {<!-- -->Statement 2}
else
  {<!-- -->Statement 3}

Example:

awk 'BEGIN{
test=100;
if(test>90){
  print "very good";
  }
  else if(test>60){
    print "good";
  }
  else{
    print "no pass";
  }
}'

Output:

very good

Each command statement can be ended with ; semicolon.

Loop statement

while statement

while(expression)
  {<!-- -->Statement}

Example:

awk 'BEGIN{
test=100;
total=0;
while(i<=test){
  total + =i;
  i + + ;
}
print total;
}'

Output:

for loop

There are two formats for for loops:

Format 1:

for(variable in array)
  {<!-- -->Statement}

Example:

awk 'BEGIN{
for(k in ENVIRON){
  print k"="ENVIRON[k];
}

}'

Output:

TERM=linux
G_BROKEN_FILENAMES=1
SHLVL=1
pwd=/root/text
...
logname=root
HOME=/root
SSH_CLIENT=192.168.1.21 53087 22

Note: ENVIRON is an awk constant and a sub-typical array.

Format 2:

for(variable; condition; expression)
  {<!-- -->Statement}

Example:

awk 'BEGIN{
total=0;
for(i=0;i<=100;i + + ){
  total + =i;
}
print total;
}'

Output:

do loop

do
{<!-- -->statement} while (condition)

example:

awk 'BEGIN{
total=0;
i=0;
do {total + =i;i + + ;} while(i<=100)
  print total;
}'

Output:

Other statements

break When the break statement is used in a while or for statement, it causes the program loop to exit.
continue When the continue statement is used in a while or for statement, causes the program loop to move to the next iteration.
next causes the next line of input to be read and returns to the top of the script. This avoids performing additional operations on the current input line.
The exit statement causes the main input loop to exit and transfers control to END, if END exists. If no END rule is defined, or an exit statement is applied in END, the execution of the script is terminated.

Array application

Arrays are the soul of awk, and the most indispensable part of text processing is its array processing. Because array indexes (subscripts) can be numbers and strings, arrays in awk are called associative arrays (associative arrays). Arrays in awk do not have to be declared in advance, nor do they have to declare their size. Array elements are initialized with 0 or the empty string, depending on the context.

Definition of array

Numbers are used as array indexes (subscripts):

Array[1]="sun"
Array[2]="kai"

String as array index (subscript):

Array["first"]="www"
Array"[last"]="name"
Array["birth"]="1987"

When using print Array[1], it will print out sun; when using print Array[2], it will print out kai; when using print[“birth”], it will get 1987.

Read the value of the array

{<!-- --> for(item in array) {<!-- -->print array[item]}; } #The order of output is random
{<!-- --> for(i=1;i<=len;i + + ) {<!-- -->print array[i]}; } #Len is the length of the array

Array related functions

Get the array length:

awk 'BEGIN{info="it is a test";lens=split(info,tA," ");print length(tA),lens;}'

Output:

4 4

length returns the length of the string and array, split splits the string into an array, and returns the length of the array obtained by splitting.

awk 'BEGIN{info="it is a test";split(info,tA," ");print asort(tA);}'

Output:

asort sorts the array and returns the array length.

Output array contents (unordered, ordered output):

awk 'BEGIN{info="it is a test";split(info,tA," ");for(k in tA){print k,tA[k];}}'

Output:

4 test
1 it
2 is
3 a

for…in output, because the array is an associative array, it is unordered by default. So what you get through for…in is an unordered array. If you need to get an ordered array, you need to get it through subscripting.

awk 'BEGIN{info="it is a test";tlen=split(info,tA," ");for(k=1;k<=tlen;k + + ){print k,tA[k ];}}'

Output:

1 it
2 is
3 a
4 tests

Note: Array subscripts start from 1, which is different from C arrays.

Determine the existence of key value and delete key value:

Wrong judgment method:

awk 'BEGIN{tB["a"]="a1";tB["b"]="b1";if(tB["c"]!="1"){print "no found"; };for(k in tB){print k,tB[k];}}'

Output:

no found
a a1
b b1
c

A strange problem occurs above. tB[“c”] is not defined, but when looping, it is found that the key value already exists and its value is empty. It should be noted here that the awk array is an associative array. As long as its key is referenced through the array, it will The sequence will be automatically created.

Correct way to judge:

awk 'BEGIN{tB["a"]="a1";tB["b"]="b1";if( "c" in tB){print "ok";};for(k in tB ){print k,tB[k];}}'

Output:

a a1
b b1

if(key in array) uses this method to determine whether the array contains the key value.

Delete key value:

awk 'BEGIN{tB["a"]="a1";tB["b"]="b1";delete tB["a"];for(k in tB){print k,tB[k ];}}'

Output:

b b1

delete array[key] can be deleted, corresponding to the sequence value of the array key.

Using two-dimensional and multi-dimensional arrays

Awk’s multi-dimensional array is essentially a one-dimensional array. To be more precise, awk does not support multi-dimensional arrays in storage. Awk provides an access method that logically simulates a two-dimensional array. For example, access such as array[2,4]=1 is allowed. Awk uses a special string SUBSEP (?34) as the split field. In the above example, the key value stored in the associative array array is actually 2?344.

Similar to member testing of one-dimensional arrays, syntax such as if ((i,j) in array) can be used for multi-dimensional arrays, but the subscripts must be placed in parentheses. Similar to the iterative access of one-dimensional arrays, multi-dimensional arrays use syntax such as for (item in array) to traverse the array. Unlike one-dimensional arrays, multi-dimensional arrays must use the split() function to access the individual subscript components.

awk 'BEGIN{
for(i=1;i<=9;i + + ){
  for(j=1;j<=9;j + + ){
    tarr[i,j]=i*j; print i,"*",j,"=",tarr[i,j];
  }
}
}'

Output:

1 * 1 = 1
1 * 2 = 2
1 * 3 = 3
1 * 4 = 4
1 * 5 = 5
1 * 6 = 6
...
9 * 6 = 54
9 * 7 = 63
9 * 8 = 72
9 * 9 = 81

The array contents can be obtained through array[k,k2] reference.

Another way:

awk 'BEGIN{
for(i=1;i<=9;i + + ){
  for(j=1;j<=9;j + + ){
    tarr[i,j]=i*j;
  }
}
for(m in tarr){
  split(m,tarr2,SUBSEP); print tarr2[1],"*",tarr2[2],"=",tarr[m];
}
}'

Learn `python`

from scratch

[Learn python from scratch] 92. Use Python’s requests library to send HTTP requests and process responses
[Learn python from scratch] 91. A simple web application that uses decorators and dictionaries to manage request paths
[Learn python from scratch] 93. Use dictionaries to manage request paths
[Learn python from scratch] 89. Use WSGI to build a simple and efficient web server
[Learn python from scratch] 88. Detailed explanation of WSGI interface: achieving simple and efficient web development
[Learn python from scratch] 87. Manually build Python implementation of HTTP server and multi-thread concurrent processing
[Learn python from scratch] 86. In-depth understanding of the HTTP protocol and its role in browser and server communication
[Learn python from scratch] 85. Application of parallel computing technology in Python process pool
[Learn python from scratch] 84. In-depth understanding of threads and processes
[Learn python from scratch] 83. Python multi-process programming and the use of process pools
[Learn python from scratch] 82. Implementation of chat program based on multi-threading
[Learn python from scratch] 81. Application of Python multi-thread communication and queue
[Learn python from scratch] 80. Thread access to global variables and thread safety issues
[Learn python from scratch] 79. Thread access to global variables and thread safety issues
[Learn python from scratch] 78. File download case
[Learn python from scratch] 77. TCP server programming and precautions
[Learn python from scratch] 76. Server and client: key components of network communication
[Learn python from scratch] 75. TCP protocol: a reliable connection-oriented transport layer communication protocol
[Learn python from scratch] 74. UDP network program: detailed explanation of port issues and binding information
[Learn python from scratch] 73. UDP network program-send data
[Learn python from scratch] 72. In-depth understanding of Socket communication and methods of creating sockets
[Learn python from scratch] 71. Network ports and their functions
[Learn python from scratch] 70. Network communication methods and their applications: from direct communication to routers connecting multiple networks
[Learn python from scratch] 69. Network communication and IP address classification analysis
[Learn python from scratch] 68. Greedy and non-greedy modes in Python regular expressions
[Learn python from scratch] 67. The re module in Python: regular replacement and advanced matching technology
[Learn python from scratch] 66. In-depth understanding of regular expressions: a powerful tool for pattern matching and text processing
[Learn python from scratch] 65. Detailed explanation of Python regular expression modifiers and their applications
[Learn python from scratch] 64. Detailed explanation of the use of re.compile method in Python regular expressions
[Learn python from scratch] 63. Introduction to the re.Match class and its attributes and methods in regular expressions
[Learn python from scratch] 62. Python regular expressions: a powerful string matching tool
[Learn python from scratch] 61. Detailed explanation and application examples of property attributes in Python
[Learn python from scratch] 60. Exploration generator: a flexible tool for iteration
[Learn python from scratch] 59. Iterator: an efficient tool for optimizing data traversal
[Learn python from scratch] 58. Custom exceptions in Python and methods of raising exceptions
[Learn python from scratch] 57. Use the with keyword in Python to correctly close resources
[Learn python from scratch] 56. The importance and application of exception handling in programming
[Learn python from scratch] 55. Serialization and deserialization in Python, application of JSON and pickle modules
[Learn python from scratch] 54. Write data in memory
[Learn python from scratch] 53. CSV files and Python’s CSV module
[Learn python from scratch] 52. Reading and writing files – Python file operation guide
[Learn python from scratch] 51. Opening and closing files and their application in Python
[Learn python from scratch] 49. Object-related built-in functions in Python and their usage
[Learn python from scratch] 48. Detailed explanation of inheritance and multiple inheritance in Python
[Learn python from scratch] 47. The concept and basic use of inheritance in object-oriented programming
[Learn python from scratch] 46. Analysis of __new__ and __init__ methods and singleton design pattern in Python
[Learn python from scratch] 45. Class methods and static methods in Python
[Learn python from scratch] 44. Private properties and methods in object-oriented programming
[Learn python from scratch] 43. Instance attributes and class attributes in Python object-oriented programming
[Learn python from scratch] 42. Built-in properties and methods in Python
[Learn python from scratch] 41. python magic method (2)
[Learn python from scratch] 40. python magic method (1)
[Learn python from scratch] 39. Basic object-oriented syntax and application examples
[Learn python from scratch] 38. How to use and import Python packages
[Learn python from scratch] 37. The use and precautions of Python custom modules
[Learn python from scratch] 36. Methods and techniques for using pip for third-party package management in Python
[Learn python from scratch] 35. Python common system modules and their usage
[Learn python from scratch] 34. Detailed explanation of how to import and use Python modules
[Learn python from scratch] 33. The role of decorators (2)
[Learn python from scratch] 32. The role of decorators (1)
[Learn python from scratch] 31. In-depth understanding of higher-order functions and closures in Python
[Learn python from scratch] 30. In-depth understanding of recursive functions and anonymous functions
[Learn python from scratch] 29. “Detailed explanation of function parameters” – Understand the different uses of Python function parameters
[Learn python from scratch] 28. Local variables and global variables in Python
[Learn python from scratch] 27. Use of Python functions and nested calls
[Learn python from scratch] 25. Function: a powerful tool to improve the efficiency of code writing
[Learn python from scratch] 24. String operations and traversal methods in Python
[Learn python from scratch] 23. How to use sets (set) and common operations in Python
[Learning python from scratch] 22. Addition, deletion, modification, and dictionary variables in Python
[Learn python from scratch] 21. Tuples and dictionaries in Python
[Learn python from scratch] 20. Python list operation skills and examples
[Learn python from scratch] 19. Application of looping through lists and list nesting
[Learn python from scratch] 18. Detailed explanation of the basic operations of Python lists (1)
[Learn python from scratch] 17. Python string format method (2)
[Learn python from scratch] 16. Python string format method (1)
[Learn python from scratch] 15. In-depth understanding of string and character set encoding
[Learn python from scratch] 14. Common operations on Python strings (2)
[Learn python from scratch] 13. Common operations on Python strings (1)
[Learn python from scratch] 12. Python string operations and applications
[Learn python from scratch] 11. Python loop statements and control flow
[Learn python from scratch] 10. Detailed explanation of Python conditional statements and if nesting
[Learn python from scratch] 09. Conditional judgment statements in Python
[Learn python from scratch] 08. Python understands bitwise operators and operator precedence
[Learn python from scratch] 07. Detailed explanation of Python operators: assignment, comparison and logical operators
[Learn python from scratch] 06. Use arithmetic operators for calculation and string concatenation in Python
[Learn python from scratch] 05. Output and input in Python
[Learn python from scratch] 04. Python programming basics: variables, data types and identifiers
[Learn python from scratch] 03. Python interactive programming and detailed explanation of comments
[Learn python from scratch] 02. Introduction to development tools
[Learn python from scratch] 01. Install and configure python