Introduction
In development and operation and maintenance, we often need to monitor and analyze the interface traffic size of the server, especially for servers where Nginx is deployed. This article will introduce how to use Golang to collect Nginx interface traffic size, and show how to monitor and analyze these data in real time.
Step 1: Preparation
Before we begin, we need to make some preparations.
-
Install Golang: First, make sure you have Golang installed on your server. It can be downloaded from the Golang official website and installed according to the official documentation.
-
Install Nginx: If Nginx is not already installed on your server, install Nginx according to your operating system documentation. Make sure Nginx is running correctly and listening on the appropriate interface.
-
Install the statistics module: We will use Nginx’s statistics module to collect traffic data. You can find the relevant statistics module on Nginx’s third-party module page and follow the instructions to install and configure it.
Step 2: Write code
Now, we can start writing the code to collect the traffic volume of the Nginx interface.
1. Get Nginx statistics
We first need to obtain data from the statistics interface of Nginx. You can use Golang’s net/http package to send HTTP requests and get response data.
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {<!-- -->
resp, err := http.Get("http://localhost/nginx_status")
if err != nil {<!-- -->
fmt.Println("Failed to obtain Nginx statistics:", err)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {<!-- -->
fmt.Println("Failed to read Nginx statistics:", err)
return
}
fmt.Println(string(body))
}
In the above code, we use the http.Get method to send an HTTP GET request, and use the ioutil.ReadAll method to read the response data into the variable body middle. You need to replace http://localhost/nginx_status with your actual Nginx statistics interface address.
2. Analyze traffic data
Next, we need to parse the traffic size from the Nginx statistics. According to the documentation of the Nginx statistics module, we can extract the information we need from the response data.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
)
func main() {<!-- -->
resp, err := http.Get("http://localhost/nginx_status")
if err != nil {<!-- -->
fmt.Println("Failed to obtain Nginx statistics:", err)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {<!-- -->
fmt.Println("Failed to read Nginx statistics:", err)
return
}
stats := string(body)
lines := strings.Split(stats, "\
")
for _, line := range lines {<!-- -->
if strings.Contains(line, "Active connections") {<!-- -->
fmt.Println(line)
} else if strings.HasPrefix(line, "server accepts handled") {<!-- -->
parts := strings.Fields(line)
fmt.Println("Number of accepted connections:", parts[2])
fmt.Println("Number of connections processed:", parts[3])
fmt.Println("Number of completed requests:", parts[4])
} else if strings.HasPrefix(line, "Reading: ") {<!-- -->
parts := strings.Fields(line)
fmt.Println("Number of connections being read:", parts[1])
} else if strings.HasPrefix(line, "Writing: ") {<!-- -->
parts := strings.Fields(line)
fmt.Println("Number of connections being written:", parts[1])
} else if strings.HasPrefix(line, "Waiting: ") {<!-- -->
parts := strings.Fields(line)
fmt.Println("Number of waiting connections:", parts[1])
}
}
}
The above code first splits the response data by rows and parses them row by row. According to the documentation of the Nginx statistics module, we can determine what traffic data the content of each line is and output the corresponding information.
3. Collect data regularly
In order to achieve real-time monitoring and analysis, we can use Golang’s time package to regularly collect data and save the data to a file or send it to the server for further analysis.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
"time"
)
func main() {<!-- -->
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
for range ticker.C {<!-- -->
resp, err := http.Get("http://localhost/nginx_status")
if err != nil {<!-- -->
fmt.Println("Failed to obtain Nginx statistics:", err)
continue
}
body, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {<!-- -->
fmt.Println("Failed to read Nginx statistics:", err)
continue
}
stats := string(body)
lines := strings.Split(stats, "\
")
for _, line := range lines {<!-- -->
// Parse traffic data...
}
}
}
The above code uses time.NewTicker to create a timer that collects data every 10 seconds. Each time the timer fires, an HTTP request is sent to obtain the data, parsed and processed. You can adjust the timer interval as needed.
Step 3: Data monitoring and analysis
Finally, we can monitor and analyze the collected data in real time. You can use various tools and libraries to achieve this, such as ECharts, Grafana, etc. Here, we take ECharts as an example to show how to display traffic data in real time.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
"time"
"github.com/go-echarts/go-echarts/charts"
"github.com/go-echarts/go-echarts/opts"
"github.com/go-echarts/go-echarts/templates"
)
var (
activeConnections[]opts.LineData
accepts[]opts.LineData
handled[]opts.LineData
requests[]opts.LineData
reading[]opts.LineData
writing[]opts.LineData
waiting []opts.LineData
)
func main() {<!-- -->
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {<!-- -->
page := charts.NewPage()
line := charts.NewLine()
line.SetGlobalOptions(
charts.InitOpts{<!-- -->PageTitle: "Interface Traffic Monitoring"},
charts.ToolboxOpts{<!-- -->Show: true},
charts.YAxisOpts{<!-- -->Name: "Quantity"},
charts.XAxisOpts{<!-- -->Name: "Time"},
)
line.AddXAxis(timeValues())
line.AddYAxis("Active Connections", activeConnections)
line.AddYAxis("Accepts", accepts)
line.AddYAxis("Handled", handled)
line.AddYAxis("Requests", requests)
line.AddYAxis("Reading", reading)
line.AddYAxis("Writing", writing)
line.AddYAxis("Waiting", waiting)
page.Add(line)
w.Header().Set("Content-Type", "text/html; charset=utf-8")
page.Render(w)
})
go func() {<!-- -->
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
for range ticker.C {<!-- -->
resp, err := http.Get("http://localhost/nginx_status")
if err != nil {<!-- -->
fmt.Println("Failed to obtain Nginx statistics:", err)
continue
}
body, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {<!-- -->
fmt.Println("Failed to read Nginx statistics:", err)
continue
}
stats := string(body)
lines := strings.Split(stats, "\
")
for _, line := range lines {<!-- -->
// Parse traffic data...
}
line := charts.NewLine()
line.SetGlobalOptions(
charts.InitOpts{<!-- -->PageTitle: "Interface Traffic Monitoring"},
charts.ToolboxOpts{<!-- -->Show: true},
charts.YAxisOpts{<!-- -->Name: "Quantity"},
charts.XAxisOpts{<!-- -->Name: "Time"},
)
line.AddXAxis(timeValues())
line.AddYAxis("Active Connections", activeConnections)
line.AddYAxis("Accepts", accepts)
line.AddYAxis("Handled", handled)
line.AddYAxis("Requests", requests)
line.AddYAxis("Reading", reading)
line.AddYAxis("Writing", writing)
line.AddYAxis("Waiting", waiting)
page := charts.NewPage()
page.Add(line)
s := templates.MustString(echartsTemplate, page)
ioutil.WriteFile("index.html", []byte(s), 0644)
}
}()
http.ListenAndServe(":8080", nil)
}
func timeValues() []string {<!-- -->
t := time.Now().Format("2006-01-02 15:04:05")
return []string{<!-- -->t}
}
const echartsTemplate = `
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>{<!-- -->{<!-- -->"{<!-- -->{.PageTitle}}"}}</title>
{<!-- -->{<!-- --> template "asset" . }}
</head>
<body>
{<!-- -->{<!-- --> template "js" . }}
</body>
</html>
The above code uses the go-echarts library to generate ECharts instances and pass real-time data to the corresponding ECharts instances for chart display. After collecting the data, we can output the page to an HTML file and then open the file using a browser to view real-time data charts.
Conclusion
Through the above steps, we successfully used Golang to collect the Nginx interface traffic size, and used ECharts to monitor and analyze the data in real time. You can further refine and extend these codes according to your needs and preferences. Hope this article can help you realize your monitoring needs!