1. Standard log library log
In daily development, logging is an essential function. Although sometimes you can use the fmt library to output some information, it is not flexible enough. The Go standard library provides a logging library log.
1. Quick use
Log is provided by the Go standard library and does not require additional installation.
package main
import (
"log"
)
typeUser struct {
Name string
Age int
}
func main() {
u := User{
Name: "test",
Age: 18,
}
log.Printf("%s login, age:%d", u.Name, u.Age)
log.Panicf("Oh, system error when %s login", u.Name)
log.Fatalf("Danger! hacker %s login", u.Name)
}
The log is output to standard error (stderr) by default, and the date and time are automatically added before each log. If the log does not end with a newline character, the log will automatically add a newline character. That is, each log will be output in a new line.
Log provides three sets of functions:
-
1. Print/Printf/Println: normal output log;
-
2. Panic/Panicf/Panicln: After outputting the log, call panic with the assembled string as parameter;
-
3. Fatal/Fatalf/Fatalln: After outputting the log, call os.Exit(1) to exit the program. The names are easier to identify. Those with f suffix have formatting function, and those with ln suffix will add a newline character after the log.
Note that in the above program, calling log.Panicf will panic, so log.Fatalf will not be called.
2. Customization options
Options
-
? Ldate: Output the date in the local time zone, such as 2020/02/07;
-
? Ltime: Output the time in the local time zone, such as 11:45:45;
-
? Lmicroseconds: The output time is accurate to microseconds. If this option is set, there is no need to set Ltime. Such as 11:45:45.123123;
-
? Llongfile: Output long file name + line number, including package name, such as github.com/darjun/go-daily-lib/log/flag/main.go:50;
-
? Lshortfile: Output short file name + line number, excluding package name, such as main.go:50;
-
? LUTC: If Ldate or Ltime is set, UTC time will be output instead of the local time zone.
log.SetFlags(log.Lshortfile | log.Ldate | log.Lmicroseconds)
log.SetPrefix("Debug: ")
3. Output to file
file := "./" + "message" + ".txt"
logFile, err := os.OpenFile(file, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0766)
if err != nil {
panic(err)
}
log.SetOutput(logFile) //Set the file as the log output file
log.SetPrefix("[qcpz]")
log.SetFlags(log.LstdFlags | log.Lshortfile | log.Ldate | log.Ltime)
4. Customized output
In fact, the log library defines a default Logger for us, named std, which means standard log. The method of the log library that we call directly, internally is the corresponding method of calling std:
// src/log/log.go
var std = New(os.Stderr, "", LstdFlags)
func Printf(format string, v ...interface{}) {
std.Output(2, fmt.Sprintf(format, v...))
}
func Fatalf(format string, v ...interface{}) {
std.Output(2, fmt.Sprintf(format, v...))
os.Exit(1)
}
func Panicf(format string, v ...interface{}) {
s := fmt.Sprintf(format, v...)
std.Output(2, s)
panic(s)
}
log.New accepts three parameters:
-
? io.Writer: Logs will be written to this Writer;
-
? prefix: prefix, which can also be set by calling logger.SetPrefix later;
-
? flag: option, which can also be set by calling logger.SetFlag later.
You can use io.MultiWriter to achieve multi-destination output
package main
import (
"bytes"
"io"
"log"
"os"
)
typeUser struct {
Name string
Age int
}
func main() {
u := User{
Name: "test",
Age: 18,
}
writer1 := & amp;bytes.Buffer{}
writer2 := os.Stdout
writer3, err := os.OpenFile("log.txt", os.O_WRONLY|os.O_CREATE, 0755)
if err != nil {
log.Fatalf("create file log.txt failed: %v", err)
}
logger := log.New(io.MultiWriter(writer1, writer2, writer3), "", log.Lshortfile|log.LstdFlags)
logger.Printf("%s login, age:%d", u.Name, u.Age)
}
2. Use of logrus
1. golang log library
The logging framework of the golang standard library is very simple. It only provides three functions: print, panic, and fatal. It does not provide support for more precise log levels, log file segmentation, and log distribution. Therefore, many third-party logging libraries have been spawned, but in the world of golang, no logging library has the absolute dominance in Java like slf4j.
In golang, popular logging frameworks include logrus, zap, zerolog, seelog, etc..
-
? Logrus is currently the log library with the largest number of stars on Github. Logrus is powerful, efficient, and highly flexible, providing custom plug-in functionality. Many open source projects, such as docker, prometheus, etc., use logrus to record their logs. Source: Official account [Advanced Notes on Programming for Coders]
-
? zap is a fast, structured hierarchical log library launched by Uber. It has powerful ad-hoc analysis capabilities and has a flexible dashboard.
-
? Seelog provides flexible asynchronous scheduling, formatting and filtering functions.
2. Logrus features
GitHub access address: https://github.com/sirupsen/logrus
logrus has the following features:
-
? Fully compatible with the golang standard library log module: logrus has six log levels: debug, info, warn, error, fatal and panic, which is a superset of the API of the golang standard library log module. If your project uses the standard library log module, you can migrate it to logrus at the lowest cost.
-
? Extensible hook mechanism: allows users to distribute logs to any place through hooks, such as local file system, standard output, logstash, elasticsearch or mq, etc., or define log content and format through hooks.
-
? Optional log output format: logrus has two built-in log formats, JSONFormatter and TextFormatter. If these two formats do not meet the needs, you can implement the interface Formatter yourself to define your own log format.
-
? Field mechanism: Logrus encourages refined and structured logging through the Field mechanism instead of logging through lengthy messages.
-
? Logrus is a pluggable, structured logging framework.
Although logrus has many advantages, for the sake of flexibility and scalability, the official has also cut many practical functions, such as:
-
? No support for line numbers and file names is provided
-
? Output to local file system does not provide log splitting function
-
? The official does not provide the function of outputting to log processing centers such as ELK, but these functions can be realized through custom hooks.
3. Log format
For example, we agree that the log format is Text and contains the following fields:
Request time, log level, status code, execution time, request IP, request method, and request routing.
4. How to use
package main
import (
"flag"
"fmt"
"os"
"path"
"runtime"
"strings"
"time"
"github.com/Sirupsen/logrus"
)
func logrus_test() {
fmt.Printf("<<<<<<<<<logrus test>>>>>>>>>>>>\\
")
logrus.WithFields(logrus.Fields{
"sb": "sbvalue",
}).Info("A walrus appears")
log1 := logrus.New()
fmt.Printf("log1 level: %d\\
", log1.Level)
log1.Debug("log1 debug")
log1.Debugf("log1 debug f, %d", 10)
log1.Info("log1 info")
log1.Warn("log1 warn")
log1.Error("log1 error")
// log1.Panic("log1 panic")
log1.SetLevel(logrus.ErrorLevel)
fmt.Printf("after set log1 level to errorlevel\\
")
log1.Debug("log1 debug")
fmt.Printf("-------------test formater-------------\\
")
log1.SetLevel(logrus.DebugLevel)
log1.Formatter = & amp;logrus.TextFormatter{
DisableColors: true,
FullTimestamp: true,
DisableSorting: true,
}
log1.Debug("log text formatter test")
fmt.Printf("----------json formatter-------------\\
")
log1.Formatter = & amp;logrus.JSONFormatter{}
log1.Debug("log json formatter test")
fmt.Printf("-----------log to file test-----------\\
")
log2 := logrus.New()
log2.SetLevel(logrus.DebugLevel)
log2.Formatter = & amp;logrus.TextFormatter{
DisableColors: true,
FullTimestamp: true,
DisableSorting: true,
}
logger_name := "logrus"
cur_time := time.Now()
log_file_name := fmt.Sprintf("%s_ d- d- d- d- d.txt",
logger_name, cur_time.Year(), cur_time.Month(), cur_time.Day(), cur_time.Hour(), cur_time.Minute())
log_file, err := os.OpenFile(log_file_name, os.O_CREATE|os.O_APPEND|os.O_WRONLY, os.ModeExclusive)
if err != nil {
fmt.Printf("try create logfile[%s] error[%s]\\
", log_file_name, err.Error())
return
}
defer log_file.Close()
log2.SetOutput(log_file)
for i := 0; i < 10; i + + {
log2.Debugf("logrus to file test %d", i)
}
}
5. Simple example
package main
import (
"os"
log "github.com/sirupsen/logrus"
)
func init() {
//Set the log format to json format
log.SetFormatter( & amp;log.JSONFormatter{})
// Set the log output to the standard output (the default output is stderr, standard error)
// Log message output can be of any io.writer type
log.SetOutput(os.Stdout)
//Set the log level to warn or above
log.SetLevel(log.WarnLevel)
}
func main() {
log.WithFields(log.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
log.WithFields(log.Fields{
"omg": true,
"number": 122,
}).Warn("The group's number increased tremendously!")
log.WithFields(log.Fields{
"omg": true,
"number": 100,
}).Fatal("The ice breaks!")
}
6. Logger
Logger is a relatively advanced usage. For a large project, a global logrus instance, that is, a logger object, is often needed to record all logs of the project. like
package main
import (
"github.com/sirupsen/logrus"
"os"
)
// Logrus provides the New() function to create a logrus instance.
// In the project, any number of logrus instances can be created. Source: Official account [Advanced Notes on Programming for Coders]
var log = logrus.New()
func main() {
// Set the message output for the current logrus instance, similarly,
// You can set the output of the logrus instance to any io.writer
log.Out = os.Stdout
//Set the message output format to json format for the current logrus instance.
//Similarly, you can also set the log level and hook separately for a logrus instance, which will not be described in detail here.
log.Formatter = & amp;logrus.JSONFormatter{}
log.WithFields(logrus.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
}
7. Fields
Logrus does not recommend using lengthy messages to record running information. It recommends using Fields to record detailed and structured information.
For example, the following log recording method:
log.Fatalf("Failed to send event %s to topic %s with key %d", event, topic, key)
//alternative plan
//Source: Official account [Advanced Notes on Programming for Coders]
log.WithFields(log.Fields{
"event": event,
"topic": topic,
"key": key,
}).Fatal("Failed to send event")
8. Use of gin framework log middleware
package middleware
import (
"fmt"
"ginDemo/config"
"github.com/gin-gonic/gin"
rotatelogs "github.com/lestrrat-go/file-rotatelogs"
"github.com/rifflock/lfshook"
"github.com/sirupsen/logrus"
"os"
"path"
"time"
)
// Log to file
func LoggerToFile() gin.HandlerFunc {
logFilePath := config.Log_FILE_PATH
logFileName := config.LOG_FILE_NAME
// log file
fileName := path.Join(logFilePath, logFileName)
// write to file
src, err := os.OpenFile(fileName, os.O_APPEND|os.O_WRONLY, os.ModeAppend)
if err != nil {
fmt.Println("err", err)
}
// instantiate
logger := logrus.New()
//Set the output
logger.Out = src
//Set log level
logger.SetLevel(logrus.DebugLevel)
// Set rotatelogs
logWriter, err := rotatelogs.New(
//Split file name
fileName + ".%Y%m%d.log",
// Generate a soft link pointing to the latest log file
rotatelogs.WithLinkName(fileName),
//Set the maximum storage time (7 days) Source: Official account [Advanced Notes on Programming for Coders]
rotatelogs.WithMaxAge(7*24*time.Hour),
//Set the log cutting interval (1 day)
rotatelogs.WithRotationTime(24*time.Hour),
)
writeMap := lfshook.WriterMap{
logrus.InfoLevel: logWriter,
logrus.FatalLevel: logWriter,
logrus.DebugLevel: logWriter,
logrus.WarnLevel: logWriter,
logrus.ErrorLevel: logWriter,
logrus.PanicLevel: logWriter,
}
lfHook := lfshook.NewHook(writeMap, & amp;logrus.JSONFormatter{
TimestampFormat:"2006-01-02 15:04:05",
})
//Add Hook
logger.AddHook(lfHook)
return func(c *gin.Context) {
// Starting time
startTime := time.Now()
// handle the request
c.Next()
// End Time
endTime := time.Now()
// execution time
latencyTime := endTime.Sub(startTime)
// Request method
reqMethod := c.Request.Method
//Request routing
reqUri := c.Request.RequestURI
// status code
statusCode := c.Writer.Status()
//Request IP
clientIP := c.ClientIP()
//Log format
logger.WithFields(logrus.Fields{
"status_code" : statusCode,
"latency_time" : latencyTime,
"client_ip" : clientIP,
"req_method" : reqMethod,
"req_uri" : reqUri,
}).Info()
}
}
// Log to MongoDB
func LoggerToMongo() gin.HandlerFunc {
return func(c *gin.Context) {
}
}
// Log records to ES Source: Official account [Advanced Notes on Programming for Coders]
func LoggerToES() gin.HandlerFunc {
return func(c *gin.Context) {
}
}
// Log to MQ
func LoggerToMQ() gin.HandlerFunc {
return func(c *gin.Context) {
}
}
9. Simple log cutting
Need to introduce external components
package main
import (
"time"
rotatelogs "github.com/lestrrat-go/file-rotatelogs"
log "github.com/sirupsen/logrus"
)
func init() {
path := "message.log"
/* Log rotation related functions
`WithLinkName` creates a soft link for the latest log
`WithRotationTime` sets the time for log splitting and how often it should be split.
Only one of WithMaxAge and WithRotationCount can be set
`WithMaxAge` sets the maximum storage time before files are cleaned
`WithRotationCount` sets the maximum number of files saved before cleaning
*/
// The following configuration log rotates a new file every 1 minute, retains the last 3 minutes of log files, and automatically cleans up the excess.
writer, _ := rotatelogs.New(
path + ".%Y%m%d%H%M",
rotatelogs.WithLinkName(path),
rotatelogs.WithMaxAge(time.Duration(180)*time.Second),
rotatelogs.WithRotationTime(time.Duration(60)*time.Second),
)
log.SetOutput(writer)
//log.SetFormatter( & amp;log.JSONFormatter{})
}
func main() {
for {
log.Info("hello, world!")
time.Sleep(time.Duration(2) * time.Second)
}
}
10. How to use ZAP (highest performance)
package main
import (
"flag"
"fmt"
"os"
"path"
"runtime"
"strings"
"time"
"github.com/golang/glog"
)
func zap_log_test() {
fmt.Printf("<<<<<<<<<zap log test>>>>>>>>>>\\
")
logger := zap.NewExample()
defer logger.Sync()
const url = "http://example.com"
// In most circumstances, use the SugaredLogger. It's 4-10x faster than most
// other structured logging packages and has a familiar, loosely-typed API.
sugar := logger.Sugar()
sugar.Infow("Failed to fetch URL.",
// Structured context as loosely typed key-value pairs.
"url", url,
"attempt", 3,
"backoff", time.Second,
)
sugar.Infof("Failed to fetch URL: %s", url)
// In the unusual situations where every microsecond matters, use the
// Logger. It's even faster than the SugaredLogger, but only supports
// structured logging.
logger.Info("Failed to fetch URL.",
// Structured context as strongly typed fields.
zap.String("url", url),
zap.Int("attempt", 3),
zap.Duration("backoff", time.Second),
)
}