[Go]–Golang handles HTTP/HTTPS requests

1. Supplementary HTTP knowledge points

http operation method

HTTP defines different methods for interacting with the server. There are four basic methods, namely GET, POST, PUT, and DELETE. The full name of URL is resource descriptor. We can think of it this way: a URL address is used to describe a resource on the network, and GET, POST, PUT, and DELETE in HTTP correspond to the four operations of checking, modifying, adding, and deleting this resource. At this point, everyone should have a general understanding. GET is generally used to obtain/query resource information, while POST is generally used to update resource information. So, in addition to the four methods mentioned above, are there other methods for HTTP? In fact, the following request methods are defined in HTTP:

GET method;
POST method;
PUT method;
DELETE method.
HEAD method;
TRACE method;
OPTIONS method;

1. Get is the most commonly used method, usually used to request the server to send a certain resource, and should be safe and idempotent.

(1). The so-called security means that the operation is used to obtain information rather than modify information. In other words, GET
Requests should generally not have side effects. That is to say, it only obtains resource information, just like a database query, and does not modify or add data, nor affect the status of resources.

Note: The meaning of security here only refers to non-modified information.

(2). Idempotence means that multiple requests to the same URL should return the same result.

The POST method submits data to the server, such as completing the submission of form data and submitting the data to the server for processing.
The PUT method tells the server to use the body part of the request to create a new document named by the requested URL; if that document exists, use the body to replace it.
The DELETE method is to request the server to delete the resource corresponding to the specified URL. However, the client has no guarantee that the delete operation will be performed because the HTTP specification allows the server to revoke the request without notifying the client.
The HEAD method behaves similarly to the GET method, but the server returns only the body part of the entity in the response. This allows the client to inspect the header of the resource without obtaining the actual resource.

Using HEAD, we can complete the following tasks more efficiently:

①. Understand some information about the resource, such as resource type, without obtaining the resource;
②. By checking the status code in the response, you can determine whether the resource exists;
③. Test whether the resource has been modified by checking the header.

The TRACE method will initiate a “loopback” diagnosis on the destination server. We all know that when the client initiates a request, the request may have to pass through a firewall, proxy, gateway, or other applications. Each node in between may modify the original HTTP request, and the TRACE method allows the client to determine what the request looks like when it is finally sent to the server. Due to a “loopback” diagnosis, when the request finally reaches the server, the server will bounce a TRACE response and carry in the response body the final appearance of the original request message it received. In this way, the client can check whether the HTTP request message has been modified while being sent.
The OPTIONS method is used to obtain the methods supported by the current URL. If the request is successful, it will include a header named “Allow” in the HTTP header, and the value is the supported method, such as “GET, POST”.

2. GET request

Basic get request

//Basic GET request
package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main() {<!-- -->
    resp, err := http.Get("http://httpbin.org/get")
    if err != nil {<!-- -->
        fmt.Println(err)
        return
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
    fmt.Println(resp.StatusCode)
    if resp.StatusCode == 200 {<!-- -->
        fmt.Println("ok")
    }
}

Get request with parameters

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main(){<!-- -->
    resp, err := http.Get("http://httpbin.org/get?name=zhangsan & amp;age=23")
    if err != nil {<!-- -->
        fmt.Println(err)
        return
    }
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
 
}

3. But what if we want to make some parameters into variables instead of putting them directly in the URL? The code example is as follows:

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
    "net/url"
)
 
func main(){<!-- -->
    params := url.Values{<!-- -->}
    Url, err := url.Parse("http://httpbin.org/get")
    if err != nil {<!-- -->
        return
    }
    params.Set("name","zhangsan")
    params.Set("age","23")
    //If there are Chinese parameters in the parameters, this method will URLEncode
    Url.RawQuery = params.Encode()
    urlPath := Url.String()
    fmt.Println(urlPath) // https://httpbin.org/get?age=23 & name=zhangsan
    resp,err := http.Get(urlPath)
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
}

Parsing the return result of JSON type

package main
 
import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
)
 
type result struct {<!-- -->
    Args string `json:"args"`
    Headers map[string]string `json:"headers"`
    Origin string `json:"origin"`
    Url string `json:"url"`
}
 
func main() {<!-- -->
    resp, err := http.Get("http://httpbin.org/get")
    if err != nil {<!-- -->
        return
    }
    defer resp.Body.Close()
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
    var res result
    _ = json.Unmarshal(body, & amp;res)
    fmt.Printf("%#v", res)
}

5.Add request header to GET request

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main() {<!-- -->
    client := & amp;http.Client{<!-- -->}
    req,_ := http.NewRequest("GET","http://httpbin.org/get",nil)
    req.Header.Add("name","zhangsan")
    req.Header.Add("age","3")
    resp,_ := client.Do(req)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Printf(string(body))
}

3. POST request

Basic post request

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
    "net/url"
)
 
func main() {<!-- -->
    urlValues := url.Values{<!-- -->}
    urlValues.Add("name","zhangsan")
    urlValues.Add("age","22")
    resp, _ := http.PostForm("http://httpbin.org/post",urlValues)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
}

The result is as follows:

{<!-- -->
  "args": {<!-- -->},
  "data": "",
  "files": {<!-- -->},
  "form": {<!-- -->
    "age": "22",
    "name": "zhangsan"
  },
  "headers": {<!-- -->
    "Accept-Encoding": "gzip",
    "Content-Length": "19",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "httpbin.org",
    "User-Agent": "Go-http-client/1.1"
  },
  "json": null,
  "origin": "211.138.20.170, 211.138.20.170",
  "url": "https://httpbin.org/post"
}

another way

package main
 
import (
    "fmt"
    "io/ioutil"
    "net/http"
    "net/url"
    "strings"
)
 
func main() {<!-- -->
    urlValues := url.Values{<!-- -->
        "name":{<!-- -->"zhangsan"},
        "age":{<!-- -->"23"},
    }
    reqBody:= urlValues.Encode()
    resp, _ := http.Post("http://httpbin.org/post", "text/html",strings.NewReader(reqBody))
    body,_:= ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
}

result

{<!-- -->
  "args": {<!-- -->},
  "data": "age=23 & amp;name=zhangsan",
  "files": {<!-- -->},
  "form": {<!-- -->},
  "headers": {<!-- -->
    "Accept-Encoding": "gzip",
    "Content-Length": "19",
    "Content-Type": "text/html",
    "Host": "httpbin.org",
    "User-Agent": "Go-http-client/1.1"
  },
  "json": null,
  "origin": "211.138.20.170, 211.138.20.170",
  "url": "https://httpbin.org/post"
}

Post request to send JSON data

package main
 
import (
    "bytes"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "net/http"
)
 
func main() {<!-- -->
    client := & amp;http.Client{<!-- -->}
    data := make(map[string]interface{<!-- -->})
    data["name"] = "zhangsan"
    data["age"] = "23"
    bytesData, _ := json.Marshal(data)
    req, _ := http.NewRequest("POST","http://httpbin.org/post",bytes.NewReader(bytesData))
    resp, _ := client.Do(req)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))
 
}

result

{<!-- -->
  "args": {<!-- -->},
  "data": "{"age":"23","name":"zhangsan"}\ ",
  "files": {<!-- -->},
  "form": {<!-- -->},
  "headers": {<!-- -->
    "Accept-Encoding": "gzip",
    "Content-Length": "29",
    "Host": "httpbin.org",
    "User-Agent": "Go-http-client/1.1"
  },
  "json": {<!-- -->
    "age": "23",
    "name": "zhangsan"
  },
  "origin": "211.138.20.170, 211.138.20.170",
  "url": "https://httpbin.org/post"
}

4. HTTPS request

The data transmitted by the HTTP protocol is unencrypted, that is, in plain text. Therefore, it is very unsafe to use the HTTP protocol to transmit private information. In order to ensure that these private data can be encrypted and transmitted, Netscape designed the SSL (Secure Sockets Layer) protocol. To encrypt the data transmitted by HTTP protocol, HTTPS was born. Simply put, the HTTPS protocol is a network protocol built from the SSL + HTTP protocol that can perform encrypted transmission and identity authentication. It is more secure than the http protocol.

The main differences between HTTPS and HTTP are as follows:

1. The https protocol requires applying for a certificate from CA. Generally, there are fewer free certificates, so a certain fee is required.

2. http is a hypertext transfer protocol, and information is transmitted in plain text, while https is a secure SSL encrypted transmission protocol.

3. http and https use completely different connection methods and use different ports. The former is 80 and the latter is 443.

4. The http connection is very simple and stateless; the HTTPS protocol is a network protocol built from the SSL + HTTP protocol that can perform encrypted transmission and identity authentication, and is more secure than the http protocol.

For https requests, we cannot get the data we want without a certificate. Therefore, the certificate must be added at the go code level.

Add certificate code

package xxxxxxxxxxxxxxxxxxxxxxxxxxx
 
import (
"crypto/tls"
"crypto/x509"
"io/ioutil"
"net/http"
"time"
)
 
func GetHttps(url, caCertPath, certFile, keyFile string) ([]byte, error) {<!-- -->
 
//Create certificate pool and various objects
var pool *x509.CertPool // We want to save some certificates in this pool
var client *http.Client
var resp *http.Response
var body []byte
var err error
 
var caCrt []byte // Root certificate
caCrt, err = ioutil.ReadFile(caCertPath)
pool = x509.NewCertPool()
if err != nil {<!-- -->
return nil, err
}
pool.AppendCertsFromPEM(caCrt)
\t
var cliCrt tls.Certificate // Specific certificate loading object
cliCrt, err = tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {<!-- -->
return nil, err
}
 
// Pass the above preparation content into the client
client = &http.Client{<!-- -->
Transport: & amp;http.Transport{<!-- -->
TLSClientConfig: & amp;tls.Config{<!-- -->
RootCAs: pool,
Certificates: []tls.Certificate{<!-- -->cliCrt},
},
},
}
 
// Get request
resp, err = client.Get(url)
if err != nil {<!-- -->
return nil, err
}
defer resp.Body.Close()
 
body, err = ioutil.ReadAll(resp.Body)
if err != nil {<!-- -->
return nil, err
}
defer client.CloseIdleConnections()
return body, nil
}

We take the certificate file from the server locally and pass it into the program as a parameter.

func TestGetHttps(t *testing.T) {<!-- -->
resp, err := GetHttps("https://xx.xx.xxx.xxx:xxxx/metrics",
"C:/Users/Desktop/ca.crt",
"C:/Users/Desktop/healthcheck-client.crt",
"C:/UsersDesktop/healthcheck-client.key")
if err != nil {<!-- -->
fmt.Println(err)
}
fmt.Println(string(resp))
}

The “crypto” standard library is used here, where we use

The tls.LoadX509KeyPair() method reads the certificate path and converts it into a certificate object;
The x509.NewCertPool() method creates a certificate pool;
The pool.AppendCertsFromPEM(caCrt) method adds the root certificate to the certificate pool.

2. Add token in the header of the HTTPS request

After we have added the certificate, we still cannot get the data for some web pages that require certification. What should we do? Add certification.

func GetHttpsSkip(url, token string) ([]byte, error) {<!-- -->
 
//Create various objects
var client *http.Client
var request *http.Request
var resp *http.Response
var body []byte
var err error
 
`Please note here that use InsecureSkipVerify: true to skip certificate verification`
client = & amp;http.Client{<!-- -->Transport: & amp;http.Transport{<!-- -->
TLSClientConfig: & amp;tls.Config{<!-- -->
InsecureSkipVerify: true,
},
}}
 
// Get request request
request, err = http.NewRequest("GET", url, nil)
 
if err != nil {<!-- -->
log.Println("GetHttpSkip Request Error:", err)
return nil, nil
}
 
//Add token
request.Header.Add("Authorization", token)
resp, err = client.Do(request)
if err != nil {<!-- -->
log.Println("GetHttpSkip Response Error:", err)
return nil, nil
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
defer client.CloseIdleConnections()
return body, nil
}

Pass in token, verify

func TestGetHttps(t *testing.T) {<!-- -->
resp, err := GetHttpsSkip("https://10.10.102.91:10250/metrics",
"Bearer eyxxxxxxxxxxxxxxxxxxxx....xxxxx")
if err != nil {<!-- -->
fmt.Println(err)
}
fmt.Println(string(resp))
}