After using TCP to simulate an HTTP proxy last time, I felt that it was still too crude. I wondered if I could use a framework to make something of practical use. So, just thinking about how to use golang’s Gin framework to implement one? Well, you heard it right, it is the gin framework. You may be wondering, isn’t it a web framework? How can it be used as proxy software? Haha, actually you will understand if you think about it carefully. I have already said that the essence of an HTTP proxy is actually an HTTP server! So, I just need to find a way to let it handle all the routing!
After thinking about it, I thought of 404. Usually for a web service, it is a non-existent route. That is to say:
Existing routes + non-existing routes = all routes
For a Web service, we have a clear interface (routing), so many existing routes will be defined. But for a proxy server, it doesn’t care what your route is, and it doesn’t need existing routes (you don’t know which routes users will access), so we only need to deal with non-existent routes. The routes that do not exist here refer to routes that are not processed by the web service.
This description may be a little confusing, so let’s just say it. My idea is: in the Gin framework, no routing is defined, so that all routes will be considered non-existent routes, and then in a special method to handle 404 Process all routes. It is equivalent to using a clever method to achieve the purpose of processing all routes, so that there is no problem in using it as an HTTP proxy.
Code
package main
import (
"fmt"
"io"
"log"
"net/http"
"strings"
"github.com/gin-gonic/gin"
)
func main() {<!-- -->
r := gin.Default()
r.NoRoute(routeProxy) // Routing proxy handler
r.GET("/", routeProxy) // There is a / route by default, so it is also processed in the routing proxy
r.Run(":8000") // listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
}
// This will handle all routing situations
func routeProxy(c *gin.Context) {<!-- -->
//The proxy receives the request, sends it out, and then returns the corresponding response.
req := c.Request
go resolveReq(req) // See what this request does. Be careful not to proxy yourself, otherwise there will be problems.
newReq, _ := http.NewRequest(req.Method, req.URL.String(), req.Body)
resp, err := http.DefaultClient.Do(newReq)
if err != nil {<!-- -->
log.Fatal(err)
}
defer resp.Body.Close()
data, err := io.ReadAll(resp.Body)
if err != nil {<!-- -->
log.Fatal(err)
}
code := resp.StatusCode
c.Status(code) //Response status code
for k, v := range resp.Header {<!-- -->
c.Header(k, strings.Join(v, ","))
}
c.Header("Server", "CrazyDragonHttpProxy") // Tamper with a response reply
c.Writer.Write(data) //Response data
}
func resolveReq(req *http.Request) {<!-- -->
fmt.Printf("Method: %s, Host: %s, URL: %s\\
", req.Method, req.Host, req.URL.String())
}
Note: It seems that the default / is not needed here, because there is a difference between proxy routing and non-proxy routing.
Proxy settings
Here http=127.0.0.1, this will only proxy the http protocol and will not process the https protocol. Because I’m just doing a simple demo here, I don’t want to deal with https. That would be too troublesome, and I don’t know exactly how to do it. When using Fiddler to set up packet capture, if you need to capture https packets, you need to install its certificate, which you will know is very troublesome.
Note that it is now a proxy server, so you cannot access it itself, otherwise the proxy server will proxy itself. This is not handled here, so an error will be reported.
Test
Now it is really troublesome to find an http website to test. It took me a long time to find an http website. After all, there are very few websites that still use http.
Then something went wrong, my computer’s fan started screaming and my CPU usage spiked. Then the above things are printed, so it should be that the proxy server forwards the request to itself, and then the system may maintain a large number of connections, causing the CPU usage to soar. I initially thought it was an issue with my http.DefaultClient code, since its default configuration seemed to use the system’s proxy. But I thought wrong again, because I am not in the system, I am in the container! I have now uninstalled the local development environment, so I develop in a container. However, I also thought that although I am in the container, docker is still in the system.
So this network request may look like the picture below: red is the user request, blue is the proxy request, it loops, and then causes problems.
So, I thought about it again, and the solution is to go back to running it on the Windows machine. But since there is no development environment locally, I have to find another way. I only need to execute it on Windows, and there is no need to compile it on Winwos. So let’s cross compile a version for Windows.
But this thing is in the docker container and I have to take it out. So what should I do? Have you heard of a command called docker cp? But it doesn’t need to be that troublesome at all, because my directory is mounted, so I can just go to my mounted directory, haha.
Demo
Finally, the demonstration was successful, but I found that it would still terminate because it could not handle https (if there was an error, I simply terminated the program, but of course you can just return without processing it). The connect method of https is really like a dark cloud. It is not used in normal web development, so if you encounter it, you can’t handle it (I only know that it is used by the proxy server to establish tunnels, and I don’t know the rest) . Because I set it up like that before, I thought I could skip the https protocol, and my other https pages could be accessed normally, but I don’t know why some of them are still sent to the proxy server. It can’t handle this, causing the proxy collapsed.
PS:
When I first started looking for the problem of circular requests, I found an article on a similar topic written by an old man. However, he did this much earlier, several years ago. However, my main idea here is about 404 processing, and what it does is still about the processing of designated routes. However, it is worth referencing that he used that tool to directly send requests, but since it is just a toy after all, it is better not to go too deep, haha.
golang gin proxy and package modification