Today I want to implement a function:
”
Use Java + SpringBoot server to implement interface proxy forwarding, perform secondary processing on the data, and then return it to the client.
”
Background: As the upstream server of the client, the Java server needs to be responsible for returning all request data. Even if it is not a function provided by itself, it is also responsible for requesting the corresponding server and returning the correct result.
Purpose: This can achieve unified interface services and solve front-end cross-domain problems.
Here are two scenarios.
Scenario 1: Call other servers and return the results directly to the client
The function to be implemented is
Browser request: https://10.28.10.11:8081/proxy/homeList
Want the request to be forwarded to http://10.28.11.20:4000/homeList
Source code
package com.maomao.apm.controller;
import au.com.bytecode.opencsv.CSVReader;
import com.pinganfu.dochelper.annotation.PAFDoc;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.client.ClientHttpRequest;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.util.StreamUtils;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
@RestController
public class ProxyController extends BaseController {
private String targetAddr = "https://10.28.10.11:4000";
/**
* Proxy all requests
*
* @param request
* @param response
* @throwsException
*/
@RequestMapping(value = "/proxy/**", produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public void proxy(HttpServletRequest request, HttpServletResponse response) throws IOException, URISyntaxException {
// String url = URLDecoder.decode(request.getRequestURL().toString(), "UTF-8");
URI uri = new URI(request.getRequestURI());
String path = uri.getPath();
String query = request.getQueryString();
String target = targetAddr + path.replace("/proxy", "");
if (query != null & amp; & amp; !query.equals("") & amp; & amp; !query.equals("null")) {
target = target + "?" + query;
}
URI newUri = new URI(target);
//Execute proxy query
String methodName = request.getMethod();
HttpMethod httpMethod = HttpMethod.resolve(methodName);
if (httpMethod == null) {
return;
}
ClientHttpRequest delegate = new SimpleClientHttpRequestFactory().createRequest(newUri, httpMethod);
Enumeration<String> headerNames = request.getHeaderNames();
//Set request headers
while (headerNames.hasMoreElements()) {
String headerName = headerNames.nextElement();
Enumeration<String> v = request.getHeaders(headerName);
List<String> arr = new ArrayList<>();
while (v.hasMoreElements()) {
arr.add(v.nextElement());
}
delegate.getHeaders().addAll(headerName, arr);
}
StreamUtils.copy(request.getInputStream(), delegate.getBody());
//Execute remote call
ClientHttpResponse clientHttpResponse = delegate.execute();
response.setStatus(clientHttpResponse.getStatusCode().value());
//Set response headers
clientHttpResponse.getHeaders().forEach((key, value) -> value.forEach(it -> {
response.setHeader(key, it);
}));
StreamUtils.copy(clientHttpResponse.getBody(), response.getOutputStream());
}
}
”
The principle is that when the proxy server (the server where the current code is located) receives the client’s request, it creates a new request to request the real server address, and deletes the “proxy” keyword in the client’s request address. After the real server returns the data, the data is returned directly to the client.
”
This method is unified. Assume that client A requests server B to actually obtain the content of server C. This code can enable all interfaces of server B to request server C to be proxied. Just add the word “proxy” to the interface when the client requests it.
Scenario 2: Call other servers, process the results and return them to the client
Scenario 1 can solve most agency problems. But if you need to perform secondary processing on the data returned by the C server, you only need to make a few changes.
Here is an example where server B obtains the CSV file of server C, and converts the returned CSV file stream into Json and returns it to the client.
The function to be implemented is
Browser request: https://10.28.10.11:8081/proxy/homedetail.csv
Want the request to be forwarded to http://10.28.11.187:8088/homedetail.csv
Finally parse homedetail.csv and return json array
Source code:
package com.maomao.apm.controller;
import au.com.bytecode.opencsv.CSVReader;
import com.pinganfu.dochelper.annotation.PAFDoc;
import org.apache.logging.log4j.LogManager;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.client.ClientHttpRequest;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.util.StreamUtils;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
@RestController
public class ProxyController extends BaseController {
private String targetAddrUba = "http://10.28.11.187:8088";
/**
* Proxy ubaReport requests for all requests
*
* @param request
* @param response
* @return
* @throwsException
*/
@RequestMapping(value = "/ubaProxy/**")
public List<String[]> proxyUba(HttpServletRequest request, HttpServletResponse response) throws IOException, URISyntaxException {
// String url = URLDecoder.decode(request.getRequestURL().toString(), "UTF-8");
URI uri = new URI(request.getRequestURI());
String path = uri.getPath();
String query = request.getQueryString();
String target = targetAddrUba + path.replace("/ubaProxy", "");
if (query != null & amp; & amp; !query.equals("") & amp; & amp; !query.equals("null")) {
target = target + "?" + query;
}
URI newUri = new URI(target);
//Execute proxy query
String methodName = request.getMethod();
HttpMethod httpMethod = HttpMethod.resolve(methodName);
if (httpMethod == null) {
return null;
}
ClientHttpRequest delegate = new SimpleClientHttpRequestFactory().createRequest(newUri, httpMethod);
Enumeration<String> headerNames = request.getHeaderNames();
//Set request headers
while (headerNames.hasMoreElements()) {
String headerName = headerNames.nextElement();
Enumeration<String> v = request.getHeaders(headerName);
List<String> arr = new ArrayList<>();
while (v.hasMoreElements()) {
arr.add(v.nextElement());
}
delegate.getHeaders().addAll(headerName, arr);
}
StreamUtils.copy(request.getInputStream(), delegate.getBody());
//Execute remote call
ClientHttpResponse clientHttpResponse = delegate.execute();
// response.setStatus(clientHttpResponse.getStatusCode().value());
//Set response headers
// clientHttpResponse.getHeaders().forEach((key, value) -> value.forEach(it -> {
// response.setHeader(key, it);
// }));
CSVReader csvReader = new CSVReader(new InputStreamReader(clientHttpResponse.getBody(), "utf-8"));
List<String[]> stringsList = csvReader.readAll();
return stringsList;
}
}
pom.xml
<dependency>
<groupId>net.sf.opencsv</groupId>
<artifactId>opencsv</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
A “ubaProxy” is used here as the keyword for the requested function. The principle is the same as scene one. Only after returning the data, the CSV to Json operation is performed on the data. Specific code:
CSVReader csvReader = new CSVReader(new InputStreamReader(clientHttpResponse.getBody(), "utf-8"));
List<String[]> stringsList = csvReader.readAll();
return stringsList;
”
Note that the code related to
response.setHeader()is commented out in the above code. The reason is that the length of the data returned after secondary processing is different from the data before processing. If the response header before processing is reused, thecontent-lengthvalue will be inaccurate. If the processed data is more than the original data, and thecontent-lengthvalue is still the value before processing, the data returned to the client will be incomplete. If left blank, all data will be returned by default.”
Reference documents: “Java + springboot implements nginx reverse proxy function” “JAVA: Convert CSV files to JSON”
path is printed repeatedly, causing memory overflow
In the middle, I also encountered that when printing path in the code, the value of path kept repeating in a loop, and the console kept printing, which looked like an infinite loop.
In the end, the server cannot support it and may report a memory overflow exception.
java.lang.OutOfMemoryError:Java heap space
java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3664
Solution
There are many answers on the Internet about setting the JVM memory to be larger:
Automatically set Heap size Modify TOMCAT_HOME/bin/catalina.bat and add the following line above "echo "Using CATALINA_BASE: $CATALINA_BASE"": Java code set JAVA_OPTS=%JAVA_OPTS% -server -Xms800m -Xmx800m -XX:MaxNewSize=256m
Or set the size value in application.properties to be larger.
//src/main/resources/application.properties #Threshold for writing files to disk. Values can use the suffix "MB" or "KB" to indicate megabytes or kilobytes respectively. spring.servlet.multipart.file-size-threshold=2KB #Set the size of a single file, spring.servlet.multipart.max-file-size=100MB #Maximum request size. Values can use the suffix "MB" or "KB" to indicate megabytes or kilobytes respectively. spring.servlet.multipart.max-request-size=100MB
But my problem here is not caused by insufficient memory settings.
The cause of the problem is that the @Controller annotation is used on the class name. Just change it to @RestController.
The principle is that the @Controller annotation will think that the request is for a page, not data. If the page is not found, it will continue to be forwarded to the next page. If it is still not found, it will be an infinite loop. The @RestController annotation only cares about the content of the request. If the content is requested, it will be returned directly. If the content is not requested, empty data will be returned and will not be forwarded.