After searching many articles, I finally found a solution!
Error report details
Solution
The first method (applicable to my own solution):
httpclient-4.5.jar regularly sends http packets, but suddenly an error is reported one day, which is caused by the change of http certificate.
previous code
try {
CloseableHttpClient httpClient = buildDefaultHttpClient();
String url = domain.getUrl();
HttpGet httpGet = new HttpGet(url);
httpGet.addHeader("User-Agent", NetUtil.INSPECTOR_USER_AGENT);
httpGet.addHeader("Accept", "text/html,application/xhtml + xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
httpGet.addHeader("Accept-Encoding", "gzip, deflate");
httpGet.addHeader("Accept-Language", "zh-CN,zh;q=0.9");
CloseableHttpResponse resp = httpClient.execute(httpGet);
String responseBody = EntityUtils.toString(resp.getEntity(), "utf-8");
respEnd = DateUtil.toEpochMilliseconds(LocalDateTime.now());
len = responseBody.length();
logger.info("http message response text length:{}B", len);
String extractedTitle = NetUtil.extractTitle(responseBody);
return builder.withRespEnd(respEnd).withRespLen(len)
.withHttpStatus((short) resp.getStatusLine().getStatusCode())
.withRetrieveTitle(extractedTitle)
.withTitleMatched(StringUtils.equals(domain.getTitle(), extractedTitle))
.build();
} catch (IOException e) {
}
code after
try {
CloseableHttpClient httpClient = buildDefaultHttpClientTrustSSL();//Trust certificate
String url = domain.getUrl();
HttpGet httpGet = new HttpGet(url);
httpGet.addHeader("User-Agent", NetUtil.INSPECTOR_USER_AGENT);
httpGet.addHeader("Accept", "text/html,application/xhtml + xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
httpGet.addHeader("Accept-Encoding", "gzip, deflate");
httpGet.addHeader("Accept-Language", "zh-CN,zh;q=0.9");
CloseableHttpResponse resp = httpClient.execute(httpGet);
String responseBody = EntityUtils.toString(resp.getEntity(), "utf-8");
respEnd = DateUtil.toEpochMilliseconds(LocalDateTime.now());
len = responseBody.length();
logger.info("http message response text length:{}B", len);
String extractedTitle = NetUtil.extractTitle(responseBody);
return builder.withRespEnd(respEnd).withRespLen(len)
.withHttpStatus((short) resp.getStatusLine().getStatusCode())
.withRetrieveTitle(extractedTitle)
.withTitleMatched(StringUtils.equals(domain.getTitle(), extractedTitle))
.build();
} catch (IOException e) {
}
/**
* Trust SSL certificate
* @return
*/
public static CloseableHttpClient buildDefaultHttpClientTrustSSL()
{
SSLContext sslContext = null;
try {
sslContext = SSLContextBuilder.create().useProtocol(SSLConnectionSocketFactory.SSL).loadTrustMaterial((x, y) -> true).build();
} catch (Exception e) {
e.printStackTrace();
}
RequestConfig config = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000)
.setConnectionRequestTimeout(30000)
.setContentCompressionEnabled(true)
.build();
return HttpClientBuilder.create().setDefaultRequestConfig(config).setSSLContext(sslContext).setSSLHostnameVerifier((x, y) -> true).build();
}
public static CloseableHttpClient buildDefaultHttpClient()
{
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000)
.setConnectionRequestTimeout(30000)
.setContentCompressionEnabled(true)
.setStaleConnectionCheckEnabled(true)
.build();
return HttpClients.custom().setDefaultRequestConfig(defaultRequestConfig).build();
}
For example:
This is how I originally created httpClient:
CloseableHttpClient httpClient= HttpClients.custom().setDefaultRequestConfig(RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build()).build();
After the change:
CloseableHttpClient httpClient= buildDefaultHttpClientTrustSSL();
/**
* Trust SSL certificate
* @return
*/
public static CloseableHttpClient buildDefaultHttpClientTrustSSL()
{
SSLContext sslContext = null;
try {
sslContext = SSLContextBuilder.create().useProtocol(SSLConnectionSocketFactory.SSL).loadTrustMaterial((x, y) -> true).build();
} catch (Exception e) {
e.printStackTrace();
}
RequestConfig config = RequestConfig.custom()
.setSocketTimeout(30000)
.setConnectTimeout(30000)
.setConnectionRequestTimeout(30000)
.setContentCompressionEnabled(true)
.setCookieSpec(CookieSpecs.STANDARD)
.build();
// return HttpClientBuilder.create().setDefaultRequestConfig(config).setSSLContext(sslContext).setSSLHostnameVerifier((x, y) -> true).build();
return HttpClients.custom().setDefaultRequestConfig(config).setSSLContext(sslContext).setSSLHostnameVerifier((x, y) -> true).build();
}
Second: Add certificate manually
In the https website you want to visit, F12, as shown in the figure:
View certificate (blue part)–Export certificate
Import certificate:
keytool -import -v -trustcacerts -alias taobao -file taobao.cer -storepass changeit -keystore %JAVA_HOME%/jre/lib/security/cacerts
explain:
1. taobao is a name that can be modified by yourself. taobao.cer is the exported certificate. Similarly, the name of the certificate here is also random, but the premise is to ensure that there is no cacerts file in the %JAVA_HOME%/jre/lib/security directory before. Importing a certificate with the same name is to ensure that the name of the certificate you are importing is unique.
2. changeit is the password, which is the default in java.
3. Keytool is an exe file in the bin directory of jdk. It comes with jdk by default. The directory on my computer is: /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/bin/keytool. exe
4. %JAVA_HOME%/jre/lib/security/cacerts In this path, %JAVA_HOME%/jre/lib/security/ is the path, and cacerts is the file (the file into which the certificate will be imported). Of course, make sure you have configured the java_home environment variable. My java_home environment variable is: /usr/lib/jvm/java-1.8.0-openjdk-amd64
5. Others remain unchanged.
6. If prompted: “Do you trust this certificate? [No]:”, then please enter “y”.
In fact, this method is also effective
The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. Java Skill TreeHomepageOverview 136778 people are learning the system