Foreword
Sometimes, we need to open the interfaces in our system to third-party applications or enterprises. Then the third-party system is not within our own authentication and authorized user system. At this time, how to ensure the data security and identity of our interfaces? What about identification?
When providing an interface for a third-party system, you must consider the security of the interface data, such as whether the data has been tampered with, whether the data is out of date, and whether the data can be submitted repeatedly. Among them, I think the final thing is whether the data has been tampered with.
A common practice in the industry is to assign appId and appSecret to third-party applications, and then perform interface signature verification.
Signature process
Signature rules
1. Allocate appid and appsecret offline, and allocate different appid and appsecret to different callers.
2. Add timestamp (timestamp), the data will be valid within 10 minutes
3. Add a serial number nonce (to prevent repeated submissions), which must be at least 10 digits. For the query interface, the serial number is only used for log implementation to facilitate later log verification. For management interfaces, it is necessary to verify the uniqueness of the serial number within the validity period to avoid repeated requests.
4. Add signature, the signature information of all data.
The above red fields are placed in the request header.
Signature generation
The signature field generation rules are as follows.
Data part
Path: Splice all values in the order in the path
?Query: Sort according to key dictionary order and splice all key=value
?Form: Sort according to key dictionary order, and splice all key=value
?Body：
Json: Sort according to key dictionary order, and concatenate all key=value (for example {“a”:”a”,”c”:”c”,”b”:{\ “e”:”e”}} => a=ab=e=ec=c)
String: The entire string is concatenated as a
If there are multiple data forms, the splicing is performed in the order of path, query, form, and body to obtain the spliced value of all data.
The value of the above splicing is recorded as Y.
Request header part
X=”appid=xxxnonce=xxxtimestamp=xxx”
Generate signature
Final splicing value = XY
Finally, the final spliced value is encrypted as follows to obtain the signature.
?signature=org.apache.commons.codec.digest.HmacUtils.hmacSha256Hex(app secret, spliced value);
Signature algorithm implementation
Specify which interfaces or entities need to be signed
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
@Target({TYPE, METHOD})
@Retention(RUNTIME)
@Documented
public @interface Signature {
String ORDER_SORT = "ORDER_SORT";//Sort by order value
String ALPHA_SORT = "ALPHA_SORT";//lexicographic order
boolean resubmit() default true;//Allow repeated requests
String sort() default Signature.ALPHA_SORT;
}
Specify which fields need to be signed
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
import static java.lang.annotation.ElementType.FIELD;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
@Target({FIELD})
@Retention(RUNTIME)
@Documented
public @interface SignatureField {
//Signature order
int order() default 0;
//Field name custom value
String customName() default "";
//Field value custom value
String customValue() default "";
}
Core algorithm
/**
* Generate all splices marked with SignatureField attribute key=value
*/
public static String toSplice(Object object) {
if (Objects.isNull(object)) {
return StringUtils.EMPTY;
}
if (isAnnotated(object.getClass(), Signature.class)) {
Signature sg = findAnnotation(object.getClass(), Signature.class);
switch (sg.sort()) {
case Signature.ALPHA_SORT:
return alphaSignature(object);
case Signature.ORDER_SORT:
return orderSignature(object);
default:
return alphaSignature(object);
}
}
return toString(object);
}
private static String alphaSignature(Object object) {
StringBuilder result = new StringBuilder();
Map<String, String> map = new TreeMap<>();
for (Field field : getAllFields(object.getClass())) {
if (field.isAnnotationPresent(SignatureField.class)) {
field.setAccessible(true);
try {
if (isAnnotated(field.getType(), Signature.class)) {
if (!Objects.isNull(field.get(object))) {
map.put(field.getName(), toSplice(field.get(object)));
}
} else {
SignatureField sgf = field.getAnnotation(SignatureField.class);
if (StringUtils.isNotEmpty(sgf.customValue()) || !Objects.isNull(field.get(object))) {
map.put(StringUtils.isNotBlank(sgf.customName()) ? sgf.customName() : field.getName()
, StringUtils.isNotEmpty(sgf.customValue()) ? sgf.customValue() : toString(field.get(object)));
}
}
} catch (Exception e) {
LOGGER.error("Signature splicing (alphaSignature) exception", e);
}
}
}
for (Iterator<Map.Entry<String, String>> iterator = map.entrySet().iterator(); iterator.hasNext(); ) {
Map.Entry<String, String> entry = iterator.next();
result.append(entry.getKey()).append("=").append(entry.getValue());
if (iterator.hasNext()) {
result.append(DELIMETER);
}
}
return result.toString();
}
/**
* Processing for array, collection, simple property, map
*/
private static String toString(Object object) {
Class<?> type = object.getClass();
if (BeanUtils.isSimpleProperty(type)) {
return object.toString();
}
if (type.isArray()) {
StringBuilder sb = new StringBuilder();
for (int i = 0; i < Array.getLength(object); + + i) {
sb.append(toSplice(Array.get(object, i)));
}
return sb.toString();
}
if (ClassUtils.isAssignable(Collection.class, type)) {
StringBuilder sb = new StringBuilder();
for (Iterator<?> iterator = ((Collection<?>) object).iterator(); iterator.hasNext(); ) {
sb.append(toSplice(iterator.next()));
if (iterator.hasNext()) {
sb.append(DELIMETER);
}
}
return sb.toString();
}
if (ClassUtils.isAssignable(Map.class, type)) {
StringBuilder sb = new StringBuilder();
for (Iterator<? extends Map.Entry<String, ?>> iterator = ((Map<String, ?>) object).entrySet().iterator(); iterator.hasNext(); ) {
Map.Entry<String, ?> entry = iterator.next();
if (Objects.isNull(entry.getValue())) {
continue;
}
sb.append(entry.getKey()).append("=").append(toSplice(entry.getValue()));
if (iterator.hasNext()) {
sb.append(DELIMETER);
}
}
return sb.toString();
}
return NOT_FOUND;
}
Verification of signature
The parameters in the header are as follows
?
Signature entity
import com.google.common.base.MoreObjects;
import com.google.common.collect.Sets;
import org.hibernate.validator.constraints.NotBlank;
import java.util.Set;
@ConfigurationProperties(prefix = "wmhopenapi.validate", exceptionIfInvalid = false)
@Signature
public class SignatureHeaders {
public static final String SIGNATURE_HEADERS_PREFIX = "wmhopenapi-validate";
public static final Set<String> HEADER_NAME_SET = Sets.newHashSet();
private static final String HEADER_APPID = SIGNATURE_HEADERS_PREFIX + "-appid";
private static final String HEADER_TIMESTAMP = SIGNATURE_HEADERS_PREFIX + "-timestamp";
private static final String HEADER_NONCE = SIGNATURE_HEADERS_PREFIX + "-nonce";
private static final String HEADER_SIGNATURE = SIGNATURE_HEADERS_PREFIX + "-signature";
static {
HEADER_NAME_SET.add(HEADER_APPID);
HEADER_NAME_SET.add(HEADER_TIMESTAMP);
HEADER_NAME_SET.add(HEADER_NONCE);
HEADER_NAME_SET.add(HEADER_SIGNATURE);
}
/**
* Value assigned offline
* The client and server each save the appSecret corresponding to the appId.
*/
@NotBlank(message = "Missing in Header" + HEADER_APPID)
@SignatureField
private String appid;
/**
* Value assigned offline
* The client and server are saved separately, corresponding to the appId
*/
@SignatureField
private String appsecret;
/**
* Timestamp, unit: ms
*/
@NotBlank(message = "Missing in Header" + HEADER_TIMESTAMP)
@SignatureField
private String timestamp;
/**
* Serial number [prevent repeated submission]; (Note: For the query interface, the serial number is only used for log implementation to facilitate later log verification; for the management interface, the uniqueness of the serial number within the validity period needs to be verified to avoid repeated requests)
*/
@NotBlank(message = "Missing in Header" + HEADER_NONCE)
@SignatureField
private String nonce;
/**
* sign
*/
@NotBlank(message = "Missing in Header" + HEADER_SIGNATURE)
private String signature;
public String getAppid() {
return appid;
}
public void setAppid(String appid) {
this.appid = appid;
}
public String getAppsecret() {
return appsecret;
}
public void setAppsecret(String appsecret) {
this.appsecret = appsecret;
}
public String getTimestamp() {
return timestamp;
}
public void setTimestamp(String timestamp) {
this.timestamp = timestamp;
}
public String getNonce() {
return nonce;
}
public void setNonce(String nonce) {
this.nonce = nonce;
}
public String getSignature() {
return signature;
}
public void setSignature(String signature) {
this.signature = signature;
}
@Override
public String toString() {
return MoreObjects.toStringHelper(this)
.add("appid", appid)
.add("appsecret", appsecret)
.add("timestamp", timestamp)
.add("nonce", nonce)
.add("signature", signature)
.toString();
}
}
Generate SignatureHeaders entities based on the header value in the request
private SignatureHeaders genrateSignatureHeaders(Signature signature, HttpServletRequest request) throws Exception {//NOSONAR
Map<String, Object> headerMap = Collections.list(request.getHeaderNames())
.stream()
.filter(headerName -> SignatureHeaders.HEADER_NAME_SET.contains(headerName))
.collect(Collectors.toMap(headerName -> headerName.replaceAll("-", "."), headerName -> request.getHeader(headerName)));
PropertySource propertySource = new MapPropertySource("signatureHeaders", headerMap);
SignatureHeaders signatureHeaders = RelaxedConfigurationBinder.with(SignatureHeaders.class)
.setPropertySources(propertySource)
.doBind();
Optional<String> result = ValidatorUtils.validateResultProcess(signatureHeaders);
if (result.isPresent()) {
throw new ServiceException("WMH5000", result.get());
}
//Get the appsecret corresponding to the appid from the configuration
String appSecret = limitConstants.getSignatureLimit().get(signatureHeaders.getAppid());
if (StringUtils.isBlank(appSecret)) {
LOGGER.error("The appSecret corresponding to appId was not found, appId=" + signatureHeaders.getAppid());
throw new ServiceException("WMH5002");
}
//Other legality checks
Long now = System.currentTimeMillis();
Long requestTimestamp = Long.parseLong(signatureHeaders.getTimestamp());
if ((now - requestTimestamp) > EXPIRE_TIME) {
String errMsg = "The request time exceeds the specified range by 10 minutes, signature=" + signatureHeaders.getSignature();
LOGGER.error(errMsg);
throw new ServiceException("WMH5000", errMsg);
}
String nonce = signatureHeaders.getNonce();
if (nonce.length() < 10) {
String errMsg = "The length of the random string nonce must be at least 10 digits, nonce=" + nonce;
LOGGER.error(errMsg);
throw new ServiceException("WMH5000", errMsg);
}
if (!signature.resubmit()) {
String existNonce = redisCacheService.getString(nonce);
if (StringUtils.isBlank(existNonce)) {
redisCacheService.setString(nonce, nonce);
redisCacheService.expire(nonce, (int) TimeUnit.MILLISECONDS.toSeconds(RESUBMIT_DURATION));
} else {
String errMsg = "Duplicate requests are not allowed, nonce=" + nonce;
LOGGER.error(errMsg);
throw new ServiceException("WMH5000", errMsg);
}
}
//Set appsecret
signatureHeaders.setAppsecret(appSecret);
return signatureHeaders;
}
There are several steps required before generating a signature, as follows.
(1) Is the appid legal?
(2) Get the appsecret from the configuration center based on the appid
(3) Whether the request has expired, the default is 10 minutes
(4) Is the random string legal?
(5) Whether to allow repeated requests
Generate header information parameter splicing
String headersToSplice = SignatureUtils.toSplice(signatureHeaders);
Generate parameters in header and splice parameters in mehtod
private List<String> generateAllSplice(Method method, Object[] args, String headersToSplice) {
List<String> pathVariables = Lists.newArrayList(), requestParams = Lists.newArrayList();
String beanParams = StringUtils.EMPTY;
for (int i = 0; i < method.getParameterCount(); + + i) {
MethodParameter mp = new MethodParameter(method, i);
boolean findSignature = false;
for (Annotation anno : mp.getParameterAnnotations()) {
if (anno instanceof PathVariable) {
if (!Objects.isNull(args[i])) {
pathVariables.add(args[i].toString());
}
findSignature = true;
} else if (anno instanceof RequestParam) {
RequestParam rp = (RequestParam) anno;
String name = mp.getParameterName();
if (StringUtils.isNotBlank(rp.name())) {
name = rp.name();
}
if (!Objects.isNull(args[i])) {
List<String> values = Lists.newArrayList();
if (args[i].getClass().isArray()) {
//array
for (int j = 0; j < Array.getLength(args[i]); + + j) {
values.add(Array.get(args[i], j).toString());
}
} else if (ClassUtils.isAssignable(Collection.class, args[i].getClass())) {
//gather
for (Object o : (Collection<?>) args[i]) {
values.add(o.toString());
}
} else {
//single value
values.add(args[i].toString());
}
values.sort(Comparator.naturalOrder());
requestParams.add(name + "=" + StringUtils.join(values));
}
findSignature = true;
} else if (anno instanceof RequestBody || anno instanceof ModelAttribute) {
beanParams = SignatureUtils.toSplice(args[i]);
findSignature = true;
}
if (findSignature) {
break;
}
}
if (!findSignature) {
LOGGER.info(String.format("Unrecognized annotation with signature, method=%s, parameter=%s, annotations=%s", method.getName(), mp.getParameterName(), StringUtils.join(mp .getMethodAnnotations())));
}
}
List<String> toSplices = Lists.newArrayList();
toSplices.add(headersToSplice);
toSplices.addAll(pathVariables);
requestParams.sort(Comparator.naturalOrder());
toSplices.addAll(requestParams);
toSplices.add(beanParams);
return toSplices;
}
Regenerate signature information for the final splicing result
SignatureUtils.signature(allSplice.toArray(new String[]{}), signatureHeaders.getAppsecret());
Depends on third-party toolkits
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
</dependency>
Usage examples
Generate signature
//Initialize request header information
SignatureHeaders signatureHeaders = new SignatureHeaders();
signatureHeaders.setAppid("111");
signatureHeaders.setAppsecret("222");
signatureHeaders.setNonce(SignatureUtils.generateNonce());
signatureHeaders.setTimestamp(String.valueOf(System.currentTimeMillis()));
List<String> pathParams = new ArrayList<>();
//Initialize the data in path
pathParams.add(SignatureUtils.encode("18237172801", signatureHeaders.getAppsecret()));
//Call the signature tool to generate a signature
signatureHeaders.setSignature(SignatureUtils.signature(signatureHeaders, pathParams, null, null));
System.out.println("Signature data: " + signatureHeaders);
System.out.println("Request data: " + pathParams);
Output results
Splicing result: appid=111^_^appsecret=222^_^nonce=c9e778ba668c8f6fedf35634eb493af6304d54392d990262d9e7c1960b475b67^_^timestamp=1538207443910^_^w8rAwcXDxcD KwsM=^_^
Signature data: SignatureHeaders{appid=111, appsecret=222, timestamp=1538207443910, nonce=c9e778ba668c8f6fedf35634eb493af6304d54392d990262d9e7c1960b475b67, signature=0a7d0b5e802eb5e 52ac0cfcd6311b0faba6e2503a9a8d1e2364b38617877574d}
Request data: [w8rAwcXDxcDKwsM=]
Introducing HmacSHA1
HmacSHA1 is an implementation based on the Hash Message Authentication Code (HMAC) algorithm. HMAC is a key-dependent hash algorithm used to verify message integrity and authentication.
HmacSHA1 combines the use of the SHA-1 hash function and a key to generate a message authentication code (MAC) with a fixed-length output. This MAC can be used to verify whether the data has been tampered with or impersonated during transmission.
The workflow of the HmacSHA1 algorithm is as follows:
-
Prepare message and key: Prepare the message and key for authentication.
-
Key padding: If the key length is less than 64 bytes, pad it to 64 bytes; if the key length is greater than 64 bytes, use SHA-1 hashing to shorten it to 64 bytes.
-
Internal secret hash: XOR each key byte with 0x36, then SHA-1 hash with the message.
-
Final hash: Process the hash result obtained in step 3 with the original key again. XOR each key byte with 0x5C and then SHA-1 hash it with the hash result from step 3.
-
Output authentication code: The final SHA-1 hash result is the HmacSHA1 authentication code.
HmacSHA1 provides a secure message authentication code algorithm that not only relies on the hashing properties of SHA-1, but also takes advantage of the mixing of keys. Therefore, even if an attacker is able to obtain the message and authentication code, it will be difficult to crack the original key, thereby protecting the integrity and security of the data.
When using HmacSHA1 in Java to interface-sign GET and POST request data and perform signature verification in an interceptor or filter, you can write the code as follows:
-
Create an interceptor or filter class that implements the
javax.servlet.Filterinterface or theorg.springframework.web.servlet.HandlerInterceptorinterface. -
In the interceptor or filter, obtain relevant data of the GET or POST request, including URL, request parameters, etc.
-
Get the appId, timestamp, nonce and signature in the request header.
-
According to the rules defined by the interface, the appId, timestamp and nonce are spliced together in the specified order, and then the key is used to perform HmacSHA1 signature on the spliced string to generate a signature result.
-
Compare the generated signature result with the signature in the request header. If they are consistent, the verification passes, otherwise the verification fails.
The following is a sample code that shows how to use HmacSHA1 in Java for interface signing and verification of GET and POST requests:
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import javax.servlet.*;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.Base64;
@WebFilter(urlPatterns = "/api/*")
public class SignatureFilter implements Filter {
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";
private static final String SECRET_KEY = "yourSecretKey";
@Override
public void init(FilterConfig filterConfig) throws ServletException {
//Initialization operation
}
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
String method = httpRequest.getMethod();
String url = httpRequest.getRequestURL().toString();
// Get the parameters in the request header
String appId = httpRequest.getHeader("appId");
String timestamp = httpRequest.getHeader("timestamp");
String nonce = httpRequest.getHeader("nonce");
String signature = httpRequest.getHeader("signature");
try {
String signPayload = appId + timestamp + nonce;
if (method.equalsIgnoreCase("POST")) {
// Get the request body data of the POST request
// It is assumed here that the request body is in JSON format. Because it is read in the form of a string, there is no problem of key disorder.
String body = httpRequest.getReader().lines().reduce("", (accumulator, actual) -> accumulator + actual);
signPayload + = body;
} else if (method.equalsIgnoreCase("GET")) {
// Get the parameters of the GET request
String queryString = httpRequest.getQueryString();
if (queryString != null) {
signPayload + = queryString;
}
}
String generatedSignature = generateHmacSHA1Signature(signPayload);
if (generatedSignature.equals(signature)) {
// Signature verification passed, continue processing the request
chain.doFilter(request, response);
} else {
//Failed to verify the signature and return an error response
HttpServletResponse httpResponse = (HttpServletResponse) response;
httpResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
httpResponse.getWriter().write("Invalid signature");
}
} catch (NoSuchAlgorithmException | InvalidKeyException e) {
e.printStackTrace();
// Handle exceptions
}
}
@Override
public void destroy() {
// Destroy operation
}
private String generateHmacSHA1Signature(String data) throws NoSuchAlgorithmException, InvalidKeyException {
byte[] secretKeyBytes = SECRET_KEY.getBytes();
SecretKeySpec secretKeySpec = new SecretKeySpec(secretKeyBytes, HMAC_SHA1_ALGORITHM);
Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
mac.init(secretKeySpec);
byte[] dataBytes = data.getBytes();
byte[] signatureBytes = mac.doFinal(dataBytes);
return Base64.getEncoder().encodeToString(signatureBytes);
}
}
SECRET_KEY in the above code example is the key used to generate HmacSHA1 signature. Please replace it with your own key. You can add additional validation logic and error handling logic based on your application needs.
Please note that in actual applications, it is also necessary to ensure the legitimacy of the appId, timestamp, and nonce in the request header, and to add appropriate timeliness checks and replay attack protection measures.