Original Shepherd Shepherd Advanced Notes 2023-05-30 08:50 Published in Zhejiang
JJ Lin-Exchanging the rest of my life, Shepherd advanced notes, 4 minutes
1. Overview
Recently, we have been focusing on the topic of how to ensure data security when using Spring Boot to develop business systems. Most of the current B/S architecture systems are also Developed based on the Spring Boot + SpringMVC three-tier architecture, we have previously summarized how to strengthen data security at the interface level: How Spring Boot elegantly improves interface data security can be considered as SpringMVC in the three-layer architecture of /code> performs secure processing operations on interface data. To put it more directly, the interface request parameters are passed in for logical processing or the response parameters are output to Data is processed before the page is displayed, so security is only reinforced in one layer of the three-layer architecture of SpringMVC, which is not very stable. Today we will talk about the security of SpringMVCHow to implement data security reinforcement in the other layer of the three-tier architecture? Before today’s topic, let’s take a look at what is the SpringMVC architecture?
What is SpringMVC three-tier architecture?
The engineering structure of SpringMVC is generally divided into three layers. From bottom to top are the Model layer (model, data access layer), Controller layer (control, logical control layer), and View layer (view, page display layer). The Model layer It is divided into two layers: dao layer and service layer. The main function of MVC architecture layering is decoupling. The benefits of adopting a layered architecture are generally accepted that system layering is beneficial to system maintenance and system expansion. It is to enhance the maintainability and scalability of the system. For a framework like Spring, the (View\Web) presentation layer calls the control layer (Controller), the control layer calls the business layer (Service), and the business layer calls the data access layer (Dao). It can be said that now more than 90% of business systems It is developed based on the three-layer architecture model. This architecture model is also said to be one of the design patterns. It can be seen that its importance is self-evident, so we need to pay attention to it.
We all know that data security is very important in the daily development of systems. Especially in today’s Internet era, personal privacy security is extremely important. Once personal user data is attacked and leaked, it will cause catastrophic accidents. All our previous data security processing based on the interface layer is far from enough. Today we will talk about how the Model layer (data access layer) can achieve elegant data encryption storage, fuzzy matching and desensitized display. The theme of this article: Data encrypted storage, fuzzy matching and desensitized display.
The banking system’s requirements for data security are second to none among business systems, so today we will use common personal bank account data: passwords, mobile phone numbers, detailed addresses, bank card numbers and other information fields as examples to give a brief introduction to the topic. analysis.
Project Recommendation: Based on the underlying framework encapsulation of SpringBoot2. Unified management of the framework. The idea of componentization is introduced to achieve high cohesion, low coupling, high configurability, and pluggability. Strictly control package dependencies and unified version management to minimize dependencies. Focus on code specifications and comments, very suitable for personal learning and corporate use
Github address: https://github.com/plasticene/plasticene-boot-starter-parent
Gitee address: https://gitee.com/plasticene3/plasticene-boot-starter-parent
2. Data encryption storage
What we have summarized before is to perform data encryption, decryption and transmission at the interface layer. We have also emphasized that this method cannot guarantee the absolute security of data. It only effectively improves the security of interface data and raises the threshold for data capture. So next we will talk about how to ensure the security of data at the source storage layer. We all know that some core private fields, such as passwords and mobile phone numbers, cannot be stored in clear text when stored in the database layer. They must be encrypted to ensure that even if the database is leaked, the data will not be easily exposed.
2.1 Elegant implementation of database field encryption and decryption principles
Mybatis-plus provides enterprise advanced features and supports data encryption and decryption, but it is charged. . . But we can explore its principles in detail to achieve self-realization of functions.
In fact, the data encryption and decryption functions have been elegantly integrated in the rapid development framework we recommended above. EncryptTypeHandler: implements database field encryption and decryption.
By default, the base64 encryption algorithm Base64EncryptService and the AES encryption algorithm AESEncryptService are provided. Of course, the business side can also customize the encryption algorithm. This requires implementing the interface EncryptService and injecting the implementation class into the container. Encryption function core logic
@Bean
@ConditionalOnMissingBean(EncryptService.class)
public EncryptService encryptService() {
Algorithm algorithm = encryptProperties.getAlgorithm();
EncryptService encryptService;
switch (algorithm) {
case BASE64:
encryptService = new Base64EncryptService();
break;
case AES:
encryptService = new AESEncryptService();
break;
default:
encryptService = null;
}
return encryptService;
}
Next, you can extend mybatis’s typeHandler to encrypt and decrypt entity field data based on the encryption algorithm: EncryptTypeHandler
public class EncryptTypeHandler<T> extends BaseTypeHandler<T> {
@Resource
private EncryptService encryptService;
@Override
public void setNonNullParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
ps.setString(i, encryptService.encrypt((String)parameter));
}
@Override
public T getNullableResult(ResultSet rs, String columnName) throws SQLException {
String columnValue = rs.getString(columnName);
return StrUtil.isBlank(columnValue) ? (T)columnValue : (T)encryptService.decrypt(columnValue);
}
@Override
public T getNullableResult(ResultSet rs, int columnIndex) throws SQLException {
String columnValue = rs.getString(columnIndex);
return StrUtil.isBlank(columnValue) ? (T)columnValue : (T)encryptService.decrypt(columnValue);
}
@Override
public T getNullableResult(CallableStatement cs, int columnIndex) throws SQLException {
String columnValue = cs.getString(columnIndex);
return StrUtil.isBlank(columnValue) ? (T)columnValue : (T)encryptService.decrypt(columnValue);
}
}
2.2 Encryption and decryption examples
First create a user table:
CREATE TABLE `user` ( `id` bigint(20) NOT NULL, `name` varchar(255) DEFAULT NULL COMMENT 'name', `phone` varchar(255) DEFAULT NULL COMMENT 'Mobile phone number', `id_card` varchar(255) DEFAULT NULL COMMENT 'ID card number', `bank_card` varchar(255) DEFAULT NULL COMMENT 'bank card number', `address` varchar(255) DEFAULT NULL COMMENT 'Address', PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
At this time we insert a piece of data normally:
@Test
public void test() {
User user = new User();
user.setName("shepherd");
user.setMobile("17812345678");
user.setIdCard("213238199601182111");
user.setBankCard("3222022046741500");
user.setAddress("Future Science and Technology City, Yuhang District, Hangzhou City");
userDAO.insert(user);
}
The database stores the query results as follows:
| id | name | mobile | id_card | bank_card | address |
|---|---|---|---|---|---|
| 1567402046481436673 | shepherd | 17812345678 | 213238199601182111 | 3222022046741500 | Future Science and Technology City, Yuhang District, Hangzhou |
This is the result of our usual unencrypted storage query. The ID here is automatically generated through the distributed ID algorithm.
Next, let’s take a look at encrypting data. We only need to configure which encryption algorithm to use in the configuration file and add annotations to the field attributes of the entity class @TableField(typeHandler = EncryptTypeHandler.class) That’s it.
Here we use the aes encryption algorithm:
ptc:
encrypt:
algorithm: aes
Entity class:
@Data
@TableName(autoResultMap = true)
public class User {
private Long id;
private String name;
@TableField(typeHandler = EncryptTypeHandler.class)
private String mobile;
@TableField(typeHandler = EncryptTypeHandler.class)
private String idCard;
@TableField(typeHandler = EncryptTypeHandler.class)
private String bankCard;
@TableField(typeHandler = EncryptTypeHandler.class)
private String address;
}
Insert data again, and the database stores the query results as follows:
| id | name | mobile | id_card | bank_card | address |
|---|---|---|---|---|---|
| 1567405175268642818 | shepherd | 9MgWngwLcd/vbYYYpG9pGQ== | 97vlZQahK + y548ofQbXlW9JUwuzuj3xCkNF/is1KLa4= | 2oQv5 + y4 + rVyN23IzudtOz + Zd7Aj1Bv2toBzmnwTXxo= | 0Wj7qqLl6jWkBu + TcxuwGYcdI jv + zIJHDM7d1dU/c8D2jc2wLp + zVvpSwBKWjX44 |
Then we can test querying this data:
@Test
public void get() {
User user = userDAO.selectById(1567405175268642818l);
System.out.println(user);
}
The result is as follows:
User(id=1567405175268642818, name=shepherd, mobile=17812345678, idCard=213238199601182111, bankCard=3222022046741500, address=Future Technology City, Yuhang District, Hangzhou)
Based on the above, the data encryption storage and decryption query are perfectly demonstrated.
2.3 How to perform fuzzy matching after data encryption
Passwords, mobile phone numbers, detailed addresses, bank card numbers and other information have different requirements for encryption and decryption. For example, we need to encrypt and store passwords. Generally, irreversible slow hash algorithms are used. Slow hash algorithms can avoid brute force cracking (typical Trade time for security).
During retrieval, we do not need to decrypt or fuzzy search, and directly use the ciphertext to completely match. However, we cannot do this with the mobile phone number, because we need to view the original information for the mobile phone number, and the mobile phone number also needs to support fuzzy search, so we Today, we will support fuzzy queries for reversible encryption and decryption of data to see what implementation methods are available.
Let’s take a look at the conventional approach, which is also the most widely used method. This method meets the requirements of data security and is query-friendly.
-
Implement the encryption algorithm function in the database and use
decode(key) like '%partial%during fuzzy queryImplement the encryption and decryption algorithm consistent with the program in the database, modify the fuzzy query conditions, and use the database encryption and decryption function to decrypt first and then fuzzy search. The advantage of this is low implementation cost, low development and use cost, and only need to slightly modify the previous fuzzy search. This can be achieved by modifying it, but the shortcomings are also obvious. In this way, the index of the database cannot be used to optimize the query. Some databases may not even be able to guarantee the same encryption and decryption algorithms as the program, but for conventional encryption and decryption algorithms, they can be guaranteed to be consistent with the program. The application is consistent. If the query performance requirements are not particularly high and the data security requirements are average, common encryption and decryption algorithms such as AES and DES are also a good choice.
-
Perform word segmentation and combination of ciphertext data, encrypt the result sets of the word segmentation combination respectively, and then store them in extended columns. When querying, use
key like '%partial%'[First Group characters into fixed-length groups and split a field into multiple ones. For example, 4 English characters (half-width) and 2 Chinese characters (full-width) are used as a search condition. For exampleshepherduses a group of 4 characters for encryption, the first group is shep, the second group is heph, the third group is ephe, the fourth group is pher… and so on.If you need to retrieve all data containing 4 characters of the search condition, such as: pher, encrypt the characters and use
key like “%partial%”to search the database.Word segmentation encryption implementation
public static String splitValueEncrypt(String value, int splitLength) { //Check whether the parameters are legal if (StringUtils.isBlank(value) & amp; & amp; splitLength <= 0) { return null; } String encryptValue = ""; //Get the number of character substrings that the entire string can be cut into int n = (value.length() - splitLength + 1); //Word segmentation (rule: the length of the word segmentation is based on [splitLength] and the start and end subscripts of each split are added by one) for (int i = 0; i < n; i + + ) { String splitValue = value.substring(i, splitLength + + ); encryptValue + = encrypt(splitValue); } return encryptValue; } /** * Get the encrypted value * * @param value encrypted value * @return */ private static String encrypt(String value) { //Encrypt here return null; }Based on the above word segmentation, encryption is saved to the extension column. At the same time, the original field is required to be corrected, deleted, modified, and checked to adapt to its corresponding extended column. It should also be noted that the length of the extended column may be several times or even dozens of times that of the original field after word segmentation. , so be sure to select and appropriate word segmentation length and encryption algorithm before development. Once encryption starts, the cost of changing it will be higher. For example, if the mobile phone number only supports the last 8 digits of the search, the ID card number only supports the last 4 digits of the search. In this way, we can intercept the last digits of the original field and directly encrypt and store them in the extended column without the need for word segmentation.
3. Data desensitization
In the actual business development process, we often need to desensitize users’ private data. The so-called desensitization process is actually to obfuscate and hide the data, such as displaying user mobile phone information
178****5939, to avoid leaking personal privacy information.3.1 Implementation Ideas
The idea is relatively simple: Before the interface returns the data, desensitize the data as required and then return to the front end.
At first I planned to use @ControllerAdvice to implement it, but I found that I need to go to the reflection class to get the annotations. When the returned object is more complex and needs to be reflected recursively, the performance will suddenly decrease, so I changed my mind and thought of the @JsonFormat I usually use. Very similar to my current scenario, fields are custom parsed through custom annotations and field parsers.
Desensitized field type enumeration
public enum MaskEnum { /** * Chinese name */ CHINESE_NAME, /** * ID number */ ID_CARD, /** * Landline number */ FIXED_PHONE, /** * Phone number */ MOBILE_PHONE, /** * address */ ADDRESS, /** * Email */ EMAIL, /** * bank card */ BANK_CARD }Desensitization annotation class: used on desensitized fields
@Retention(RetentionPolicy.RUNTIME) @JacksonAnnotationsInside @JsonSerialize(using = MaskSerialize.class) public @interface FieldMask { /** * Type of desensitization * @return */ MaskEnum value(); }Desensitized serialization class
public class MaskSerialize extends JsonSerializer<String> implements ContextualSerializer { /** * Type of desensitization */ private MaskEnum type; @Override public void serialize(String s, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException { switch (this.type) { case CHINESE_NAME: { jsonGenerator.writeString(MaskUtils.chineseName(s)); break; } case ID_CARD: { jsonGenerator.writeString(MaskUtils.idCardNum(s)); break; } case FIXED_PHONE: { jsonGenerator.writeString(MaskUtils.fixedPhone(s)); break; } case MOBILE_PHONE: { jsonGenerator.writeString(MaskUtils.mobilePhone(s)); break; } case ADDRESS: { jsonGenerator.writeString(MaskUtils.address(s, 4)); break; } case EMAIL: { jsonGenerator.writeString(MaskUtils.email(s)); break; } case BANK_CARD: { jsonGenerator.writeString(MaskUtils.bankCard(s)); break; } } } @Override public JsonSerializer <?> createContextual(SerializerProvider serializerProvider, BeanProperty beanProperty) throws JsonMappingException { // Skip directly if empty if (beanProperty == null) { return serializerProvider.findNullValueSerializer(beanProperty); } // Non-String classes are skipped directly if (Objects.equals(beanProperty.getType().getRawClass(), String.class)) { FieldMask fieldMask = beanProperty.getAnnotation(FieldMask.class); if (fieldMask == null) { fieldMask = beanProperty.getContextAnnotation(FieldMask.class); } if (fieldMask != null) { // If the annotation can be obtained, pass the annotation value into MaskSerialize return new MaskSerialize(fieldMask.value()); } } return serializerProvider.findValueSerializer(beanProperty.getType(), beanProperty); } public MaskSerialize() {} public MaskSerialize(final MaskEnum type) { this.type = type; } }3.2 Usage Example
Desensitize the mobile phone number on the interface for sending SMS records:
@FieldMask(MaskEnum.MOBILE_PHONE) private String mobile;The data returned by calling the interface is as follows:
{ "code": 200, "msg": "OK", "data": { "list": [ { "id": 1565599123774607362, "signId": 8389008488923136, "templateId": 8445337328943104, "templateType": 1, "content": "Lovely ${name}, the blog post has been uploaded and updated at ${submitTime}, please take the time to browse.", "channelType": 0, "mobile": "178****5939", "sendStatus": 0, "receiveStatus": 0 } ], "total": 19, "pages": 19 } }4. Summary
Based on the above content, we summarize how to perform data security reinforcement at the data storage layer to achieve a more secure system. It can be said that there is no most secure system, only a safer system. Therefore, we will spend our whole life to strengthen system security performance during the development process. Of course, there are many ways to enhance system security. We have recently focused on the effective and elegant implementation of data security based on the
Spring BootandSpringMVCframeworks. Interested parties Friends can learn about other reinforcement methods by themselves. (Recently, I have returned to positive state, and my condition is not very good, and my efficiency is not high, so I am a bit delayed in updating.)