Apache ShardingSphere 5.4.1 version is released, data encryption has been upgraded to provide a higher level of data protection!

04a479a5c8ba7d8e99c2585e079cc034.jpeg

Summary

ShardingSphere

Apache ShardingSphere ushered in the release of version 5.4.1 this week. This version lasted for more than two months and merged a total of 967 PRs from global teams and individuals. The new version has improved functionality, performance, testing, and documentation. , examples and other aspects have been greatly optimized. This update contains a lot of content that can improve user experience and solve user pain points. This article will give you a quick preview:

  • Encryption: Encryption rules do not have the same name as logical columns except plaintext columns and ciphertext columns.

  • Encryption: Proxy prohibits ciphertext from DDL

  • Encryption: Derived column format and length fixed varchar 4000

  • Adaptation: Proxy connection pool adaptation (druid, dbcp, c3p0)

5946a8a41a236e370963496a2dcf33d0.jpeg

Feature Preview

ShardingSphere

01Encryption rules do not have the same names as logical columns except plaintext columns and ciphertext columns

  • Background

The current encryption rules do not verify whether the field name of the derived column in the configuration has the same name as the logical column. This time, a new derived column name verification is added. If it has the same name as the logical column, an error message will be reported.

  • Technical implementation

Added a new derived column check item for Encryption Rule, the prompt information is:

Assisted query column or like query column conflicts with logic column
private void checkColumnNames(final CreateEncryptRuleStatement sqlStatement) {
    for (EncryptRuleSegment each : sqlStatement.getRules()) {
        ShardingSpherePreconditions.checkState(isColumnNameNotConflicts(each),
                () -> new InvalidRuleConfigurationException("encrypt", "assisted query column or like query column conflicts with logic column"));
    }
}


private boolean isColumnNameNotConflicts(final EncryptRuleSegment rule) {
    for (EncryptColumnSegment each : rule.getColumns()) {
        if (null != each.getLikeQuery() & amp; & amp; each.getName().equals(each.getLikeQuery().getName())) {
            return false;
        }
        if (null != each.getAssistedQuery() & amp; & amp; each.getName().equals(each.getAssistedQuery().getName())) {
            return false;
        }
    }
    return true;
}

02 Proxy prohibits ciphertext DDL

  • Background

After currently configuring the encryption rules, DDL of the encrypted table is not intercepted. If DDL is executed on the encrypted column/plain text column (such as modifying the field type, length, or even directly dropping the field), it may directly affect data writing and reading, causing failures. Questions etc.

This time, new ALTER statement type judgment for encrypted tables is added, ALTER xxx ADD statements are released, ALTER xxx CHANGE/MODIFY/DROP statements are intercepted, and the interception prompt message: Unsupported operation ‘xxx’ for the cipher column”

  • Technical implementation

All Alter statements for encrypted tables need to be rewritten through EncryptAlterTableTokenGenerator to generate actual execution statements. The current logic will generate statements for ciphertext columns/plaintext columns.

Implementing interception requires adding judgment before generating Token. If the encrypted column/plaintext column is operated, an exception will be thrown.

ShardingSpherePreconditions.checkState(!encryptTable.isEncryptColumn(columnName) & amp; & amp; !encryptTable.isPlainColumn(columnName),
        () -> new UnsupportedOperationException("Unsupported operation 'modify' for the cipher column and plain column"));

03 Derived column format and fixed length varchar 4000

  • Background

In the encryption scenario, the ciphertext generated by the existing algorithm is a string, but the user’s logical column may be of type such as number or time, which will cause the generated derived column data type to be consistent with the logical column, that is, ciphertext column, auxiliary query The data type of columns and fuzzy query columns may be int, datetime, etc., causing the ciphertext to be unable to be written. Need to set default text type for encrypted derived columns

  • Technical implementation

The generated ciphertext column, auxiliary query column, and fuzzy query column all use the default data type VARCHAR(4000)

04 Proxy connection pool adaptation (druid, dbcp, c3p0)

  • Background

Currently, the Hikari connection pool is used globally by default. Users cannot choose or change the connection pool by themselves.

a2cc328bf6f4384e1786c5eadc39efaa.png

  • Technical implementation

1. New field dataSourceClassName in ConnectionConfiguration class

public final class ConnectionConfiguration {
    
    private final String dataSourceClassName;
    
    private final String url;
    
    private final String username;
    
    private final String password;
}

2. Construct DataSourceProperties from fixed com.zaxxer.hikari.HikariDataSource to obtain dataSourceClassName

public static DataSourceProperties create(final DataSourceConfiguration dataSourceConfig) {
    return new DataSourceProperties(dataSourceConfig.getConnection().getDataSourceClassName(), createProperties(dataSourceConfig));
}

3. Implement the DataSourcePoolMetaData SPI of the connection pool. Currently Hikari, DBCP, and C3P0 have been implemented

4. Implement connection pool DataSourcePoolFieldMetaData, DataSourcePoolPropertiesValidator, DataSourcePoolActiveDetector

2aa8cd4f54da3b73a31c618864f6017a.jpeg

Update log

ShardingSphere

01 New Features

  1. Proxy connection pool adaptation (druid, dbcp, c3p0)

  2. Metadata: Standalone mode adapts to new version metadata

02 Function enhancement

  1. Mode: Optimize Standalone mode JDBC type initialization data reset

  2. Scaling: Isolate the storage of sharded data sources in Standalone mode without affecting each other.

  3. Scaling: Disable system-schema-metadata-enabled to improve performance

  4. JDBC: Register ShardingSphereDriver as JDBC java.sql.Driver SPI for user convenience

  5. Scaling: SHOW MIGRATION CHECK ALGORITHMS New verification algorithm column

  6. Scaling: SHOW MIGRATION CHECK ALGORITHMS New type_aliases column

  7. Encryption: Except for the ciphertext column, it cannot have the same name as the logical column. When adding encryption rules, an error should be reported when encountering the same name.

  8. Encryption: Proxy prohibits ciphertext from DDL

  9. Encryption: Derived column format and length fixed varchar 4000

03 Problem fixes

  1. Scaling: Fixed an issue: Wrong full task progress may be obtained when resuming a breakpoint when there are similar table names.

  2. Scaling: Fixed the problem: when the first full task is completed, full breakpoint resume cannot be enabled (CDC Importer is not started)

  3. Single Table: Fixed the problem of not switching versions when modifying Single rules in CREATE/DROP table

  4. JDBC: Fixed the MySQL JDBC memory leak problem in the previous version

d87cfc00292a580de108aa635ce9a35a.jpeg

Related links

ShardingSphere

Download link

https://shardingsphere.apache.org/document/current/cn/downloads/

Update log

https://github.com/apache/shardingsphere/blob/master/RELEASE-NOTES.md

project address

https://shardingsphere.apache.org/

Cloud sub-project address

https://github.com/apache/shardingsphere-on-cloud

edeb6e55329eebfbad25514eaa3883a2.jpeg

Community Contribution

ShardingSphere

For the release of Apache ShardingSphere 5.4.1, a total of 47 Contributors submitted 967 PRs. We are very grateful to the community partners for their strong support. We also welcome more and more developers to actively participate in the construction of the Apache ShardingSphere community and build on pure technology. In the atmosphere, you can improve your personal skills and gain self-growth. Welcome more open source technology enthusiasts to join the Apache ShardingSphere official exchange group, grow together with global technology partners, and build a community ecosystem!

787f3c3e715eaf09d9b75f7cfcb3795b.jpeg

How to join the ShardingSphere community and become a contributor?

  1. Community Q&A: Actively answer questions in the community, share technologies, and help other open source enthusiasts in the group solve problems.

  2. Code contribution: The community has compiled simple and easy-to-use tasks, which is very suitable for newcomers to make code contributions. You can check out the newbie task list:

    https://github.com/apache/shardingsphere/issues?q=is:open + is:issue + label:”good + first + issue”,discussion + no:assignee

  3. Content contribution: Publish ShardingSphere-related content, such as installation and deployment tutorials, usage experience, case practices, etc., in any form. You are welcome to scan the QR code and submit contributions to the community assistant.

  4. Community evangelism: Actively participate in community activities, become a community volunteer, help community publicity, provide effective suggestions for community development, etc.

  5. Official document contribution: Participate in community contributions by discovering document deficiencies, optimizing documents, and continuously updating documents. Through document contribution, developers can become familiar with how to submit PRs and truly participate in community building.

48e64a0744ea9940d46393be01ea1c1b.jpeg

Long press to identify and reply “Volunteer” to learn more~

About Apache ShardingSphere

Apache ShardingSphere is a distributed SQL transaction and query engine that can enhance any database through data sharding, elastic scaling, encryption and other capabilities.

Click to read the original text and download it to experience it~