On July 10, 2023, TiDB Serverless was officially launched for commercial use. This is a fully managed database as a service platform (DBaaS) that provides flexible cluster configuration and usage-based payment model. Immediately afterwards, the beta version of TiDB Serverless Branching was also released.
The TiDB Serverless Branching feature enables users to create branches for their TiDB Serverless clusters. These branches can enable parallel development, promote rapid iteration of new features, and troubleshoot faults without interrupting the operation of the production database. This feature not only simplifies the development and deployment process, but also maintains the stability and reliability of the database in production environments.
What is a branch?
To a cluster, a branch is an independent instance that contains a snapshot of the original cluster’s data. It establishes an isolated environment so that various operations can be performed without affecting the original cluster.
When a branch is created for a cluster, the data in that branch begins to fork from the original cluster, which means that subsequent changes made in the original cluster or the branch will not be synchronized.
TiDB Serverless uses copy-on-write technology to create branches quickly and smoothly. This approach allows data to be shared between the original cluster and its branches. This operation can usually be completed within a few minutes, is imperceptible to the user, and will not affect the performance of the original cluster.
Balancing software development speed and quality
In software development, the delicate balance between rolling out new features quickly and testing them thoroughly is a challenge. Finding the right balance enables agile development, faster iterations, and receiving valuable customer feedback in a timely manner without compromising the quality and reliability of your software. TiDB Serverless Branching provides a way to find this optimal balance.
Provides an independent environment for developers
When using databases in daily development and testing activities, developers often face challenges such as quota restrictions, high costs, resource constraints, and data quality. So it’s usually more practical to share the database within the team. But shared databases often lead to environmental conflicts, and developers have to spend extra effort to add some isolation logic to the application.
TiDB Serverless Branching solves these problems by providing each developer with an independent development and testing environment. Increase productivity and promote efficient team collaboration by eliminating resource sharing and task distractions.
Efficient testing with production-like branches
In order to eliminate the troubles caused by resource sharing and interference between tasks, developers usually use a separate database environment when doing development work, such as setting up their own database on the server or starting a database container on Docker. However, these environments differ significantly from production environments, making it difficult to simulate actual performance and latency conditions. For functional testing, a lot of data preparation work is always required, but it may still not match the actual production environment. To ensure data quality, developers can choose to take a snapshot of the production database and restore it in a test environment, or painstakingly build mock data. However, both methods are cumbersome and will significantly reduce development efficiency.
With TiDB Serverless Branching, developers will be able to quickly create branches that are identical to the production environment in minutes. These branches facilitate testing with the latest production data and detecting issues quickly. Additionally, these branches are completely isolated from the production cluster, allowing for safer functional testing and troubleshooting.
Seamless integration with Continuous Integration and Continuous Deployment (CICD) processes
Automation scenarios require advanced environmental management and quality control. TiDB Serverless Branching can be easily integrated into automated CICD workflows. Through branch integration, code quality control and testing processes become smoother. This ensures the quality of the product while following efficient software development practices.
Branch management integrated with GitHub
Combined with the TiDB Serverless Branching function, we launched the TiDB Cloud branch management GitHub App. If developers use GitHub flow ( https://docs.github.com/en/get-started/quickstart/github-flow ), the application can greatly simplify the integration of branches into CI pipelines. For more details about the GitHub App, see our documentation ( https://docs.pingcap.com/tidbcloud/branch-github-integration ).
Integrate TiDB Serverless Branching with GitHub CI Pipeline
We will use tidbcloud-branch-gorm-example (https://github.com/shiyuhang0/tidbcloud-branch-gorm-example) to demonstrate the integration process. tidbcloud-branch-gorm-example (https://github.com/go-gorm/playground) is a branch version of the gorm playground project suitable for TiDB Serverless Branching. Here’s how to do it:
- Follow the instructions to connect your TiDB Serverless cluster with your GitHub repository. This will install the TiDB Cloud Branch Management GitHub App on your GitHub account. Each time a pull request is created, it will trigger the creation of a new corresponding TiDB Serverless branch.
- Adjust the CI pipeline and use the newly created TiDB Serverless branch to run tests.
name: tests
?
on:
pull_request:
branches: [master]
?
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: tidbcloud/wait-for-tidbcloud-branch@v0
id: wait-for-branch
with:
token: ${<!-- -->{ secrets.GITHUB_TOKEN }}
public-key: ${<!-- -->{ secrets.TIDB_CLOUD_PUBLIC_KEY }}
private-key: ${<!-- -->{ secrets.TIDB_CLOUD_PRIVATE_KEY }}
timeout-seconds: 600
- name: Set up Go 1.19
uses: actions/setup-go@v2
with:
go-version: 1.19
?
- name: go mod pakcage cache
uses: actions/cache@v2
with:
path: ~/go/pkg/mod
key: ubuntu-latest-go-1.19-${<!-- -->{ hashFiles('go.mod') }}
?
- name: Tests
run: |
GORM_ENABLE_CACHE=true GORM_DIALECT=tidb GORM_DSN="${<!-- -->{ steps.wait-for-branch.outputs.username }}:${<!-- -->{ steps.wait-for-branch .outputs.password }}@tcp(${<!-- -->{ steps.wait-for-branch.outputs.host }}:${<!-- -->{ steps.wait-for-branch .outputs.port }})/test?parseTime=true & amp;tls=tidb" ./test.sh
To simplify the CI pipeline, we developed wait-for-tidbcloud-branch ( https://github.com/tidbcloud/wait-for-tidbcloud-branch ), a secondary GitHub Action. It will wait until the TiDB Serverless branch corresponding to the pull request is ready, and then create a new database user for subsequent use.
After completing the setup, whenever a pull request is created, the TiDB Cloud branch management GitHub App will generate a TiDB Serverless branch for the pull request. The CI process will then wait for the branch to be ready and use it for testing. This ensures that any tests you perform using these branches can be mapped to data in the production database, making it easier to catch any potential issues in a timely manner.
A specific development case
For example, you are developing a function that adds a unique index, but there are already duplicate rows in the production database. With Ti DB Serverless Branching, the CI pipeline can quickly identify such issues. This is because the test load is executed in the same branch as the production database image, allowing potential issues to be detected early. However, if you are testing on simulated data, such issues can easily be overlooked and may even be missed in the production environment.
In the above example, we performed a simple DDL change in the database. TiDB supports online DDL (https://docs.pingcap.com/tidb/stable/ddl-introduction#execution-principles-and-best-practices-of-ddl-statements) changes, so users do not need to worry about affecting application workload. However, as the application expands and tests involve more business logic, branching becomes increasingly beneficial for efficiency and quality assurance.
Use TiDB Serverless Branching to customize CI/CD workflow
TiDB Cloud Branching workflows are specifically designed to be compatible with popular application frameworks, allowing developers to simply integrate them into existing CI pipelines. If TiDB Serverless Branching GitHub App cannot fully meet your needs, you can use TiDB Cloud CLI (https://docs.pingcap.com/tidbcloud/get-started-with-cli), setup-tidbcloud-cli (https://github .com/tidbcloud/setup-tidbcloud-cli ) and the help of GitHub Actions, you can also customize the branch’s CI/CD workflow.
Conclusion
With TiDB Serverless Branching, developers can work in parallel and quickly iterate new features without affecting the normal and stable operation of the production database. This feature simplifies the development and deployment process while ensuring the stability and reliability of production databases.
We hope that more developers can use TiDB Serverless Branching, and we will continue to improve and expand this feature to meet your changing needs, so stay tuned!
Click the link to try TiDB Serverless Branching now for a more streamlined and faster application development experience!
TiDB Cloud
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Cloud native entry-level skills treeHomepageOverview 15194 people are learning the system