1Foreword
As back-end development programmers, we often have some relatively complex logic. For example, we need to write a calling interface for the front end. This interface needs to perform relatively complex business logic operations, such as query and remote interface. Or some logic such as local interface call, update, insertion, calculation, etc., the final interface return result is given to the front end. After such a series of business logic operations, the interface operates the DB, judges the code business logic, and makes interface calls. It all takes time, and as long as this is a transaction operation, each interaction with the database will generate a transaction record.
Then this will have an impact on the efficiency returned by our interface, and this impact will grow with the increase in data volume. At this time, we need to split an entire large transaction to improve the efficiency of the overall interface.
2What is a big event
Take an interface I recently developed and written as an example. The logic is roughly like this. I need to generate a payment slip based on the data submitted on the page. The overall business handled by the interface is as follows. I wrote them in an interface. It can be understood that this is a big thing. The execution time of this interface is relatively long, and it is not reasonable to write all these logics in one interface.
Some problems existing in the three major affairs
Concurrent data is inconsistent
Without locking, due to various reasons, the first call to the interface has not been completed and is still waiting for a third-party call to write back the data. The second call comes in and changes the data. The second call is executed first. Finished. At this time, the first interface call got the return of the third-party interface. When I went to write back the status, I found that it had been updated, resulting in an invalid operation.
Locking is easy to block
In the case of locking, there will be no data inconsistency. However, due to the long execution time of large transactions, it is easy to cause lock timeout and lock too much data to cause blocking, seriously affecting efficiency.
Undo logo transaction log performance issues
It is easy to cause a large amount of Undo logo log data, which reduces the query performance of the log, including the efficiency of transaction rollback.
The concurrent database is under too much pressure
When the amount of concurrency reaches a certain level, it will put a lot of pressure on database reading and writing, and a large number of waiting threads will accumulate.
4How to optimize large transactions
Do not make remote RPC calls in transactions
First, remote interface calls are made in transactions. If the distributed transaction framework is not used, there will be transaction inconsistencies, and data rollback operations cannot be performed. In concurrent situations, the remote service does not respond in time, and interface return inconsistencies will occur. Of course, asynchronous calls must be used, which will be mentioned later.
Programmatic transactions are more flexible
Declarative transactions only need to add the @Transactional annotation to the method header to start the transaction, but it is still not very flexible, which means that the database operation performed by the entire method must be added to the transaction. Of course, a query must also enter the transaction. This is not What we want is to perform transaction operations on update and insert operations to facilitate rollback.
public Boolean transactionCommit(String userName) {
//Query user
SysUser sysUser = userMapper.selectUserByUserName(userName,null);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
@Override
protected void doInTransactionWithoutResult(TransactionStatus transactionStatus) {
try {
if (null != sysUser) {
//User information status update status is updated to 1
userMapper.updateStatus(userName);
}
} catch (Exception e){
//rollback
transactionStatus.setRollbackOnly();
}
}
});
//Query again
SysUser sysUser1 = userMapper.selectUserByUserName(userName,"1");
/log/.info("User information with status 1" + JSON./toJSONString/(sysUser1));
return true;
}
The flexibility of programmatic transactions is that you can control the transaction execution method and use the transactionTemplate class to perform transaction operations. The query operations can be written outside, so that the query to obtain data will not enter the MySQL transaction table.
Data batch processing
For transaction updates or insertions, there may be batch operations on the front end. Batch updates and insertions of large-scale data will also have an impact on the transaction interface. Once there is an update or insertion failure, in order to ensure the consistency of the transaction, the entire operation must be performed. rollback;
-
Front-end: You can limit data, access to the back-end interface, paginate data, and make multiple requests to avoid transactions from submitting large amounts of data.
-
Backend: You can also perform paging processing of data. For example, you can limit 50 operations at a time. If you are adding new logic, use Mybatis’s batch update to greatly improve efficiency.
List<List<ReceivableFeeSaveDTO>> partition = Lists.partition(receivableFeeSaveDTOList, 50);
Split large transactions into small transactions
A transaction interface can be split into multiple transaction interfaces, and each transaction interface only does one thing, such as the above payment receipt generation interface, amount writeback, third-party interface call, and call result writeback. It can be drawn into a small transaction interface.
It’s like doing a very complicated thing. It looks complicated at first glance, but if we divide this complicated step into multiple steps and complete the things at each stage, the whole process can be simplified. It doesn’t look that complicated.
Asynchronous parallel processing
The most important thing is that if remote calls cannot be avoided in the transaction, then asynchronous calls must be made, because the timely responsiveness of the remote interface cannot be guaranteed. The CompletableFuture asynchronous orchestration feature can be used. After task1 and task2 tasks are completed, task3 is executed.
CompletableFuture<Object> task1 =CompletableFuture.supplyAsync(() -> {
System.out.println("single number check thread" + Thread.currentThread().getId());
//Single number check interface throws an exception if verification fails
return "Billing entity information";
}, executor);
CompletableFuture<Object> task2 = CompletableFuture.supplyAsync(() -> {
System.out.println("Receipt Form Generation Thread" + Thread.currentThread().getId());
try {
//Generate payment slip
return "Bill Number";
Thread.sleep(3000);
System.out.println("Task 2 ends:");
} catch (InterruptedException e) {
e.printStackTrace();
}
}, executor);
//After task1 and task2 are executed, task3 is executed and the execution results of task1 and task2 need to be sensed.
CompletableFuture<Boolean> future = task1.thenCombineAsync(task2, (t1, t2) -> {
System.out.println("Bill amount write-back thread" + Thread.currentThread().getId());
// t1, t2 return judgment
//Write back the result
return true;
}, executor);
5 Summary
It can be seen that large transactions are the culprit of the inefficiency of our interfaces. Sometimes in order to quickly implement functions, we may ignore some things related to performance, and these things are an opportunity to improve our capabilities.
As you progress, you may have questions about why you wrote the code like this before. When you feel this way, congratulations, you are already standing on another hill, overlooking the mountain. Everything is so small and not much. He said I would optimize the interface first~
Source: juejin.cn/post/7213636024110956599
Back-end exclusive technology group
To build a high-quality technical exchange community, HR personnel engaged in programming development and technical recruitment are welcome to join the group. Everyone is also welcome to share their own company’s internal information, help each other and make progress together!
Speak civilly, use
exchange technology, and leadrecommendations for positionsindustry discussions
Advertisers are not allowed to enter. Please trust your contacts in private messages to prevent being deceived.
