Implementation ideas
Our check-in function can be completely completed through mysql.
CREATE TABLE `sign_record` ( `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'primary key', `user_id` bigint NOT NULL COMMENT 'userid', `year` year NOT NULL COMMENT 'Sign in year', `month` tinyint NOT NULL COMMENT 'Month of check-in', `date` date NOT NULL COMMENT 'check-in date', `is_backup` bit(1) NOT NULL COMMENT 'Whether to re-sign', PRIMARY KEY (`id`), ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Sign-in record table';
However, if a user checks in 100 times a year and the website has 1 million users, 100 million records will be generated. As the number of users increases and time goes by, there will only be more and more data in this table, and the space it takes up will also become larger and larger. Is there any way to reduce the number of sign-in data records and reduce space usage?
In fact, we can consider a very common solution when we were young. When we were young, we prepared a small card. As long as you signed in, you would put a check mark. I would finally judge whether you signed in. In fact, you only need to look at the small card to know.
If we can use a program to simulate such a check-in card and use one row of data to record a user’s check-in status for a month, it is conceivable that it will save a lot of space compared to this database method.
We count user check-in information on a monthly basis. The sign-in record is 1, and the non-sign-in record is 0. Each bit corresponds to each day of the month, forming a mapping relationship. Using 0 and 1 to mark the business status is called BitMap. In this way, we can use a very small space to represent a large amount of data.
BitMap usage
Redis provides the BitMap structure and some related operation commands. https://redis.io/commands/?group=bitmap
- offset: which bit of data is to be modified?
- value: 0 or 1
If you want to sign in, you can use the above command. For example, if you sign in on the 1st, 2nd, 3rd, 6th, 7th, and 8th of this month, you can do this:
# Sign in on day 1 SETBIT bm 0 1 # Sign in on day 2 SETBIT bm 1 1 # Sign in on the 3rd day SETBIT bm 2 1
Let’s try it in the redis console:
What if we want to query the check-in record?
That is to read the data in BitMap, you can use this command: This command is relatively complex and is a combination command that can implement various operations such as query and modification. But we only care about reading, so we only look at the first operation, GET:
BITFIELD key GET encoding offset
- key: is the key of BitMap
- GET: stands for query
- encoding: the encoding method of the returned result. BitMap saves it in binary, and the returned result will be converted to decimal, but a conversion rule is required, which is the encoding method here.
- u: unsigned integer, such as u2, which means reading 2 bits, converting to an unsigned integer and returning
- i: A signed integer, such as i2, which means reading 2 bits, converting it to a signed integer and returning it
- offset: which bit to start reading from, for example 0: means starting from the first bit
For example, if I want to query the check-in records from day 1 to day 3, I can do this:
BITFIELD bm GET u3 0
You can see the return results:
The returned result is 7. Why is it 7?
The sign-in record is 11100111. Starting from 0, taking 3 bits, it happens to be 111. Convert to an unsigned integer, it happens to be 7.
Note: The calculation method for converting binary to decimal is
1. Unsigned integer, from right to left, multiply the sum of the binary digits by the nth power of 2 (n is greater than or equal to 0)
2. Signed binary integer, except for the highest sign bit (1 is a negative number, 0 is a positive number), the rest is the same as the method of converting unsigned binary to decimal:
For example: 111 (binary) to decimal
Unsigned conversion rule: (12 to the power 2 + 12 to the power 1 + 1*2 to the power 0) = 7
**Signed conversion returns. First check whether the highest bit is 1. If it is 1, it means it is a negative number. Take the inverse of the other bits and add one, ** and then convert to decimal in the normal way (add a negative sign to decimal). If the highest bit is 0, convert directly to decimal in the normal way. So 111 (binary) converted to signed is: -1
Sign-in interface
We plan to generate an independent KEY for each user every month, so the KEY must contain user information and month information, which looks like this:
sign:uid:xxx:202401
It can be seen from the structure of KEY that in order to sign in, you must know who signed in on which day, that is, two pieces of information:
- Current user
- current time
We can obtain both of these information ourselves, so when signing in, the front end does not need to pass any parameters. So do I need to return data after signing in?
The requirements say that there will be points rewards for continuous sign-ins, so in order to improve the user experience, should the number of consecutive check-in days and points rewards obtained be returned after the user successfully signs in?
Entity class
Regarding the return value of the check-in function, I think we only need these three fields for the entity class:
- signDays: Number of consecutive sign-in days
- signPoints: sign-in score, fixed at 1
- rewardPoints: reward points for consecutive check-ins
Because we do not implement sign-in in mysql, we implement it in redis. So we only need a class that returns the front end.
Continuous sign-in statistics
Definition: Count forward from the last check-in until the first failure to check in. Calculate the total number of check-ins, which is the number of consecutive check-in days.
Java logic code: Get the last check-in data of the current month, define a counter, and then keep counting forward until the first non-0 is obtained Thenumber is enough. Every time you get a non-zero number, the counter + 1, until all the data is traversed, you can get the total number of check-in days in the current month.
So how do we traverse each bit from back to front?
Note: The data returned by bitMap is in decimal. If redis returns a number 8, how do I know which ones are 0 and which ones are 1? We just need to AND the decimal number we get with 1, because 1 is 1 only when it meets 1, and other numbers are all 0. We AND the sign-in result with 1 , every time it is used, move the check-in result one position to the right, and push it in sequence, we can complete the effect of traversing one by one.
Concrete implementation of the project
Create a new springboot project:
application.yaml
spring:
redis:
database: 3
host: localhost
port: 6379
domain:
package com.zd.domain;
import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.Data;
@Data
public class Result {<!-- -->
//Number of consecutive check-in days
private Integer signDays;
//Sign-in score
private Integer signPoints = 1;
// Continuous sign-ins will award points. Only if you sign-in continuously for more than 7 days will you be rewarded.
private Integer rewardPoints;
}
UserController:
package com.zd.controller;
import com.zd.domain.Result;
import com.zd.service.UserService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class UserController {<!-- -->
@Autowired
private UserService userService;
@PostMapping("/sign")
public Result sign(){<!-- -->
return userService.sign();
}
}
UserService：
package com.zd.service;
import com.zd.domain.Result;
public interface UserService {<!-- -->
//Sign in
Result sign();
}
UserServiceImpl:
package com.zd.service.impl;
import com.zd.domain.Result;
import com.zd.service.UserService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.connection.BitFieldSubCommands;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;
import java.time.LocalDate;
import java.time.format.DateTimeFormatter;
import java.util.List;
@Service
public class UserServiceImpl implements UserService {<!-- -->
@Autowired
private StringRedisTemplate redisTemplate;
@Override
public Result sign() {<!-- -->
//todo here should first get the user id
//splicing key
LocalDate now =LocalDate.now();//Get the current year and month
String format = now.format(DateTimeFormatter.ofPattern(":yyyyMM"));//Get the string with colon year and month
String key= "sign" + format;
//Use the bitset command to save the check-in record into the bitmap structure of redis. You need to verify whether you have signed in.
int offset=now.getDayOfMonth()-1;//The current day number-1
Boolean setBit = redisTemplate.opsForValue().setBit(key, offset, true);//The last parameter is whether to sign in
if(setBit){<!-- -->
throw new RuntimeException("Cannot check in repeatedly");
}
//Calculate the number of consecutive check-in days
int days =countSignDays(key,now.getDayOfMonth());
//Calculate reward points for consecutive sign-ins
int rewardPoints=0;//Represents continuous check-in reward points
switch(days){<!-- -->
case 7:
rewardPoints=10;
break;
case 14:
rewardPoints=20;
break;
case 28:
rewardPoints=40;
break;
}
//At first I wanted to use if, but I thought it was too cumbersome, so I changed to switch.
// if(days==7){<!-- -->
// rewardPoints=10;
// }else if(days==14){<!-- -->
// rewardPoints=20;
// }else if(days==28){<!-- -->
// rewardPoints=40;
// }
//todo save points
//Encapsulate vo return
Result vo=new Result();
vo.setSignDays(days);
vo.setRewardPoints(rewardPoints);
returnvo;
}
/**
* Calculate how many days the check-in lasts
* @param key
* @param dayOfMonth The number of days from this month to today
* @return
*/
private int countSignDays(String key, int dayOfMonth) {<!-- -->
//Find all check-in data from the first day of this month to the current day
List<Long> bitField = redisTemplate.opsForValue().bitField(key,
BitFieldSubCommands.create().
get(BitFieldSubCommands.BitFieldType.unsigned(dayOfMonth)).valueAt(0));
if(bitField.isEmpty()){<!-- -->
return 0;
}
Long num = bitField.get(0);//Check-in data from the first day of this month to the current day
//Num is converted to binary, and how many 1s are there when pushing forward from back to front?
int counter = 0;//Counter
while((num & amp;1)==1){<!-- -->//Start traversing, do AND operation with 1 and shift one position to the right
counter + + ;
num=num>>>1;
}
return counter;
}
}
Start the project, open postman, and visit http://localhost:8080/sign
After receiving the return value, check if there is any data in redis:
Redis has data. Will it throw an exception if I try again?
Regarding the continuous check-in function, I modified the date data of continuous check-in, and the result is also correct.
At this point, we have successfully completed the check-in function in redis.
Extended content of bitmap
Redis has only 5 basic data types: String, List, Set, SortedSet, and Hash. Most of the other special data structures are based on the above 5 data types.
BitMap is no exception, it is based on String structure. Because the underlying String type of Redis is SDS, there will also be a byte array used to save data. Redis provides several commands for bitwise manipulation of data in this array to achieve the BitMap effect.
Since the maximum space of the String type is 512MB, which is 2 to the 31st power of bits, the amount of data that can be saved is very scary.
To understand the SDS structure of Redis, you can refer to the following video: https://www.bilibili.com/video/BV1cr4y1671t?p=146 & amp;vd_source=1ff0c1b434581723cf696ccc2f59ceaa
bitmap operation command:
- SETBIT: Store a 0 or 1 at the specified location (offset)
- GETBIT: Get the bit value at the specified position (offset)
- BITCOUNT: Counts the number of bits with a value of 1 in the BitMap
- BITFIELD: operate (query, modify, increment) the value of the specified position (offset) in the bit array in BitMap
- BITFIELD_RO: Get the bit array in BitMap and return it in decimal form
- BITOP: Perform bit operations (AND, OR, XOR) on the results of multiple BitMap
- BITPOS: Find the position where the first 0 or 1 appears in the specified range in the bit array
Reference documentation
Bitmap based on redis to implement sign-in function (backend)_redis continuous sign-in-CSDN blog
https://redis.io/commands/?group=bitmap