1. Foreword
I have heard of MMKV before, but I haven’t had time to read it. I found it interesting after simply looking at its related content some time ago. Then I wanted to replace SP with MMKV. At this time, I thought of some The problem of data migration, so this time I will briefly talk about SharedPreferences and MMKV. I mainly want to talk about the problem of data migration.
2. MMKV
Tencent’s MMKV is awesome. Why is it awesome? It’s very thoughtful. This also reflects from the side that you want to make something awesome. You have to dare to think, and then you can realize it after you come up with a plan. . Maybe you look at its principle and you think it’s okay, and it’s not too complicated, but can you come up with this plan from 0 to 1 and then implement it?
First of all, we need to know why it was designed. According to the official introduction: a general-purpose key-value storage component with very high performance is needed. We inspected common components such as SharedPreferences, NSUserDefaults, and SQLite, and found that none of them can meet such stringent performance requirements. Obviously to improve performance
Does that mean I think MMKV performs better than SP, so I just use it? This is not the case. If you only use key-value components to store a small amount of data such as state, and do not read and write frequently, then SP is completely sufficient, and there is no need to introduce MMKV. But if the data you store is complex and reads and writes frequently, if you don’t finish writing the data this time and start writing the next time, there will be performance problems. At this time, it is completely wrong to use MMKV to replace SP. A great program.
Because my current project does not have such a requirement and has not reached such an order of magnitude, so I don’t need to use MMKV for the time being, but I briefly read its principle.I think there are only two core ideas: mmap and protobuf , other appends are further optimized operations on this basis, the core is mmap and protobuf, especially mmap. So why do you say awesome, because if you do it, without reference, can you come up with a solution like mmap to optimize?
What is mmap, memory mapping mmap, if you know the Binder mechanism, you should have some impressions of it, if you don’t know what memory mapping is, it is recommended to look at the Binder mechanism first, understand the concept of the next copy, and then come back You can know what the operation is by looking at mmap, and you will know why it uses this idea to improve performance.
Let’s look at another point, protobuf. Protobuf is a data storage format. It occupies less space, so it is also an optimization point. The smaller the space, the smaller the space required for storage, and the faster the transmission. quick.
2. SharedPreferences
A component that is often used by android, I like to use it because it is convenient to use. You can simply take a look at how it is implemented, and then compare it with the MMKV above.
Generally, we call commit() or apply of SharedPreferences.Editor, and then click to see that Editor is an interface, and SharedPreferences is also an interface. Click its class to see where to get it and find it in Context
public abstract SharedPreferences getSharedPreferences(File file, @PreferencesMode int mode);
See its subclass implementation in ContextWrapper
@Override
public SharedPreferences getSharedPreferences(File file, int mode) {
return mBase.getSharedPreferences(file, mode);
}
mBase is the Context, and after clicking it, it jumps to the Context again. It’s over, Barbie Q is gone, the endless loop, and the implementation class of SharedPreferences can’t be found. Why do you want to talk about this? In fact, if you look at the source code more, you will find that there is a habit. Generally, the specific implementation class is added after the abstract interface. Impl, so we look for SharedPreferencesImpl. Of course, you still have a way to find it. It is Baidu. Then look at the commit method of SharedPreferencesImpl
@Override
public boolean commit() {<!-- -->
?…
MemoryCommitResult mcr = commitToMemory();
SharedPreferencesImpl.this.enqueueDiskWrite(
mcr, null /* sync write on this thread okay */);
try {<!-- -->
mcr.writtenToDiskLatch.await();
} catch (InterruptedException e) {<!-- -->
return false;
} finally {<!-- -->
if (DEBUG) {<!-- -->
Log.d(TAG, mFile.getName() + ":" + mcr.memoryStateGeneration
+ " committed after " + (System. currentTimeMillis() - startTime)
+ "ms");
}
}
notifyListeners(mcr);
return mcr.writeToDiskResult;
}
The commitToMemory just packs the data into MemoryCommitResult, and then gives the enqueueDiskWrite method
private void enqueueDiskWrite(final MemoryCommitResult mcr,
final Runnable postWriteRunnable) {<!-- -->
?…
final Runnable writeToDiskRunnable = new Runnable() {<!-- -->
@Override
public void run() {<!-- -->
synchronized (mWritingToDiskLock) {<!-- -->
writeToFile(mcr, isFromSyncCommit);
}
?…
}
};
?…
QueuedWork.queue(writeToDiskRunnable, !isFromSyncCommit);
}
QueuedWork.queue is put into the queue for operation, so I won’t talk about this, let’s look at writeToFile (it’s quite long, I intercepted the middle part)
try {
FileOutputStream str = createFileOutputStream(mFile);
if (DEBUG) {
outputStreamCreateTime = System. currentTimeMillis();
}
if (str == null) {
mcr.setDiskWriteResult(false, false);
return;
}
XmlUtils.writeMapXml(mcr.mapToWriteToDisk, str);
writeTime = System. currentTimeMillis();
FileUtils. sync(str);
fsyncTime = System. currentTimeMillis();
str. close();
ContextImpl.setFilePermissionsFromMode(mFile.getPath(), mMode, 0);
if (DEBUG) {
setPermTime = System. currentTimeMillis();
}
try {
final StructStat stat = Os.stat(mFile.getPath());
synchronized(mLock) {
mStatTimestamp = stat.st_mtim;
mStatSize = stat.st_size;
}
} catch (ErrnoException e) {
// Do nothing
}
if (DEBUG) {
fstatTime = System. currentTimeMillis();
}
// Writing was successful, delete the backup file if there is one.
mBackupFile.delete();
if (DEBUG) {
deleteTime = System. currentTimeMillis();
}
mDiskStateGeneration = mcr. memoryStateGeneration;
mcr.setDiskWriteResult(true, true);
if (DEBUG) {
Log.d(TAG, "write: " + (existsTime - startTime) + "/"
+ (backupExistsTime - startTime) + "/"
+ (outputStreamCreateTime - startTime) + "/"
+ (writeTime - startTime) + "/"
+ (fsyncTime - startTime) + "/"
+ (setPermTime - startTime) + "/"
+ (fstatTime - startTime) + "/"
+ (deleteTime - startTime));
}
long fsyncDuration = fsyncTime - writeTime;
mSyncTimes.add((int) fsyncDuration);
mNumSync++;
if (DEBUG || mNumSync % 1024 == 0 || fsyncDuration > MAX_FSYNC_DURATION_MILLIS) {
mSyncTimes.log(TAG, "Time required to fsync " + mFile + ": ");
}
return;
} catch (XmlPullParserException e) {
Log.w(TAG, "writeToFile: Got exception:", e);
} catch (IOException e) {
Log.w(TAG, "writeToFile: Got exception:", e);
}
In fact, it is obvious at first glance that it is directly written to the file with FileOutputStream, and then XmlUtils writes the file into xml. In fact, SharedPreferences stores data in xml format. I believe everyone understands it. I just go through the process simply through the code.
You can see the difference between SharedPreferences and MMKV. SP uses FileOutputStream to write data into the book, while MMKV uses memory mapping. MMKV is obviously faster. In terms of the format of stored data, SP uses xml format, and MMKV uses protobuf, obviously MMKV will be smaller.
Although SharedPreferences is convenient to call, it also has some disadvantages. In a more multi-process environment, such as using apply in some fast read and write environments. Does that mean I have to use MMKV instead of SharedPreferences? In fact, it is not. Your function does not involve a multi-process environment, and does not involve frequent reading and writing of large amounts of data. For example, only one state is stored in storage, or I only read and write data with a small amount of data once in a while, so use it directly. SharedPreferences will not be a problem either. There’s no need to go to war, but to kill a chicken with a sledgehammer?
3. Data Migration
This is the point I want to talk about. What is data migration, and what does it have to do with SharedPreferences and MMKV? Data migration is a way to solve problems. It has nothing to do with SP and MMKV, but I use them to An example will be better explained.
Although MMKV is easy to use, if you have a scenario where SP really can’t support your business, use MMKV instead, but your old version still uses SP to store data, and directly overwrites and upgrades without deleting the disk data, then you have to migrate the data in xml format stored in the SP to MMKV, which is a process of local data migration.
If you migrate from SP to MMKV, it should be quite simple. I believe that there is a corresponding method in MMKV for you. I think Tencent developed it, and it will definitely take this into consideration. If not, you can write the migration logic yourself. Disaster. Moreover, SP is a component provided by android natively, so operations such as deleting components will not be involved. But if, I said if, byte also has a key-value component, such as ByteKV, if it does not use protobuf, it is another format that can compress data to a smaller size. At this time, you use MMKV, and you want to replace it with ByteKV, how do you do it.
Some people said that if there is such a situation, they will also consider compatibility with other components. If not, then manually write the migration logic, which is not complicated. The logic of handwriting migration is not complicated, but have you ever thought about a problem, You need to delete the previous library, for example, if you used to rely on MMKV, after you change to ByteKV, you no longer need to rely on MMKV , otherwise you will re-depend on every time you change to a new library, and you will not delete the old dependencies.
For example, your version 1.0 relies on MMKV, and version 2.0 uses ByteKV. Do you still rely on MMKV while relying on ByteKV? SP does not have this problem because it is native code.
I have thought of a way for you. If the 1.0 version depends on MMKV, my 2.0 version is a transitional version that relies on ByteKV and MMKV, and I will remove the MMKV dependency in 3.0, okay? Of course not, then some users directly upgrade from 1.0 to 3.0, so the data that has not been migrated will disappear
How to deal with this is actually simple.You know which file MMKV saves data in locally. You also know that it uses protobuf to store data. Then this matter is over, you know Where and how the file is stored, then you can read the content, which has nothing to do with the process of saving. So you don’t need MMKV at all when you read the content of this file. You only need to judge that there is this file in this folder, and this file is in a certain format, and then manually migrate it. File deleted. If you don’t know where the framework you use will store the data and in what format, it’s easy, just look at its source code.
Here we take MMKV as an example, and the database is the same. It doesn’t matter if you change a different database framework. You know where it exists and how it is stored. Then you can extract the data without using the corresponding library.
This is actually the principle of data migration. I don’t care what inventory you use. What your library does is just optimize the storage process and determine the data format.
Another point to note is that data is not migrated at one time, but in parts. You migrate a part first, and then delete the data in the old file.
Summary
In fact, this article mainly wants to briefly introduce the difference between SP and MMKV, understand why MMKV was designed, and think from the perspective of a developer. If it were you, how can you design such a A set of ideas.
The second is the issue of local data migration. If we look at the essence through the phenomenon, we usually use a lot of libraries written by others. Why do we use them? Because others have written well, and I can’t design them like them. So good, so use them too. But I also need to know the principles of this and how they are implemented.
This article is participating in the “Golden Stone Project”