Unity AssetBundle packaging

1, Concept and function of AssetBundle

AssetBundle is an archive file, a resource compression package provided by Unity for storing resources, which can include models, textures, audio, prefabs, etc.
The AssetBundle system in Unity is an extension of resource management. By distributing resources in different AB packages, it can minimize the memory pressure at runtime. It can dynamically load and unload AB packages, and then selectively load content. .
The most important thing is that AssetBundle can be used for hot updates and is Unity’s main tool for updating non-code content.

2, composed of AssetBundle

AssetBundle mainly consists of two parts: file header and data segment

The file header contains the id, compression type, and index list, which is the same lookup table as Resources that records the byte offsets in the serialized file. This table is a balanced search tree for most platforms, and a red-black tree for Windows and OSX series (including iOS). As the number of objects in the AssetBundle increases, the time required to construct the list will grow faster than the linear growth rate.
The data segment contains the serialized original data of the Asset. You can also choose whether to compress the data. If LZMA compression is used, the byte arrays of all Assets will be compressed as a whole; if LZ4 compression is used, each Asset will be compressed individually; if not Compression, the data remains the original byte stream

3, compression mode

  • AssetBundle provides three compression formats
  • NoCompression: No compression, fast decompression, large package size, not recommended.
  • LZMA: has the smallest compression and slow decompression. To use one resource, you need to decompress all resources in the package.
  • LZ4: The compression is slightly larger, the decompression is faster, you can use whatever you use to decompress, and the memory usage is low. This is generally recommended.

4. Construction of AssetBundle

4.1 Two construction methods:

BuildPipeline.BuildAssetBundles(string outputPath, AssetBundleBuild[] builds, BuildAssetBundleOptions assetBundleOptions, BuildTarget targetPlatform);
BuildPipeline.BuildAssetBundles(string outputPath, BuildAssetBundleOptions assetBundleOptions, BuildTarget targetPlatform);

In the first method, you need to manually collect all AssetBundleBuilds when packaging.
The second method is to generate in advance or manually fill in all the AssetBundleBuild that needs to be packaged.

The second method is recommended. You can see all the ABs to be packaged in the Unity editor, which is more intuitive.

4.2 Subcontracting strategy

  • Group objects that need to be loaded at the same time into a group, such as a model, and its required materials and animations into a group
  • If multiple objects in multiple AssetBundles reference a single Asset in other AssetBundles, separate the dependencies into separate packages to reduce duplication
  • Make sure that two sets of objects that are completely impossible to load at the same time are not in the same pack, such as low-definition and high-definition texture packs
  • If less than half of the objects in a package are loaded frequently, they can be split
  • Merge some small packages (less than 5 to 10 resources) that are loaded at the same time
How big is a single ab bag?

Generally, it should be controlled below 10MB, and it is best not to exceed 20MB. Compared with the total size, it is particularly important to control the number of resources of a single ab. If most ABs in the project only contain a single resource, it will be too scattered and too many ABs need to be loaded, resulting in increased IO pressure; if a single AB contains too many resources, first of all, it will easily increase the amount of downloaded resources during hot updates, and it will also easily cause each time Loading has most of the unused resources, and leads to an increase in the complexity of the ab file header, and a relative increase in interpretation time. Therefore, it is necessary to make a detailed analysis based on the project situation, and balance is the best. According to experience, it is generally appropriate for a single ab to contain about 10 resources.

A recommended subcontracting strategy:

Each folder is an ab package, and the ab name is based on the folder path. Under such rules, the project organization can also be classified by folders, which is more clear. You can also add some flexible rules, such as filtering some file suffixes or folders that do not need to be packaged. Each file in a folder is a separate AB.

4.3 AssetBundle parameters (BuildAssetBundleOptions)

When we call Unity’s API BuildPipeline.BuildAssetBundles to build AssetBundle, there are actually many parameters for us to choose from. If the appropriate parameters are not selected, it may cause a lot of waste in terms of package body, memory and loading time.
In fact, there are a few we often use:

  • ChunkBasedCompression: This parameter is used to compress AssetBundle. As mentioned earlier, Android’s StreamingAssets are not compressed. In order to reduce the package size, you can use this parameter to compress the AssetBundle. It is actually a LZ4 modified by Unity, making its algorithm more consistent with the way Unity is used.
  • DisableWriteTypetree: This is actually a parameter that will be ignored by many developers. It is very useful and can help us reduce the size of the AssetBundle package. It can also reduce the memory and reduce the CPU time when we load this AssetBundle.
  • DisableLoadAssetByFileName, DisableLoadAssetByFileNameWithExtension: When we load an AssetBundle and then use LoadAsset to load the Asset, we need to pass the path name of the Asset.

4.4 Automatic build process

  • Pre-build checks, such as checking various resource formats, to see if they comply with the agreement
  • lua script packaging
  • Automatically generate AssetBundle name
  • Build and generate AssetBundle package
  • AssetBundle package encryption
  • Generate AssetBundle md5
  • Delete old ab resources
  • Copy the first package ab resource to StreamingAssets
  • Generate various platform packages, exe, apk, ios projects, etc.

5. AssetBundle loading

5.1 Loading method

We can use four different methods to load an AssetBundle.

  • AssetBundle.LoadFromMemory and AssetBundle.LoadFromMemoryAsync
  • AssetBundle.LoadFromFile and AssetBundle.LoadFromFileAsync (recommended)
  • WWW.LoadfromCacheOrDownload(5.6 and previous versions)
  • UnityWebRequestAssetBundle and DownloadHandlerAssetBundle (WebGL or network loading required)

AssetBundle.LoadFromMemory
Create an AssetBundle from the memory area and load the AB package completely through byte[]. Generally used for applications that require high encryption, or WebGL. Disadvantages: high memory usage, taking up two copies of memory.

AssetBundle.LoadFromFile
This method can efficiently load uncompressed or LZ4 compressed Assetbundle from the hard disk. When loading the LZMA compressed package, it will be decompressed first and then loaded into the memory. When loading LZ4, it will only load the header of the AB package. What resources are needed later will load that part of the AB. Package chunk. From this we can see the advantages of using LZ4 to build AssetBundle.

public class LoadFromFileExample : MonoBehaviour {
    function Start() {
        var myLoadedAssetBundle
            = AssetBundle.LoadFromFile(Path.Combine(Application.streamingAssetsPath, "myassetBundle"));
        
        if (myLoadedAssetBundle == null) {
            Debug.Log("Failed to load AssetBundle!");
            return;
        }
        var prefab = myLoadedAssetBundle.LoadAsset.<GameObject>("MyObject");
        Instantiate(prefab);
    }
}

5.2 AssetBundle Asset loading

After the AssetBundle is loaded, it cannot be used directly. You still need to load the required Asset from the ab.

  • AssetBundle.LoadAsset (LoadAssetAsync)
  • AssetBundle.LoadAllAssets (LoadAllAssetsAsync)

LoadAsset loads the specified Asset from the ab package and returns Object
LoadAllAssets loads all Assets from the ab package and returns Object[], which can be used when loading a Sprite atlas.

5.3 Synchronous and asynchronous loading

Whether it is AssetBundle loading or Asset loading in AssetBundle, Unity provides synchronous and asynchronous loading methods, so how to choose?
In fact, this is just a matter of strategy, and there is no better one. The biggest advantage of synchronization is that it is fast, because in this frame all the CPU of the main thread will be used by you, and all the time slices will be used by you. It can finish this thing wholeheartedly and then do other things. But the problem with synchronization is that it will cause the main thread to freeze. Asynchronous can be simply understood as multi-threading (actually there is a slight difference). The biggest advantage is that it does not cause the main thread to get stuck (it does not mean that it does not get stuck at all), and the main thread can run as smoothly as possible without getting stuck.
In other words, synchronization can be used when lag is not sensitive, and asynchronous can be used for lag-sensitive scenes such as combat scenes.
Recommended practices:

  • General UI interfaces, UI textures, and sound effect resources can all be loaded synchronously, which usually does not cause lag and is also convenient for UI logic processing.
  • Special effects, character models, scenes, and background music resources are loaded asynchronously
  • Avoid synchronization and asynchronous execution at the same time. For example, there are multiple Assets in an ab. When loading one Asset asynchronously and loading another Asset synchronously at the same time, it is best not to put the assets loaded synchronously and asynchronously in the same ab package. .

5.4 Editor loading resources and Resources loading

AssetBundle is generally not used to load resources when running under Unity Editor. On the one hand, AssetBundle needs to be packaged in advance, and on the other hand, most shaders have display problems.
Then under the Editor, you generally use AssetDatabase.LoadMainAssetAtPath or AssetDatabase.LoadAllAssetsAtPath, which corresponds to AssetBundle.LoadAsset and AssetBundle.LoadAllAssets.
Loading AssetDatabase requires knowing the resource path, which can be obtained through AssetDatabase.GetAssetPathsFromAssetBundleAndAssetName
AssetDatabase only loads synchronously, so for the asynchronous loading of AssetBundle, it is best to simulate the asynchronous loading of AssetDatabase under the Editor, for example, you can wait for a few frames.

Resourcesload
Resources.Load can only load resources from the Resources folder, which must be included in the first package and cannot be hot-changed. The most important thing is that it is automatically loaded at startup, which increases the startup time. Therefore, the shortcomings of Resources.Load are very obvious, and it can generally be used to load resources that must be used in the first package, such as configuring resources.

6, AssetBundle dependency

A very useful feature of using AssetBundle is the dependency of AssetBundle. When there is a resource that needs to be reused in Unity, the resource can be generated into a reused ab package, so that other ab packages that reference the resource will be automatically associated when the AssetBundle is built. Rely on it.

So what impact does AssetBundle dependency have on loading?
When an ab package has other dependent packages, if only the ab package is loaded, the instantiated object will lose resources. Therefore, before ab loads Asset, all its dependencies ab must be loaded. Note that all dependencies must be loaded recursively.
AssetBundleManifest.GetAllDependencies can obtain all dependency levels of AssetBundle, manifest is the main package

7. AssetBundle uninstall

7.1 AssetBundle.Unload(bool)

The parameters are divided into true and false. If it is true, the AssetBundle and the Assets loaded by it will be killed together. This may cause resource loss and pink phenomenon at inappropriate times. If it is false, then only the AssetBundle is thrown away, and the Asset will not be thrown away. Then when you load the same AssetBundle for the second time, there will be two copies of the Asset in the memory, because when the AssetBundle is unloaded, its relationship with the corresponding Asset is cut off. Therefore, AssetBundle does not know whether the previous Asset is still in the memory or whether it was loaded from itself, which can easily lead to memory leaks. Therefore, using AssetBundle.Unload is a test of game planning.

It is recommended to use AssetBundle.Unload(true), reason: the program should manage and maintain the AssetBundle by itself, and the AssetBundle should be unloaded only after all the assets that reference the AssetBundle are removed.

7.2 Resources.UnloadUnusedAssets

If the application must use AssetBundle.Unload(false), or after the Asset loaded by Resources.Load is removed, Resources.UnloadUnusedAssets can be used, which can unload the useless Asset and clear it from the memory. Resources.UnloadUnusedAssets may easily cause lag. You need to pay attention to the calling timing. Unity will automatically call UnloadUnusedAssets when switching Scene.

7.3 AssetBundle management and Asset management

To properly handle the loading and unloading of AssetBundle, a complete management system is indispensable. In actual use, the program only cares about the acquisition of Assets. AssetBundle should be completely transparent, so the management of AssetBundle and Asset should be independent.

Recommended practices:

  • The AssetBundle management AssetBundleManager is placed in the C# layer, and the Asset management AssetManager is placed in the Lua layer.
  • AssetBundleManager and AssetManager each have their own set of reference counting algorithms.
  • Obtain Asset: Obtain Asset through AssetManager, Asset reference count + 1; AssetManager obtains Asset from AssetBundleManager, AssetBundleManager calculates the AB to which the Asset belongs, loads the corresponding AB, and then loads the corresponding Asset through AB, AB reference count + 1
  • Release Asset: Release the Asset through AssetManager, and the Asset reference count is -1; when the Asset reference count is 0, you can notify the AssetBundleManager at the appropriate time to release the Asset. AssetBundleManager calculates the AB to which the Asset belongs, and the AB reference count is -1. When the AB reference count is 0 , AB can be uninstalled when appropriate.
  • Assets usually only need one copy, such as: textures, audios, fonts, text, animations. AssetManager only needs to obtain it from AssetBundleManager once, and use the reference count + 1 each time. It should be noted that each AssetBundle may have multiple Assets and reference counts. should be its sum. In addition, Prefab is a commonly used Asset. It only requires one Asset, but GameObject.Instantiate(asset) needs to be instantiated every time it is used. To avoid frequent instantiation, it is best to use an object pool to manage GameObject.
  • When memory is not tight, Assets can be released uniformly and AssetBundles can be unloaded when switching scenes.

8, AssetBundle encryption

Unity’s AssetBundle is very easy to crack, and original resources, such as AssetStudio tools, can be easily obtained. Therefore, mature projects need to consider the encryption of AssetBundle. A more direct and customizable encryption method is to use AssetBundle.LoadFromMemory(Async)

using UnityEngine;
using UnityEngine.Networking;
using System.Collections;

public class ExampleClass : MonoBehaviour
{
    byte[] MyDecription(byte[] binary)
    {
        byte[] decrypted = new byte[1024];
        return decrypted;
    }

    IEnumerator Start()
    {
        var uwr = UnityWebRequest.Get("http://myserver/myBundle.unity3d");
        yield return uwr.SendWebRequest();
        byte[] decryptedBytes = MyDecription(uwr.downloadHandler.data);
        AssetBundle.LoadFromMemory(decryptedBytes);
    }
}

However, the cost of using AssetBundle.LoadFromMemory(Async) is very high. Not only does it increase the memory, but it also takes time to decrypt. It is generally not recommended.

Recommended practices:
AssetBundle LoadFromFile(string path, uint crc, ulong offset);
The offset parameter can be customized and refers to the offset of the AssetBundle content. As long as the AssetBundle is built,

foreach (var abName in manifest.GetAllAssetBundles())
{
string filePath = outputPath + abName;
int offset = Utility.GetAssetBundlesOffset(abName);
var fileContent = File.ReadAllBytes(filePath);
int filelen = offset + fileContent.Length;
byte[] buffer = new byte[filelen];
fileContent.CopyTo(buffer, offset);

FileStream fs = File.OpenWrite(filePath);
fs.Write(buffer, 0, filelen);
fs.Close();
}

Utility.GetAssetBundlesOffset is a custom offset algorithm. It is calculated based on the ab name. It can also be calculated based on other parameters, such as hashcode.

Load AssetBundle based on offset:

public AssetBundle LoadAssetBundle(string abName)
{
string path = abPath + abName;
int offset = Utility.GetAssetBundlesOffset(abName);
AssetBundle ab = AssetBundle.LoadFromFile(path, 0, (ulong)offset);
return ab;
}

In this way, the encryption effect is achieved. As long as the offset algorithm is not known, it cannot be cracked. Moreover, this approach only makes an offset and basically does not increase performance consumption.