Z's Blog

Welcome to my blog. This is a technical site focused on game development. My open-source projects on GitHub: github.com/hxhb and Open-Source projects.
The articles in the blog include a C++ series, Hot Update for UnrealEngine, Plugins and tools development, Game Resource Management series, and thoughts series, etc.

In my spare time, I research and develop technical solutions and preview the results: Showcase. If you are interested in any of the projects, feel free to email me at imzlp@foxmail.com.

UE as Lib

Dictionary-based Shader Compression

Remake PAK at Runtime

Multi-stage Resource Inspection

Splitting Game Resource

Human Pose Tracking in UE

In addition to this site, I also created a community-driven Chinese knowledge base site for Unreal Engine https://ue5wiki.com/, hosted on GitHub Pages. Contributions of technical content are welcome to help build the Chinese Unreal Engine technology community! Please see the submission rules: Wiki Content Writing Format and Rules .

For game projects, we always hope players can experience the latest games at the lowest cost. This includes CDN costs during operation, as well as players’ data and time costs during download. Smaller, faster, and more economical is an eternal pursuit.

So, when a game’s installation package is updated (the so-called major version), does the player need to download all resources completely again?
Large mobile games in the current market have a huge resource footprint, reaching over 10GB. If players download the full package every time an update occurs, it’s a poor choice for both operational costs and user experience.
When switching versions, players actually have the complete resources from the previous version locally. If these can be utilized, the portion that needs to be downloaded can be significantly reduced. Therefore, a mechanism capable of reusing old version resources is needed!

This article will address this pain point by sharing a resource reuse solution for major version updates, minimizing the resources players need to download, and making it easy to publish and maintain.
It has achieved excellent results in actual online project operations.

I have an old laptop from 10 years ago that has been idle for several years. Recently, I tidied it up and repurposed it, deploying a Proxmox environment on the bare metal to turn it into a simple home Homelab server.
It runs Ubuntu and FnOS virtual machines, and with intranet penetration, it can be used as a long-term online VPS or to deploy a NAS system for backing up files and photos.

This article will introduce the complete deployment solution for PVE + Virtual Machines + FRP intranet penetration + Nginx reverse proxy, which can securely expose local area network services for public domain access.

随着游戏规模和资产精度的提升，以及Nanite等技术的应用，项目工程规模急剧膨胀，达到数百G乃至上T的量级，完整拉取的耗时可能数小时。
而每个人在实际开发中，能够用到的资源只占其中的一小部分，所以怎么样减少工程拉取量级，使工程轻量化是需要优化的问题。
在UE5中，官方推出了虚拟资产(VirtualAssets)机制，与P4结合能够做到这一点。

本篇文章会介绍UE5中虚拟资产的配置流程、资产的虚拟化过程与加载代码分析，以及断网可用性的测试。

In the process of game development, hot update is a very important capability that allows us to update features and fix bugs without replacing the entire package. However, there aren’t many detailed articles currently introducing what can be hot-updated, what cannot, and what carries risks.

In this article, I will introduce my thoughts on hot update capabilities and hot update security during the development of HotPatcher and project hot update practices. This ensures that the hot update capabilities and stability are guaranteed when the project goes online, and that it’s clear what can be hot-updated during the hot update iteration phase, allowing for a good assessment of update risks.
Additionally, by building peripheral capabilities, conducting risk assessments in advance, and automating the hot update process, only the version PM needs to control the patch construction and release timing, with no need for programmers to be involved in the hot update process at all.

When developing plugins, as plugin functionalities become increasingly complex, it’s often necessary to provide some extensibility support to implement custom extended features that enrich the plugin’s capabilities, and also to facilitate users in customizing projects without modifying the core program.
In previous articles (HotPatcher’s Modular Refactoring and Development Planning), the support for HotPatcher and extension modules, and how to develop a new module based on HotPatcher were introduced. This article primarily details the specific implementation methods within UE.

Simply put, it means building one’s own extension system on top of UE’s existing plugin system. This article will introduce some of my thoughts and techniques on improving plugin extensibility, and how to conveniently decouple these extension features from the main plugin body for easier maintenance and management.

Runtime Retrieval of Decryption Key

For the encryption and signing of Pak, it is necessary to acquire the corresponding key at runtime.

As seen in the definitions of UE_REGISTER_ENCRYPTION_KEY and UE_REGISTER_SIGNING_KEY above, there are two extern declarations:

1
2

extern void RegisterEncryptionKeyCallback(void (*)(unsigned char OutKey[32])); 
extern void RegisterSigningKeyCallback(void (*)(TArray<uint8>&, TArray<uint8>&));

These two functions are methods for the engine to obtain the key:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

typedef void(*TSigningKeyFunc)(TArray<uint8>&, TArray<uint8>&);
typedef void(*TEncryptionKeyFunc)(unsigned char[32]);

void RegisterSigningKeyCallback(TSigningKeyFunc InCallback)
{
	FCoreDelegates::GetPakSigningKeysDelegate().BindLambda([InCallback](TArray<uint8>& OutExponent, TArray<uint8>& OutModulus)
	{
		InCallback(OutExponent, OutModulus);
	});
}

void RegisterEncryptionKeyCallback(TEncryptionKeyFunc InCallback)
{
	FCoreDelegates::GetPakEncryptionKeyDelegate().BindLambda([InCallback](uint8 OutKey[32])
	{
		InCallback(OutKey);
	});
}

When these two functions are invoked (that is, from the constructors of FEncryptionKeyRegistration and FSigningKeyRegistration).

Recall the initialization process for the Monolithic module described in a previous article: UE Plugins and Tool Development: Basic Concepts #Monolithic Mode.
They will be created at the time the module starts.

When loading PAKE, FCoreDelegates::GetPakEncryptionKeyDelegate() is called to obtain the keys:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

void FPakPlatformFile::GetPakEncryptionKey(FAES::FAESKey& OutKey, const FGuid& InEncryptionKeyGuid)
{
	OutKey.Reset();

	if (!GetRegisteredEncryptionKeys().GetKey(InEncryptionKeyGuid, OutKey))
	{
		if (!InEncryptionKeyGuid.IsValid() && FCoreDelegates::GetPakEncryptionKeyDelegate().IsBound())
		{
			FCoreDelegates::GetPakEncryptionKeyDelegate().Execute(OutKey.Key);
		}
		else
		{
			UE_LOG(LogPakFile, Fatal, TEXT("Failed to find requested encryption key %s"), *InEncryptionKeyGuid.ToString());
		}
	}
}

This key is then used to decrypt data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

void DecryptData(uint8* InData, uint32 InDataSize, FGuid InEncryptionKeyGuid)
{
   if (FPakPlatformFile::GetPakCustomEncryptionDelegate().IsBound())
   {
      FPakPlatformFile::GetPakCustomEncryptionDelegate().Execute(InData, InDataSize, InEncryptionKeyGuid);
   }
   else
   {
      SCOPE_SECONDS_ACCUMULATOR(STAT_PakCache_DecryptTime);
      FAES::FAESKey Key;
      FPakPlatformFile::GetPakEncryptionKey(Key, InEncryptionKeyGuid);
      check(Key.IsValid());
      FAES::DecryptData(InData, InDataSize, Key);
   }
}

This completes the whole process from compile-time encryption to runtime decryption. ## Serialization Structure of Pak

In a previous article, it was mentioned that UE’s PAK is a typical archive structure:

Generated through specific serialization rules, and reading at runtime also follows formatting rules:

1. When mounting the PAK, first read the PakInfo from the end of the file, then retrieve the index’s offset.
2. Obtain the EncodePakEntrys based on the index offset, which contains the PakEntry information for each file.
3. Each PakEntry records the current file’s information to guide how to read and decompress.
   It is also an important part of UE’s own virtual file system.

However, since the UE source code is open, the serialization structure of the PAK is not confidential. If the encryption key is acquired through reverse engineering, the files within the PAK can be decrypted directly using the logic of the official UnrealPak engine.

Additionally, some third-party tools, like UModel or UnrealPakViewer, can also achieve this. Therefore, aside from key and decryption obfuscation, it is also necessary to modify the serialization structure of the PAK itself.

Hardening Solutions

Disadvantages of the Default Solution

Based on the previous sections, the following conclusions can be drawn:

1. The decryption key is directly compiled into code as a macro definition.
2. Through engine source code + static analysis, it’s relatively easy to locate the function that retrieves the key.
3. Exported symbols in the code can directly expose the location of decryption functions.
4. PAK strings in the code (logs/ProfilingTag) can assist in locating key decryption functions.
5. AES encryption is used by default, with obvious code characteristics that are easy to analyze.
6. The serialization format of the PAK is quite generic, and given the key, it can be decrypted by the official UnrealPak or tools like UModel/UnrealPakViewer.

Therefore, based on these situations, we need to implement various modifications to the encryption scheme, increasing the cost of decryption as much as possible. However, due to security requirements, I can only introduce some encryption ideas without providing specific code.

Key Hardening

As mentioned in the first section, the key is generated by Base64 encoding into a macro definition, and retrieved at runtime. We can implement some unique key obfuscation logic to add a layer of processing to the key itself, preventing the encoded key from being easily obtained through static analysis.

Symbol Stripping

By default, if symbols are not stripped, using reverse engineering tools like IDA to open the executable can directly reveal function names:

This makes it easy to see the decryption function, and therefore obtain the key. Therefore, all distribution packages need to have symbols stripped.
After stripping, the symbol situation is:

Refer to the content of my previous article: Ultimate Optimization of UE Android APK Size

PAK String Stripping

For the same reasons, the engine logs extensively during the mounting of PAK. Analyzing the accessed locations of these log strings can assist in locating the decryption logic:

After finding the decryption function, attaching a debugger can reveal the actual key stored in memory.

Therefore, stripping PAK-related strings is crucial.

Log Stripping

The removal strategy should retain logs in Dev but strip them in Shipping:

1
2
3
4
5
6
7
8
9
10

//++[lipengzha] Remove PAK related strings in shipping
#ifndef DISABLE_PAK_LOG_IN_SHIPPING
	#define DISABLE_PAK_LOG_IN_SHIPPING 0
#endif
#if UE_BUILD_SHIPPING && DISABLE_PAK_LOG_IN_SHIPPING
	#define UE_PAK_LOG(...)
#else
	#define UE_PAK_LOG(LogCategory,LogLevel,Format, ...) UE_LOG(LogCategory,LogLevel,Format,##__VA_ARGS__)
#endif
//--

Thus, the UE_LOG macro can be wrapped such that it is forwarded in Dev but replaced with an empty macro in Shipping.

Boot Timing Stripping

The SCOPED_BOOT_TIMING introduced strings also need to be stripped:

1
2
3
4
5
6
7
8
9
10
11
12

//++[lipengzha] Remove PAK related strings in shipping
#ifndef REMOVE_PAK_TEXT_IN_SHIPPING
	#define REMOVE_PAK_TEXT_IN_SHIPPING 0
#endif
#if UE_BUILD_SHIPPING && REMOVE_PAK_TEXT_IN_SHIPPING
	#define UE_PAK_LOG(...)
	#define UE_PAK_SCOPED_BOOT_TIMING(...)
#else
	#define UE_PAK_LOG(LogCategory,LogLevel,Format, ...) UE_LOG(LogCategory,LogLevel,Format,##__VA_ARGS__)
	#define UE_PAK_SCOPED_BOOT_TIMING(x) SCOPED_BOOT_TIMING(x)
#endif
//--

Then replace all UE_LOG with UE_PAK_LOG, and all SCOPED_BOOT_TIMING with UE_PAK_SCOPED_BOOT_TIMING in both IPlatformFilePak.cpp and IPlatformFilePak.h.

This will ensure that these strings are automatically stripped in Shipping.

PAK Structure Obfuscation

In the previous section (Serialization Structure of Pak), I introduced the serialization method of the PAK structure itself. Thus, for commercial projects, obfuscating the structure itself is also necessary.

Improvements can be made to the serialization of the PAK based on the existing UFS file system, modifying the organization of the structure and encrypting or altering data to prevent the public version engine and open-source tools from being able to decrypt after obtaining the key. This can also ensure the uniqueness of each PAK structure and decryption, avoiding a scenario where one being compromised leads to a mass breach.

This content won’t be elaborated on further, as each project needs to implement it uniquely.

Decompression Algorithm Obfuscation

Because files are usually compressed with a compression algorithm after being packaged into the PAK, they need to be decompressed at runtime. The selected compression algorithm for each file is recorded in the PakInfo and noted in each PakEntry’s index.

Therefore, some processing can also be implemented in this area. Even if the key and PAK structure are reverse engineered, additional obfuscation can be added when decompressing actual files:

Replacing AES Algorithm

As AES is standard in UE, it can be located by feature codes during reverse engineering. Thus, AES can be replaced or the code logic can be modified. The specific algorithm to replace it with depends on the project’s situation.

Dynamic Key Registration

According to the previous analysis, if all assets use a single default key, it is directly compiled into the code. If that key is compromised, all resources are at risk of exposure. Local decryption indeed has low security.

In such cases, for critical assets or files (such as unreleased commercial content), sometimes assets are pre-embedded in the client before their release time. This situation presents a significant risk if the same key is shared.

The engine itself provides logic for dynamically registering decryption keys:

1
2
3
4
5
6
7

class CORE_API FCoreDelegates  
{  
public:
	// Callback for registering a new encryption key  
	DECLARE_DELEGATE_TwoParams(FRegisterEncryptionKeyDelegate, const FGuid&, const FAES::FAESKey&);
	// ...
};

Thus, a dynamic key mechanism can be implemented in the update process. The encrypted PAK does not contain a decryption key in the program itself; only after the designated version releases the key can the client use it, ensuring no premature unpacking and leakage occurs. This can be combined according to the project’s needs.

Conclusion

This article introduces the default encryption logic of UE, analyzes potential risks, and provides some feasible hardening strategies for reference.

Note: The article analyzes the engine’s encryption process and reverse engineering techniques solely to explore safer methods, not as a crack tutorial. Any behavior related to game reverse engineering is not related to the author. Thank you!

When developing UE plugins, we often provide a large number of configurable parameters for flexible control, used to manage the specific execution logic and behavior of the plugins.

This article is the eighth in my UE Plugin Development Series, and will introduce my thoughts and implementations regarding the configurability of plugins during the development process, along with practices in project configuration, task configuration, dynamic parameter replacement, etc., to make the plugin configuration process as flexible and easy to use as possible.

In game projects, when we package for various platforms, we always hope that the package for each platform can be minimized for easier distribution, and there are specific size requirements for some platforms.

For UE, it contains massive code and numerous plugins, and during the Build phase, it generates a lot of reflection glue code, resulting in a significant increase in code segments during compilation. Taking the Android platform as an example, this leads to a sharp increase in the size of libUE4.so, putting pressure on both the package size and runtime memory.

Moreover, some necessary and additional resources brought in by the engine can take up hundreds of MB, making the size of an empty APK easily reach several hundred MB! Not only to meet the platform’s requirements, but it is also necessary to trim down the size of the UE package from the perspective of package size and memory optimization.

This article will take Android as an example and introduce optimization ideas and practices for the cuttable parts in the UE package from various aspects, optimizing both the APK size and the runtime memory usage of native libraries. The strategies can also be reused on other platforms.

Recently, I encountered an extremely bizarre bug involving two maps. One map, A, can be accessed with its PAK placed in the engine’s automatic mount directory, but it cannot be accessed from the hot update directory. The other map, B, behaves completely oppositely: it is abnormal in the automatic mount directory but works normally in the hot update directory.

At first glance, the issue appears entirely elusive, with two mutually exclusive behaviors occurring within the same logical framework. Moreover, the hot update mount and the automatic mount only differ in timing and priority, so this problem shouldn’t theoretically exist.

While the issue can ultimately be resolved on the business logic side, this behavior involves another very obscure path within the engine. Understanding why and how it works is crucial. Therefore, I analyzed the engine’s code based on this behavior, came to a reasonable conclusion, and devised a method to detect and mitigate this issue.

This article assumes that readers have some basic knowledge of UE hot updates; if in doubt, please refer to other articles in this blog’s hot update series for more information.