Z's Blog

In game development, program performance is a crucial issue that needs to be emphasized. To cover the largest possible user base, we must consider the operational effects on mid to low-end devices and ensure compatibility with a wide range of hardware specifications. In this context, it is key to analyze and optimize the performance bottlenecks of the game.

Loading more resources into memory at runtime is essentially a trade-off of space for time. Frequent I/O from the disk is quite time-consuming, and pre-loading resources into memory allows for fast retrieval. However, memory resources are also limited and cannot be used without restrictions, especially for some mid to low-end mobile devices where devices with 4GB or even less memory still have a significant market share. Therefore, it is important to avoid wasting memory, as excessive memory usage may lead to termination by the system.

Memory optimization fundamentally seeks to strike a balance between loading efficiency and memory consumption, aiming to use as much available memory as possible while ensuring that more low-end devices run normally without triggering OOM.

I plan to write a few articles related to performance optimization. This article will start with UE memory analysis, introducing commonly used memory analysis tools and methods, as well as organizing the memory optimization techniques that can be applied in UE projects. This part was previously recorded in the form of notes at notes/ue, and subsequent memory-related content will be added to this article.

HotPatcher project has been open source for over a year, undergoing numerous updates and optimizations. It has increasingly been chosen by developers as a hot update solution for their projects. During this time, many have inquired about common issues related to UE4 hot updates, which I noticed frequently repeated, so I decided to organize these common questions to help newcomers quickly troubleshoot the UE4 hot update solution.

This article will continually update the Q&A content related to UE4 hot updates and HotPatcher. If you have any questions, feel free to comment directly on this article. I will periodically consolidate responses. You can also join my UE4 hot update group to discuss encountered issues (QQ group 958363331).

In the previous series of articles on hot updates, the process and packaging details of UE hot updates were introduced. In fact, there are some engineering practices for optimizing hot update patches that I think can be discussed in detail.

This article starts with the generation of Shader patches, aiming to reduce the size of shaders during each hot update. It will analyze some internal implementation details of the engine, address issues related to Shader patches in the engine, and automate the Shader patch process based on HotPatcher.

2021.11.02 UPDATE: HotPatcher now supports collecting all compiled shaders of Cooked resources in the patch and packaging them into a Shader Library, which can replace the Shader Patch mechanism. For more details, see the article: UE Hot Updates: Shader Update Strategy.

I previously wrote two articles analyzing the implementation of reflection in UE, introducing the basic concepts of reflection in UE and some of the C++ features it relies on. This article will begin analyzing the specific process of UE’s reflection implementation.

The C++ standard does not have reflection features, and the reflection used in UE is a mechanism based on marker syntax and UHT scanning to generate auxiliary code. As David Wheeler famously said, “All problems in computer science can be solved by another level of indirection,” UHT does just that: it analyzes the marked code before actual compilation and generates real C++ code, collecting metadata regarding reflection types for runtime use.

UHT generates a large amount of code. To avoid organizational confusion in this article, I will mainly discuss the actual C++ code generated in generated.h after UHT processes reflection markers like GENERATED_BODY and UFUNCTION.

The code generated by UHT is located in generated.h and gen.cpp. The code in generated.h mostly defines some macros added to declared classes through compiler preprocessing to introduce common members, while the code in gen.cpp represents the specific code generated by UHT to describe class reflection information, maintaining separation of declaration and definition.

In the previous article, the rules and implementation of basic package splitting were introduced. After the basic packaging rules are stabilized, the focus in daily development shifts to asset management, package resource auditing, and analyzing asset sizes and redundancy in the project.

This article introduces resource packaging configuration in UE, commonly used asset management methods, and asset auditing tools in engineering practice. It is also a supplement to the previous article UE4 Hot Update: Splitting Basic Packages in the series on resource management for hot updates.

In previous articles, we introduced the implementation mechanism of UE hot updates and the automation process of hot updates. Recently, we plan to continue writing a few articles to discuss the process and rules of resource package management in UE hot updates.

Of course, different types of projects will have different packaging strategies, and resource management does not have a universal best strategy. This article mainly introduces the engineering practice of splitting the base package in the hot update process, involving the modification of the engine to implement a common splitting method for Android/iOS. We hope to provide some useful ideas for projects in different businesses.

In practical engineering, we abandoned the default package splitting scheme provided by UE and implemented a flexible package splitting scheme based on the HotPatcher framework. For details, see the article: Resource Management: Reshaping UE’s Package Splitting Scheme.

HotPatcher is an open-source version management and resource packaging tool for UE4 hot updates that I developed earlier. It allows for convenient differential analysis between versions and pak packaging. Previous articles have visually introduced this based on manual configuration and packaging within the editor. In actual engineering practice, it’s preferable to automate repetitive operations to avoid manual involvement. I added commandlet support to the plugin earlier, and recently, I have fixed some issues and added many optimizations for commandlet. This article discusses the engineering practice of the automated hot update process based on HotPatcher.

Open source is not just about making the code available; it’s about how to use your code to address real-world needs, continuously iterate and update, and build a community for open source projects. Based on my experiences with open source over the past couple of years, I want to share my views on open source from a developer’s perspective and some thoughts on open source behavior.

In the previous article, the basic concepts of UE’s reflection were introduced. This article begins to explore the specific implementation of UE’s reflection mechanism.

Before introducing UE’s code, some C++ features need to be highlighted. Although UE’s reflection implementation relies heavily on UHT’s code generation, it also requires support from C++ syntax features. Only by understanding these features and their underlying meanings can we better grasp UE’s reflection mechanism.

The C++ features and standards described in this article are based on ISO/IEC 14882:2014, which is the C++14 standard.

Reflection refers to the ability of a program to perform self-inspection at runtime, which is very useful in the editor’s property panel, serialization, GC, and other areas. However, C++ does not support reflection features inherently. UE implements the generation of reflection information through UHT based on the syntax of C++, thereby achieving the goal of runtime reflection.

In previous articles, some content related to UE’s build system and reflection was discussed.

Articles related to UE’s build system:

• Build flow of the Unreal Engine4 project
• UE4 Build System: Target and Module
• Differences and Connections Between UEC++ and Standard C++

Articles on using UE’s reflection mechanism for various tricks:

• UE4: Hook UObject

The implementation of UE’s reflection relies on UHT in the build system to perform code generation. This article provides a basic concept introduction to UE’s reflection, and subsequent articles will thoroughly introduce the implementation mechanisms of reflection in UE.