Z's Blog

DevOps is a philosophy based on automation for Continuous Integration (CI) and Continuous Deployment (CD), optimizing all aspects of development, testing, and operations. It emphasizes the integration of software development, testing, and operations, reducing communication costs between departments or processes to achieve rapid and high-quality releases and iterations.

Although the development situation in the gaming field differs significantly from traditional internet development, leveraging the concept of DevOps to integrate all production elements of a project into automation can greatly enhance efficiency.

This article will introduce the basic concepts of automation in UE and share some of my attempts and actual engineering practices regarding automation builds in Unreal Engine.

Recently, there is often a strange sense of anxiety. After a period of self-observation and reflection, I believe it is necessary to confront these issues and attempt to provide answers.

This article is a thought experiment that examines and analyzes the current state of life and its problems, summarizing the ideological tools for guiding life and work. As a personal methodology, it contains a large amount of subjective content.

In game project development, a large number of developers from various aspects such as development, art, planning, and outsourcing will be involved. For reasons of resource security and information confidentiality, complex permission controls and synchronization logic are usually implemented.

However, leaving aside permission control and focusing on the resources themselves, in UE, resources are upward compatible, meaning resources can be opened directly with the same version engine or a higher version engine. This means that distributed resources can be used directly in other projects. Therefore, ensuring the security of resources during the development phase is a key focus.

This article provides an idea for encrypting the original UASSET resources in the project and introduces the basic principles that can be used for resource encryption. However, publicly disclosing the specific encryption implementation would be akin to running naked, so this article will only provide an analysis of the uasset resource structure and ideas for implementing encryption, but will not provide specific implementation code.

When using the Unreal Engine development tool, a significant number of tasks involve resource processing and data export needs. These tasks need to be performed frequently and automatically, often integrated into a CI/CD system.

In the specific implementation, the Commandlet mechanism of UE is utilized to drive the engine in a command-line manner, performing custom behaviors.

Taking the Commandlet features supported in the plugin I developed as an example:

1. HotPatcher: Export basic package information, pack patches
2. ResScannerUE: Incremental scanning of changed resources
3. HotChunker: Standalone packaging of Chunk
4. libZSTD: Training Shader dictionaries
5. ExportNavMesh: Export NavMesh data

Commandlet allows for easier integration into CI/CD to achieve automation.

In this article, I will primarily introduce the Commandlet mechanism of UE, analyze its implementation principles, and provide some development tips and insights from my development process.

Additionally, this is the second article in my UE Plugin and Tool Development series, which will continue to be updated, so stay tuned.

I previously wrote an article introducing UE’s default unpacking method (UE Hot Update: Splitting the Base Package), but the default unpacking method in UE is not flexible enough for large-scale resource projects and cannot meet the refined management needs of super-large-scale resource projects.

To address the issues with UE’s default packaging and resource splitting pain points, I developed an extension based on HotPatcherCore that can solve the drawbacks of the default unpacking process, making it sufficiently flexible and powerful. There’s an article on the blog that introduces its overall implementation mechanism: A Flexible and Non-Intrusive Base Package Splitting Solution.

HotChunker can be integrated into UE’s default packaging process non-intrusively, requiring no manual handling. As long as the default UE packaging is done, it will automatically trigger HotChunker’s unpacking process, achieving resource cooking, packaging, and automatic copying to the StagedBuilds directory.

This article will focus on how to configure resource splitting using HotChunker, automate builds, minimize resource redundancy, and implement parallel packaging.

Memory optimization is a frequent focus in game development. To avoid excessive memory allocation that triggers an OOM (Out of Memory) to be forcibly terminated by the system, common optimization methods typically start from the usage perspective, enhancing memory utilization efficiency by trimming unnecessary features and controlling resource loading.

However, there is another approach to increase the threshold for triggering OOM, allowing the system to permit our app to allocate more memory. In the new iOS versions, Apple has introduced new memory features for apps, allowing them to extend addressable space and increase allocatable memory.

In this article, I will explore how to leverage these features in UE to enhance the total allocatable memory for games on the iOS platform.

During the process of project development using Unreal Engine, various types of plugins are often developed and integrated to extend the engine, thereby fulfilling different requirements. For developers, understanding the operational mechanism is more important than using the tools. Thus, it is necessary to understand the principles behind the integration and development of plugins.

Previously, I also developed some tools and plugins for UE, and I hope to write a series of related articles summarizing the common technical content related to UE plugin and tool development and sharing some of my thoughts and functional scaffolding during plugin development. The goal is to achieve the desired functionality with minimal code, minimal invasiveness, and the best implementation strategies.

With the development of HotPatcher and my research on resource management technology, more and more features are being implemented around HotPatcher, and the implementation is becoming increasingly complex. Coupled with continuous updates of the engine, maintaining compatibility for a large and comprehensive plugin has become more difficult.

With the arrival of the new generation engine UE5, HotPatcher will also follow up with support for UE5 and develop new features based on UE5’s characteristics. I hope it will not only be a resource packaging tool but also include a comprehensive resource management solution encompassing resource management, auditing, packaging, hot updates, package optimization, and build enhancements, and serve as an open resource processing framework.

This article will introduce the modular progress of HotPatcher, support for existing features, and how to use the powerful packaging capability of HotPatcher to customize modular extensions based on project needs. Finally, I will discuss the future development plans for the HotPatcher project.

The Unreal Open Day 2022 was hosted by Epic in the form of an offline event combined with online live streaming. I am very happy to have attended the offline Unreal Open Day event this year. I recorded an online technical presentation and also participated in the live event in Shanghai. I am honored to have once again received the Outstanding Community Contributor award granted by Epic, which is both recognition and motivation for me. I look forward to the UE tech community continuing to grow and becoming the most vibrant game developer community. This article serves as a simple record, organizing the materials from UOD 2022, summarizing my participation in UOD, my presentation PPT, and related materials, as well as some photos taken at the UOD live event.

UE’s default resource management is relatively complex. By default, the resource packaging process is executed based on the configurations of maps, directories, and PrimaryAsset specified in ProjectSetting, as well as a combination of condition checks. Furthermore, UE’s Cook dynamically adds resources to the package based on runtime loading, making the resource packaging process almost a black box.

The blog introduces articles on default resource packaging rules and basic package splitting:

• UE Resource Management: Engine Packaging Resource Analysis
• UE Hot Update: Splitting Basic Packages

This article presents a new idea, utilizing HotPatcher‘s precise Cook and packaging mechanism to implement a HotChunker Mod that can perform non-intrusive, direct reuse of UE’s default packaging process and clearly split the basic package. This article will specifically introduce the usage and implementation principles.