Z's Blog

BuildGraph：构建支持多平台打包的二进制引擎

Posted on Edited on In , Views:
Symbols count in article: 13k Reading time ≈ 32 mins.

Typically, there are two ways for UE4 developers to obtain the UE4 engine:

  1. Install from Epic Games Launcher
  2. Clone the code from Github for local compilation

The version installed from Epic Games Launcher is the standard engine, which cannot be modified or recompiled, but you can choose platforms for installation support, debug symbols, etc. Cloning the code from Github and compiling it results in the source version of the engine, with some functionalities only available in the source version (like Standalone Application). However, if you modify the engine’s code in your project, it requires everyone to clone and compile the source again, which is very time-consuming. Additionally, the source version of the engine occupies a significant amount of disk space, reaching hundreds of gigabytes. When deploying the engine to multiple build machines, the time and space for compiling the engine become redundant. Therefore, it’s necessary to compile the engine on one machine, and the others can simply pull the compiled engine, achieving the same behavior as the installed version.

This article mainly introduces the workflow of BuildGraph and analyzes the engine’s default build script InstalledEngineBuild.xml; how to use BuildGraph to configure, compile, and export a binary engine supporting Android/iOS packaging, and how to cut down and accelerate the overall build process.

UE provides the BuildGraph functionality, allowing us to build an installed version of the engine from source code similar to the one from Epic Games Launcher.

The parameters are as follows:

1
RunUAT.bat BuildGraph -target="Make Installed Build Win64" -script=Engine/Build/InstalledEngineBuild.xml -set:WithMac=false -set:WithAndroid=false -set:WithIOS=false -set:WithTVOS=false -set:WithLinux=false -set:WithHTML5=false -set:WithSwitch=false -set:WithDDC=false -set:WithWin32=false -set:WithLumin=false -set:WithPS4=false -set:WithXboxOne=false -set:WithHoloLens=false -set:GameConfigurations=Development

This executes the Make Installed Build Win64 from the InstalledEngineBuild.xml script, realizing a one-stop process for engine compilation and binary export.

However, there are several issues: the default build can only package for Windows, not for Android/iOS.

How to build a binary engine supporting Android/iOS packaging will be discussed later, first let’s analyze the default InstalledEngineBuild.xml script of UE.

Optimizing the BuildGraph Build Process

InstalledEngineBuild.xml is the BuildGraph build script located in the engine’s Engine/Build/ directory, primarily implementing the functionality to export a binary engine via code, supporting many platforms. The official introduction and syntax of BuildGraph can be found here: BuildGraph.

We mainly use Make Installed Build Win64, which is a node implementation.

The main process is as follows:

  1. Compile UBT and other build tools
  2. Compile NotForLicence tools
  3. Compile Editor
  4. Compile supported platforms (default Win64)
  5. Make Feature Packs
  6. Copy build results

If you want to modify the export path of the built binary engine, you can change the BuiltDirectory value in InstalledEngineBuild.xml.

1
2
3
4
5
6
7
<!-- Ouput directory for the build -->
<Option Name="BuiltDirectory" DefaultValue="$(RootDir)/LocalBuilds/Engine" Description="Directory for outputting the built engine"/>

<!-- The local output directory -->
<Property Name="LocalInstalledDir" Value="$(BuiltDirectory)/Windows"/>
<Property Name="LocalInstalledDirMac" Value="$(BuiltDirectory)/Mac"/>
<Property Name="LocalInstalledDirLinux" Value="$(BuiltDirectory)/Linux"/>

Moreover, upon analyzing Make Installed Build Win64, many of the compilation processes are redundantly executed.

Take compiling UE4Game Win64 as an example:

1
2
3
4
5
6
7
8
<Node Name="Compile UE4Game Win64" Requires="Compile UnrealHeaderTool Win64" Produces="#UE4Game Win64;#UE4Game Win64 Unstripped;#UE4Game Win64 Stripped;#UE4Game Win64 Unsigned;#UE4Game Win64 Signed">
    <ForEach Name="Target" Values="UE4Game;$(OptionalClientTarget);$(OptionalServerTarget)">
        <ForEach Name="Configuration" Values="$(GameConfigurations)">
            <Compile Target="$(Target)" Platform="Win64" Configuration="$(Configuration)" Tag="#UE4Game Win64" Arguments="-precompile -allmodules -nolink $(VSCompilerArg) $(TargetDebugInfoArg)"/>
            <Compile Target="$(Target)" Platform="Win64" Configuration="$(Configuration)" Tag="#UE4Game Win64" Arguments="-precompile $(VSCompilerArg) $(TargetDebugInfoArg)" Clean="false"/>
        </ForEach>
    </ForEach>
</Node>

It can be seen that the same target is compiled with different compile parameters twice:

  1. With parameters -allmodules, -nolink, etc.
  2. Without the above parameters

The -allmodules parameter means that all modules in the engine must be recompiled, leading to a complete engine compilation. During the next execution, the entire engine will still be recompiled; the UBT contains a description of -allmodules:

-allmodules is defined in UBT’s TargetRules.cs:

UnrealBuildTool/Configuration/TargetRules.cs
1
2
3
4
5
/// <summary>
/// Build all the modules that are valid for this target type. Used for CIS and making installed engine builds.
/// </summary>
[CommandLine("-AllModules")]
public bool bBuildAllModules = false;

It’s specifically for building the installed version of the engine, and I believe we can turn it off to use incremental compilation, otherwise, it will be too slow on CI.

Also, this script executes twice for each platform: once without the -nolink parameter and once with it, which can be turned off based on your requirements. UE’s documentation describes it like this (Unreal Engine 4.14 Released!):

New: Added a -nolink command line option for Unreal Build Tool, which enables compiling a target without linking it. This is useful for automated build tests, such as non-unity compiles.

When using BuildGraph to build the engine, by default, the formula for calculating the compilation frequency of the engine is:

1
2
UE4Editor DebugGame/Development 2 times
UE4Game Win64/Android/IOS/...   Each platform 2*Configuration times (-allmodules -nolink)

If we use BuildGraph to build the installed engine via Make Installed Build Win64 (supporting Win/Android/IOS packaging), we need to compile at least 2+3*2=8 times, leading to compiling the engine code eight times! Each execution requires a full compilation of all modules, which takes a very long time.

Therefore, trimming is very necessary, as mentioned above:

  1. Remove the -allmodules parameter for incremental compilation
  2. Remove the non-linking builds (depends on needs)
  3. Reduce the number of required build platforms

After trimming, the number of required compilations becomes:

  1. UE4Editor Development 1 time
  2. UE4Game Win64/Android/IOS/… Each platform 1*Configuration times

This leads to 4 compilations, cutting it down by half from the default, and allowing for incremental compilation, greatly speeding up the process.

Building the Engine Supporting Android Packaging

To use Make Installed Build Win64 to build an engine that supports Android packaging, the -set:WithAndroid=true parameter must be added when executing the BuildGraph script, as WithAndroid is a parameter defined in InstalledEngineBuild.xml and can be passed via the command line.

However, just specifying this on the command line is not enough. Before executing BuildGraph, the Android development environment needs to be properly configured, as BuildGraph will look for the paths of JDK/Android NDK/Android SDK and gradle in the system PATH during the compilation process; you need to configure this in advance:

Note: Although the environment will be checked during compilation and gradle will be downloaded automatically, due to firewall issues, this may not succeed.

The Android environment should be added to the system environment variable:

Note: When using some CI tools, they can only see the environment variables of the user the service is running as, so it’s better to add them to the system environment variables to avoid additional issues.

By default, an engine that supports packaging for Android architectures armv7 and arm64 will be compiled. If you do not need a specific architecture, you can choose to comment it out in InstalledEngineBuild.xml.

Configuring the Android development environment is straightforward; once done, you can directly export an engine that can package for Android:

Building the Engine Supporting iOS Packaging

As mentioned earlier, building a binary engine supporting Android packaging is relatively simple, but for the iOS platform, it’s more complex.

First, to build an engine supporting iOS packaging, a Mac must perform the remote build since BuildGraph needs to compile the iOS platform’s UE4Game, and cannot compile the iOS libraries on Windows, thus a Mac is mandatory.

  1. There is a Mac accessible in the local area network.
  2. Have a mobile provision.
  3. Generate an SSH Key.

As for “Why do we need to support iOS on Windows even with a Mac?” The primary reasons are:

  1. The Mac machine has limited capabilities. When packaging directly on Mac, both compilation and cooking operations occur there. By using remote builds, the Mac only handles code compilation, without needing to execute the cooking process. This reduces dependence on Mac resources and can enhance build efficiency.
  2. Using Windows uniformly for builds across all platforms (Win/Android/iOS).

First, SSH keys for the Mac need to be generated on Windows, as BuildGraph requires SSH keys to compile the engine for iOS. The generation method has been documented in my previous blog, and I will not elaborate again: UE4 Development Notes: Mac/iOS Section#Remote Build Configuration

Once the SSH key has been generated, the following configurations need to be modified in the engine’s Engine/Config/BaseEngine.ini:

1
2
3
4
5
6
[/Script/IOSRuntimeSettings.IOSRuntimeSettings]
RemoteServerName=
RSyncUsername=
SSHPrivateKeyOverridePath=
mobileprovision=
SigningCertificate=

Note: The mobileprovision and SigningCertificate can be specified by name, such as:

1
2
MobileProvision=com.XXXX.XXXX.fmgame_Development_SignProvision.mobileprovision
SigningCertificate=iPhone Developer: Created via API (JDPXHYVWYZ)

Also, SSHPrivateKeyOverridePath path doesn’t need to be specified, as RemoteMac.cs will look for the SSH key in three automatically searched paths for the engine directory:

1
2
3
4
5
6
if (ProjectFile != null)
{
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Build", "NotForLicensees"));
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Build", "NoRedist"));
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Build"));
}

Place the SSH Key in any of these three directories using the following naming format:

1
SSHKeys/IP_ADDR/USER_NAME/RemoteToolChainPrivate.key

For example:

1
SSHKeys/192.168.1.123/buildmachine/RemoteToolChainPrivate.key

In engine versions 4.26 and later, the search path for SSH key has changed. See: ToolChain/RemoteMac.cs

1
2
3
4
5
6
7
8
9
if (ProjectFile != null)
{
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Restricted", "NotForLicensees", "Build"));
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Restricted", "NoRedist", "Build"));
    Locations.Add(DirectoryReference.Combine(ProjectFile.Directory, "Build"));
}
Locations.Add(DirectoryReference.Combine(UnrealBuildTool.EngineDirectory, "Restricted", "NotForLicensees", "Build"));
Locations.Add(DirectoryReference.Combine(UnrealBuildTool.EngineDirectory, "Restricted", "NoRedist", "Build"));
Locations.Add(DirectoryReference.Combine(UnrealBuildTool.EngineDirectory, "Build"));

For example:

1
[PROJECT_DIR]\Restricted\NotForLicensees\Build\SSHKeys\192.168.31.55\lipengzha\RemoteToolChainPrivate.key

Remote Build UBT Path Exception

Errors in RemoteMac.cs during execution:

// RemoteMac.cs
1
2
3
4
5
6
7
8
9
10
11
12
13
/// <summary>
/// Attempts to get the SSH private key from the standard locations
/// </summary>
/// <param name="OutPrivateKey">If successful, receives the location of the private key that was found</param>
/// <returns>True if a private key was found, false otherwise</returns>
private bool TryGetSshPrivateKey(out FileReference OutPrivateKey)
{
    // Build a list of all the places to look for a private key
    List<DirectoryReference> Locations = new List<DirectoryReference>();
    Locations.Add(DirectoryReference.Combine(DirectoryReference.GetSpecialFolder(Environment.SpecialFolder.ApplicationData), "Unreal Engine", "UnrealBuildTool"));
    Locations.Add(DirectoryReference.Combine(DirectoryReference.GetSpecialFolder(Environment.SpecialFolder.Personal), "Unreal Engine", "UnrealBuildTool"));
    // ...
}

In the above code, there’s no check during Combine, thus Environment.SpecialFolder.ApplicationData may not be found; therefore, this portion of code needs to be modified in UBT's RemoteMac.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
private bool TryGetSshPrivateKey(out FileReference OutPrivateKey)
{
    // Build a list of all the places to look for a private key
    List<DirectoryReference> Locations = new List<DirectoryReference>();
    DirectoryReference ApplicationData = DirectoryReference.GetSpecialFolder(Environment.SpecialFolder.ApplicationData);

    DirectoryReference Personal = DirectoryReference.GetSpecialFolder(Environment.SpecialFolder.Personal);
    if (ApplicationData != null)
        Locations.Add(DirectoryReference.Combine(ApplicationData, "Unreal Engine", "UnrealBuildTool"));
    if (Personal != null)
        Locations.Add(DirectoryReference.Combine(Personal, "Unreal Engine", "UnrealBuildTool"));
    
    // ...
}

The above operation will only resolve the path error when looking for the SSH key.

SSH Connection Error During Remote Build

After resolving this issue, there’s another:

1
2
3
4
5
6
7
8
9
10
11
****** [4/11] Compile UnrealHeaderTool Mac

Reading local file list from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Update Version Files\Tag-Update Version Files.xml
Running: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Binaries\DotNET\UnrealBuildTool.exe UnrealHeaderTool Mac Development  -NoUBTMakefiles -nobuilduht -precompile -allmodules -Manifest=C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Programs\AutomationTool\Saved\Logs\UBT-UnrealHeaderTool-Mac-Development.txt"
  [Remote] Using remote server 'xx.xx.xx.xx' on port 2222 (user 'buildmachine')
  [Remote] Using private key at C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Build\NotForLicensees\SSHKeys\xx.xx.xx.xx\buildmachine\RemoteToolChainPrivate.key
  ERROR: Unable to determine home directory for remote user. SSH output:
           Host key verification failed.
Took 0.6103776s to run UnrealBuildTool.exe, ExitCode=6
UnrealBuildTool failed. See log for more details. (C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Programs\AutomationTool\Saved\Logs\UBT-UnrealHeaderTool-Mac-Development.txt)

The error indicates that SSH key verification failed; however, it’s not the key itself that’s the problem (this is quite frustrating). The actual issue arises because of how the SSH connection command is coded in RemoteMac.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class RemoteMac
{
    // ...
    /// <summary>
    /// The authentication used for SSH (probably similar to RsyncAuthentication).
    /// </summary>
    [XmlConfigFile]
    private string SshAuthentication = "-i '${CYGWIN_SSH_PRIVATE_KEY}'";
    // ...
};

public RemoteMac(FileReference ProjectFile)
{
    // ...
    SshAuthentication = ExpandVariables(SshAuthentication);

    // Build a list of arguments for SSH
    CommonSshArguments = new List<string>();
    CommonSshArguments.Add("-o BatchMode=yes");
    CommonSshArguments.Add(SshAuthentication);
    CommonSshArguments.Add(String.Format("-p {0}", ServerPort));
    CommonSshArguments.Add(String.Format("\"{0}@{1}\"", UserName, ServerName));
    // ...
}

The command generated gives:

1
\Engine\Extras\ThirdPartyNotUE\DeltaCopy\Binaries\ssh.exe -o BatchMode=yes -i \Engine\Build\NotForLicensees\SSHKeys\xx.xx.xx.xx\buildmachine\RemoteToolChainPrivate.key -p 22 buildmachine@xx.xx.xx.xx

The critical issue lies in BatchMode=yes; when we connect to a host for the first time via SSH, the following prompt appears:

1
2
3
4
5
6
$ Engine\Extras\ThirdPartyNotUE\DeltaCopy\Binaries\ssh.exe -p 22 buildmachine@192.168.1.123 
The authenticity of host '[192.168.1.123]:22' can't be established.
RSA key fingerprint is e0:8d:b9:7c:65:c7:9e:18:94:12:ed:ef:40:1a:15:47.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[192.168.1.123]:22' (RSA) to the list of known hosts.
Password:

A prompt will ask you to verify the host (manual input of yes is required), but enabling BatchMode=yes will disable all interactive prompts, causing the connection to fail upon any interaction! This is why we receive the Host key verification failed. message despite having the correct key.

Knowing this issue, there are two ways to resolve it:

  1. Disable host verification for SSH connections.
  2. Before building, manually connect using the SSH command, allowing for interactive verification (initial verification required only once).
  3. It might also be due to overly permissive permissions on the key file on the client, so permission adjustments may be necessary.

To disable host verification during the SSH connection, you need to edit the Mac’s /etc/ssh/ssh_config file to modify or add the following:

1
StrictHostKeyChecking no

Then restart the sshd service:

1
2
sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist

Alternatively, modify the SSH connection command to add the following parameters:

1
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null user@192.168.1.123

The specific location for this is:

RemoteMac.cs
1
2
3
4
5
6
CommonSshArguments = new List<string>();
//++[lipengzha] skip ssh new host auth
CommonSshArguments.Add("-o StrictHostKeyChecking=no");
CommonSshArguments.Add("-o UserKnownHostsFile=/dev/null");
CommonSshArguments.Add("-q"); // dont print shh warning log, for rsync GetRemotePath
//--[lipengzha]

Since rsync also uses the configured SSH information for file transfers during remote builds, the rsync command should also be modified:

RemoteMac.cs
1
2
3
//++[RSTDIO][lipengzha] skip ssh new host auth
BasicRsyncArguments.Add(String.Format("--rsh=\"{0} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -q -p {1}\"", RsyncAuthentication, ServerPort));
//--[RSTUDIO]

The original rsync command:

1
rsync --compress --verbose --rsh="./ssh -i '/cygdrive/D/Projects/Client/Build/NotForLicensees/SSHKeys/xx.xx.xx.xx/buildmachine/RemoteToolChainPrivate.key' -p 2222" --chmod=ugo=rwx --recursive --delete --delete-excluded --times --omit-dir-times --prune-empty-dirs --copy-links --rsync-path="mkdir -p /Users/buildmachine/UE4/Builds/lipengzha-PCb/D/Projects/Client && rsync" --filter="merge /cygdrive/D/Projects/Client/Build/Rsync/RsyncProject.txt" --filter="merge /cygdrive/D/Projects/Engine/Engine/Build/Rsync/RsyncProject.txt" --exclude='*' "/cygdrive/D/Projects/Client/" "buildmachine@xx.xx.xx.xx":'/Users/buildmachine/UE4/Builds/lipengzha-PCb/D/Projects/Client/'

The modified command:

1
rsync --compress --verbose --rsh="./ssh -i '/cygdrive/D/Projects/Client/Build/NotForLicensees/SSHKeys/xx.xx.xx.xx/buildmachine/RemoteToolChainPrivate.key' -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -q -p 2222" --chmod=ugo=rwx --recursive --delete --delete-excluded --times --omit-dir-times --prune-empty-dirs --copy-links --rsync-path="mkdir -p /Users/buildmachine/UE4/Builds/lipengzha-PCb/D/Projects/Client && rsync" --filter="merge /cygdrive/D/Projects/Client/Build/Rsync/RsyncProject.txt" --filter="merge /cygdrive/D/Projects/Engine/Engine/Build/Rsync/RsyncProject.txt" --exclude='*' "/cygdrive/D/Projects/Client/" "buildmachine@xx.xx.xx.xx":'/Users/buildmachine/UE4/Builds/lipengzha-PCb/D/Projects/Client/'

Error of Missing Import for Provision

Other compilation errors:

1
2
3
4
5
6
7
8
9
10
11
****** [6/11] Compile UE4Game IOS

Reading local file list from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Compile UnrealHeaderTool Mac\Tag-Compile UnrealHeaderTool Mac.xml
Reading local file list from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Update Version Files\Tag-Update Version Files.xml
Reading shared manifest from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Compile UnrealHeaderTool Mac\Manifest.xml
Running: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Binaries\DotNET\UnrealBuildTool.exe UE4Game IOS Development  -NoUBTMakefiles -nobuilduht -precompile -allmodules -nolink -nodebuginfo -Manifest=C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Programs\AutomationTool\Saved\Logs\UBT-UE4Game-IOS-Development.txt"
  [Remote] Using remote server 'xx.xx.xx.xxx' on port 22 (user 'buildmachine')
  [Remote] Using private key at C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Build\NotForLicensees\SSHKeys\10.75.27.129\buildmachine\RemoteToolChainPrivate.key
  [Remote] Using base directory '/Users/buildmachine/UE4/Builds/lipengzha-PC2'
  ERROR: Unable to find mobile provision for UE4Game. See log for more information.
Took 4.0300959s to run UnrealBuildTool.exe, ExitCode=6

This is because the provision was not configured when compiling the engine. You need to set it in Engine/Config/BaseEngine.ini.

Here you can only specify the name of the provision file, and the code in UEBuildIOS.cs will look for it in the path C:\Users\lipengzha\AppData\Local\Apple Computer\MobileDevice\Provisioning Profiles.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
protected IOSProvisioningData(IOSProjectSettings ProjectSettings, bool bIsTVOS, bool bForDistribtion)
{
    // ...
	if(!string.IsNullOrEmpty(MobileProvision))
	{
		DirectoryReference MobileProvisionDir;
		if(BuildHostPlatform.Current.Platform == UnrealTargetPlatform.Mac)
		{
			MobileProvisionDir = DirectoryReference.Combine(new DirectoryReference(Environment.GetEnvironmentVariable("HOME")), "Library", "MobileDevice", "Provisioning Profiles");
		}
		else
		{
			MobileProvisionDir = DirectoryReference.Combine(DirectoryReference.GetSpecialFolder(Environment.SpecialFolder.LocalApplicationData), "Apple Computer", "MobileDevice", "Provisioning Profiles");
		}

		FileReference PossibleMobileProvisionFile = FileReference.Combine(MobileProvisionDir, MobileProvision);
		if(FileReference.Exists(PossibleMobileProvisionFile))
		{
			MobileProvisionFile = PossibleMobileProvisionFile;
		}
	}
    // ...
}

This operation can also be imported through the engine, not only requiring the import of the provision but also the certificate (which can also be imported via iPhonePackager):

If not imported, the following errors will occur:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
****** [7/12] Compile UE4Game IOS

Reading local file list from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Compile UnrealHeaderTool Mac\Tag-Compile UnrealHeaderTool Mac.xml
Reading local file list from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Update Version Files\Tag-Update Version Files.xml
Reading shared manifest from C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Saved\BuildGraph\Compile UnrealHeaderTool Mac\Manifest.xml
Running: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Binaries\DotNET\UnrealBuildTool.exe UE4Game IOS Development  -NoUBTMakefiles -nobuilduht -precompile -allmodules -nolink -nodebuginfo -Manifest=C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Programs\AutomationTool\Saved\Logs\UBT-UE4Game-IOS-Development.txt"
  [Remote] Using remote server '192.168.1.123' on port 22 (user 'buildmachine')
  [Remote] Using private key at C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Build\NotForLicensees\SSHKeys\192.168.1.123\buildmachine\RemoteToolChainPrivate.key
  [Remote] Using base directory '/Users/buildmachine/UE4/Builds/lipengzha-PC3'
  [Remote] Uploading C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Remote\UE4Game\IOS\Development\com.xxxxx.xxxx.xx_Development_SignProvision.mobileprovision
  [Remote] Exporting certificate for C:\Users\lipengzha\AppData\Local\Apple Computer\MobileDevice\Provisioning Profiles\com.xxxxx.xxxx.xx_Development_SignProvision.mobileprovision...
  Executing iPhonePackager ExportCertificate C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Source -provisionfile C:\Users\lipengzha\AppData\Local\Apple Computer\MobileDevice\Provisioning Profiles\com.xxxxx.xxxx.xx_Development_SignProvision.mobileprovision -outputcertificate C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Remote\UE4Game\IOS\Development\Certificate.p12
  CWD: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Binaries\DotNET\IOS
  Initial Dir: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Source
  Env CWD: C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Binaries\DotNET\IOS
  BranchPath = lipengzha-PC3/C/BuildAgent/workspace/FGameEngine/Engine/Engine/Binaries --- GameBranchPath = lipengzha-PC3/C/BuildAgent/workspace/FGameEngine/Engine/Engine/Binaries
  
  ----------
  Executing command 'ExportCertificate' '' ...
    Looking for a certificate that matches the application identifier '9TV4ZYSS4J.com.xxxxx.xxxx.xx'
    .. Provision entry SN '61B440405D86B84D' matched 0 installed certificate(s)
    .. Failed to find a valid certificate that was in date
  IPP ERROR: Failed to find a valid certificate
  
  ERROR: IphonePackager failed.
Took 2.9281688s to run UnrealBuildTool.exe, ExitCode=6

After resolving the above issues, the BuildGraph process can be completed normally, exporting the binary engine compiled based on the engine code. Once started, you can see that packaging for Windows/Android/iOS is possible.

## Error Troubleshooting

No certificate for team xxxx matching

This issue is likely due to the mismatch between the project’s configured certificate and the one in the system (or the same mobileProvision corresponds to multiple certificates). Follow the steps below, and a restart is required for changes to take effect.

If you receive the following error message:

1
Code Signing Error: No certificate for team '9TV4ZYSS4J' matching 'iPhone Developer: Created via API (JDPXHYVWYZ)' found: Select a different signing certificate for CODE_SIGN_IDENTITY, a team that matches your selected certificate, or switch to automatic provisioning.

Solution:

  1. Clear out excess mobileprovision files in ~/Library/MobileDevice/Provisioning\ Profiles on Mac.
  2. Clear out expired developer certificates in the Mac Keychain.
  3. Re-import mobileprovision and certificate.

Note: The imported mobileprovision file name must match the MobileProvision specified in BaseEngine.ini.

errSecInternalComponent error

Method 1

This occurs because calling /usr/bin/codesign via SSH does not have permission to access the Keychain. You can use the following command to unlock it during an SSH session:

1
security unlock-keychain -p password login.keychain

In UE remote builds, you can first execute this command to unlock the Keychain in the current SSH environment, allowing subsequent signing to be executed properly. Modify the Engine\Build\BatchFiles\Mac\Build.sh file to write the following content before calling UBT compile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#!/bin/sh

cd "`dirname "$0"`/../../../.."

# Setup Mono
source Engine/Build/BatchFiles/Mac/SetupMono.sh Engine/Build/BatchFiles/Mac

if [ "$4" == "-buildscw" ] || [ "$5" == "-buildscw" ]; then
	echo Building ShaderCompileWorker...
	mono Engine/Binaries/DotNET/UnrealBuildTool.exe ShaderCompileWorker Mac Development
fi
echo unlock mac keychain...
security unlock-keychain -p password login.keychain
echo Running command : Engine/Binaries/DotNET/UnrealBuildTool.exe "$@"
mono Engine/Binaries/DotNET/UnrealBuildTool.exe "$@"

ExitCode=$?
if [ $ExitCode -eq 254 ] || [ $ExitCode -eq 255 ] || [ $ExitCode -eq 2 ]; then
	exit 0
else
	exit $ExitCode
fi

Since the Build.sh will be transmitted to Mac via RSync during compilation, you can observe the following log:

1
2
3
4
5
6
7
8
9
10
11
[Remote] Executing build
  Running bundled mono, version: Mono JIT compiler version 5.16.0.220 (2018-06/bb3ae37d71a Fri Nov 16 17:12:11 EST 2018)
  unlock mac keychain...
  Running command : Engine/Binaries/DotNET/UnrealBuildTool.exe UnrealHeaderTool Mac Development -SkipRulesCompile -XmlConfigCache=/Users/buildmachine/UE4/Builds/lipengzha-PC2/C/BuildAgent/workspace/FGameEngine/Engine/Engine/Intermediate/Build/XmlConfigCache.bin -precompile -allmodules -Log=/Users/buildmachine/UE4/Builds/lipengzha-PC2/C/BuildAgent/workspace/FGameEngine/Engine/Engine/Programs/AutomationTool/Saved/Logs/UBT-UnrealHeaderTool-Mac-Development_Remote.txt -Manifest=/Users/buildmachine/UE4/Builds/lipengzha-PC2/C/BuildAgent/workspace/FGameEngine/Engine/Engine/Intermediate/Remote/UnrealHeaderTool/Mac/Development/Manifest.xml
  Target is up to date
  Deploying UnrealHeaderTool Mac Development...
  Deploying now!
  Total execution time: 1.01 seconds
[Remote] Downloading C:\BuildAgent\workspace\FGameEngine\Engine\Engine\Intermediate\Remote\UnrealHeaderTool\Mac\Development\Manifest.xml
[Remote] Downloading build products
  receiving file list ... done

Thus, each compilation will unlock the Keychain, avoiding signing errors caused by lack of access to codesign when using SSH.

Note: Also check if the SigningCertificate value in BaseEngine.ini has been specified.

Method 2

If an Apple Developer Intermediate Certificate in the system has expired, this would also lead to the issue. The solution is to import a new Apple Developer Root Certificate:

Delete the Apple Worldwide Developer Relations Certification Authority from the Keychain - System, and then download the new one from the links above and re-import it.

Method 3

Check if the certificate was imported under Login in the Keychain, if so, delete it and re-import it under System.

Invalid trust settings

If the log contains the following error:

1
2
Code Signing Error: Invalid trust settings. Restore system default trust settings for certificate "iPhone Developer: Created via API (JDPXHYVWYZ)" in order to sign code with it.
Code Signing Error: Code signing is required for product type 'Application' in SDK 'iOS 13.6'

This is because the certificate settings in the Mac Keychain have been changed to “Always Trust”. Change it back to “Use System Defaults”.

unable to build chain to self-signed root for signer

Check if the certificate is imported under Login or System categories in the Keychain. If it’s under Login, delete it and re-import it under System.

Copy Custom Directories

While building the binary engine, by default, it is consistent with the installation from Epic Launcher, but if you have modified the engine and want to package some new directories in.

Modify the InstalledEngineFilters.xml file by adding directories under CopyEditorFilter:

InstalledEngineFilters.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
<!-- Define Editor Filters -->
<Property Name="CopyEditorFilter">
	<!-- This assembly is normally embedded into the UBT executable, but it can technically be rebuilt from an installed build -->
	Engine/Binaries/ThirdParty/Newtonsoft/...
	Engine/Binaries/ThirdParty/VisualStudio/...

	<!-- In-editor documentation -->
	Engine/Documentation/Source/Shared/...
	Engine/Documentation/Extras/...

	<!-- Content folders -->
	Engine/Content/...
	<!-- Shaders folders -->
	Engine/Shaders/...
	<!-- Source code -->
	Engine/Source/UE4Game.Target.cs
	Engine/Source/UE4Editor.Target.cs

	<!-- Starter content -->
	Samples/StarterContent/Content/...
	Samples/MobileStarterContent/Content/...

	<!-- Templates -->
	Templates/TemplateCategories.ini
	Templates/FP_FirstPerson/...
	Templates/FP_FirstPersonBP/...
	Templates/Media/...
	Templates/TP_Blank/...
	Templates/TP_BlankBP/...
	Templates/TP_FirstPerson/...
	Templates/TP_FirstPersonBP/...
	Templates/TP_Flying/...
	Templates/TP_FlyingBP/...
	Templates/TP_HandheldARBP/...
	Templates/TP_ProductConfigBP/...
	Templates/TP_Rolling/...
	Templates/TP_RollingBP/...
	Templates/TP_SideScroller/...
	Templates/TP_SideScrollerBP/...
	Templates/TP_ThirdPerson/...
	Templates/TP_ThirdPersonBP/...
	Templates/TP_TopDown/...
	Templates/TP_TopDownBP/...
	Templates/TP_TwinStick/...
	Templates/TP_TwinStickBP/...
	Templates/TP_Vehicle/...
	Templates/TP_VehicleBP/...
	Templates/TP_Puzzle/...
	Templates/TP_PuzzleBP/...
	Templates/TP_2DSideScroller/...
	Templates/TP_2DSideScrollerBP/...
	Templates/TP_VehicleAdv/...
	Templates/TP_VehicleAdvBP/...
	Templates/TP_VirtualRealityBP/...

	<!-- Enterprise Templates -->
	Templates/TP_AEC_BlankBP/...
	Templates/TP_AEC_ArchvisBP/...
	Templates/TP_PhotoStudioBP/...

	Templates/TP_ME_BlankBP/...
	Templates/TP_ME_VProdBP/...

	Templates/TP_CollaborativeBP/...

	<!-- Shared template resources -->
	Templates/TemplateResources/...

	<!-- Build files -->
	Engine/Build/Build.version
	Engine/Build/Target.cs.template
</Property>

Modify according to your needs.

Incremental Build of Binary Engine

To avoid recompiling Engine modules every time, which causes a lot of dependent modules to be recompiled, modify Engine.Build.cs in the engine:

1
2
3
4
5
6
PrivateIncludePathModuleNames.AddRange(
			new string[] {
				"MessagingRpc",
				"PortalRpc",
				"PortalServices",
			}

Change to:

1
2
3
4
5
6
PublicDependencyModuleNames.AddRange(
			new string[] {
				"MessagingRpc",
				"PortalRpc",
				"PortalServices",
			}

Related discussion on UDN: Compile UE4Game Win64 particularly slow issue in Make Installed Build Win64

Conclusion

This article primarily introduces the workflow of BuildGraph, optimization for build speed, and support for packaging on Android/iOS.

It is important to note that the command used at the beginning of the article:

1
RunUAT.bat BuildGraph -target="Make Installed Build Win64" -script=Engine/Build/InstalledEngineBuild.xml -set:WithMac=false -set:WithAndroid=false -set:WithIOS=false -set:WithTVOS=false -set:WithLinux=false -set:WithHTML5=false -set:WithSwitch=false -WithDDC=false -set:WithWin32=false -set:WithLumin=false -set:WithPS4=false -set:WithXboxOne=false -set:WithHoloLens=false -set:GameConfigurations=Development

The GameConfigurations was set to only Development, so the engine compiled will only support packaging Development. If you want to support Shipping packaging, you need to specify in BuildGraph: -set:GameConfigurations=Development;Shipping. However, this will increase the build time since both Development and Shipping need to be compiled separately, adding a time overhead for compiling the engine for all supported platforms, which is necessary, but can be selectively enabled during everyday development to save build time.

The article is finished. If you have any questions, please comment and communicate.

Scan the QR code on WeChat and follow me.

Title:BuildGraph：构建支持多平台打包的二进制引擎
Author:LIPENGZHA
Publish Date:2020/11/08 15:40
Update Date:2023/10/22 12:35
World Count:13k Words
Link:https://en.imzlp.com/posts/11956/
License: CC BY-NC-SA 4.0
Reprinting of the full article is prohibited.
Your donation will encourage me to keep creating!