Posted onEdited onViews: Symbols count in article: 6.8kReading time ≈17 mins.
UE builds projects through UBT (whether using Build in VS or Compile in the Editor, it eventually calls UBT). UBT and UHT are the cornerstones of the UE toolchain, and due to the vast amount of content, it is impossible to analyze everything at once. Let’s outline the main points now and supplement them later as time allows.
Here’s a general introduction to the responsibilities of UBT and UHT: UBT:
Scans solution directory for modules and plug-ins
Determines all modules that need to be rebuilt
Invokes UHT to parse C++ headers
Creates compiler & linker options from .Build.cs & .Target.cs
Executes platform specific compilers (VisualStudio, LLVM)
UHT:
Parses all C++ headers containing UClasses
Generates glue code for all Unreal classes & functions
Generated files stored in Intermediates directory
VS
Back to the topic. First, to start from scratch, the first step is to create a C++ project (choose either BasicCode or ThridPerson) and open VS.
After opening VS, we can see this Solution structure:
Right-click on the created Project in the Solution, and select Properties:
You can see that the Build Command under NMake-General uses the .bat files found in the Engine\Build\BatchFiles directory (on Windows platform):
@echo off setlocal enabledelayedexpansion REM The %~dp0 specifier resolves to the path to the directory where this .bat is located in. REM We use this so that regardless of where the .bat file was executed from, we can change to REM directory relative to where we know the .bat is stored. pushd "%~dp0\..\..\Source" REM %1 is the game name REM %2 is the platform name REM %3 is the configuration name
IFEXIST ..\..\Engine\Binaries\DotNET\UnrealBuildTool.exe ( ..\..\Engine\Binaries\DotNET\UnrealBuildTool.exe %* -DEPLOY popd REM Ignore exit codes of 2 ("ECompilationResult.UpToDate") from UBT; it's not a failure. if "!ERRORLEVEL!"=="2" ( EXIT /B 0 )
EXIT /B !ERRORLEVEL! ) ELSE ( ECHO UnrealBuildTool.exe not found in ..\..\Engine\Binaries\DotNET\UnrealBuildTool.exe popd EXIT /B 999 )
It can be seen that Build.bat forwards the received parameters to UnrealBuildTool.exe:
Building a project through UnrealBuildTool requires passing parameters:
%1 is the game name
%2 is the platform name
%3 is the configuration name
%4 is the ProjectPath
1 2
# Example UnrealBuildTool.exe ThridPerson420 Win64 Development "C:\Users\visionsmile\Documents\Unreal Projects\Examples\ThridPerson420\ThridPerson420.uproject" -WaitMutex -FromMsBuild
Next, let’s take a look at how UnrealBuildTools processes this:
// Engine\Source\Programs\UnrealBuildTools\UnrealBuildTool.cs privatestaticintMain(string[] Arguments) { // make sure we catch any exceptions and return an appropriate error code. // Some inner code already does this (to ensure the Mutex is released), // but we need something to cover all outer code as well. try { returnGuardedMain(Arguments); } catch (Exception Exception) { if (Log.IsInitialized()) { Log.TraceError("UnrealBuildTool Exception: " + Exception.ToString()); } if (ExtendedErrorCode != 0) { return ExtendedErrorCode; } return (int)ECompilationResult.OtherCompilationError; } }
You can see the parameters passed in. After a bunch of checks on the engine and incoming parameters in GuardedMain, it will call RunUBT:
RulesAssembly
In RunUBT, there’s a crucial function call UEBuildTarget.CreateTarget:
if (UBTMakefile != null && !bIsGatheringBuild && bIsAssemblingBuild) { // If we've loaded a makefile, then we can fill target information from this file! Targets = UBTMakefile.Targets; } else { DateTime TargetInitStartTime = DateTime.UtcNow;
ReadOnlyBuildVersion Version = newReadOnlyBuildVersion(BuildVersion.ReadDefault());
if (UnrealBuildTool.bPrintPerformanceInfo) { double TargetInitTime = (DateTime.UtcNow - TargetInitStartTime).TotalSeconds; Log.TraceInformation("Target init took " + TargetInitTime + "s"); } }
// other code ... }
The definition of UEBuildTarget.CreateTarget is found in Configuration/UEBuildTarget.cs. It constructs a RulesAssembly object inside, which is responsible for reading and constructing the target.cs and Module’s build.cs in the project. The call stack for constructing RulesAssembly is as follows:
The constructor of RulesAssembly accepts a variety of parameters:
Plugins: Absolute paths of all .uplugin files for all plugins the project depends on
ModuleFiles: Absolute paths of all .build.cs files for all Modules (Game Module and all internal modules of plugins) in the current Target
TargetFiles: Absolute paths of all target.cs files in the current Target
ModuleFileToPluginInfo: Mapping of plugin information; mapping of a Module’s build.cs file to the module’s basic information
AssemblyFileName: Absolute path of the project’s BuildRules DLL file (this file is generated when calling UnrealVersionSelector to create the VS project, located in the Intermediate/Build/BuildRules directory)
The RulesAssembly class defines two crucial members: TargetNameToTargetFile and ModuleNameToModuleFile, to which all defined TargetRules and ModuleRules of the current project are added in the constructor.
///<summary> /// Constructor. Compiles a rules assembly from the given source files. ///</summary> ///<param name="BaseDir">The base directory for this assembly</param> ///<param name="Plugins">All the plugins included in this assembly</param> ///<param name="ModuleFiles">List of module files to compile</param> ///<param name="TargetFiles">List of target files to compile</param> ///<param name="ModuleFileToPluginInfo">Mapping of module file to the plugin that contains it</param> ///<param name="AssemblyFileName">The output path for the compiled assembly</param> ///<param name="bContainsEngineModules">Whether this assembly contains engine modules. Used to initialize the default value for ModuleRules.bTreatAsEngineModule.</param> ///<param name="bUseBackwardsCompatibleDefaults">Whether modules in this assembly should use backwards-compatible defaults.</param> ///<param name="bReadOnly">Whether the modules and targets in this assembly are installed, and should be created with the bUsePrecompiled flag set</param> ///<param name="bSkipCompile">Whether to skip compiling this assembly</param> ///<param name="Parent">The parent rules assembly</param> publicRulesAssembly(DirectoryReference BaseDir, IReadOnlyList<PluginInfo> Plugins, List<FileReference> ModuleFiles, List<FileReference> TargetFiles, Dictionary<FileReference, PluginInfo> ModuleFileToPluginInfo, FileReference AssemblyFileName, bool bContainsEngineModules, bool bUseBackwardsCompatibleDefaults, bool bReadOnly, bool bSkipCompile, RulesAssembly Parent) { this.BaseDir = BaseDir; this.Plugins = Plugins; this.ModuleFileToPluginInfo = ModuleFileToPluginInfo; this.bContainsEngineModules = bContainsEngineModules; this.bUseBackwardsCompatibleDefaults = bUseBackwardsCompatibleDefaults; this.bReadOnly = bReadOnly; this.Parent = Parent;
// Find all the source files List<FileReference> AssemblySourceFiles = new List<FileReference>(); AssemblySourceFiles.AddRange(ModuleFiles); AssemblySourceFiles.AddRange(TargetFiles);
// Setup the module map foreach (FileReference ModuleFile in ModuleFiles) { string ModuleName = ModuleFile.GetFileNameWithoutAnyExtensions(); if (!ModuleNameToModuleFile.ContainsKey(ModuleName)) { ModuleNameToModuleFile.Add(ModuleName, ModuleFile); } }
// Setup the target map foreach (FileReference TargetFile in TargetFiles) { string TargetName = TargetFile.GetFileNameWithoutAnyExtensions(); if (!TargetNameToTargetFile.ContainsKey(TargetName)) { TargetNameToTargetFile.Add(TargetName, TargetFile); } }
// ignore other code..
}
Compilation Environment: Constructing Target and Executing
The main role of Target is to collect and configure compilation information for compiling the executable settings, similar to project settings in VS.
In RunUBT, the incoming parameters (Platform/Configuration, etc.) are extracted, and a series of parameters are added. Then, by calling UEBuildTarget.CreateTarget, a UBuildTarget object is created:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
// UnrealBuildTool/Configuration/UEBuildTarget.cs // Executes logic in `target.cs` and constructs a URBduileTarget object
In CreateTarget, RulesAssembly.CreateTargetRules is called (to get the target.cs file and construct the basic compilation environment information into a TargetInfo passed to CreateTargetRulesInstance for creating a TargetRules object):
///<summary> /// Creates a target rules object for the specified target name. ///</summary> ///<param name="TargetName">Name of the target</param> ///<param name="Platform">Platform being compiled</param> ///<param name="Configuration">Configuration being compiled</param> ///<param name="Architecture">Architecture being built</param> ///<param name="ProjectFile">Path to the project file for this target</param> ///<param name="Version">The current build version</param> ///<param name="Arguments">Command line arguments for this target</param> ///<param name="TargetFileName">The original source file name of the Target.cs file for this target</param> ///<returns>The build target rules for the specified target</returns> public TargetRules CreateTargetRules(string TargetName, UnrealTargetPlatform Platform, UnrealTargetConfiguration Configuration, string Architecture, FileReference ProjectFile, ReadOnlyBuildVersion Version, string[] Arguments, out FileReference TargetFileName) { // ignore conditional check code... // Return the target file name to the caller TargetFileName = TargetNameToTargetFile[TargetName]; // Currently, we expect the user's rules object type name to be the same as the module name + 'Target' string TargetTypeName = TargetName + "Target"; // The build module must define a type named '<TargetName>Target' that derives from our 'TargetRules' type. // 通过构造TargetInfo对象,将基本的编译环境信息传递给`target.cs`中定义的构造函数 return CreateTargetRulesInstance(TargetTypeName, new TargetInfo(TargetName, Platform, Configuration, Architecture, ProjectFile, Version), Arguments); }
The *.Target.cs file for the project is obtained, and then CreateTargetRulesInstance is invoked to construct a TargetRules object (which is instantiated from the type defined in target.cs), executing the code in the constructor of target.cs.
// UnrealBuildTool/System/RulesAssembly.cs // Obtain the TargetRules object defined in target.cs and invoke its constructor
/// Construct an instance of the given target rules ///<param name="TypeName">Type name of the target rules</param> ///<param name="TargetInfo">Target configuration information to pass to the constructor</param> ///<param name="Arguments">Command line arguments for this target</param> ///<returns>Instance of the corresponding TargetRules</returns> protected TargetRules CreateTargetRulesInstance(string TypeName, TargetInfo TargetInfo, string[] Arguments) { // The build module must define a type named '<TargetName>Target' that derives from our 'TargetRules' type. Type RulesType = CompiledAssembly.GetType(TypeName); if (RulesType == null) { thrownew BuildException("Expecting to find a type to be declared in a target rules named '{0}'. This type must derive from the 'TargetRules' type defined by Unreal Build Tool.", TypeName); }
// Create an instance of the module's rules object, and set some defaults before calling the constructor. TargetRules Rules = (TargetRules)FormatterServices.GetUninitializedObject(RulesType); Rules.bUseBackwardsCompatibleDefaults = bUseBackwardsCompatibleDefaults;
// Find the constructor ConstructorInfo Constructor = RulesType.GetConstructor(new Type[] { typeof(TargetInfo) }); if(Constructor == null) { thrownew BuildException("No constructor found on {0} which takes an argument of type TargetInfo.", RulesType.Name); }
// Invoke the regular constructor try { Constructor.Invoke(Rules, newobject[] { TargetInfo }); } catch (Exception Ex) { thrownew BuildException(Ex, "Unable to instantiate instance of '{0}' object type from compiled assembly '{1}'. Unreal Build Tool creates an instance of your module's 'Rules' object in order to find out about your module's requirements. The CLR exception details may provide more information: {2}", TypeName, Path.GetFileNameWithoutExtension(CompiledAssembly.Location), Ex.ToString()); }
// ignore other code... }
After these operations, we have obtained the TargetRules object from target.cs, which represents the compilation environment of the current project. The code is extensive and a bit tedious; the call stack looks like this:
Compilation Target: Module
Module is a small target file used for execution in UE, compiling to exe or DLL (the startup module compiles to exe, while non-startup modules compile to DLL or are statically linked to exe).
After the execution above, UBT will start reading and analyzing ModuleRules in the project, constructing UEBuildModule for subsequent compilation processing.
In RunUBT->UEBuildTarget.Build->PreBuidSetup, the actual execution logic can be understood to occur in PreBuildSetup:
///<summary> /// Setup target before build. This method finds dependencies, sets up global environment etc. ///</summary> publicvoidPreBuildSetup(UEToolChain TargetToolChain) { // Describe what's being built. Log.TraceVerbose("Building {0} - {1} - {2} - {3}", AppName, TargetName, Platform, Configuration);
// Setup the target's binaries. SetupBinaries();
// Setup the target's plugins SetupPlugins();
// Setup the custom build steps for this target SetupCustomBuildSteps();
// Add the plugin binaries to the build foreach (UEBuildPlugin Plugin in BuildPlugins) { foreach(UEBuildModuleCPP Module in Plugin.Modules) { AddModuleToBinary(Module); } }
// Add all of the extra modules, including game modules, that need to be compiled along // with this app. These modules are always statically linked in monolithic targets, but not necessarily linked to anything in modular targets, // and may still be required at runtime in order for the application to load and function properly! AddExtraModules();
// Create all the modules referenced by the existing binaries foreach(UEBuildBinary Binary in Binaries) { Binary.CreateAllDependentModules(FindOrCreateModuleByName); }
// Bind every referenced C++ module to a binary for (int Idx = 0; Idx < Binaries.Count; Idx++) { List<UEBuildModule> DependencyModules = Binaries[Idx].GetAllDependencyModules(true, true); foreach (UEBuildModuleCPP DependencyModule in DependencyModules.OfType<UEBuildModuleCPP>()) { if(DependencyModule.Binary == null) { AddModuleToBinary(DependencyModule); } } }
// Add all the modules to the target if necessary. if(Rules.bBuildAllModules) { AddAllValidModulesToTarget(); }
// Add the external and non-C++ referenced modules to the binaries that reference them. foreach (UEBuildModuleCPP Module in Modules.Values.OfType<UEBuildModuleCPP>()) { if(Module.Binary != null) { foreach (UEBuildModule ReferencedModule in Module.GetUnboundReferences()) { Module.Binary.AddModule(ReferencedModule); } } }
if (!bCompileMonolithic) { if (Platform == UnrealTargetPlatform.Win64 || Platform == UnrealTargetPlatform.Win32) { // On Windows create import libraries for all binaries ahead of time, since linking binaries often causes bottlenecks foreach (UEBuildBinary Binary in Binaries) { Binary.SetCreateImportLibrarySeparately(true); } } else { // On other platforms markup all the binaries containing modules with circular references foreach (UEBuildModule Module in Modules.Values.Where(x => x.Binary != null)) { foreach (string CircularlyReferencedModuleName in Module.Rules.CircularlyReferencedDependentModules) { UEBuildModule CircularlyReferencedModule; if (Modules.TryGetValue(CircularlyReferencedModuleName, out CircularlyReferencedModule) && CircularlyReferencedModule.Binary != null) { CircularlyReferencedModule.Binary.SetCreateImportLibrarySeparately(true); } } } } }
// On Mac AppBinaries paths for non-console targets need to be adjusted to be inside the app bundle if (Platform == UnrealTargetPlatform.Mac && !Rules.bIsBuildingConsoleApplication) { TargetToolChain.FixBundleBinariesPaths(this, Binaries); } }
Where:
1 2 3 4 5 6 7 8 9
// Setup the target's binaries. // Read LaunchModule, the executable target exe to be compiled, create the startup module's UEBuildModuleCPP and UEBuildBinary (compile exe) // UEBuildBinary is only created here SetupBinaries();
// Setup the target's plugins // Construct all plugin Modules and create the compilation object UEBuildModuleCPP SetupPlugins();
They directly or indirectly call FindOrCreateCppModuleByName, which ultimately calls CreateModuleRulesAndSetDefaults to construct the real ModuleRules object, and creates UEBuildModuleCPP for the module to be compiled:
CreateModuleRulesAndSetDefaults then calls RulesAssembly.CreateModuleRules (note that at this point, the execution flow has returned to RulesAssembly).
RulesAssembly.CreateModuleRules retrieves the *.build.cs file using ModuleNameToModuleFile established during construction, then calls the constructor of ModuleRules similarly to the constructor of TargetRules, and passes the constructed TargetRules to ModuleRules.
///<summary> /// Creates an instance of a module rules descriptor object for the specified module name ///</summary> ///<param name="ModuleName">Name of the module</param> ///<param name="Target">Information about the target associated with this module</param> ///<param name="ReferenceChain">Chain of references leading to this module</param> ///<returns>Compiled module rule info</returns> public ModuleRules CreateModuleRules(string ModuleName, ReadOnlyTargetRules Target, string ReferenceChain) { // Currently, we expect the user's rules object type name to be the same as the module name string ModuleTypeName = ModuleName;
// Make sure the module file is known to us FileReference ModuleFileName; if (!ModuleNameToModuleFile.TryGetValue(ModuleName, out ModuleFileName)) { if (Parent == null) { thrownew BuildException("Could not find definition for module '{0}' (referenced via {1})", ModuleName, ReferenceChain); } else { return Parent.CreateModuleRules(ModuleName, Target, ReferenceChain); } }
// The build module must define a type named 'Rules' that derives from our 'ModuleRules' type. Type RulesObjectType = GetModuleRulesTypeInternal(ModuleName); if (RulesObjectType == null) { thrownew BuildException("Expecting to find a type to be declared in a module rules named '{0}' in {1}. This type must derive from the 'ModuleRules' type defined by Unreal Build Tool.", ModuleTypeName, CompiledAssembly.FullName); }
// Create an instance of the module's rules object try { // Create an uninitialized ModuleRules object and set some defaults. ModuleRules RulesObject = (ModuleRules)FormatterServices.GetUninitializedObject(RulesObjectType); RulesObject.Name = ModuleName; RulesObject.File = ModuleFileName; RulesObject.Directory = ModuleFileName.Directory; ModuleFileToPluginInfo.TryGetValue(RulesObject.File, out RulesObject.Plugin); RulesObject.bTreatAsEngineModule = bContainsEngineModules; RulesObject.bUseBackwardsCompatibleDefaults = bUseBackwardsCompatibleDefaults && Target.bUseBackwardsCompatibleDefaults; RulesObject.bPrecompile = (RulesObject.bTreatAsEngineModule || ModuleName.Equals("UE4Game", StringComparison.OrdinalIgnoreCase)) && Target.bPrecompile; RulesObject.bUsePrecompiled = bReadOnly;
// Call the constructor ConstructorInfo Constructor = RulesObjectType.GetConstructor(new Type[] { typeof(ReadOnlyTargetRules) }); if(Constructor == null) { thrownew BuildException("No valid constructor found for {0}.", ModuleName); } Constructor.Invoke(RulesObject, newobject[] { Target });
return RulesObject; } catch (Exception Ex) { Exception MessageEx = (Ex is TargetInvocationException && Ex.InnerException != null)? Ex.InnerException : Ex; thrownew BuildException(Ex, "Unable to instantiate module '{0}': {1}\n(referenced via {2})", ModuleName, MessageEx.ToString(), ReferenceChain); } }
We execute the code in all GameMode and build.cs in all Plugins at this time. ### Launch Module Compilation
The above mentioned compiling Module, there is one Module that is special, namely the Launch module.
Every executable program has an entry point, in UE, every Target compiles an executable program. The engine starts from the Launch module, and the main function is also defined therein, so it needs to compile the main function from the Launch module into an executable program.
The startup module is specified in the TargetRules of UBT:
1 2 3 4 5 6 7 8 9 10 11
publicstring LaunchModuleName { get { return (LaunchModuleNamePrivate == null && Type != global::UnrealBuildTool.TargetType.Program)? "Launch" : LaunchModuleNamePrivate; } set { LaunchModuleNamePrivate = value; } }
You can specify a launch module using LaunchModuleNamePrivate in TargetRules. If not specified and the Target type is not Program, it uses the Launch module, otherwise it uses the specified module. This means that the startup module is Launch for Game/Editor/Server/Client Targets. However, since LaunchModuleNamePrivate is defined as a private member in TargetRules, it cannot be assigned a value in our inherited TargetRules, so it currently serves no purpose.
It is used in UEBuildTarget.SetupBinaries (mentioned above, UEBuildBinary is the executable exe compilation object for the startup module and is only created here):
///<summary> /// Sets up the binaries for the target. ///</summary> protectedvoidSetupBinaries() { // If we're using the new method for specifying binaries, fill in the binary configurations now if(Rules.LaunchModuleName == null) { thrownew BuildException("LaunchModuleName must be set for all targets."); }
// Create the launch module UEBuildModuleCPP LaunchModule = FindOrCreateCppModuleByName(Rules.LaunchModuleName, TargetRulesFile.GetFileName());
// Add the launch module to it LaunchModule.Binary = Binary; Binary.AddModule(LaunchModule);
// Create an additional console app for the editor if (Platform == UnrealTargetPlatform.Win64 && Configuration != UnrealTargetConfiguration.Shipping && TargetType == TargetType.Editor) { Binary.bBuildAdditionalConsoleApp = true; } }
The execution environment is as follows:
You can see that the output file here is the project exe we compiled.
UHT
Then UHT is called to generate code: The function called is ExecuteHeaderToolIfNecessary (System/ExternalExecution.cs):
If the previous step generates successfully through UHT, the compilation action will be executed (ActionGraph.ExecuteActions in System/ActionGraphs.cs):
It will further check a bunch of engine build configurations (e.g: Engine/Saved/UnrealBuildTool/BuildConfiguration.xml):
I kept the default engine build configuration, so a ParallelExecutor (System/ParallelExecutor.cs) is created, and then executed:
The current compilation tasks are created with multiple Actions and executed:
Start compiling code:
Postscript
Based on the analysis above, the build path in UE is:
Click Build in VS, which calls build.bat
build.bat calls UBT
UBT executes the logic in target.cs and all Module‘s build.cs
UBT calls UHT (generating code based on UE’s macro tags)
After UHT finishes generating, UBT calls the compiler
Preprocessing
Compilation
Linking
The key point of this process is that UBT calls UHT for generation before preprocessing by the compiler, which means we cannot wrap UE’s macros (in fact, UCLASS / UFUNCTION and similar should not be called macros, but rather tags), because UE’s macros are preprocessed by UHT before the compiler.