Introducing QBDL: how to run the NVIDIA NGX SDK under Linux

Symbols resolution

The three executable formats ELF, PE and, Mach-O supported by QBDL use different mechanisms to bind imported symbols. This section introduces these differences and how they are managed in QBDL's loaders.

ELF: plt/got

In the ELF format, when the binary is compiled to use the plt/got mechanisms, the imported symbols are associated with a specific relocation type: R_X86_64_JUMP_SLOT. This relocation provides information about the offset of the Global Offset Table (GOT) that is used for the plt/got trampoline mechanism.

$ readelf -r ./elf.bin

Relocation section '.rela.plt' at offset 0x348 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000006008f8  000100000007 R_X86_64_JUMP_SLO 0000000000000000 printf@GLIBC_2.2.5 + 0

In this case, the offset of the relocation (0x6008f8) points to an entry in the .got.plt that can hold two values depending on the loading parameters:

1. Lazy bind: *(0x6008f8) contains the address of the push/jmp instructions that are used to resolve the function.

2. Bind now: *(0x6008f8) contains the resolved address of the function.

When loading an ELF binary with QBDL, we ask the user to provide the addresses of the imported functions through the symlink() function. These values are used by the QBDL's loader to fill the address associated with the relocation described previously.

In the case where the binary is compiled without the plt/got mechanism, we have to deal with another kind of relocation (R_X86_64_GLOB_DAT) but the process remains the same:

const uintptr_t addr_target = base_address_ + reloc.address();

// [...]

case RELOC_x86_64::R_X86_64_GLOB_DAT: {
    const uintptr_t addr = engine_->symlink(*this, reloc.symbol());
    engine_->mem().write_ptr(binarch, addr_target, addr + reloc.addend());
    break;
  }

Mach-O: Binding bytecode

The process to resolve imported symbols in the Mach-O format is very close to the ELF one. Nonetheless, instead of having a relocation table that provides information about the symbols, we have to interpret a bytecode. This bytecode is composed of few opcodes, associated with ULEB-128 integers and other primitives. It can be seen as a DSL (Domain Specific Language) that embeds, and compresses, the information about the relocations and the symbols resolutions.

The following figure shows the differences between the Mach-O bindings bytecode and the ELF relocations table.

In QBDL, we process these bindings with the following piece of code:

for (const BindingInfo &info : binary.dyld_info().bindings()) {

  // [...]

  const auto &sym = info.symbol();
  const uint64_t ptrRVA = get_rva(binary, info.address());
  const uint64_t ptrAddr = base_address_ + ptrRVA;
  const uint64_t symAddr = engine_->symlink(*this, sym);

  engine_->mem().write_ptr(binarch, ptrAddr, symAddr);
}

PE: IAT

Finally, PE binaries handle imported functions differently from ELF and Mach-O. In PE binaries, the imported functions are associated with two tables:

1. Lookup Table.

2. Import Address Table (IAT).

Before loading the PE binary, these two tables contain the same values: the function names offsets (or the ordinal value if loaded by ordinal). When the binary is loaded, the Lookup Table keeps the function names offsets while the values of the Import Address Table are replaced with the absolute address of the resolved function.

Therefore, the PE loader of QBDL updates the Import Address Table with the value returned by the symlink function:

for (const Import &imp : binary.imports()) {
  for (const ImportEntry &entry : imp.entries()) {
    const uint64_t iatAddr = entry.iat_address();

    LIEF::Symbol sym{entry.name()};
    const uintptr_t symAddr = engine_->symlink(*this, sym);
    engine_->mem().write_ptr(binarch, base_address_ + iatAddr, symAddr);
  }
}

Let's know dig into a concrete use case of QBDL.

What is NVIDIA NGX?

NVIDIA NGX is an SDK provided by NVIDIA that uses machine learning techniques to provide image and video enhancement algorithms, like upscaling, slow motion or image reconstruction. It is also the framework used by DLSS, an upscaling technology tailored for video games.

In order to download the SDK, you need to register to the NVIDIA Developer program [1]. At the time of this writing, it is free, and access is granted directly. Unfortunately, only a Windows version is provided.

That being said, NVIDIA announced in 2020 [2] that the NGX library was shipped to Linux starting from the 450.51 driver. Let's understand what is really shipped and what is missing, by first understanding how all of this works.

NGX architecture

First of all, NGX is only made of userland components, no specific driver interface is involved. It relies on the Direct3D, Vulkan and/or CUDA APIs (depending on the component).

The first main component is the NGX shared library, which is:

• nvngx.dll on Windows.

• libnvidia-ngx.so on Linux (can be found on Debian systems in /usr/lib/x86_64-linux-gnu/libnvidia-ngx.so).

These components expose the final user API, that is for instance used in the sample provided in the NGX SDK archive. Then, each NGX "feature" is implemented in a dedicated shared library:

• Image Super Resolution (isr): nvngx_dlisr.dll.

• In-Painting (inpainting): nvngx_dlinpainting.dll.

• Video Super Resolution (vsr): nvngx_dlvsr.dll.

• Slow Motion (slowmo): nvngx_dlslomo.dll.

The Linux NGX implementation (libnvidia-ngx.so) looks for libnvidia-ngx-dl{feature}.so files (e.g. libnvidia-ngx-dlisr.so). These ELF shared libraries are not publicly provided by NVIDIA.

The last component of the NGX framework is the updater. It is implemented within the ngx-updater binary under Linux, and is transparently launched by the libnvidia-ngx.so component. According to the documentation [3], its role is to download updated machine learning models for each feature. We didn't try to reverse engineer that part.

So the missing bits with the current NGX Linux version are the implementation of the actual features (the libnvidia-ngx-dl*.so files). In the next sections, we will use QBDL to implement them based on their Windows counterparts.

Challenges

We started with the NGX feature that looked like the easiest one: Image Super Resolution (nvngx_dlisr.dll). Indeed, in terms of data, it looks like it takes an image, upscales it, and regenerates an image. No video processing in the middle.

After a quick look at nvngx_dlisr.dll libraries, we can see that:

• It imports functions from the cuDNN [8] library. Good news for us, it also exists natively for Linux.

• It imports functions from the Windows API (kernel32.dll, user32.dll & so on). We will have to implement some of them, or at least emulate the behavior we need.

• The Visual C runtime library (aka MSVCRT) is statically linked.

• The NVIDIA CUDA runtime is also statically linked.

So we have a few challenges ahead, and others we discovered while making this happen:

• Signature verification on shared libraries.

• Setting up a valid Windows environment for MSVCRT to work (or, more exactly, how not to do it).

• Making the CUDA runtime work.

Let's describe these in the following sections.

Bypassing signature verification

The libnvidia-ngx.so main library verifies a signature within the nvngx_dl*.so files, which should be in a .nvsig section. We didn't take the time to reverse engineer the format of this signature. Indeed, even if we knew how to sign, chances are we wouldn't have a valid private key to sign our wrapping binary. So, whatever happens, we either need to patch libnvidia-glx.so to change a potential public key/root of trust that is used to verify the signature, or to just bypass that signature. We chose the second option.

This is basically only modifying a flag passed to the function at address 0x6EF00, which is responsible for loading these .so plugins. This is done by patching the code in memory.

The static library problem

The NGX runtime libraries are linked with the static Visual C RT library. This means that all the necessary C and C++ library implementations are shipped within the shared libraries. While this is theoretically fine, these implementations rely on many components of the Windows OS (e.g. the PEB). Even if it is possible to setup a PEB-like structure under Linux [4] - as Wine does -, it is not something QBDL provides at this point.

The same kind of issue arises with the CUDA runtime: it is also statically linked. The CUDA runtime makes calls to the CUDA userspace driver, but some structures are not defined and would require further reverse engineering. wine-staging contains such a wrapper [6], but using them involved many CUDA-related failures in our case. It would be better to directly call the Linux CUDA runtime libraries.

One idea to fix these problems is to identify the entry points of these static libraries within the NGX's DLL, and hook them. This is far from ideal, as it binds our QBDL wrapper to a specific version of the NGX libraries (e.g. with this version-specific list of offsets), but it made the whole thing work in the end.

IDA provides a signature mechanism named FLIRT [5]. It first creates a signature database from a static library with symbols, and then searches for these signatures in our DLLs. How to do this is out of the scope of this blog post, but this article is a good reference.

Putting everything together

The final wrapper can be downloaded here: https://github.com/quarkslab/nvidia-ngx-wrapper. Please follow the README for build & usage instructions.

In the end, we generate a libnvidia-ngx-dlisr.so shared library that uses QBDL to load nvngx_dlisr.dll, and then uses the Linux implementation of cuDNN & the CUDA runtime. This is all called through the NGX Linux interface implementation (in libnvidia-ngx.so). One last detail we didn't mention yet is that we need to adapt the interface of an NGX C++ virtual interface (that is used to basically describe options) from the MSVC C++ ABI to the Linux one [7]. This is necessary because the order of the functions in the vtable is not the same between the two ABIs. Having tools that automate that kind of wrapping would greatly help (an idea for yet another Clang-based project?).

In the end, here is an example of the wrapper in action that uses NGX to upscale an image:

With the original and upscaled image: