Tutorial:Upscale texture using ai

From Total War Modding

What is this about ?

In this article, we will explore a method for upscaling game textures using AI. This process can be partially automated, enabling batch processing of textures.

Warhammer3 2023-10-08 16-08-18.png

Setting up the work environment

Before proceeding, it's essential to set up the correct environment with all the necessary tools and configurations.

Note that you can adapt tools and methods based on your preferences and expertise.

Required software

Bellow are the required tools and their configuration:

  1. Ubuntu WSL
  2. Gimp
  3. Paint.Net
  4. AutoHotKey
  5. Upscayl
  6. RPFM or AssetEditor
  7. Python: Any form will suffice, I personally use the Python binary built-in Ubuntu WSL

Software configuration & plugins

I. Gimp batch processing plugin

Later on, we will need to convert DDS textures to PNG to allow Upscayl to work its magic. Therefore, we'll utilize a custom Python-fu plugin to perform batch texture conversion instead of doing it manually.

To do so, put the code bellow in the following folder (make sure to add your Windows Username):


Click to toggle code
from gimpfu import *

def batch_dds_to_png(directory):
    import os

    pdb.gimp_message("Processing directory: " + directory)

    # Find all DDS files in the directory
    filelist = [f for f in os.listdir(directory) if f.lower().endswith('.dds')]

    pdb.gimp_message("List of files to convert: " + ", ".join(filelist))

    if len(filelist) == 0:
        pdb.gimp_message("No DDS files found in the specified directory.")

    pdb.gimp_message("Initiating batch conversion from DDS to PNG...")

    for filename in filelist:
        filepath = os.path.join(directory, filename)

        pdb.gimp_message("Currently converting: " + filename)

        # Open the DDS file
        image = pdb.file_dds_load(filepath, filepath, 1, 0)

        # Get the active layer
        drawable = pdb.gimp_image_get_active_layer(image)

        # Save as PNG
        png_filepath = os.path.join(directory, os.path.splitext(filename)[0] + '.png')
        pdb.file_png_save_defaults(image, drawable, png_filepath, png_filepath)

        # Delete the image to free memory

    pdb.gimp_message("Batch conversion completed.")

    "Batch DDS to PNG",
    "Convert all DDS files in a directory to PNG",
    "Batch DDS to PNG...",
        (PF_DIRNAME, "directory", "Directory", "")


II. AutoHotKey script

Since we cannot re-use Gimp to convert from PNG to DDS, we will have to automate the manual conversion through Paint.Net. To do so, put the following code in :

C:\Users\<username>\Documents\AutoHotke\/Paint.Net PNG to DDS.ahk**

Click to toggle code
; Specify the directory and file extension to search for
directory := "C:\\Users\\Guild\\Documents\\4k_demo\\base_colour_upscayled_RealESRGAN_General_x4_v3_x4\\"
extension := "*.png"

; Loop through each file in the directory
Loop, Files, % directory extension
    ; Open the file with Paint.NET
    Run, "C:\\Program Files\\paint.net\\PaintDotNet.exe" "%A_LoopFileFullPath%"

    ; Wait for Paint.NET to become active
    Sleep, 1500

    ; Save As dialog (Ctrl+ALT+S)
        Send, ^+s

    ; wait for the dialog to open
        Sleep, 500

    ; Tab to the "Save as type" dropdown field
        Send, {Tab}

    ; Type in the desired format (.dds) and press Enter
        Send, d{Enter}

    ; Wait for export option dialog
        Sleep, 500
        Send, {Enter}

    ; Wait for the export to be done
        Sleep, 7000

    ; ; Close the active file (Ctrl+W)
        Send, ^w

    ; ; Wait for the file to close
        Sleep, 500

III. Custom Python script to replace old texture

This script will be used in the later part of this tutorial to replace old textures with the newly upscaled ones. Don't worry about it for now; its usage will make more sense when you reach that part of the tutorial.

For now, just create a new Python file and paste the following code:

Click to toggle code
import os
import shutil

def move_matching_files(src_dir, dst_dir):
        # Loop through all filenames in src_dir
        for filename in os.listdir(src_dir):
            src_filepath = os.path.join(src_dir, filename)

            # Skip directories
            if os.path.isdir(src_filepath):

            # Search recursively for matching files in dst_dir
            for root, dirs, files in os.walk(dst_dir):
                if filename in files:
                    dst_filepath = os.path.join(root, filename)

                    # Move the file to the matched place in dst_dir
                    shutil.move(src_filepath, dst_filepath)
                    print(f"Moved {src_filepath} to {dst_filepath}")
                    break  # Stop search as file is moved

if __name__ == "__main__":
    src_dir = "/mnt/c/Users/Guild/Documents/4k_demo/final"
    dst_dir = "/mnt/c/Users/Guild/Documents/4k_demo/grn"
    move_matching_files(src_dir, dst_dir)

Getting started

In this example, we will upscale the textures of Greenskins' trolls, including troll river, troll river hag, and their respective weapons.

Step 1: Extracting Textures

The first step is to extract all the textures that need upscaling. In this case, we'll focus on the base_colour and normal textures. To achieve this, we'll use the AssetEditor and apply a filter to find the desired textures: ^grn_troll.*[normal|base_colour].dds$

Additionally, ensure that you do not export low-poly textures, as we are only interested in upscaling high-quality textures.


Step 2: Centralizing Textures

Next, we need to centralize all the textures in the same folder or directory. Since we're using different models for each type of texture (normal and base_colour), it's a good practice to keep them separate. Below are the commands to achieve this using the shell in Ubuntu WSL:

# Create new folders to store edited textures
mkdir -p /path/to/wip/normal /path/to/wip/base_colour

# Move all the **normal** textures to the designated folder
cp **/*normal.dds /path/to/wip/normal

# Move all the **base_colour** textures to the designated folder
cp **/*base_colour.dds /path/to/wip/base_colour

Step 3: Converting DDS to PNG

Now that we have all the vanilla DDS files, we need to convert them to PNG format for upscaling.

  1. Launch Gimp and ensure you have installed the custom plugin mentioned at the beginning of this guide.
  2. . In Gimp, select "Batch DDS to PNG" from the options, as shown in the image below
    Gimp custom plugin usage.jpg
  3. A pop-up window will appear. Select the folder containing the vanilla normal textures and begin the conversion process, as indicated in the image bellow
    Gimp custom plugin usage 2.jpg
  4. Once the conversion is complete, you will see the message Batch conversion completed in the Error Console Panel as shown bellow
    Gimp custom plugin finished.jpg
  5. Once done with the normal textures, make sure to repeat this step for the base_colour folder to convert those textures to PNG format as well.

Step 4: Upscaling the textures

Now it's time to upscale the textures using your chosen models. Here are the models I personally recommend, but feel free to experiment or import your custom models:

  • Normal Textures: Ultramix Balanced
  • Base Colour Textures: Fast Real-ESRGAN

Let's get started:

  1. To prepare for upscaling, we need to remove or move the vanilla DDS textures to avoid conflicts. You can do this using the following commands:
    cd /path/to/wip/ && rm normal/*dds rm base_colour/*dds
Remove non texture files.jpg
  1. We are now ready to launch upscayl.
  2. In the left option panel, toggle the batch upscayl option.
  3. Select the folder containing your normal textures.
  4. Choose the model you want to use (in this case, Ultramix Balanced).
  5. Do not specify an output folder.
    Upscayl config .jpg
  6. Click the purple Upscayl button
  7. Wait for the processing to complete.
    Upscayl done.jpg
  8. By default Upscayl will have created a new folder containing the upscaled textures
    Upscayl generated folder.jpg
  9. After completing this step, be sure to repeat the process with the base_colour folder and adjust the model accordingly.

Step 5: Putting the Upscaled Textures Back into the Game

Congratulations, you have successfully upscaled the textures! Now, it's time to put these textures back into the game ; since GIMP doesn't support the necessary compression, we will use Paint.Net and automate the process with AutoHotKey.

  1. Open Paint.Net
  2. Open one of the normal textures (any will do)
  3. Click Save As or Ctrl+Shift+S
  4. In the Save as type option, change it from PNG (*.png*) to Direct Draw Surface (DDS) (*.dds)
    Paint.Net export option.jpg
  5. In the Export Configuration, make sure to select the following settings:
    1. Normal Textures: BC3 (sRGB, DX 10+)
    2. Base Colour Textures: BC1 (sRGB, DX 10+)
      Paint.Net DDS export option config.jpg
  6. Finally, click OK

Now, Paint.Net is configured to convert your normal textures, as such we can automate this process to convert all the extures using AutoHotKey:

  1. If you followed the installation process at the beginning of this demo, you should have a script called Paint.Net PNG to DDS.ahk.
  2. Open that file and make the following changes
    1. In line 2, put the path to your DDS normal texture folder. Make sure to escape all backslashes as shown below.
    2. In line 32, if your GPU is not high-end, you can increase the waiting time (7 seconds in the screenshot below).
AutoHotKey script configuration.jpg
  1. Once you've edited the script, open the folder containing the script
  2. Double-click on it
  3. AutoHotKey will start simulating the process of converting the upscaled PNG to DDS. Do not interrupt the process by using your mouse or keyboard, as the script cannot recover from interruptions.
  4. Once you are done with the normal textures, you can repeat the entire Step 5, but this time focus on the base_colour textures. Make sure to select the BC1 (sRGB, DX 10+) compression for the base_colour textures during the conversion.

Step 6: Replacing Vanilla Textures with Upscaled Ones

At this point, you should have all the upscaled DDS textures, as shown below.

Upscaled textures as dds.jpg

It is now time to replace all the vanilla textures in the exported folder (see Step 1) with the upscaled ones. However, if you used the AssetEditor to export the textures, it may have also exported other files in the folder. Let's first remove those unnecessary files using the following commands:

cd /path/to/exported/textures
find . -name 'low_poly' -type d -exec rm -rf {} \;
find . -name '*lowlod*' -delete
find . -name '*.anim' -delete
find . -name '*.bone_inv_trans_mats' -delete
find . -name '*base_colour*' -delete
find . -name '*xml.material' -delete
find . -name '*material_map.dds' -delete
find . -name 'materials' -type d -delete
find . -name '*.wsmodel' -delete
find . -name '*mask.dds' -delete
find . -name '*rigid_model_v2' -delete
Delete useless exported files.jpg

Once you've completed this step, you'll need to edit the python script transfert.py as set up at the beginning of this tutorial.

Make sure to replace the following variables:

  • line 24: src_dir: the folder where you can find the Upscaled DDS textures.
  • line 25: dst_dir: the folder exported through the AssetEditor.
Python script to replace old textures with upscaled one.jpg

After editing the script, launch it and verify that the upscaled textures have been successfully moved.

Transfert.py script execution.jpg

Step 7: Importing Upscaled Textures with RPFM / AssetEditor

Now, you're almost done! At this stage, you can use RPFM or AssetEditor to import the folder containing the Upscaled DDS textures:

  1. Open RPFM
  2. Create a new packfile
  3. Add the folder containing the upscaled textures
    Rpfm add folder demo.jpg
  4. You'll see the end result as shown bellow
    RPFM add folder done.jpg
  5. Save as usual

Step 8: Profit

Congratulations! You've successfully upscaled and integrated your textures back into the game. Enjoy the enhanced graphics!

If you have any more questions or need further assistance, please feel free to contact me on Discord: toumai#3164

Example of upscaled texture mod.gif