Help:Nope: Difference between revisions

From Total War Modding
(Blanked the page)
Tag: Blanking
Line 1: Line 1:
== What is this about ? ==


In this article, we will explore a method for upscaling game textures using AI.
This process can be partially automated, enabling batch processing of textures.
[[File:Warhammer3 2023-10-08 16-08-18.png|centre|920px]]
== Setting up the work environment ==
Before proceeding, it's essential to set up the correct environment with all the necessary tools and configurations.
'''''Note that you can adapt tools and methods based on your preferences and expertise.''
'''
=== Required software ===
Bellow are the required tools and their configuration:
# [https://ubuntu.com/tutorials/install-ubuntu-on-wsl2-on-windows-10 Ubuntu WSL]
# [https://www.gimp.org/downloads Gimp]
# [https://www.dotpdn.com/downloads/pdn.html Paint.Net]
# [https://www.autohotkey.com AutoHotKey]
# [https://www.upscayl.org/#download Upscayl]
# [https://github.com/Frodo45127/rpfm RPFM] or [https://tw-modding.com/wiki/Tutorial:AssetEditor AssetEditor]
# [https://www.python.org/downloads/ Python]: Any form will suffice, I personally use the Python binary built-in Ubuntu WSL
=== Software configuration & plugins ===
==== I. Gimp batch processing plugin ====
Later on, we will need to convert DDS textures to PNG to allow Upscayl to work its magic. Therefore, we'll utilize a custom Python-fu plugin to perform batch texture conversion instead of doing it manually.
To do so, put the code bellow in the following folder (''make sure to add your Windows Username''):
<syntaxhighlight inline lang="html">C:\Users\<username>\AppData\Roaming\GIMP\2.10\plug-ins\dds_to_png.py</syntaxhighlight>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">Click to toggle code </div>
<div class="mw-collapsible-content">
<syntaxhighlight lang="python">
from gimpfu import *
def batch_dds_to_png(directory):
    import os
    pdb.gimp_message("Processing directory: " + directory)
    # Find all DDS files in the directory
    filelist = [f for f in os.listdir(directory) if f.lower().endswith('.dds')]
    pdb.gimp_message("List of files to convert: " + ", ".join(filelist))
    if len(filelist) == 0:
        pdb.gimp_message("No DDS files found in the specified directory.")
        return
    pdb.gimp_message("Initiating batch conversion from DDS to PNG...")
    for filename in filelist:
        filepath = os.path.join(directory, filename)
        pdb.gimp_message("Currently converting: " + filename)
        # Open the DDS file
        image = pdb.file_dds_load(filepath, filepath, 1, 0)
        # Get the active layer
        drawable = pdb.gimp_image_get_active_layer(image)
        # Save as PNG
        png_filepath = os.path.join(directory, os.path.splitext(filename)[0] + '.png')
        pdb.file_png_save_defaults(image, drawable, png_filepath, png_filepath)
        # Delete the image to free memory
        pdb.gimp_image_delete(image)
    pdb.gimp_message("Batch conversion completed.")
register(
    "python-fu-batch-dds-to-png",
    "Batch DDS to PNG",
    "Convert all DDS files in a directory to PNG",
    "Toumai",
    "Toumai",
    "2023",
    "Batch DDS to PNG...",
    "",
    [
        (PF_DIRNAME, "directory", "Directory", "")
    ],
    [],
    batch_dds_to_png,
    menu="<Image>/File/Batch/"
)
main()
</syntaxhighlight>
</div></div>
==== II. AutoHotKey script ====
Since we cannot re-use Gimp to convert from PNG to DDS, we will have to automate the manual conversion through Paint.Net.
To do so, put the following code in :
<syntaxhighlight inline lang="html">C:\Users\<username>\Documents\AutoHotke\/Paint.Net PNG to DDS.ahk**</syntaxhighlight>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">Click to toggle code </div>
<div class="mw-collapsible-content">
<syntaxhighlight lang="elisp">
; Specify the directory and file extension to search for
directory := "C:\\Users\\Guild\\Documents\\4k_demo\\base_colour_upscayled_RealESRGAN_General_x4_v3_x4\\"
extension := "*.png"
; Loop through each file in the directory
Loop, Files, % directory extension
{
    ; Open the file with Paint.NET
    Run, "C:\\Program Files\\paint.net\\PaintDotNet.exe" "%A_LoopFileFullPath%"
    ; Wait for Paint.NET to become active
    Sleep, 1500
    ; Save As dialog (Ctrl+ALT+S)
        Send, ^+s
    ; wait for the dialog to open
        Sleep, 500
    ; Tab to the "Save as type" dropdown field
        Send, {Tab}
    ; Type in the desired format (.dds) and press Enter
        Send, d{Enter}
    ; Wait for export option dialog
        Sleep, 500
        Send, {Enter}
    ; Wait for the export to be done
        Sleep, 7000
    ; ; Close the active file (Ctrl+W)
        Send, ^w
    ; ; Wait for the file to close
        Sleep, 500
}
</syntaxhighlight>
</div></div>
==== III. Custom Python script to replace old texture ====
This script will be used in the later part of this tutorial to replace old textures with the newly upscaled ones.
Don't worry about it for now; its usage will make more sense when you reach that part of the tutorial.
For now, just create a new Python file and paste the following code:
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">Click to toggle code </div>
<div class="mw-collapsible-content">
<syntaxhighlight lang="python">
import os
import shutil
def move_matching_files(src_dir, dst_dir):
        # Loop through all filenames in src_dir
        for filename in os.listdir(src_dir):
            src_filepath = os.path.join(src_dir, filename)
            # Skip directories
            if os.path.isdir(src_filepath):
                continue
            # Search recursively for matching files in dst_dir
            for root, dirs, files in os.walk(dst_dir):
                if filename in files:
                    dst_filepath = os.path.join(root, filename)
                    # Move the file to the matched place in dst_dir
                    shutil.move(src_filepath, dst_filepath)
                    print(f"Moved {src_filepath} to {dst_filepath}")
                    break  # Stop search as file is moved
if __name__ == "__main__":
    src_dir = "/mnt/c/Users/Guild/Documents/4k_demo/final"
    dst_dir = "/mnt/c/Users/Guild/Documents/4k_demo/grn"
    move_matching_files(src_dir, dst_dir)</syntaxhighlight>
</div></div>
== Getting started ==
In this example, we will upscale the textures of Greenskins' trolls, including ''troll river'', ''troll river hag'', and their ''respective weapons''.
=== Step 1: Extracting Textures ===
The first step is to extract all the textures that need upscaling.
In this case, we'll focus on the ''base_colour'' and ''normal'' textures. To achieve this, we'll use the AssetEditor and apply a filter to find the desired textures: <syntaxhighlight inline lang="html">^grn_troll.*[normal|base_colour].dds$</syntaxhighlight>
Additionally, ensure that you do not export low-poly textures, as we are only interested in upscaling high-quality textures.
[[File:Step1.jpg|centre]]
=== Step 2: Centralizing Textures ===
Next, we need to centralize all the textures in the same folder or directory.
Since we're using different models for each type of texture (''normal'' and ''base_colour''), it's a good practice to keep them separate.
Below are the commands to achieve this using the shell in Ubuntu WSL:
<syntaxhighlight lang="sh">
# Create new folders to store edited textures
mkdir -p /path/to/wip/normal /path/to/wip/base_colour
# Move all the **normal** textures to the designated folder
cp **/*normal.dds /path/to/wip/normal
# Move all the **base_colour** textures to the designated folder
cp **/*base_colour.dds /path/to/wip/base_colour
</syntaxhighlight>
[[File:Step2.jpg|center]]
==== Step 3: Converting DDS to PNG ====
Now that we have all the vanilla DDS files, we need to convert them to PNG format for upscaling.
# Launch Gimp and ensure you have installed the custom plugin mentioned at the beginning of this guide.
# . In Gimp, select "Batch DDS to PNG" from the options, as shown in the image below [[File:Gimp custom plugin usage.jpg|center]]
# A pop-up window will appear. Select the folder containing the vanilla ''normal'' textures and begin the conversion process, as indicated in the image bellow [[File:Gimp custom plugin usage 2.jpg|center|850px]]
# Once the conversion is complete, you will see the message ''Batch conversion completed'' in the ''Error Console Panel'' as shown bellow [[File:Gimp custom plugin finished.jpg|center]]
# Once done with the ''normal'' textures, make sure to repeat this step for the ''base_colour'' folder to convert those textures to PNG format as well.
==== Step 4: Upscaling the textures ====
Now it's time to upscale the textures using your chosen models. Here are the models I personally recommend, but feel free to experiment or import your custom models:
* '''Normal Textures''': ''Ultramix Balanced''
* '''Base Colour Textures''': ''Fast Real-ESRGAN''
Let's get started:
# To prepare for upscaling, we need to remove or move the vanilla DDS textures to avoid conflicts. You can do this using the following commands: <syntaxhighlight lang="sh">cd /path/to/wip/ && rm normal/*dds rm base_colour/*dds</syntaxhighlight > 
[[File:Remove non texture files.jpg|center]]
# We are now ready to launch ''upscayl''.
# In the left option panel, toggle the ''batch upscayl'' option.
# Select the folder containing your ''normal'' textures.
# Choose the model you want to use (in this case, ''Ultramix Balanced'').
# Do not specify an output folder. [[File:Upscayl config .jpg|center|850px]]
# Click the purple '''Upscayl''' button
# Wait for the processing to complete. [[File:Upscayl done.jpg|center]]
# By default Upscayl will have created a new folder containing the upscaled textures [[File:Upscayl generated folder.jpg|center]]
# After completing this step, be sure to repeat the process with the ''base_colour'' folder and adjust the model accordingly.
==== Step 5: Putting the Upscaled Textures Back into the Game ====
'''Congratulations, you have successfully upscaled the textures!'''
Now, it's time to put these textures back into the game ; since GIMP doesn't support the necessary compression, we will use Paint.Net and automate the process with AutoHotKey.
# Open Paint.Net
# Open one of the ''normal'' textures (any will do)
# Click ''Save As'' or ''Ctrl+Shift+S''
# In the ''Save as type'' option, change it from ''PNG (*.png*)'' to ''Direct Draw Surface (DDS) (*.dds)'' [[File:Paint.Net export option.jpg|center|850px]]
# In the Export Configuration, make sure to select the following settings:
## '''Normal Textures''': ''BC3 (sRGB, DX 10+)''
## '''Base Colour Textures''': ''BC1 (sRGB, DX 10+)'' [[File:Paint.Net DDS export option config.jpg|center|850px]]
# Finally, click '''OK'''
Now, '''Paint.Net''' is configured to convert your ''normal'' textures, as such we can automate this process to convert all the extures using AutoHotKey:
# If you followed the installation process at the beginning of this demo, you should have a script called ''Paint.Net PNG to DDS.ahk''.
# Open that file and make the following changes
## In '''line 2''', put the path to your ''DDS normal texture folder''. Make sure to escape all backslashes as shown below.
## In '''line 32''', if your GPU is not high-end, you can increase the waiting time (7 seconds in the screenshot below).
[[File:AutoHotKey script configuration.jpg|center]]
# Once you've edited the script, open the folder containing the script
# Double-click on it
# AutoHotKey will start simulating the process of converting the upscaled PNG to DDS. '''Do not interrupt the process by using your mouse or keyboard, as the script cannot recover from interruptions.'''
# Once you are done with the ''normal textures'', you can repeat the entire ''Step 5'', but this time focus on the ''base_colour textures''. Make sure to select the ''BC1 (sRGB, DX 10+)'' compression for the ''base_colour textures'' during the conversion.
==== Step 6: Replacing Vanilla Textures with Upscaled Ones ====
At this point, you should have all the upscaled DDS textures, as shown below.
[[File:Upscaled textures as dds.jpg|center|850px]]
It is now time to replace all the vanilla textures in the exported folder (see ''Step 1'') with the upscaled ones.
However, if you used the AssetEditor to export the textures, it may have also exported other files in the folder. Let's first remove those unnecessary files using the following commands:
<syntaxhighlight lang="sh">
cd /path/to/exported/textures
find . -name 'low_poly' -type d -exec rm -rf {} \;
find . -name '*lowlod*' -delete
find . -name '*.anim' -delete
find . -name '*.bone_inv_trans_mats' -delete
find . -name '*base_colour*' -delete
find . -name '*xml.material' -delete
find . -name '*material_map.dds' -delete
find . -name 'materials' -type d -delete
find . -name '*.wsmodel' -delete
find . -name '*mask.dds' -delete
find . -name '*rigid_model_v2' -delete
</syntaxhighlight>
[[File:Delete useless exported files.jpg|center]]
Once you've completed this step, you'll need to edit the python script ''transfert.py'' as set up at the beginning of this tutorial.
Make sure to replace the following variables:
* ''line 24'': ''src_dir'': the folder where you can find the Upscaled DDS textures.
* ''line 25'': ''dst_dir'': the folder exported through the AssetEditor.
[[File:Python script to replace old textures with upscaled one.jpg|center]]
After editing the script, launch it and verify that the upscaled textures have been successfully moved.
[[File:Transfert.py script execution.jpg|center|850px]]
==== Step 7: Importing Upscaled Textures with RPFM / AssetEditor ====
Now, you're almost done! At this stage, you can use RPFM or AssetEditor to import the folder containing the Upscaled DDS textures:
# Open RPFM
# Create a new packfile
# Add the folder containing the upscaled textures [[File:Rpfm add folder demo.jpg|center]]
# You'll see the end result as shown bellow [[File:RPFM add folder done.jpg|center]]
# Save as usual
==== Step 8: Profit ====
Congratulations! You've successfully upscaled and integrated your textures back into the game. Enjoy the enhanced graphics!
If you have any more questions or need further assistance, please feel free to contact me on Discord: toumai#3164
[[File:Example of upscaled texture mod.gif|center|256px|link=https://steamcommunity.com/sharedfiles/filedetails/?id=3047798343]]

Revision as of 07:21, 1 January 2024

Retrieved from "https://tw-modding.com/index.php?title=Help:Nope&oldid=1047"