Aspect ratio alignment options for less-sensitive models

[InResponse]

Feature Idea

Even a little stretching or squeezing due to an aspect ratio mismatch annoys me endlessly. I've found myself constantly using the custom resolution field just to get aspect ratios that are listed in the dropdown. So I have to ask:

Could we have the option to use aspect ratios that (a) are actually what they are labeled as and (b) approximate the 1:1 pixel count for a given side length?

I'd happily make a PR, but I didn't want to bother if there was a reason you chose to go this direction rather than another. In which case, I'm content to stick with patches on my personal machines.

Other

The resolution for 3:2 (for a 512x512 model) is set to 608x416, but that's not really 3:2, is it? Valid resolutions for that aspect ratio (and near that pixel count) would be 600x400, 624x416, or 648x432. Except for 1:1, all predefined aspect ratios show similar stretching/squeezing.

predefined ratios in params.js and T2IParamInput.cs

["1:1"] = (512, 512)

["4:3"] = (576, 448)
["3:4"] = (448, 576)

["3:2"] = (608, 416)
["2:3"] = (416, 608)

["8:5"] = (608, 384)
["5:8"] = (384, 608)

["16:9"] = (672, 384)
["9:16"] = (384, 672)

["21:9"] = (768, 320)
["9:21"] = (320, 768)

After some digging, I realized that most of these dimensions are exactly half those of certain image resolutions present in the SDXL training set for mixed–aspect ratio finetuning according to their paper.

• 512x512 -> 1024x1024
• 576x448 -> 1152x896
• 416x608 -> 1216x832
• 672x384 -> 1344x768
• 768x320 -> 1536x640.

(8:5/5:8 is the outlier.) Is the idea to match the SDXL training resolutions rather than to give the resolutions that come closest to the model's trained pixel count? If so, could we possibly have the option for the latter? That is, a 1024x1024 model's standard pixel count is 2^20, so selecting a 2:3 aspect ratio when using that model would result in a resolution of 832x1248, as that is the 2:3 aspect ratio resolution with pixel count closest to 2^20.

I tested this idea on my own machines by making the following changes. They're working for me with no apparent issues across SD 1.5, SDXL, Hunyuan, and Wan.

Why I didn't hardcode dimensions

Merely doubling the dimensions of the resolutions for a 512x512 model doesn't always result in the correct dimensions for a 1024x1024 model, never mind for 960x960 or something. For example, the optimal 16x9 resolution for 512 is 768x432, but the optimal resolution for 1024 is 1280x720. (Assuming that dimensions must be multiples of 16. Values change if only restricting to multiples of 8.)

AR	SideLen	Multiple	Resolution
4:3	512	16	576x432
3:2	512	16	624x416
8:5	512	16	640x400
16:9	512	16	768x432
21:9	512	16	672x288
4:3	1024	16	1152x864
3:2	1024	16	1248x832
8:5	1024	16	1280x800
16:9	1024	16	1280x720
21:9	1024	16	1680x720
4:3	512	8	576x432
3:2	512	8	624x416
8:5	512	8	640x400
16:9	512	8	640x360
21:9	512	8	840x360
4:3	1024	8	1184x888
3:2	1024	8	1248x832
8:5	1024	8	1280x800
16:9	1024	8	1408x792
21:9	1024	8	1512x648

Tested code changes

Note: This implementation is more verbose than necessary in order to make it clear what's going on while minimizing the total scope of alteration.

In params.js:

class AspectRatio {
    constructor(id, width, height, altLogic = null) {
        this.id = id;
        this.width = width;
        this.height = height;
        this.ratio = width / height;
        this.altLogic = altLogic;
    }

    read(inWidth, inHeight, doAltLogic = true) {
        // this logic assumes the aspect ratio is given in the form of a reduced fraction. It can break otherwise.
        let depth = inWidth * inHeight;
        let multiple = 16; // require that dimensions be multiples of this
        let factor = Math.round(Math.sqrt( depth / (multiple ** 2 * this.width * this.height) ));
        let width = this.width * multiple * factor;
        let height = this.height * multiple * factor;
        return [width, height];
    }
}

let aspectRatios = [
    new AspectRatio("1:1",1,1),

    new AspectRatio("4:3",4,3),
    new AspectRatio("3:2",3,2),
    new AspectRatio("16:9",16,9),
    new AspectRatio("8:5",8,5),
    new AspectRatio("21:9",7,3),

    new AspectRatio("3:4",3,4),
    new AspectRatio("2:3",2,3),
    new AspectRatio("9:16",9,16),
    new AspectRatio("5:8",5,8),
    new AspectRatio("9:21",3,7)
];

In T2IParamInput.cs:

    /// <summary>Reference sheet of 512x512 aspect ratio approximations for custom Aspect Ratio selection.</summary>
    public static Dictionary<string, (int, int)> ResolutionAspectReferences = new()
    {
        ["1:1"] = (1,1),
        ["4:3"] = (4,3),
        ["3:4"] = (3,4),
        ["3:2"] = (3,2),
        ["2:3"] = (2,3),
        ["8:5"] = (8,5),
        ["5:8"] = (5,8),
        ["16:9"] = (16,9),
        ["9:16"] = (9,16),
        ["21:9"] = (7,3),
        ["9:21"] = (3,7)
    };

    /// <summary>Gets the desired image width.</summary>
    public int GetImageWidth(int def = 512)
    {
        if (TryGet(T2IParamTypes.RawResolution, out string res))
        {
            return int.Parse(res.Before('x'));
        }
        if (TryGet(T2IParamTypes.SideLength, out int sideLen) && TryGet(T2IParamTypes.AspectRatio, out string aspect) && ResolutionAspectReferences.TryGetValue(aspect, out (int, int) resRef))
        {
            // NOTE: This math must match params.js AspectRatio
            // return (int)Utilities.RoundToPrecision(resRef.Item1 * (sideLen / 512.0), 16);
            int depth = System.Math.Pow(sideLen, 2);
            int multiple = 16; // require that dimensions be multiples of this
            int factor = System.Math.Round(System.Math.Sqrt(depth / (System.Math.Pow(multiple, 2) * resRef.Item1 * resRef.Item2)));
            return multiple * factor * resRef.Item1;
        }
        return Get(T2IParamTypes.Width, def);
    }

    /// <summary>Gets the desired image height, automatically using alt-res parameter if needed.</summary>
    public int GetImageHeight(int def = 512)
    {
        if (TryGet(T2IParamTypes.RawResolution, out string res))
        {
            return int.Parse(res.After('x'));
        }
        if (TryGet(T2IParamTypes.AltResolutionHeightMult, out double val) && TryGet(T2IParamTypes.Width, out int width))
        {
            return (int)(val * width);
        }
        if (TryGet(T2IParamTypes.SideLength, out int sideLen) && TryGet(T2IParamTypes.AspectRatio, out string aspect) && ResolutionAspectReferences.TryGetValue(aspect, out (int, int) resRef))
        {
            // return (int)Utilities.RoundToPrecision(resRef.Item2 * (sideLen / 512.0), 16);
            int depth = System.Math.Pow(sideLen, 2);
            int multiple = 16; // require that dimensions be multiples of this
            int factor = System.Math.Round(System.Math.Sqrt(depth / (System.Math.Pow(multiple, 2) * resRef.Item1 * resRef.Item2)));
            return multiple * factor * resRef.Item2;
        }
        return Get(T2IParamTypes.Height, def);
    }

[mcmonkey4eva]

Several models actually target x64, is why the rounding is that way - SDXL especially, that's how they calculated their sizes, but some later models too. They can work off-scale but will misbehave, as the reason they work is only comfy adding special compensation code to jank tighter resolutions into the model.

The latest models (eg Wan) have much more generalized resolution capabilities (I think at least x32 is frequently preferred still? x16 is the bare minimum where a lot of code fully breaks, x8 is mandatory for most VAEs).

There's a few model-specific res behavior changes, notably a few for the models like wan that have a precise 720 definition in their trained resolutions, but there's a fair argument that perhaps the code for newer models should be indeed set to true full close-as-possible-to-correct.

[InResponse]

Multiples of 64? That makes for some rather huge jumps with certain ratios. If different models assume different multiples, however, it shouldn't be difficult to calculate that programmatically if we know what any given base model targets. I mean, a new GetModelScale() method returning the proper target for a given model (or custom value like SideLength) could easily set the value of multiple in my sample.

The 608x416 resolution doesn't look like an x16/x32/x64 rounding issue, though. It looks suspiciously more like there was an off-by-one error somewhere in someone's scripts when prepping the training set, because 1248x832 works at x16, and the 3:2 x64 ideal would have been 1344x896. Really weird choices they made.

I'll freely admit that this is all a minor quibble, all things considered, but I'll also freely admit that I get disproportionately fixated on minor things. I really appreciate and am truly grateful for all your work on this project!

[AshtakaOOf]

Maybe a user setting to offer an option to choose which multiplier to use ?