>>67166782Ah, I see. That might be where your problem is coming from? Some (all?) of the upscalers you use with a Load Upscale Model node have a certain amount of sizing built into them already. If I feed a 1024x1024 first pass image into the arrangement in picrel, it becomes 4096x4096. I guess it makes sense since the upscaler does have 4x in the name, after all. But the point is, if I then also use an Upscale Image By node, I better put the scale factor below 1.0 or the image will become super large and when I try to convert it into latent and sample again there is no way I will have enough VRAM for it.
When I do a process like you do, I tend to scale it down to 0.4 or 0.5 before encoding back into latent. You should do a test with just your upscaler to see how much it resizes it without you actually specifying anything
