Quoted By:
Did some tests with SDXL style loras. Prodigy seems really good! Requires quite a lot of steps tho with 16 dim. Higher dim could work better but getting OOM so might switch to adafactor to see if the memory requirements are lower. Scale Weight Norms also adds training time but seems to perform way better with character loras so might be worth it. Some strangeness with higher lora weights like mutated hands but requires more testing with other epochs to see if it's only an outlier.
