Quoted By:
A few threads ago I mentioned that NAI has a different training method for characters, it seems I was correct and kohya also referred to it in the Finetune trainer documentation.
They might have improved it now and they are using a new base model but trying their old method shows some nice results.
What their method does is instead of training a lora, they finetune the whole model on a set of images. Then the lora can be extracted from the finetuned model by making a difference with the base one used for training.
Weirdly, even with my very bad settings which overcooked the model after 8/20 epochs the results look promising, with the character learned well and being flexible. I need to test it some more but this looks nice 1st try.