Quoted By:
So I have a new theory of why the embed has to be put in front of the prompt list in order to work.
I think its because the other embeds already kind of know the characters somehow and part of their weights gets put into the embed allowing it to work better with other prompts.
But these are just so recent that the model has no weights at all to go on so even prompts like best quality and masterpiece messes with it to much unless it is put at the beginning of the prompt list.