>>65625364DPO used Pick-a-pick v2 which is a dataset resulted from the scoring of a million images.
I picked a random image from their dataset and I do not agree with what their pick was.
Anime girl on a baroque style chair, long red hair, with piercing blue eyes, extremely light realistic skin all in a dark place
Their pick: https://files.catbox.moe/8u6m5v.png
The other option: https://files.catbox.moe/d0p6jq.png
You can observe how the picker didn't give a shit about the 'Anime girl' part of the prompt.
As for the dataset, you could make one yourself. Do batches of 4 on the same prompt and pick only one. Collect 500k of them and then finetune the model with them.
>>65625347It was made by wildcards so if you see anything weird there, don't blame it on me.
https://files.catbox.moe/wz7u29.jpg 