>>35711069Try reading the guide, if you havent yet
https://rentry.org/hypernetwork4dumdumsAlso did you use new options, like layer normalization?
From my testing, with layer normalization on it seems to never do anything. With normalization off and "relu" - decent results after 20k steps, with normalization off and "linear" - decent after 10k steps. I used training rate 5e-6:8000, 2e-6