Quoted By:
>doing something completely unrelated to ML in general
watching uncle dane the pyro main on the you tubes
>tism starts randomly thinking of pyg generating short messages
uh
matrixbros i dont know if yall still lurk here but couldnt yall just
>add a negative logit bias for <END_OF_MESSAGE> token
?
yes i know kobold doesnt 'have' the ability to change logit biases (atleast directly, you can do this through soft prompts) like OAI does but same shit
>manually initialize the <END_OF_MESSAGE> token bias at a decent negative number at training start
yes i also know
>its not that easy
just feed in [...0,0,0,-50,0,0...] as init biases and let RNJesus take the wheel