>>52510592 good morning I should really upgrade my model its been months
>>52509196Thanks for your work. I will give this a try
I think local text ai is in its early nai leak days. I looked into a bit yesterday. 4bit and soon 3bit quantization and now Exlama. Models now can have 8k token size instead of 2k. I tried a 13B model and in run with less then 10GB vram and outputs in 2 seconds.
