>>73941729Huh? They have 400B in the works. That isn't an issue but they are claiming GPT-4 performance which is 250Bx8 MOE for 1.8T total. That is a good improvement.
>>73943547Context is now 128k. I will post a screenshot of a Meta employee who summarized the changes in a linked post.