>>36827113>be able to run at home>needs the power of several Nvidea A100 to workAnon I don't think you get it how much horsepower this needs
If this were in the nineties, and we were still on Voodoo2 GPUs, you would need the power jump equivalent of a 1080ti at least to run this on a single GPU device
The Nvidia A100, the only card they are probably using it, costs a fuckton and one alone although more powerful than a 3090, it is not by that more powerful, you need to pair it with at least 3 A100 (a cluster of 4) so the AI can actually work well
This is all because to run these models you need not only beefy as fuck GPUs, but also loads and loads of VRAM, something that gaming GPUs don't come with usually
The biggest of the 30s cards, the 3090, can be maxed out with 24GB, and a single A100 has 40GB, considering most AI models start to benefit from A100s once you have 4 or more of them, that means that you need 160GB AT LEAST
>s-so lets just build rigs like mining rigs that have a shit ton of gaming GPUsThat is an idea, yes, but a very inefficient one, gaming GPUs are best at rendering graphics and have been tuned to work as single devices, not sharing its load well (look how dual GPU systems have become less and less viable in the last few gens), meanwhile the A100, a render server GPU is made to share the fuck of its resources with other A100s
We won't see any chances of this becoming able to be run at home unless AI accelerator hardware becomes the norm (Just like 2D Graphics accelerator cards did in the 80s, and then 3D Graphics accelerator cards did in the 90s and recently, Ray tracing accelerator components becoming popular in GPUs)
But rejoice, the way the future looks like to be AI-powered <Insert whatever you want here>, we will probably start seeing AI accelerator hardware on the next-gen GPUs
Nvidia is already toying with AI-powered real-time up resing and interpolation on their most powerful cards, running the AI on the Ray Tracing cores (that sit mostly unused because RayTracing was a big wet fart that everyone disables if they can get a few more FPS)