The $1.30 Video: Why Transformers are a Digital Dead End
We have been told that Sora and the big AI models are the pinnacle of intelligence. But if you look at the energy bill, the truth is a lot more embarrassing. We are basically trying to brute force ...

Source: DEV Community
We have been told that Sora and the big AI models are the pinnacle of intelligence. But if you look at the energy bill, the truth is a lot more embarrassing. We are basically trying to brute force "thought" using the digital equivalent of tin cans and a very expensive string. The Quadratic Trap The Transformer architecture is the engine behind everything from ChatGPT to Sora. It has a massive math flaw called quadratic complexity. To make a short video, every "patch" of pixels has to talk to every other patch. If you double the detail, you do not double the work. You quadruple it. This is why OpenAI reportedly burns over a dollar in electricity and hardware wear for a single ten second clip. It isn't actually "smart." It is just a massive, high speed game of telephone where everyone is shouting at once. The Brute Force Wall Compare this to a human brain. Your brain runs on about 20 watts. That is less power than a dim porch light. Your brain is efficient because it uses sparsity. Your