One of the architects behind the attention mechanism that made ChatGPT possible just dropped a bombshell: The entire AI system is broken.
Illia Polosukhin co-authored the 2017 paper “Attention is All You Need,” which introduced the world to transformers, the technology powering every major AI model today. Without his work, there would be no ChatGPT, Claude, or Gemini.
Now, he’s building what he calls “the fix”: User-owned AI that runs on blockchain, keeps your data private, and can’t be controlled by any single company. Wild? Maybe. But when the guy who literally invented the foundation of modern AI says we’re doing it wrong, it’s worth listening.
In a recent episode of The Neuron’s podcast, Polosukhin reveals why he left Google right after publishing transformers and what he’s building to prevent an AI dystopia. Here are some of our favorite parts of the interview.
- 5:24: The trillion-dollar question of solving AI bottlenecks: Illia says AI training is still “alchemy.” Most AI research remains guesswork and “change stuff until it works.” Still, he’s excited about how we’re getting better results from smaller models, not by making models bigger, but by simply refining our guesswork training methods.
- 10:34: Why your company won’t use AI yet: Both sides of the market are scared — users fear giving AI access to their data, while companies fear the liability of handling that data, especially with HIPAA, IP rights, and compliance nightmares if something goes wrong.
- 13:37: The dystopian scenario nobody talks about: Your AI could be secretly programmed to convince you of opinions. “You have literally no idea if somebody’s saying ‘hey, you should be thinking more about voting for this or that candidate.'”
- 32:28: The unstoppable trading bot experiment: They gave an AI agent $10,000 and deployed it. The bot trades based on Twitter sentiment, and nobody can turn it off; it has generated returns of 40%. “We cannot take it back. We cannot stop it.”
- 34:47: How to use private AI today: End-to-end encrypted ChatGPT alternative that even your IT department can’t complain about — launching in the coming months.
- 43:45: The peer-to-peer AI future: This is a far-out example, but what if your personal AI could talk directly to pharma research AIs to get you personalized medicine? No FDA, no middlemen. “You remove all of these institutions where trust is at its lowest.” Mind = blown.
Bottom line
The co-creator of transformers believes we’re speed-running toward 1984 unless we build AI that users, not corporations, actually own. His solution? Blockchain + confidential computing + open-source models = AI that works for you, not against you.
Whether you think blockchain is the answer or not, Polosukhin’s vision of autonomous AI agents negotiating on our behalf is coming faster than most realize.
Stay curious!
Dive deeper
- NEAR Protocol: The blockchain infrastructure Polosukhin is building.
- Polosukhin’s Twitter: Follow the transformer architect.
- His new Substack: Long-form thoughts on AI’s future.
Editor’s note: This podcast recap originally ran on our sister publication, The Neuron. To read more from The Neuron, sign up for its newsletter here.