Leading AI results can be achieved through clever engineering, not just scaling up training.
I am tracking a newly published paper titled “The Illusion of Deep Learning Architectures” that I consider "new gold in AI!" and could accelerate a shift from transformer models to more traditional AI architectures.
The paper might push AI transformers into traditional AI architectures sooner.
"Nested learning" could be the new transformers.
I plan to monitor the paper’s arguments and evidence, and explore how "Nested learning" differs technically from existing transformer mechanisms.