@VictorTaelin Agree, but also worth noting: Attention mechanism obviously learns erasure - as the query-answer mechanism figures what’s relevant and what’s not.

Source

• URL: https://x.com/ozataman/status/2078157738723401873
• Author: Ozgun Ataman (@ozataman)
• Posted: 2026-07-17 16:39:49

Branch

1/ @ozataman

@VictorTaelin

Agree, but also worth noting: Attention mechanism obviously learns erasure - as the query-answer mechanism figures what’s relevant and what’s not. Compaction is the next layer of erasure. Your point is to repeat that at meta levels of granularity. “Context rot”, basically. 👍

Related

• Spine: 2026-07-17-read-this-and-your-next-model-will-be-10x-smarter