Source
- URL: https://x.com/asefweca/status/2078153493148528732
- Author: asefweca (@asefweca)
- Posted: 2026-07-17 16:22:57
Branch
1/ @asefweca
@VictorTaelin
I think they already do erasure a lot, on every token. The MLP deletes information because it’s not a bijective function. The token then supplied with fresh information from its neighbors through attention