Source

Branch

1/ @sprloveless

I have a somewhat similar idea. The model can forget alright, it’s just that the benchmarks for context/memory handling are a mess right now. I don’t know why a model should remember some trivial facts from nowhere while such capacity should be reserved for reasoning. Hallucinations are a feature, not a bug.