Can you delete one specific fact from an LLM?hard
Not cleanly. Because knowledge is distributed across many entangled parameters rather than stored as explicit key-value entries, there's no "delete fact" operation. Research approaches like machine unlearning and model editing (ROME, MEMIT) exist but are imperfect and can affect related knowledge.
Removing "Person X works at Company Y" may require changing weights that also encode many related facts, since the model doesn't store facts in a clean key-value format. Research directions include machine unlearning (attempting to remove training influence, still an active research area), model editing techniques like ROME or MEMIT that try to directly patch specific facts in weights, fine-tuning on examples that override the fact, and refusal tuning that trains the model to decline answering about a topic — all partial and risk side effects on related knowledge. What actually works better in production is not trying to edit weights at all: manage sensitive or changing facts in external memory, RAG, or access-controlled databases, add output filters or guardrails to block specific content before it reaches the user, and avoid relying on model weights for facts that need to be updated or deleted.