LLMs suffer from the 'reversal curse' where they know 'Tom Cruise's mother is Mary Lee Pfeiffer' but fail to infer 'Mary Lee Pfeiffer's son is Tom Cruise'. This reveals a fundamental limitation in their reasoning ability.
A phenomenon called 'The Reversal Curse' has been discovered, where LLMs fail to infer reverse relationships even after learning a directional fact. For example, if an LLM learns that 'Tom Cruise's mother is Mary Lee Pfeiffer,' it cannot deduce that 'Mary Lee Pfeiffer's son is Tom Cruise.' Researchers confirmed this across multiple models including GPT-3, GPT-4, and LLaMA.
Previous studies evaluated LLMs' ability to store and reason with knowledge, but the symmetric reasoning of directional relationships was not highlighted. This research systematically demonstrates that LLMs learn unidirectional relationships and fail at reverse inference.
This phenomenon suggests that LLMs do not perform logical reasoning like humans but rely on patterns in training data. It raises the need for symmetric learning of directional relationships to improve future models.