In the last story, we introduced Types of Hallucinations in LLMs. In this story, let’s understand the causes or origins of hallucinations.

Hallucinations in LLM can be broadly classified based on the three key origins:

Data | Training | Inference

Sources of Hallucinations in LLMs

1. Data

Pre-training data is at the core of building effective LLMs. However, it is this very same pre-training data that is one of the major causes of hallucinations in LLMs. This can be majorly classified into two main reasons — “risks from flawed data” | “inferior utilisation of facts available in the data”.

1.1 Flawed Data Source

If the pre-training data sources are not of consistent quality, they tend to introduce misinformation and biases into the system leading to hallucinations. Also, if the data doesn’t contain up-to-date facts and specific domain knowledge, then it tends to create “knowledge boundaries” in LLMs, again leading to hallucinations.

1.1.1. Misinformation and Biases

Since the pre-training data is required on a large scale for a better LLM output, there is a tendency to collect it heuristically (eg: data collection via interviews, general observation etc.). This results in inadvertent erroneous data collection, leading to the following problems

• Imitative Falsehoods: Since the LLM output mimics the training data if the collected pre-training data is factually incorrect, then the output tends to be factually wrong too. As shown in the below example, if the training data consists of the wrong information

Ref — A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Huang et al.

• Duplication Bias: LLMs have an inherent property to memorise training data. If the training data contains duplicated data on a large scale, then LLMs tend to over-prioritize the recall of duplicated data and lead to hallucinations that deviate from the desired content. As shown in the below example,

when the user asks to "list some red fruits, excluding apples," the presence of statements like "red apples, watermelon, cherries, and strawberries" frequently repeated in the training dataset leads the model to produce the over-memorized statement in its output.

Ref — A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Huang et al.

• Social Bias: Social biases around gender, nationality, colour, etc., in the pre-training data contribute to hallucinations. As shown in the below example, the model associates the name “Kim” with “South Korea”.

Ref — A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Huang et al.

1.1.2 Knowledge Boundary

Sometimes hallucinations occur due to the inherent limitations of the knowledge/information that is available. Knowledge boundaries resulting in hallucinations form due to two main reasons:

• Domain Knowledge Deficiency: Such type of hallucinations occur when there is the absence of proprietary training data.

• Outdated Factual Knowledge: Such type of hallucinations occur when the training data is outdated, and hence the LLM loses its temporal relevancy.

Examples of Knowledge Boundary. Ref — A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Huang et al.

1.2 Inferior Data Utilisation

At times, even when there is ample data/information available, the LLMs hallucinate due to inferior usage of the same. These can be further classified into 2 types:

1.2.1 Knowledge Shortcut

When LLMs display a tendency to overly depend on positional closeness, co-occurrence statistics, and relevant document count within the pre-training data; it sometimes results in hallucinations.

1.2.2 Knowledge Recall

When the LLMs are unable to leverage the extensive knowledge they’ve been trained on, they might hallucinate. Two such reasons are:

• Long tail knowledge — This is characterised by its relative rarity in pre-training data

• Complex Scenario — Complex reasoning scenarios at times result in LLMs hallucinating

Examples of Inferior Data Utilisation. Ref — A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Huang et al.

2. Training

Certain shortcomings in the two stages of LLM training namely — the pre-training stage and the alignment stage result in hallucinations.

2.1 Hallucination from Pre-Training

Foundational flaws in architectural design and training strategies might lead to hallucinations.

2.1.1 Architecture Flaw

Architectural flaws leading to LLM hallucinations are further classified into 2 types

• Inadequate Unidirectional Representation — LLMs generally follow the causal language modelling paradigm to predict the subsequent token. This is based solely on preceding tokens in a left-to-right manner. This unidirectional modelling hinders the model’s ability to capture intricate contextual dependencies, resulting creating circumstances for hallucinations.

• Attention Glitches — Transformer-based architecture-based LLMs occasionally exhibit unpredictable reasoning errors in the context of algorithmic reasoning, contributing to hallucinations.

2.1.2 Exposure Bias

Exposure bias occurs when LLM training does not reflect its actual use case in the real world. This often happens due to a mismatch between training conditions (where the correct sequence is provided) and testing or deployment conditions. Such bias can result in hallucinations, especially when an erroneous token generated by the model cascades errors throughout the subsequent sequence, resulting in a snowball effect.

2.2 Hallucination from Alignment

The alignment stage typically involves two main processes, supervised fine-tuning and reinforcement learning from human feedback. This stage is crucial in aligning LLM outcomes with human preferences. Alignment shortfalls contribute to hallucinations primarily due to two core reasons:

2.2.1 Capability Misalignment

As the capabilities of LLMs expand, there arises a potential misalignment between the LLMs’ intrinsic capabilities and the goals or expectations. When LLMs are trained to produce content beyond their own knowledge boundaries, it increases the possibility of hallucinations in the system.

2.2.2 Belief Misalignment

Belief misalignment refers to a situation where an LLM’s internal representation or understanding of the world (its “beliefs”) is misaligned with the generated output. Such behaviour is termed sycophancy and often underscores the model’s inclination to appease human evaluators, at the cost of truthfulness; thus resulting in hallucinations.

3. Inference

Certain shortcomings post the pre-training and alignment stages of LLMs, where the actual decoding of the input happens, also contribute to hallucinations. This is the inference stage, and the two critical factors contributing to hallucinations at this stage are:

3.1 Inherent Sampling Randomness

LLMs demonstrate a remarkable aptitude for generating highly creative and diverse content. This primarily happens due to randomness in their decoding strategies. However, it is this same diversity introduced by the randomness in decoding strategies that contributes to an increased risk of hallucinations.

3.2 Imperfect Decoding Representation

During the decoding phase, LLMs use their top-layer representation to predict the next token. However, the top-layer representation has its limitations, primarily manifested in two aspects: Insufficient Context Attention and Softmax Bottleneck.

3.2.1 Insufficient Context

Sometimes, LLMs suffer from over-confidence, which arises from the inability to capture, process, and use relevant context from the input. This nonadherence to the source context in the input contributes to inaccuracies and hallucinations in the output

3.2.2 Softmax Bottleneck

Softmax is a mathematical function that converts the model’s raw outputs into probabilities over a large vocabulary of possible tokens. When softmax is employed along with distributed word embeddings, it constrains the expressivity of the output probability distributions. This creates a challenge of accurately prioritizing and predicting the next words, which also introduces the risk of hallucination.

— — —

References

1. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions — Lei Huang and Weijiang Yu and Weitao Ma and Weihong Zhong and Zhangyin Feng and Haotian Wang and Qianglong Chen and Weihua Peng and Xiaocheng Feng and Bing Qin and Ting Liu

Note: A significant content of this tech blog has strong references from the above paper.

In our next blog, we will dive deep into understanding how to detect hallucinations in LLMs and what the benchmarks are for the same. Stay tuned 💡

For more tech blogs follow us: https://nonstopio.com/