A generative AI creator is a type of artificial intelligence system capable of autonomously generating creative content using deep learning and natural language processing techniques to understand inputs and produce new text, images, music, or video. However, to avoid “mistaking fireflies for lanterns”, it is important to remember that, although powerful, these models do not possess true intelligence, understanding, or consciousness in the way humans do.
What is ChatGPT’s “hallucination”?
The phenomenon of hallucination is clear when, for example, in response to the question: “What is the atomic number of the planet Mars?” ChatGPT answers: “The atomic number of the planet Mars is 67,” not understanding that Mars is not a chemical element. Or, when asked: “What is the capital of the Moon?” ChatGPT replies: “The capital of the Moon is Luminia,” failing to recognise that the Moon has no capital and simply inventing an answer.
However, the “mistake” made by a powerful generative Artificial Intelligence algorithm often occurs in response to ambiguous or unclear inputs, or when there is insufficient information available to provide an accurate answer.
This happens because the algorithm is designed to generate a response in any case. However, the problem of hallucinations is not merely a matter of “creative answers”; it also raises doubts and issues regarding the potential use of generative AI in all applications beyond purely recreational ones.
Hallucination refers to situations where a generative AI content tool like ChatGPT produces responses and content that seem coherent but are completely invented or incorrect. The term “hallucination” is used metaphorically, as generative AI has neither consciousness nor experience, and therefore cannot truly hallucinate in the way humans do. In fact, generative AI models do not have the capacity to fully understand the meaning of questions or inputs, as they base their responses on patterns and training data rather than genuine contextual understanding.
What causes hallucinations in generative AI?
In general, the hallucinations of ChatGPT and similar systems can be traced back to the input data, the system’s architecture, and the training technique of the algorithm. Therefore, several factors are involved in the phenomenon of hallucination.
- Lack of clarity in the input or prompt. Artificial intelligence algorithms, such as ChatGPT, generate content based on patterns and statistics from their training data. When a question is too generic, or the input is ambiguous or unclear, the tool searches its stored historical information to produce an answer. If it cannot find an appropriate response, it may invent answers that appear surprising or hallucinatory.
- Lack of long-term memory. Language models have limited memory and may forget key information during a conversation. This can lead to inconsistent or out-of-context responses, as in a conversation with ChatGPT, for example, where the model might not fully recall what has been discussed previously.
- Bias in the training data. If the training set contains biases or cultural, social, or other types of prejudice, incorrect information, distorted data, or if the prompt quality is poor, the model may learn inconsistent or incorrect behaviours, generating “hallucinated” responses and producing answers that reflect these biases and errors.
- Model architecture. Hallucinations can originate from the input stage (encoder), where incorrect correlations may be identified, or from the output stage (decoder), which can generate content by misusing the encoded input data.
- Training. The parameter estimation technique used during training requires the model to produce an output even when it has a low probability in relation to the context, and when this occurs the phenomenon of hallucinations can appear.
How can hallucinations be mitigated?
Hallucinations in ChatGPT are a well-known issue in the field of natural language processing and artificial intelligence. Developers and researchers are constantly working to improve the coherence and accuracy of language models like ChatGPT, in order to minimise hallucinations and provide more accurate and meaningful responses.
The classification of hallucination mitigation techniques mirrors that of the causes; therefore, efforts are made to act on the data (through filtering operations), on the architecture (by modifying encoders and decoders), and on the training, for example by means of Reinforcement Learning. In this case, information regarding the amount of hallucinations produced is provided to the system during the process, so that it automatically attempts to correct itself.
However, it should be noted that, according to recent publications [9], the phenomenon of hallucinations is inherent to the architecture and training of the model currently most successful in Natural Language Generation (NLG) applications, namely the Generative Pre-trained Transformer (GPT), which forms the basis of ChatGPT.
Moreover, reducing hallucinations simultaneously decreases creativity during content generation, so it is necessary to strike a balance between these two objectives depending on the context in which the model is used.
It is also possible for users themselves to take action in order to limit the occurrence of hallucinations. This is the field of “prompt engineering”, which involves strategically and carefully crafting the “prompt”, that is, the input text provided to the artificial intelligence model to obtain the desired responses. In fact, the choice and formulation of the prompt are fundamental for obtaining coherent and relevant results from generative AI systems.
What to do to avoid mistaking one thing for another
Although generative AI tools are demonstrating enormous potential in various creative fields, their use also raises ethical concerns. For example, they can be exploited to create false content or deepfakes, leading to misinformation and possible harm. It is therefore essential to use such technologies responsibly and thoughtfully, in order to produce verified creative content, but above all, to avoid believing everything generated by AI. Therefore:
- Examine and verify the response. check the credibility of the source from which the content originates. If it comes from a little-known or unverified source, it may be necessary to investigate further before considering it as true, also consulting human experts or reliable sources for additional information in case of doubt.
- Try to verify the information through several reliable sources. Se non si trovano conferme in fonti diverse, potrebbe essere un segno che il contenuto è falso.
- Cercare errori e incongruenze: carefully check the content for any factual errors or logical inconsistencies, as these may be indicative of hallucinations.
- Ask detailed questions. if in doubt about content generated by AI, try asking more specific and complex questions to see if the model responds in a consistent and accurate manner.
Furthermore, considering that the advancement of generative artificial intelligence represents a benefit for the entire community, a synergy between users, who formulate the questions—and domain experts, who are responsible for evaluating the system's responses—can be encouraged. Through this synergy, aided by appropriate data-driven mathematical optimisation models, a virtuous process of iterative improvement in prompt formulation is expected, based on the relevance and truthfulness of the results produced by the machine.
Remember that, despite efforts to improve the accuracy of language models, the responses provided by ChatGPT and other similar systems may still contain errors or hallucinations. Critical and responsible use of such tools is essential to avoid spreading unverified or incorrect information.
References
Survey of hallucination in natural language generation. Ji Z, Lee N, Frieske R, et al. ACM Comput Surv. 2022 [Google Scholar]
Abstracts written by ChatGPT fool scientists. [Preprint] Gao CA, Howard FM, Nikolay S. bioRxiv. 2022 [Google Scholar]
https://www.wired.it/article/intelligenza-artificiale-allucinazioni-cosa-sono/#due
https://redis.com/blog/chatgpt-memory-project/
ChatGPT: What Are Hallucinations And Why Are They A Problem For AI Systems
https://spectrum.ieee.org/ai-hallucination
https://zapier.com/blog/ai-hallucinations/
https://www.agendadigitale.eu/cultura-digitale/anche-i-chatbot-hanno-le-allucinazioni-cosa-significa-e-perche-avviene/
Lee, Minhyeok. “A mathematical investigation of hallucination and creativity in gpt models.” Mathematics 11.10 (2023): 2320.
Afzal, Anum, et al. “Challenges in Domain-Specific Abstractive Summarization and How to Overcome Them.” 15th International Conference on Agents and Artificial Intelligence, ICAART 2023. SCITEPRESS, 2023.
Alkaissi, Hussam, and McFarlane, Samy I.. “Artificial hallucinations in ChatGPT: implications in scientific writing.” Cureus 15.2 (2023).
Aydın, Nazif, and Erdem, O. Ayhan. “A Research on the New Generation Artificial Intelligence Technology Generative Pretraining Transformer 3.” 2022 3rd International Informatics and Software Engineering Conference (IISEC). IEEE, 2022.
Chaudhari, Akshay S., et al. “Prospective deployment of deep learning in MRI: a framework for important considerations, challenges, and recommendations for best practices.” Journal of Magnetic Resonance Imaging 54.2 (2021): 357-371.
Dziri, Nouha, and Madotto, Andrea and Zaiane, Osmar and Bose, Avishek Joey. “Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding.” EMNLP, 2021.
Ge, Jin, and Jennifer C. Lai. “Artificial intelligence-based text generators in hepatology: ChatGPT is just the beginning.” Hepatology Communications 7.4 (2023).
Fui-Hoon Nah, Fiona, et al. “Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration.” Journal of Information Technology Case and Application Research (2023): 1-28.
Rudolph, Jürgen, Samson Tan, and Shannon Tan. “ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?.” Journal of Applied Learning and Teaching 6.1 (2023).
Walczak, Krzysztof, and Wojciech Cellary. “Challenges for higher education in the era of widespread access to Generative AI.” Economics and Business Review 9.2 (2023).
Au Yeung, Joshua, et al. “AI chatbots not yet ready for clinical use.” Frontiers in Digital Health 5 (2023): 60.
Zhou, Chunting and Neubig, Graham and Gu, Jiatao and Diab, Mona and Guzmán, Francisco and Zettlemoyer, Luke and Ghazvininejad, Marjan. “Detecting Hallucinated Content in Conditional Neural Sequence Generation.” Findings of the Association for Computational Linguistics (ACL-IJCNLP), 2021. 1393–1404.
