It’s a rarely discussed but worrying topic: Peak Data.
The term “Peak Data” refers to the point when the availability of real-world data to train artificial intelligence models reaches its maximum, after which it becomes increasingly difficult to acquire new meaningful data.
This idea was recently highlighted by Elon Musk, who expressed concerns about the future of AI training.
Definition of “Peak Data”
“Peak Data” therefore refers to the point at which the data available for training AI models starts to dwindle.
According to Elon Musk, founder of Grok, this phenomenon is imminent, because almost all accessible human knowledge has already been used to train AI. In other words, we have “exhausted” all real-world data that can be used to train AI models.
Elon Musk’s statements about Peak Data
During a live broadcast on his own social network, Elon Musk said: ” We have essentially exhausted the sum total of human knowledge in AI training. That happened mostly last year. ”
Because of this so-called Peak Data, Elon Musk stressed the need for tech companies to turn to “synthetic data.” This data is generated by AI itself, which could lead to self-learning processes.
What is “synthetic data” in AI?
“Synthetic data” in artificial intelligence refers to information generated by AI models rather than coming from real-world observations. This data can be created using algorithms that simulate situations, behaviors, or interactions, making it possible to supplement or enrich existing datasets.
Using synthetic data is especially relevant in fields where real data is limited or hard to obtain, such as medicine, self-driving, or image recognition. AI could then learn on its own, autonomously.
But with “hallucination” risks!
However, Elon Musk also warned about the risks associated with using synthetic data, notably the phenomenon of “hallucination” in AI. Hallucinations refer to chatbots’ ability to make things up or produce incorrect information, which is a major challenge in the quest for reliable AI models.
“Hallucinations” are the Achilles’ heel of generative artificial intelligence. They are the main technological criticism leveled against it, with users being especially watchful of every error produced by these systems.
These hallucinations could lead to a loss of trust in these tools, sparking disapproval among internet users… and even the end of AI’s popularity!
Also read on this topic: Is ChatGPT reliable? 5 crucial points to watch closely
Peak Data and synthetic data: The end of generative AI?
Andrew Duncan, Director of Fundamental AI at the London-based Alan Turing Institute in the UK, confirmed the concerns raised by Elon Musk. Academic papers predict that publicly available data for AI models could indeed be exhausted by 2026.
He also warned that training based on synthetic data could lead to drift, resulting not only in incorrect outputs but also a total loss of reliability in AI systems. Language models like ChatGPT could be affected.
Andrew Duncan suggested that the only way to add new knowledge would be to use synthetic data, where AI writes content, evaluates itself, and follows a feedback learning loop.
The concept of “Peak Data” therefore raises crucial questions about the future of AI training. While we’ve reached a plateau in the use of real-world data, the era of synthetic data will be the decisive turning point for AI. Are we ready?
Challenges related to the reliability and quality of information produced by AI will be key things to watch in the coming years. This is a real topic that might be discussed at the AI summit in Paris?
Source: TechCrunch.com
