In the vast world of artificial intelligence, the ultimate goal is to create models capable of learning and generalizing, like with GPT models and other AIs you know on Yiaho.
But what else? The goal of AI is to learn from existing data to make predictions or informed decisions in new situations.
However, this learning process is not always smooth sailing…
After exploring the concept of overfitting in a previous article, where a model becomes so specialized in training data that it loses its ability to adapt to new information, let’s dive into its counterpart: underfitting.
What is underfitting? Simple definition
Imagine we are trying to build a machine learning model to predict house prices based on different characteristics. Our training data might include area, number of rooms, location, year of construction, presence of a garden, etc.
Overfitting in this context would occur if the model learned the prices of each specific house in a dataset in such detail (e.g., by memorizing the exact address and unique peculiarities of each sale) that it would become unable to correctly predict the price of a new house with similar but not identical characteristics.
It would struggle to extract general real estate market trends.
Underfitting, conversely, would manifest if the model was far too simple to grasp the complexity of the factors influencing a house’s price.
For example, if a model relied solely on area to predict price, completely ignoring crucial factors like location or number of rooms, it would make very approximate predictions, even for houses present in our training data.
It would lack the necessary “insight” to understand market dynamics.
In more formal terms, underfitting occurs when an AI model fails to establish a significant relationship between input variables (features) and the target variable (what we are trying to predict).
It is unable to capture important patterns present in the training data, resulting in poor performance, both on the data used for learning and on new data it encounters.
The root causes of Underfitting:
Several factors can lead to this lack of “understanding” on the part of the model:
- Insufficient model complexity: Using an algorithm that is intrinsically too simple for the nature of the problem. For example, attempting to model a complex, non-linear relationship (like stock price evolution) with a simple linear regression model. The latter assumes a direct and constant relationship between variables, which is rarely the case in complex systems.
- Insufficient number of relevant features: The model does not have the necessary information to make accurate predictions. In our real estate example, relying solely on area is an oversimplification. Ignoring location (a property in Paris will have a different price than one in the provinces), the condition of the property, or the presence of important amenities deprives the model of crucial information.
- Insufficient Training: Even with a potentially suitable model and rich data, training for too short a period can lead to underfitting. The model has not had enough iterations to adjust its internal parameters and discover significant relationships in the data. It’s like trying to learn a complex melody by listening to it only once.
- Excessively pre-processed data: While data preprocessing is essential, oversimplification (e.g., overly broad category grouping or overly aggressive dimensional reduction) can eliminate important information that the model could have learned.
Also read: How are AIs evaluated? Here are the 8 main tests (Turing, Winograd, GAIA)
The concrete consequences of Underfitting:
A model suffering from underfitting is often limited:
- Low accuracy: Its predictions or classifications will often be incorrect or very far from reality.
- Poor generalization: Unable to understand underlying trends, it will fail miserably when faced with new data it has never seen.
- Lack of insight: The model fails to extract useful information or significant relationships from the data, limiting its value for analysis and decision-making.
Strategies to remedy Underfitting:
To give AI more “brainpower” and avoid underfitting, several approaches can be considered:
- Increase model complexity: Choose a more sophisticated algorithm capable of modeling non-linear relationships or complex interactions between variables. For example, moving from linear regression to a polynomial model, a decision tree, or a neural network.
- Feature Engineering: Create new, more informative features from raw data. In our real estate example, we could create a feature combining area and number of rooms per floor, or a categorical variable indicating proximity to public transport.
- Increase training duration: Allow the model to train for longer on the data, adjusting its parameters until it converges to a satisfactory solution (without falling into overfitting, of course).
- Reduce model constraints: Some models have hyperparameters that limit their complexity. Adjusting these parameters to allow the model to learn more complex relationships can help overcome underfitting.
- Re-evaluate data preprocessing: Ensure that preprocessing has not eliminated important information. Sometimes, retaining more detail or using less aggressive transformation techniques can be beneficial.
Underfitting is a common pitfall in the development of artificial intelligence systems. It signals a model’s lack of ability to learn the nuances and complexities of data. Just like overfitting, it compromises the model’s performance and usefulness.
The art of building AI models lies in finding a delicate balance, where the model is complex enough to capture important relationships without getting lost in the noise and specificities of the training data. It is this precise balance that allows AI to truly shine and offer us valuable insights and reliable predictions!
If you want to learn more about the world of artificial intelligence, feel free to consult our AI dictionary.
