AI Dictionary › AI Fundamentals

Feature selection

Feature selection is the process of choosing, among all the variables available in a dataset, the subset most useful to a machine learning model, discarding those that are redundant, irrelevant, or even harmful to performance. Not every available feature helps a model: some add only noise, others duplicate information already present elsewhere, and others still slow down training with no real benefit.

Definition

How it works

There are three main approaches. Filter methods evaluate each feature individually using statistical measures, such as correlation with the target variable, before even training a model. Wrapper methods try different subsets of features by training the model repeatedly and keeping the one that performs best, a more accurate but computationally expensive approach. Embedded methods integrate selection directly into the training process, as some linear models with regularization do, automatically zeroing out the weight of less useful features.

Applications

It matters especially when data has hundreds or thousands of variables, as in genomics, text processing or systems with many sensors: reducing the number of features speeds up training, lowers the risk of overfitting, and makes the model simpler to interpret and maintain in production. It is a step that often precedes or accompanies feature engineering proper.

History & etymology

The concept has roots in mid-twentieth-century multivariate statistics, when researchers began systematically addressing which variables to include in a regression model to avoid redundancy. With the growth of machine learning and increasingly large datasets, from the 1990s onward feature selection established itself as an independent subdiscipline, with conferences and publications dedicated exclusively to comparing methods for choosing the most informative variables.

Related terms

• Feature engineering
• Regularization
• Overfitting
• Dataset
• Feature importance

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.