Landmark 1 stop to get here · leads to 10

Feature

A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.

Your route here

1 stop · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Feature · you are here ✓ understood

Picture it

  1. 01 Raw example e.g. a house listing
  2. 02 Measure properties Floor area, bedrooms, age
  3. 03 Feature vector [135, 3, 22]
  4. 04 Model Learns weights on each feature
  5. 05 Prediction e.g. sale price
Notice that the model never sees the house itself, only the measurable features chosen to describe it.

A feature is one number or category that describes an example. A house listing becomes floor area, bedroom count, year built and postcode. An email becomes the number of links it contains, whether the sender is in your contacts, and which words appear. The model never sees the house or the email. It sees only these measurements, lined up as a feature vector, and learns how much weight to give each one.

That makes the choice of features a ceiling on what the model can learn. If the thing that really drives house prices, say the school catchment area, isn’t among the features, no algorithm can recover it.

Kinds of features

  • Numeric: area, age, price. Usually rescaled with normalization so features with large values don’t swamp small ones.
  • Categorical: colour, country, product type. These have to be turned into numbers, commonly with one-hot encoding.
  • Derived: built from other features, such as price per square metre, or the day of the week taken from a timestamp.

Hand-made or learned

In classic machine learning, people design the features, and that work, feature engineering, often decides whether a project succeeds. Pedro Domingos put it bluntly in 2012: the features used are easily the most important factor, and building them is typically where most of a project’s effort goes.

Deep learning changed that for raw data like images, audio and text. A convolutional network is given raw pixels and learns its own features in its layers: edges, then textures, then object parts. This is representation learning, and it’s why “feature” also refers to the patterns a network learns internally. For tabular data, spreadsheets of customers and transactions, hand-built features still carry a lot of the weight.

The catch

More features isn’t automatically better. Each one adds a dimension, and the amount of data needed to cover the space grows very quickly; Domingos calls this the second biggest problem in machine learning after overfitting. Irrelevant features add noise, which is why feature selection exists.

Features can also leak the answer. A “refund issued” column predicts fraud almost perfectly in historical data, but at the moment you need the prediction, the refund hasn’t happened yet.

Where it sits

Before this

Dataset
Feature

Leads to

Representation Learning Curse of Dimensionality Dimensionality Reduction Feature Engineering Feature Selection Normalization

+ 4 more

Explore nearby

Foundations Feature Engineering The process of selecting, transforming, and creating input features to improve model performance. Foundations Feature Selection Choosing the most relevant features from available data to reduce dimensionality and improve model performance. Foundations Feature Importance Measures indicating which features contribute most to model predictions, useful for interpretation and selection. Foundations One-Hot Encoding Converting categorical variables into binary vectors with one element set to 1 and others to 0. Foundations Normalization Scaling features to a standard range (typically 0-1 using min-max scaling) to improve model training and convergence. Often used interchangeably with standardization (mean=0, std=1), though technically distinct.

Sources

  1. Pedro Domingos, "A Few Useful Things to Know About Machine Learning" . Communications of the ACM 55(10), 2012; doi:10.1145/2347736.2347755
  2. Bengio, Courville and Vincent, "Representation Learning: A Review and New Perspectives" . 2012