Tuesday Tip #43: Should you discretize features for Machine Learning? 🤖


Hi Reader,

Today's tip is drawn directly from my upcoming course, Master Machine Learning with scikit-learn. You can read the tip below or watch it as a video!

If you're interested in receiving more free lessons from the course (which won't be included in Tuesday Tips), you can join the waitlist by clicking here:


👉 Tip #43: Should you discretize continuous features for Machine Learning?

Let's say that you're working on a supervised Machine Learning problem, and you're deciding how to encode the features in your training data.

With a categorical feature, you might consider using one-hot encoding or ordinal encoding. But with a continuous numeric feature, you would normally pass that feature directly to your model. (Makes sense, right?)

However, one alternative strategy that is sometimes used with continuous features is to "discretize" or "bin" them into categorical features before passing them to the model.

First, I'll show you how to do this in scikit-learn. Then, I'll explain whether I think it's a good idea!


👩‍💻 How to discretize in scikit-learn

In scikit-learn, we can discretize using the KBinsDiscretizer class:

When creating an instance of KBinsDiscretizer, you define the number of bins, the binning strategy, and the method used to encode the result:

As an example, here's a numeric feature from the famous Titanic dataset:

And here's the output when we use KBinsDiscretizer to transform that feature:

Because we specified 3 bins, every sample has been assigned to bin 0 or 1 or 2. The smallest values were assigned to bin 0, the largest values were assigned to bin 2, and the values in between were assigned to bin 1.

Thus, we've taken a continuous numeric feature and encoded it as an ordinal feature (meaning an ordered categorical feature), and this ordinal feature could be passed to the model in place of the numeric feature.


🤔 Is discretization a good idea?

Now that you know how to discretize, the obvious follow-up question is: Should you discretize your continuous features?

Theoretically, discretization can benefit linear models by helping them to learn non-linear trends. However, my general recommendation is to not use discretization, for three main reasons:

(1) Discretization removes all nuance from the data, which makes it harder for a model to learn the actual trends that are present in the data.

(2) Discretization reduces the variation in the data, which makes it easier to find trends that don't actually exist.

(3) Any possible benefits of discretization are highly dependent on the parameters used with KBinsDiscretizer. Making those decisions by hand creates a risk of overfitting the training data, and making those decisions during a tuning process adds both complexity and processing time. As such, neither option is attractive to me!

For all of those reasons, I don't recommend discretizing your continuous features unless you can demonstrate, through a proper model evaluation process, that it's providing a meaningful benefit to your model.


📚 Further reading

đź”— Discretization in the scikit-learn User Guide

đź”— Discretize Predictors as a Last Resort from Feature Engineering and Selection (section 6.2.2)


đź‘‹ See you next Tuesday!

Did you like this week’s tip? Don't forget to join the waitlist for my new ML course and get more free lessons!

- Kevin

P.S. My hobby: Extrapolating​

Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week!

Learn Artificial Intelligence from Data School 🤖

Join 25,000+ intelligent readers and receive AI tips every Tuesday!

Read more from Learn Artificial Intelligence from Data School 🤖

Hi Reader, Here are your top AI stories for the week: ChatGPT can weaken your brain Claude shares nerve gas recipe Amsterdam ends AI experiment due to bias Read more below! 👇 Sponsored by: Brain.fm Transform Your Focus With Brain.fm I know you're always on the hunt for tools that genuinely improve your life—which is why I'm excited to introduce you to Brain.fm's groundbreaking focus music. Brain.fm's patented audio technology was recently validated in a top neuroscience journal, showing how...

Hi Reader, Last week, I invited you to help me test Google's Data Science Agent in Colab, which promises to automate your data analysis. Does it live up to that promise? Let's find out! 👇 Sponsored by: Morning Brew Business news you’ll actually enjoy Join 4M+ professionals who start their day with Morning Brew—a free daily newsletter that makes business, tech, and finance news genuinely enjoyable to read and hard to forget. Each morning, it breaks down complex stories in plain English—cutting...

Hi Reader, Today I'm trying something brand new! I wrote short summaries of the 5 most important AI stories this week, and also turned it into a video: Watch the video I'd love to know what you think! đź’¬ AI-generated TV ad airs during NBA finals Prediction market Kalshi just aired this AI-generated ad on network TV during the NBA finals. It was created in just two days by one person using Google's new Veo 3 video model, plus scripting help from Google's Gemini chatbot. Expect to see many more...