Tuesday Tip #43: Should you discretize features for Machine Learning? πŸ€–


Hi Reader,

Today's tip is drawn directly from my upcoming course, Master Machine Learning with scikit-learn. You can read the tip below or watch it as a video!

If you're interested in receiving more free lessons from the course (which won't be included in Tuesday Tips), you can join the waitlist by clicking here:


πŸ‘‰ Tip #43: Should you discretize continuous features for Machine Learning?

Let's say that you're working on a supervised Machine Learning problem, and you're deciding how to encode the features in your training data.

With a categorical feature, you might consider using one-hot encoding or ordinal encoding. But with a continuous numeric feature, you would normally pass that feature directly to your model. (Makes sense, right?)

However, one alternative strategy that is sometimes used with continuous features is to "discretize" or "bin" them into categorical features before passing them to the model.

First, I'll show you how to do this in scikit-learn. Then, I'll explain whether I think it's a good idea!


πŸ‘©β€πŸ’» How to discretize in scikit-learn

In scikit-learn, we can discretize using the KBinsDiscretizer class:

When creating an instance of KBinsDiscretizer, you define the number of bins, the binning strategy, and the method used to encode the result:

As an example, here's a numeric feature from the famous Titanic dataset:

And here's the output when we use KBinsDiscretizer to transform that feature:

Because we specified 3 bins, every sample has been assigned to bin 0 or 1 or 2. The smallest values were assigned to bin 0, the largest values were assigned to bin 2, and the values in between were assigned to bin 1.

Thus, we've taken a continuous numeric feature and encoded it as an ordinal feature (meaning an ordered categorical feature), and this ordinal feature could be passed to the model in place of the numeric feature.


πŸ€” Is discretization a good idea?

Now that you know how to discretize, the obvious follow-up question is: Should you discretize your continuous features?

Theoretically, discretization can benefit linear models by helping them to learn non-linear trends. However, my general recommendation is to not use discretization, for three main reasons:

(1) Discretization removes all nuance from the data, which makes it harder for a model to learn the actual trends that are present in the data.

(2) Discretization reduces the variation in the data, which makes it easier to find trends that don't actually exist.

(3) Any possible benefits of discretization are highly dependent on the parameters used with KBinsDiscretizer. Making those decisions by hand creates a risk of overfitting the training data, and making those decisions during a tuning process adds both complexity and processing time. As such, neither option is attractive to me!

For all of those reasons, I don't recommend discretizing your continuous features unless you can demonstrate, through a proper model evaluation process, that it's providing a meaningful benefit to your model.


πŸ“š Further reading

πŸ”— Discretization in the scikit-learn User Guide

πŸ”— Discretize Predictors as a Last Resort from Feature Engineering and Selection (section 6.2.2)


πŸ‘‹ See you next Tuesday!

Did you like this week’s tip? Don't forget to join the waitlist for my new ML course and get more free lessons!

- Kevin

P.S. My hobby: Extrapolating​

Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week!

Learn Artificial Intelligence from Data School πŸ€–

Join 25,000+ intelligent readers and receive AI tips every Tuesday!

Read more from Learn Artificial Intelligence from Data School πŸ€–

Hi Reader, Today I'm trying something brand new! I wrote short summaries of the 5 most important AI stories this week, and also turned it into a video: Watch the video I'd love to know what you think! πŸ’¬ AI-generated TV ad airs during NBA finals Prediction market Kalshi just aired this AI-generated ad on network TV during the NBA finals. It was created in just two days by one person using Google's new Veo 3 video model, plus scripting help from Google's Gemini chatbot. Expect to see many more...

Hi Reader, Thanks for sticking with me through last week’s course launch! πŸ™ As you may have noticed, Tuesday Tips has been on pause for a few months. I was focused on launching the course, plus I’ve been working on a book 🀫 I can’t promise a new tip every Tuesday, but I’ll do my best to provide you with valuable content as time permits. As for today’s tip, I’m trying out something new… πŸ‘‡ Sponsored by: Superhuman AI Find out why 1M+ professionals read Superhuman AI daily. AI won't take over...

Hi Reader, This week, I've got a short tip about AI agents, followed by some Data School news... πŸ‘‰ Tip #56: What are AI agents? Google is calling 2025 "the agentic era," DeepLearning.AI says "the agentic era is upon us," and NVIDIA's founder says "one of the most important things happening in the world of enterprise is agentic AI." Clearly AI agents are a big deal, but what exactly are they? Simply put, an AI agent is an application that uses a Large Language Model (LLM) to control its...