profile

Learn Data Science from Data School 📊

Tuesday Tip #41: Profile your pandas DataFrame 🔎

Published about 2 months ago • 1 min read

Hi Reader,

I just published a new blog post, Get started with conda environments. If you’re new to virtual environments in Python, give it a read!

Once you start using virtual environments, you’ll wonder how you ever got along without them!


🔗 Link of the week

Yann LeCun on the future of AI (Lex Fridman interview)

Yann LeCun is one of the “godfathers of Deep Learning”, the Chief AI Scientist at Meta, and (in my opinion) one of the clearest and most convincing thinkers on the future of AI.

It’s a long interview (2 hr 45 min), and some of the sections are highly technical, so use the timestamps to jump to the parts that interest you most.

Lex is a world-class interviewer and an AI researcher, and he was the perfect person to conduct this conversation. Highly recommended!


👉 Tip #41: Profile your pandas DataFrame in one line of code

Let’s say that you’ve got a new dataset you want to quickly explore without too much work. Here’s what to do:

Step 1: Install ydata-profiling

Step 2: Import ydata_profiling in Jupyter

Step 3: Run the ProfileReport() function and pass it any DataFrame

Voilà! It returns an interactive report in HTML format.

The report’s first section is an overview of the dataset and a list of possible issues with the data:

The second section gives a summary of each column:

The third section lets you explore feature interactions:

The fourth section visualizes missing values:

And the final section shows a sample of the dataset:

Want more tricks like this? Check out My top 25 pandas tricks on YouTube (250K+ views), or enroll in my new course, pandas in 30 days!


👋 See you next Tuesday!

Did you like this week’s tip? Please forward it to a friend or share this link with your favorite online community. It really helps me out! 🙌

- Kevin

P.S. How This Guy Makes the World’s Best Puzzle Boxes (WIRED video)

Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week!

Learn Data Science from Data School 📊

Kevin Markham

Join 25,000+ aspiring Data Scientists and receive Python & Data Science tips every Tuesday!

Read more from Learn Data Science from Data School 📊

Hi Reader, Last week, I recorded the FINAL 28 LESSONS 🎉 for my upcoming course, Master Machine Learning with scikit-learn. That's why you didn't hear from me last week! 😅 I edited one of those 28 videos and posted it on YouTube. That video is today's tip, which I'll tell you about below! 👉 Tip #45: How to read the scikit-learn documentation In order to become truly proficient with scikit-learn, you need to be able to read the documentation. In this video lesson, I’ll walk you through the five...

4 days ago • 1 min read

Hi Reader, happy Tuesday! My recent tips have been rather lengthy, so I'm going to mix it up with some shorter tips (like today's). Let me know what you think! 💬 🔗 Link of the week A stealth attack came close to compromising the world's computers (The Economist) If you haven't heard about the recent "xz Utils backdoor", it's an absolutely fascinating/terrifying story! In short, a hacker (or team of hackers) spent years gaining the trust of an open-source project by making helpful...

18 days ago • 1 min read

Hi Reader, Today's tip is drawn directly from my upcoming course, Master Machine Learning with scikit-learn. You can read the tip below or watch it as a video! If you're interested in receiving more free lessons from the course (which won't be included in Tuesday Tips), you can join the waitlist by clicking here: Yes, I want more free lessons! 👉 Tip #43: Should you discretize continuous features for Machine Learning? Let's say that you're working on a supervised Machine Learning problem, and...

25 days ago • 2 min read
Share this post