Tuesday Tip #41: Profile your pandas DataFrame šŸ”Ž


Hi Reader,

I just published a new blog post, Get started with conda environments. If youā€™re new to virtual environments in Python, give it a read!

Once you start using virtual environments, youā€™ll wonder how you ever got along without them!


šŸ”— Link of the week

ā€‹Yann LeCun on the future of AI (Lex Fridman interview)

Yann LeCun is one of the ā€œgodfathers of Deep Learningā€, the Chief AI Scientist at Meta, and (in my opinion) one of the clearest and most convincing thinkers on the future of AI.

Itā€™s a long interview (2 hr 45 min), and some of the sections are highly technical, so use the timestamps to jump to the parts that interest you most.

Lex is a world-class interviewer and an AI researcher, and he was the perfect person to conduct this conversation. Highly recommended!


šŸ‘‰ Tip #41: Profile your pandas DataFrame in one line of code

Letā€™s say that youā€™ve got a new dataset you want to quickly explore without too much work. Hereā€™s what to do:

Step 1: Install ydata-profilingā€‹

Step 2: Import ydata_profiling in Jupyter

Step 3: Run the ProfileReport() function and pass it any DataFrame

VoilĆ ! It returns an interactive report in HTML format.

The reportā€™s first section is an overview of the dataset and a list of possible issues with the data:

The second section gives a summary of each column:

The third section lets you explore feature interactions:

The fourth section visualizes missing values:

And the final section shows a sample of the dataset:

Want more tricks like this? Check out My top 25 pandas tricks on YouTube (250K+ views), or enroll in my new course, pandas in 30 days!


šŸ‘‹ See you next Tuesday!

Did you like this weekā€™s tip? Please forward it to a friend or share this link with your favorite online community. It really helps me out! šŸ™Œ

- Kevin

P.S. How This Guy Makes the Worldā€™s Best Puzzle Boxes (WIRED video)

Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week!

Learn Artificial Intelligence from Data School šŸ¤–

Join 25,000+ intelligent readers and receive AI tips every Tuesday!

Read more from Learn Artificial Intelligence from Data School šŸ¤–

Hi Reader, Last week, I encouraged you to experiment with different LLMs, since thereā€™s no one model that is superior across all use cases. Specifically, I suggested you try using Chatbot Arena, which allows you to chat with multiple models at once. Itā€™s completely free, but has two significant disadvantages: Your chats are not private and may be used for research. It lacks the feature-rich interface provided by other LLMs. Today, I want to offer you a better method for experimenting with...

Hi Reader, Over the past 50 tips, Iā€™ve touched on many different topics: Python, Jupyter, pandas, ML, data visualization, and so on. Going forward, Iā€™m planning to focus mostly on Artificial Intelligence. Iā€™m announcing this so you know what to expect, and I know what to deliver! šŸ’Œ Iā€™ll also try to make the tips shorter, so that they're easier to digest on-the-go. Finally, I plan to include an ā€œaction itemā€ each week, so that you can practice what youā€™re learning. I hope you like these...

Hi Reader, Next week, Iā€™ll be offering a Black Friday sale on ALL of my courses. Iā€™ll send you the details this Thursday! šŸšØ šŸ‘‰ Tip #50: What is a "method" in Python? In Python, a method is a function that can be used on an object because of the object's type. For example, if you create a Python list, the "append" method can be used on that list. All lists have an "append" method simply because they are lists: If you create a Python string, the "upper" method can be used on that string simply...