Learn Data Science from Data School πŸ“Š

Tuesday Tip #36: Deciphering the pandas index πŸ•΅οΈβ€β™‚οΈ

Published about 1 month agoΒ β€’Β 2 min read

Hi Reader,

Do you have a topic you'd like me to cover in a future tip? Reply and let me know! πŸ’Œ

πŸ”— Link of the week

​Deep Learning Fundamentals (free course)

Are you interested in getting started with Deep Learning in Python? I haven't taken this course, but it looks promising!

I was thinking of starting a YouTube series in which I work through this course and share some of my learnings (and questions), and then you could follow along at home. Would that be of interest to you?

πŸ‘‰ Tip #36: Eight things you need to know about the pandas index

One aspect of Python's pandas library that can be confusing to newcomers is the concept of the "index". Every pandas DataFrame or Series has an index, and it's used for identification, selection, and alignment.

Here are some key facts about the index that will help you to work with pandas objects more effectively:

1️⃣ The index is on the left

The sequence of numbers on the left side is called the index, and they're used to identify the DataFrame rows. Sometimes they're called the "row labels".

2️⃣ By default, the index is a sequence of integers

Because I didn't define an index when creating the DataFrame, it assigned the numbers 0 through 192 as the index. They're stored in a special object called the "RangeIndex".

3️⃣ The column names are stored in an "Index" object

When people talk about "the index", they're talking about the row labels. But it turns out that the column names are stored in a special type of object called an "Index"!

That being said, you should never refer to the column names as "the index"... just call them "the columns" or "the column names".

4️⃣ The labels are not included in the shape

You should think of the index and columns as labels for the DataFrame. They're not part of the DataFrame "contents", and thus they're not included in its shape.

5️⃣ The index remains with each row

If you filter the DataFrame, the index remains with each row to identify it.

6️⃣ The index can be used for selection

You can use the loc accessor to select one or more rows or columns by label.

7️⃣ You can set a different column as the index

Notice that the default integer index no longer exists.

8️⃣ Setting a meaningful index allows for easier selection

Rather than referring to Brazil as "23" (its previous row label), you can refer to it as "Brazil".

Going further

If you want to learn more about the pandas index, check out these Data School videos:

πŸ”— What do I need to know about the pandas index? (Part 1)​

πŸ”— What do I need to know about the pandas index? (Part 2)​

Are you new to pandas? Stay tuned for my upcoming course, pandas in 30 days! Coming February 2024 πŸš€

🐊 See you later, alligator!

Did you like this week’s tip? Please forward it to a friend, or share this link in your favorite Slack group. It really helps me out! πŸ™Œ

See you next Tuesday!

- Kevin

P.S. AI murder mystery​

Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week!

Learn Data Science from Data School πŸ“Š

Kevin Markham

Join 25,000+ aspiring Data Scientists and receive Python & Data Science tips every Tuesday!

Read more from Learn Data Science from Data School πŸ“Š

Hi Reader, Do any of these sound like you? You're new to the pandas library and you want to learn the fundamentals You have some experience with pandas, but you want to fill in the gaps in your knowledge You want to learn the best practices for data analysis with pandas in 2024 If so, you should enroll in my FREE course (launching today!), pandas in 30 days. Why learn pandas? pandas is a powerful, open source Python library for data analysis, manipulation, and visualization. If you're working...

9 days agoΒ β€’Β 1 min read

Hi Reader, There's a gift for you somewhere in this email... just look for the 🎁 emoji! Tip #39: Six quick Python tricks Here's what I'll cover below: Return the number of unique values Count values with Counter Better debugging with f-strings Return multiple values from a function Count while looping Create a dictionary with a comprehension Let's get started! πŸ‘‡ 1️⃣ Return the number of unique values Need to know the number of unique values in an iterable? Convert it to a set and check the...

16 days agoΒ β€’Β 2 min read

Hi Reader, My goal with Tuesday Tips is to help you get better at Data Science every week. Is there anything that would make these tips even more helpful for you? Let me know! πŸ’¬ You can find past tips at (Yes, that’s a real URL!) Tip #38: Five ways to rename your DataFrame columns Let's say that we have a simple pandas DataFrame: I prefer to use dot notation to select pandas columns, but that won't work since the column names have spaces. Let's fix this! The most flexible method...

23 days agoΒ β€’Β 1 min read
Share this post