Tuesday Tip #8: Build a dataset with regular expressions 👷‍♂️

Hi Reader,

Next week, I’m launching a NEW course, Become a Regex Superhero!

Learning regular expressions (also known as “regex”) will help you become a more versatile and valuable data scientist. My course will help you go from Zero to Hero! 💪

Stay tuned for more details about the course. (As a newsletter subscriber, you'll get a significant launch discount! 💸)

In today’s tip, I’m going to show you one of the many use cases for regular expressions. Please enjoy!

👉 Tip #8: Build a dataset with regex

Let’s say that you wanted to build a dataset of every Python version and its release date. This python.org web page has all of the data you need:

But how would you turn this into a structured dataset?

We can start by reading the source of the web page (meaning the HTML) into Python using the requests library:

Here’s a small portion of the HTML, which is stored in r.text:

In order to parse the HTML into something useful, we’ll use regular expressions!

Let me be clear: What I’m about to show you is NOT enough to “teach you” regular expressions. (That’s why I created a course!)

Instead, what I’m trying to show you is that regular expressions is not as scary as you might think! 👻

Extracting the dates

Here’s how we can use regex to extract the Python release dates:

We imported the re module, and then used the findall function to search the r.text string and find all occurrences of a regex pattern.

This is the pattern we searched for: \d+ \w+ \d{4}

Here’s how to decode the pattern:

\d means “digit character” (0 through 9)
\w means “word character” (letter, digit, or underscore)
+ means “one or more”
{4} means “exactly 4”

Thus the pattern \d+ \w+ \d{4} can be read as “1 or more digits, then space, then 1 or more word characters, then space, then 4 digits”. And that’s how it found the dates!

Extracting the version numbers

It’s a bit more complicated to extract the version numbers because some have 2 parts (1.5), some have 3 parts (1.5.1), and some have a letter (1.5.1p1):

Here’s what we’ll do:

This is the pattern we searched for: Python (\d.+?)<

Here’s how to decode this:

\d means “digit character”
. means “any character except newline”
+ means “one or more”
? means “make the plus sign lazy” (this is a tricky concept, but it basically means "make the match as short as possible")
() means “only return this part of the match”

Thus the pattern Python (\d.+?)< can be read as “Python, then space, then 1 digit character, then 1 or more of any character (lazy behavior), then <, and only return the part in parentheses”.

In case you were wondering, the angle bracket (meaning the <) helps us to find the version number since it's always right before the </a> tag in the HTML.