Hi Reader, In case you missed it, I launched a free, 7-hour pandas course! 800+ students have enrolled, and a few have already earned their certificate of completion π©βπ π Link of the weekβData Internshipsβ Looking for an internship in Data Science or Analytics? This site curates the latest internship postings and emails them to you each week! π Tip #40: Build a DataFrame from multiple filesLetβs say that your dataset is spread across multiple files, but you want to read the dataset into a single pandas DataFrame. For example, I have a tiny dataset of stock market data in which each CSV file only includes a single day. Hereβs the first day: Hereβs the second day: And hereβs the third day: You could read each CSV file into its own DataFrame, combine them together, and then delete the original DataFrames, but that would be memory inefficient and require a lot of code. A better solution is to use Pythonβs built-in glob module: You can pass a pattern to the In this case, glob returns filenames in an arbitrary order, which is why we sorted the list using Pythonβs built-in We can then use a generator expression to read each of the files using Unfortunately, there are now duplicate values in the index. To avoid that, we can tell the Pretty cool, right? Need to build a DataFrame column-wise instead? Use the same code as above, except pass axis='columns' to π Until next timeDid you like this weekβs tip? Please forward it to a friend or share this link in your favorite Slack team. It really helps me out! π See you next Tuesday! - Kevin P.S. Would you wear pajamas during a Zoom call?β Did someone AWESOME forward you this email? Sign up here to receive Data Science tips every week! |
Join 25,000+ aspiring Data Scientists and receive Python & Data Science tips every Tuesday!
Hi Reader, Regardless of whether you enrolled, thanks for sticking with me through the launch of my new course! π I've already started exploring topics for the next course... π π Link of the week git cheat sheet (PDF) A well-organized and highly readable cheat sheet from Julia Evans, the brilliant mind behind Wizard Zines! π Tip #48: Three ways to set your environment variables in Python I was playing around with Mistral LLM this weekend (via LangChain in Python), and I needed to set an...
Hi Reader, Last week, I announced that a new course is coming soon and invited you to guess the topic. Hundreds of guesses were submitted, and four people who guessed correctly got the course for free! (I've already notified the winners.) I'll tell you about the course next week. In the meantime, I've got a new Tuesday Tip for you! π π Link of the week OpenAI just unleashed an alien of extraordinary ability (Understanding AI) If you're curious about what makes OpenAI's new "o1" models so...
Hi Reader, I'm really proud of this week's tip because it covers a topic (data leakage) that took me years to fully understand. π§ It's one of those times when I feel like I'm truly contributing to the collective wisdom by distilling complex ideas into an approachable format. π‘ You can read the tip below π or on my blog. π Link of the week Building an AI Coach to Help Tame My Monkey Mind (Eugene Yan) In this short post, Eugene describes his experiences calling an LLM on the phone for coaching:...