python - Why is pandas read_csv able to find file but os.listdir isn't?

Question

I have a file at ~/Google Drive/iNat/full.csv.

iNatPATH = r"~/Google Drive/iNat/"
FILENAME = iNatPATH + "full.csv"

I can't figure out why Pandas is able to read the csv file:

import pandas as pd
raw_data = pd.read_csv(FILENAME)

# no error returned.

But os can't find it:

import os
os.path.exists(FILENAME)

# outputs False

I am running this code in a macOS system on a local runtime in Google Colab.

My working directory is (not supposed to be relevant since I am using the absolute path):

>>> os.getcwd()
...
'/Users/<myusername>/Documents'

Answer 1

Your problem is that os.path.exists() function doesn't expand the ~ character. To solve this, the same os module provides the os.path.expanduser(path) function that will expand that user variable in a path for you.

Applied to your case:

iNatPATH = r"~/Google Drive/iNat/"
FILENAME = iNatPATH + "full.csv"

# Expand the ~ character
FILENAME = os.path.expanduser(FILENAME)

# Finally you can check again with exists() function
os.path.exists(FILENAME)

Reference: https://docs.python.org/3/library/os.path.html#os.path.expanduser