python - How to avoid unsupported .nc file while reading from different directory

Question

I have several folders in a directory containing .nc files. While reading, I am getting an error: 'NETCDF can not read unsupported file'. Since there are more than 5 thousand files, I dont know which file is corrupted or unsupported. Is there anyways to read files by jumping into another supported file.

The code that I am using is:

import xarray as xr
import numpy as np
import pandas as pd


ncfile = glob.glob('mydata/****/se*')
frame = pd.DataFrame()
for i in np.arange(len(ncfile)):
    frame = frame
    for j in np.arange(len(ds.variables['time'])): 
        ds1 = xr.open_dataset(ncfile[i])
        prec = np.ravel(ds.variables['precipitation_amount'][j,:,:])
        frame[dates] = prec
    ds = xr.open_dataset(ncfile[i])

question from:https://stackoverflow.com/questions/66049237/how-to-avoid-unsupported-nc-file-while-reading-from-different-directory

Answer 1

You could do this using exception handling. I've shown this with a simple example based on your code:

import xarray as xr
import numpy as np
import pandas as pd


ncfile = glob.glob('mydata/****/se*')
frame = pd.DataFrame()
errors = []
for i in np.arange(len(ncfile)):
    frame = frame
    try:
        ds = xr.open_dataset(ncfile[i])
    except:
        errors.append(ncfile[i])