pyarrow: open_csv with an empty file

[delucchi-cmu]

I have some CSV files with header lines, and some may not contain any data rows. I'm struggling to find the right options to read such files using open_csv and have this return either an empty Table or None.

Such a file might be:

id,ra,dec,ra_error,dec_error,comment,code

Some examples of patterns I've tried, and the errors I get:

from pyarrow import csv
with csv.open_csv(file) as reader:
    for next_chunk in reader:
        print(len(next_chunk))

yields ArrowInvalid: CSV parse error: Empty CSV file or block: cannot infer number of columns

with csv.open_csv(
    file,
    read_options=csv.ReadOptions(
        column_names=["id", "ra", "dec", "ra_error", "dec_error", "comment", "code"], skip_rows=1
    ),
) as reader:
    for next_chunk in reader:
        print(len(next_chunk))

yields Could not skip initial 1 rows from CSV file, either file is too short or header is larger than block size.

Is there a way to have pyarrow.open_csv open my (mostly) empty file, and not raise an error?

[jacek-pliszka]

You can always wrap it in try except.