Convert NPY to PARQUET
Turn NPY files into PARQUET. A Parquet table with a typed column for each column the CSV conversion makes.
Converted in an isolated sandbox on our serversDeleted after 24 hours
Drop NPY files on the dialor turn it to pick what to do
A NumPy array as a typed Parquet table.Each column gets its own type. The shape of the array decides the columns.
What you get
One Parquet file with a typed column for each column the CSV conversion makes. A 1-D array becomes one column. A 2-D array becomes a table with a column for each index of its second axis, named 0, 1, 2 and so on.
A record array gets a column for each field. A field that holds an array or nested records is written as JSON text.
Arrays with more dimensions
An array of three or more dimensions becomes one row for each index of all but its last axis. Those rows are led by columns named axis0, axis1 and so on, so each row says where it came from in the array.
NaN and NaT become NULL, so missing values are empty in the table instead of a special number or date.
Arrays that are not loaded
Arrays of Python objects are not loaded. Unpickling them could run code, so the conversion stops with
unsupported_elements.If you have several Parquet files from arrays like this, you can merge PARQUET afterwards. You can also see every PARQUET conversion.
NPY and PARQUET.What each format is.
The formats
NumPy Array
A single NumPy array saved with numpy.save, with its shape and data type.
- Extensions
- .npy
- MIME type
- application
/x -npy - Kind
- Text and data
Parquet
The columnar data format of analytics. Compressed, typed and fast to scan.
- Extensions
- .parquet .pq
- MIME type
- application
/vnd .apache .parquet - Kind
- Text and data
In your code.The same NPY to PARQUET conversion through the API, for your app or your agent.
$ cv convert report.npy report.parquet$ cv convert report.npy report.parquetQuestions.What people ask about NPY to PARQUET.
It becomes a table with a column for each index of its second axis. The columns are named 0, 1, 2 and so on.
NaN becomes NULL in the Parquet file. NaT becomes NULL too.
Yes. Each field becomes a column. If a field holds an array or nested records, it is written as JSON text.
The array holds Python objects. These are not loaded, because unpickling them could run code.