Convert NPY to PARQUET

Turn NPY files into PARQUET. A Parquet table with a typed column for each column the CSV conversion makes.

Converted in an isolated sandbox on our serversDeleted after 24 hours

Drop NPY files on the dialor turn it to pick what to do

A NumPy array as a typed Parquet table.Each column gets its own type. The shape of the array decides the columns.

  1. What you get

    One Parquet file with a typed column for each column the CSV conversion makes. A 1-D array becomes one column. A 2-D array becomes a table with a column for each index of its second axis, named 0, 1, 2 and so on.

    A record array gets a column for each field. A field that holds an array or nested records is written as JSON text.

  2. Arrays with more dimensions

    An array of three or more dimensions becomes one row for each index of all but its last axis. Those rows are led by columns named axis0, axis1 and so on, so each row says where it came from in the array.

    NaN and NaT become NULL, so missing values are empty in the table instead of a special number or date.

  3. Arrays that are not loaded

    Arrays of Python objects are not loaded. Unpickling them could run code, so the conversion stops with unsupported_elements.

    If you have several Parquet files from arrays like this, you can merge PARQUET afterwards. You can also see every PARQUET conversion.

NPY and PARQUET.What each format is.

The formats

From · NPY

NumPy Array

A single NumPy array saved with numpy.save, with its shape and data type.

Extensions
.npy
MIME type
application/x-npy
Kind
Text and data
To · PARQUET

Parquet

The columnar data format of analytics. Compressed, typed and fast to scan.

Extensions
.parquet .pq
MIME type
application/vnd.apache.parquet
Kind
Text and data

In your code.The same NPY to PARQUET conversion through the API, for your app or your agent.

terminal
$ cv convert report.npy report.parquet
$ cv convert report.npy report.parquet

Questions.What people ask about NPY to PARQUET.

01OF 04

It becomes a table with a column for each index of its second axis. The columns are named 0, 1, 2 and so on.

NaN becomes NULL in the Parquet file. NaT becomes NULL too.

Yes. Each field becomes a column. If a field holds an array or nested records, it is written as JSON text.

The array holds Python objects. These are not loaded, because unpickling them could run code.

More conversions.Other ways in and out of NPY and PARQUET.