Skip to content

Add binary bulk load

Joseph Allemandou requested to merge jobar/spark-etl:add_binary_bulk_load into master

Fix #14 (closed)

This strategy uses the PgBulkInsert library to stream binary format for postgres. Accepted types are all primitive ones and Array, conversion of Map and Struct to hstore or jsonb is still to be done. Performance of data conversion is even better than the Univocity CSV generator, so if postgres can load binary faster, there might be some actual gain.

Merge request reports

Loading