Skip to content

Use coalesce in spark-postgres stream and PgBulk

Using repartition was shuffling data therefore adding a read/write step we were trying to prevent. coalesce will need data to be streamed from different HDFS nodes, but not having the write step should help.

Merge request reports