Fast upload of Parquet data to MySQL.
For fast upload we do the following:
- Batch into target number of CSV files
- Provide hooks for sorting data based on primary key
- Use LOAD command to upload batch CSVs
- Parallel number of connections to MySQL
This repository was archived by the owner on Feb 19, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 4
Uploads data from parquet files on HDFS to MySQL via Spark.
License
UnityTech/mysql-uploader
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Uploads data from parquet files on HDFS to MySQL via Spark.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published