Presented by:

5dfe20bbd837557a7745fc7b6cfb8943

Corey Huinker

Corlogic Consulting / Moat Inc

Corey Huinker is a database programmer and consultant based in New York City. He specializes in database query optimization and ETLs.

He is the author of the PGXN modules Poor Mans Parallel Processing and range_partitioning.

His past hobbies have included improvisational theater and refereeing roller derby.

ETL (Extract, Transform, Load) is the industry term for importing data from external sources into a database. However, more often the pattern is ELT - Extract, Load, Transform. This talk covers methods of loading external data into PostgreSQL and reshaping it to fit local needs. The talk addresses popular commercial tools, but focuses mostly on custom coding, specifically: *) Identifying bottlenecks *) Tuning for specific optimization goals (speed, lower resource usage, etc) *) temp tables *) foreign data wrappers *) COPY from PROGRAM *) index management *) filtration techniques *) data validation and error reporting *) importing highly variant data sources

Date:
2017 March 31 06:00
Duration:
50 min
Room:
Liberty II
Conference:
PGConf US 2017
Language:
Track:
Development
Difficulty:
Medium