Big Data Engineer

Big Data EngineerThe applicant will be responsible for managing, enhancing and updating big data pipelines where appropriate including but not limited to:Upgrades to infrastructureEnhancements to infrastructureProduction supportCreating curated versions of data as requested by our clients.The applicant will assist in the conversion of data sets into parquet with Iceberg tables on top for ingestion into a data fabric for easily accessible data for our clients.The applicant(s) will be aiding in enablement of data products that are of high quality to be ingested in a data fabric to enable investment analytics.Applicant should be familiar with the following technologies/skillsets:Big Data knowhow is mandatoryBig Data pipelinesOptimization of big data Scalability of big data across different data stores (Aurora, RedShift, S3)Database migration services (DMS) Highly skilled in Python, as a majority of our development is in Python.