GCP Data Engineer

Urgent fill -

We need someone who is current hands-on with building end-to-end data pipeline using GCP services such as Dataflow, Spark+Python (DataProc), Composer, GCS, BQ, etc. This is a more hands-on developer role.

Would love someone in Chicago, would take someone remote.

10+ years experience.

Waiting on rate information

Client -

JOB TITLE : GCP Data Engineer

POSITION SUMMARY :

The GCP Data Engineer is responsible for Analysis, Design, Development, Testing and Deployment support for new framework, enhance existing framework that enables the data platform data pipelines. This position is required to perform independently in a highly dynamic and fast paced environment. The person will work alongside Architects, engineers, analysts and PMs to deliver scalable robust innovative technical solutions. This position plays a key role in building real-time and batch data ingestion, egress frameworks, streaming analytics framework and support AI platform. The person must have similar experience in a prior job.

RESPONSIBILITIES

• Build frameworks for large-scale data processing evaluating appropriate emerging technologies, and approaches that will power data-driven capabilities across the Enterprise.
• Develops data solutions on Google Cloud Platform leveraging Google Data Flow, Data Proc, Composer, Pub / Sub, BigQuery, GCS, Cloud function, define workflows, scheduled and event driven workloads to ingest data from internal / external partners, data distribution channels, etc.
• Build features using Python in Spark, leveraging GCP's Spark Engine (Dataproc), SQL / GSQL on Google BQ.
• Build and maintain scalable data pipelines to handle high-volume data (e.g., 150 million rows).
• Collaborate with cross-functional technologists across the organization to gather requirements, solve new problems and deliver quality results.
• Coordinate with offshore engineers to get projects / tasks completed.
• Develops and executes test plans to validate the implementation and performance of frameworks and recommend performance improvements.
• Supports the operations of the deployed solutions, investigates complex issues and assists with the resolution and implementation of preventive measures.

REQUIREMENTS FOR CONSIDERATION :