Principal Data Engineer

Quest Global is an organization at the forefront of innovation and one of the world’s fastest growing engineering services firms with deep domain knowledge and recognized expertise in the top OEMs across seven industries.

We are a twenty-five-year-old company on a journey to becoming a centenary one, driven by aspiration, hunger and humility.

If you are considering sending an application, make sure to hit the apply button below after reading through the entire description.

We are looking for humble geniuses, who believe that engineering has the potential to make the impossible, possible; innovators, who are not only inspired by technology and innovation, but also perpetually driven to design, develop, and test as a trusted partner for Fortune 500 customers.

As a team of remarkably diverse engineers, we recognize that what we are really engineering is a brighter future for us all.

If you want to contribute to meaningful work and be part of an organization that truly believes when you win, we all win, and when you fail, we all learn, then we’re eager to hear from you.

Roles & Responsibilities :

As a senior / principal engineer, you will be responsible for ideation, architecture, design and development of new enterprise data platform.

You will collaborate with other cloud and security architects to ensure seamless alignment within our overarching technology strategy.

• Architect and design core components with a microservices architecture, abstracting platform, and infrastructure intricacies.
• Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
• Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.
• Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.
• Architect and design the best security patterns and practices.
• Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
• Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
• Design and develop advanced analytics and machine learning capabilities on the data platform.
• Design and develop observability and data governance frameworks and practices.
• Stay up to date with the latest data engineering trends, technologies, and best practices.
• Drive the deployment and release cycles, ensuring a robust and scalable platform.

Requirements :

• 10+ (for senior) 15+ (for principal) of proven experience in modern cloud data engineering, broader data landscape experience and exposure and solid software engineering experience.
• Prior experience architecting and building successful enterprise scale data platforms in a green field environment is a must.
• Proficiency in building end to end data platforms and data services in GCP is a must.
• Proficiency in tools and technologies : BigQuery, Cloud Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, Airflow, PubSub.
• Experience with Microservices architectures - Kubernetes, Docker and Cloud Run.
• Experience building Symantec layers.
• Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.
• Solid experience with architecting and implementing metadata management including data catalogues, data lineage, data quality and data observability for big data workflows.
• Hands-on experience with GCP ecosystem and data lakehouse architectures.
• Strong understanding of data modeling, data architecture, and data governance principles.
• Excellent experience with DataOps principles and test automation.
• Excellent experience with observability tooling : Grafana, Datadog.
• Nice to have :
• Experience with Data Mesh architecture.
• Experience building Semantic layers for data platforms.
• Experience building scalable IoT architectures.

Physical Requirements & Work Environment :

• Mostly Office Environments, Occasional Shop Floor involvement.
• Substantial amounts of telephone and computer work.
• Heavily Regulated Industries with strict adherence to procedures.
• Flexibility to meet business deadlines by staying late or arriving early.
• Typical 8 hour days plus lunch / 40 hour weeks / core (required) hours are 9AM to 4PM.
• Ability to use personal transportation to visit customer locations.

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job.

Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

Job Type : Full Time-Temporary

Full Time-Temporary

Experience Level : Senior Level

Senior Level

Total Years of Exp : 12-25

12-25

J-18808-Ljbffr