We are seeking a dynamic and highly skilled Senior Data Engineer who has extensive experience building self -service enterprise scale data platforms with microservices architecture and lead these foundational efforts.
This role demands someone who not only possesses a profound understanding of the data engineering landscape but also has a very strong software engineering background specially building microservices frameworks and architectures.
The ideal candidate will be an individual contributor as well as the technical lead and contribute significantly to platform development and actively shape our data ecosystem.
What we offer : Career Development
Career Development
Competitive Compensation and Benefits
Pay Transparency
Global Opportunities
Learn More Here :
The base pay range for this role is estimated to be $82,000-$166,000 at the time of posting. Final compensation will be determined by various factors such as work location, education, experience, knowledge, and skills.
Tasks and Qualifications :
This is What You Will do in This Role :
As a senior engineer, you will be responsible for ideation, architecture, design and development of our enterprise data platform.
Architect and design core components with a microservices architecture, abstracting platform, and infrastructure intricacies.
Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.
Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.
Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.
Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
Design and develop microservices based semantic layer and metadata management components.
Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.
Design and develop advanced analytics and machine learning capabilities on the data platform.
Design and develop observability and data governance frameworks and practices.
Stay up to date with the latest data engineering trends, technologies, and best practices.
Drive the deployment and release cycles, ensuring a robust and scalable platform.
What We are Looking For :
Bachelor's or Master's degree in Computer Science, Engineering, or related field.
8-10 years of proven experience in modern cloud data engineering, broader data landscape experience and exposure and solid software engineering experience.
Prior experience architecting and building successful self-service enterprise scale data platforms in a green field environment with microservices based architecture.
Proficiency in building end to end data platforms and data services in GCP is a must.
Proficiency in tools and technologies : BigQuery, Cloud Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, Airflow, PubSub.
Experience with Microservices architectures - Kubernetes, Docker. Our microservices are build using TypeScript, NestJS, NodeJS stack.
Prefer candidates with this experience.
Experience building Symantec layers.
Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.
Solid experience with architecting and implementing metadata management including data catalogues, data lineage, data quality and data observability for big data workflows.
Hands-on experience with GCP ecosystem and data lakehouse architectures.
Strong understanding of data modeling, data architecture, and data governance principles.
Excellent experience with DataOps principles and test automation.
Excellent experience with observability tooling : Grafana, Datadog.
Nice to have :
- Experience with Data Mesh architecture
- Experience building Semantic layers for data platforms.
- Experience building scalable IoT architectures
Senior Azure Data Engineer
Data Engineer to lead the creation of data-driven solutions using Azure technologies. ...
Senior Software Engineer - Data Platforms
Be part of a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets. Senior Software Engineer - Data Platforms. As a Capital One Data Engineer, you'll have the opportunity t...
Senior Data Platform Engineer
A company is looking for a Senior Data Platform Engineer. Key Responsibilities:Understand the overall vision of the technology platformLead cross-team technical initiativesAdvocate for new features and product decisions based on user feedbackRequired Qualifications:7+ years of software engineering e...
Senior Lead Software Engineer - Data Solutions
As a Senior Lead Software Engineer - Database Solutions at JPMorgan Chase within the Consumer and Community Banking - Risk - Corporate Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, ...
Senior Fullstack Engineer Data Team
Key Responsibilities:Make core design decisions and lead the development of large-scale data ingestion systemsIntegrate with 3rd party APIs and develop web crawlers and scrapers to ingest electronics market dataCreate tools to streamline integration with new data sources and improve system performan...
Senior Data Engineer
The Data Engineer for the UT Data Hub improves university outcomes and advances the UT mission to transform lives for the benefit of society by increasing the useability and value of institutional data. The Data to Insights (D2I) Initiative at UT Austin is an investment to build a trusted, integrate...
Senior Data & BI Engineer
Want to join a mission-driven software company? Engineer solutions and work with business stakeholders to make healthcare patients’ lives better? We are looking for a data engineer with a heavy focus on visualizations to join our team. Data Engineering (pipelines & data cleaning). Years of experienc...
Senior Data Engineer - VP
Experience with Data Modeling, Master Data Management, Data Governance, Data Cataloging, Data Replication. The Senior Data Engineer is responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. Experience with developing an...
Senior Azure Data Engineer
As the Senior Data Engineer, you’ll be primarily responsible for leading projects to automate manual processes and produce data driven insights for the team and other stakeholders across the organization, managing Data Consumption and Data Governance areas of DNA, and actively contribute to both dat...
Senior Data Engineer - Artificial Intelligence
Incorporate core data management competencies including data governance, data security and data quality. Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Senior Data Engineer you will accelerate growth and transfor...