Your future starts here
Imagine being part of a team that helps clients build better relationships with customers. When you join us, you'll help top-notch clients to execute the digital strategies of the future.
Every day, we collaborate with clients and each other to provide technology expertise, human-centered design and industry experience to deliver real business results.
Position description
- Work collaboratively with technical team members to develop, deploy and production support data engineering jobs and pipelines - spark structured stream processing jobs across one or more streams, persisted to a data lake.
- Work with leads and architects to prototype designs and technologies.
- Design and develop data engineering jobs with an eye on quality attributes (and trade-offs) such as simplicity, reliability, scalability, extensibility, performance.
Build or reuse common frameworks where possible.
- Set and adhere to coding and style guides to ensure consistency. Ensure code quality through mechanisms including code coverage, peer reviews, pair programming.
- Create and deploy streaming data quality checks with remediation runbooks. Adhere to data governance and compliance requirements and standards.
- Provide recommendations for and help curate technical backlogs for technical debt management and continuous improvement.
Required qualifications
- 5+ years of total experience working with data and data pipeline technologies at scale.
- 3+ years of solid Python development experience including : Robust unit and integration testing and Git driven CI / CD to production
- 2+ years of proven experience developing streaming data processing tasks using pySpark and SparkSQL
- Solid understanding of Spark, preferably Databricks Spark and Apache Delta
- Read streaming data from Kafka, Apache Delta or similar, merge / enrich / qualify data inline, capture operations metrics, persist to target, debug and optimize
- At scale and in production
- Experience leveraging public cloud data and analytics services. Azure preferred, but AWS / GCP also works.
- Nice to have : Super user experience with reporting tools such a PowerBI.
Required education and experience
- College / University Degree or equivalent.
Data Protection Engineer, Senior
The Data Protection Engineer Senior will independently perform Data Protection engineering activities of building, configuring, troubleshooting, integrating and administrating Data protection technologies aligned to one of the Data Protection sub-domains (Data in Transit, Data at Rest, Cryptographic...
Senior Associate Infrastructure Engineer (Data Center)
Working with the Product Owner, Service Owner, and Subject Matter Experts, validates the context of requirements and executes individual user stories relating to onsite data center systems including critical system installs, data room moves, decommissions, reconfigurations, and onsite data center/da...
Senior Big Data Engineer
In partnership with other business, platform, technology, and analytic teams across the enterprise, design, build and maintain well-engineered data solutions in a variety of environments, including traditional data warehouses, Big Data solutions, and cloud-oriented platforms. Align with security, da...
Senior Electrical Engineer - Arc Flash - Data Center (Remote)
As a Senior Electrical Engineer, you will work directly with some of the world’s largest technology companies and other mission-critical clients. We are Olsson, a team-based, purpose-driven engineering and design firm. You will serve as an electrical engineer on projects, design calculations, write ...
Senior Cloud Engineer -- Strategic Data (Hybrid)
Are you an experienced cloud engineer looking to be a part of a cloud transformation? The Hartford’s Group Benefits (GB) organization is aiming to transform our claims management systems, and we are looking for an innovative Cloud Engineer to play a key role in supporting this evolution. This work w...
Senior Electrical Engineer - Data Center
We are Olsson, a team-based, purpose-driven engineering and design firm. As an Electrical Engineer, you will work directly with some of the world’s largest technology companies and other mission-critical clients. You will serve as an electrical engineer on projects, design calculations, write techni...
Senior Data Engineer
You will have an opportunity to work in roles such as Data Scientist, Data Engineer, or Chief Data Officer covering all aspects of Data including Data Management, Data Governance, Data Intelligence, Knowledge Graphs, and IoT. You will be part of a highly collaborative and growing network of technolo...
Senior Data Engineer
As a Senior Data Engineer, you will be responsible for leading the development and implementation of data integration solutions for analytics. You will play a critical role in designing, building and managing data pipelines, ensuring data quality and driving the strategic direction of data initiativ...
Senior Data Engineer
This includes designing and building data ingestion and transformation pipelines using Azure Data engineering stack (Azure Data Factory, etc. Familiar with best practices for data flow patterns for data ingestion into Data warehouse including progression through Staging, ODS and Data Warehouse (Dims...
Cloud Data Engineer Senior Consultant
Net, C#, Qlik, Power BI, Machine Learning, Azure Data Factory, RedShift, UiPath, Cloud, RPA, AWS, Redshift, Kinesis, QuickSight, SageMaker, S3, Databricks, AWS Lake Formation, Snowflake, Python, Qlik, Athena, Data Pipeline, Glue, Star Schema, Data Modeling, SQL, SSIS, SSAS, SSRS, PySpark, Microsoft ...