Job Summary :
Learn more about the general tasks related to this opportunity below, as well as required skills.
We are seeking a skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making.
You will collaborate with data scientists, analysts, and other stakeholders to ensure data is readily available, reliable, and of high quality.
Key Responsibilities :
- Experience is a MUST with last 3-4 years.
- Should have experience in using Pyspark.
- Working experience on KUSTO for telemetry data.
- Should be able to write python code in Azure Synapse Notebook.
- Good understanding and working experience of Azure Synapse end-to-end.
- Should have experience in working with Spark structured streaming data possibly from Event Hubs.
- Should have experience in handling large volumes of data possibly in 100s of GB.
- Should be able to debug and fine-tune the jobs notebook for optimum memory consumption and processing.
- Should have knowledge of Data warehousing and data management for efficient data handling.
- Good knowledge of data modeling data engineering work.
- Experience working with large data sets using SQL, Azure Data Lake, PySpark, ADF, Synapse, etc., to derive actionable insights is highly desirable.
- They should be productive as soon as possible post joining.
Nice To Have :
- Local Redmond resource.
- PST working hour resource.
- Microsoft ecosystem knowledge.
- Azure DevOps, Azure WebApps, Functions, Azure Cosmos DB, Event Hub.
Additional Responsibilities :
- To provide technical guidance or solutions; define, advocate, and implement best practices and coding standards for the team.
- To develop and guide the team members in enhancing their technical capabilities and increasing productivity.
- To ensure process compliance in the assigned module and participate in technical discussions or reviews as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
- To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalations.
J-18808-Ljbffr
Lead Software Engineer, Data Products & Storage Program Team
Lead a portfolio of diverse technology projects and a team of developers with deep experience in distributed microservices, and full stack systems to create solutions that help meet regulatory needs for the company Share your passion for staying on top of tech trends, experimenting with and learning...
Technical Lead Data Engineer
As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making. To ensure process compliance in the assigned module and participate in technical discussions or reviews as a technical consultant ...
Lead Data Engineer (Bank Tech)
What You’ll Do: Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems Uti...
Lead Digital Data Engineer
A company is looking for a Lead Digital Data Engineer to oversee data architecture and optimization efforts. ...
Lead Data Engineer (Bank Tech) - Capital One
Delaware Avenue (18052), United States of America, Wilmington, DelawareLead Data Engineer (Bank Tech)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital On...
Lead Actuarial Data Engineer
A company is looking for a Corporate Vice President, Lead Actuarial Data Engineer/Architect. ...
Lead Data Engineer (Bank Tech)
Delaware Avenue (18052), United States of America, Wilmington, DelawareLead Data Engineer (Bank Tech)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital On...
Azure AI Data Engineer Lead
A company is looking for an Azure AI/Data Engineer Lead for a 100% remote position. ...
Sr Lead Data Platform Engineer, Shopping (Remote-Eligible)
What You’ll Do: Collaborate with and across Agile teams to design, develop, itdetest, implement, and support technical solutions in full-stack development tools and technologies Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems Util...
Technical Lead ( Data Engineer)
As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making. To ensure process compliance in the assigned module, and participate in technical discussionsorreview as a technical consultant fo...