Microsoft Azure Data Engineer

Purple DriveBOSTON, Massachusetts, USA

job_description.job_card.30_days_ago

serp_jobs.job_preview.job_type

serp_jobs.job_card.full_time

job_description.job_card.job_description

Microsoft Azure~Digital

6-9 yrs

Data Engineer - Databricks

Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL / ELT using Data pipelines, SQL / Procedure experience.
Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks / SQL.
Must have 8+ Years of hands-on development and performance tuning / enhancement experience in Scala / PySpark and SQL.
Must have 8+ Years of hands-on development experience in working on Azure Cloud.
Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
Must have experience and a good understanding on the below topics

Databricks Delta Lake storage

Unity Catalog

Autoloader

Lakehouse Architecture

Medallion Architecture

Databricks Serverless Options.

Dataset and Data frame

Databricks Clusters Configuration and Sizing

Databricks Job Orchestration

Spark Structured Streaming

Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.

Must have extensive experience in Datawarehouse / Data Lakehouse implementations.

Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.

Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.

Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.

Good to have experience with containerization technologies such as Docker and Kubernetes.

Strong critical thinking, communication, and problem-solving skills is essential.

serp_jobs.job_alerts.create_a_job

Azure Data Engineer • BOSTON, Massachusetts, USA

Job_description.internal_linking.related_jobs

serp_jobs.job_card.promoted

Azure Data Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for an Azure Developer-Data Engineer.Key Responsibilities Design, develop, and deploy scalable Azure-based data ingestion, transformation, and curation pipelines Implement d...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Principal Data Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Principal Data Engineer with extensive experience in high-transaction e-commerce environments. Key Responsibilities Develop and maintain ETL / ELT pipelines in Python and ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Engineer I

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Engineer I to develop and operationalize data pipelines for improved health outcomes.Key Responsibilities Design and implement standardized data management procedu...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Engineer (AI / ML)

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Engineer (AI / ML) 2 to support the Generative AI for Forms SBIR project by designing and maintaining data infrastructure for AI applications.Key Responsibilities De...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days

serp_jobs.job_card.promoted

Senior Data Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Senior Data Engineer to lead the development of scalable data processing and enrichment pipelines. Key Responsibilities Design and implement scalable AI-powered data pro...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Architect (MS Fabric)

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Architect (MS Fabric) - Remote.Key Responsibilities Design and implement end-to-end data architecture solutions using Microsoft Fabric Define and establish data g...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days

serp_jobs.job_card.promoted

Data Engineer Contractor

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for an Independent Contractor, Data Engineer, Business Operations.Key Responsibilities Build and maintain a scalable Databricks lakehouse for business operations data Design...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days

serp_jobs.job_card.promoted

Data Engineer III

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Engineer III to design, develop, and maintain data pipelines and systems.Key Responsibilities Design, develop, and maintain data pipelines and infrastructure using...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Lead Data and ML Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Lead Data / ML Engineer to build scalable, cost-efficient systems for transforming ad data into reliable intelligence. Key Responsibilities Design, build, and maintain sca...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days

serp_jobs.job_card.promoted

Senior Data Analytics Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Senior Data Analytics Engineer.Key Responsibilities Design and implement advanced data models and analytics solutions Develop and optimize scalable data pipelines and ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Senior Databricks Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Senior Databricks Engineer / Cloud Data Engineer (Remote).Key Responsibilities Design, build, and optimize data pipelines using Databricks, Apache Spark, and Delta Lake...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Scientist Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Scientist Engineer to join a specialized project team focused on building and maintaining production-level data pipelines. Key Responsibilities Design, develop, and...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day

serp_jobs.job_card.promoted

Principal Cloud Engineer

RaftHanscom Air Force Base, MA, United States

serp_jobs.job_card.full_time

All of the programs we support require.All work must be conducted within the continental U.Distributed Data Systems, Platforms at Scale, and Complex Application Development, with headquarters in Mc...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Engineer II

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Engineer II.Key Responsibilities Designing, developing, and maintaining software applications that process large volumes of data Collaborating with cross-function...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Science Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Science Engineer to implement data science and machine learning solutions.Key Responsibilities Collaborate with teams to understand business requirements and impro...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Lead Data Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Lead Data Engineer to architect and scale their data platform and analytics stack.Key Responsibilities Refine and evolve the AWS data platform to enhance analytical cap...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Data Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Data Engineer to design, construct, and maintain scalable data management systems.Key Responsibilities Develop and maintain data architectures, including databases and ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Senior Data Infrastructure Engineer

VirtualVocationsDorchester, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Senior Data Infrastructure Engineer to enhance its core data infrastructure and support advanced applications. Key Responsibilities Own and maintain data pipeline archit...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30

serp_jobs.job_card.promoted

Cloud Data Operations Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Cloud Data Operations Engineer (Technology Specialist I).Key Responsibilities Support the planning, design, implementation, and deployment of Data Cloud services on Mic...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days

serp_jobs.job_card.promoted

Senior Data Platform Engineer

VirtualVocationsLowell, Massachusetts, United States

serp_jobs.job_card.full_time

A company is looking for a Senior Data Platform Engineer.Key Responsibilities Enhance and improve the performance and reliability of existing Data Platform Services Design and build self-service...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30