Data Engineer - DataBricks

Intelliswift Software
CA, United States
Temporary

Title : Data Engineer

Location : San Francisco, CA (1-2 Days onsite / Week)

Duration : 12+ Month Contract

Key Responsibilities :

  • Design and implement scalable data pipelines using Databricks and Spark.
  • Develop and maintain data processing applications in Scala and Python.
  • Integrate and manage data streams using Kafka.
  • Build and maintain microservices using Spring Boot.
  • Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
  • Optimize and troubleshoot data workflows to ensure high performance and reliability.
  • Ensure data quality and integrity across various data sources.

Required Skills and Qualifications :

  • Bachelor’s degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Data Engineer or in a similar role.
  • Proficiency in Databricks, Spark, Scala, Kafka, Python, and Spring Boot.
  • Strong understanding of microservices architecture.
  • Experience with cloud platforms (e.g., AWS, Azure, GCP) is a plus.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and teamwork abilities.
  • 5 hours ago
Related jobs
Promoted
Intelliswift Software
CA, United States

Design and implement scalable data pipelines using Databricks and Spark. Collaborate with data scientists and analysts to understand data requirements and deliver solutions. Ensure data quality and integrity across various data sources. Proven experience as a Data Engineer or in a similar role. ...

Promoted
DataBricks
San Francisco, California

As a Staff Software Engineer working on the Data Platform team you will help build the Data Intelligence Platform for Databricks. You will do this using the latest, bleeding-edge Databricks product and other tools in the data ecosystem - the team also functions as a large, production, in-house custo...

Promoted
Databricks Inc.
Mountain View, California

More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. At Databricks, we are obsessed with enabling data teams to solve the world's toug...

Promoted
Diverse Lynx
San Francisco, California

Role: Data Engineer - Databricks. Any relational database Data classification/Data profiling for MLOPS use cases. Developing modern Data warehouse solutions using Databricks and AWS/Azure stack. Knowledge in Databricks DELTA lake for the Analytical data lake use case. ...

Promoted
Databricks Inc.
Mountain View, California

As a software engineer on the Runtime team at Databricks, you will be building the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support...

Promoted
DataBricks
San Francisco, California

The role that the Senior Data Engineer plays is highly collaborative and, as such, works closely with data analysts, data scientists, and other data consumers within the business in an attempt to gather and populate data warehouse table structure, which is optimized for reporting. The Senior Data En...

Ursus
Foster City, California

Our client is helping an on-demand, autonomous ride-hailing company hire an experienced Data Engineer to manage and maintain company data stack tools. Advanced understanding of SQL to extract data from the databases. In this role, you will be responsible for various duties that will include user man...

Highmark Health
CA, Working at Home, California

This role within the 'Data Engineering & Self-Service Products' team involves architecting and engineering analytic data solutions, including designing and developing data marts in Databricks using PySpark or Spark SQL, building interactive Power BI dashboards to visualize KPIs and trends, and creat...

Databricks
San Francisco, California

As a software engineer on the Runtime team at Databricks, you will be building the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support...

Databricks
Mountain View, California

As a software engineer on the Runtime team at Databricks, you will be building the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support...