Talent.com
serp_jobs.error_messages.no_longer_accepting
Data Engineer (Airflow, PySpark, Dataricks)

Data Engineer (Airflow, PySpark, Dataricks)

Robert HalfWashington, DC, US
job_description.job_card.variable_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
job_description.job_card.job_description

Job Description

Job Description

We are looking for a skilled Software Developer with expertise to join our team in Washington, DC. This is a long-term contract position offering the opportunity to work on innovative projects and contribute to the development of high-quality software solutions. The ideal candidate will possess strong problem-solving skills and collaborate with cross-functional teams to ensure seamless application performance and scalability.

As a Data Engineer, you will :

  • Design and optimize data ingestion and indexing pipelines
  • Work with structured and unstructured data to support real-time search capabilities
  • Develop and maintain workflows using Airflow , PySpark , and Databricks
  • Build and deploy infrastructure using Terraform
  • Index data into Elasticsearch or AWS Search with low latency
  • Troubleshoot networking and cloud infrastructure issues
  • Collaborate on potential API service development (in planning phase)
  • Contribute to infrastructure scalability (possible use of Kubernetes)

Tech Stack

  • Languages & Tools : Python, PySpark, Pandas
  • Cloud & Infrastructure : AWS (Glue, ECS, Lambda, S3, RDS, Athena), Airflow, Databricks, Jupyter Notebooks, Terraform, Elasticsearch / AWS Search
  • Development : Microservices, API development (FastAPI, Django – nice-to-have)
  • Containerization & Orchestration : Docker, Kubernetes (optional)

    • Required Qualifications

  • 3–5+ years of experience in data engineering
  • Strong proficiency in :
  • AWS
  • Python
  • PySpark (basic or better)
  • Airflow
  • Terraform
  • Cloud networking troubleshooting

    • Preferred Skills

  • Experience with Elasticsearch or AWS Search
  • Familiarity with Kubernetes
  • Background in microservices / API development
  • Cloud database setup and optimization
    • serp_jobs.job_alerts.create_a_job

    Data Engineer • Washington, DC, US

    Job_description.internal_linking.related_jobs
    • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    VirtualVocationsAlexandria, Virginia, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer, DEVX.Key Responsibilities Integrate diverse data sources and vendor products for analytical and operational use Automate ETL deployment tasks to enhance...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer II

    Data Engineer II

    VirtualVocationsBaltimore, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer II to develop and operationalize data pipelines for improved health outcomes.Key Responsibilities Design and implement standardized data management proced...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer (AWS, Azure, GCP)

    Data Engineer (AWS, Azure, GCP)

    CapTech ConsultingReston, VA, US
    serp_jobs.job_card.full_time
    CapTech is an award-winning consulting firm that collaborates with clients to achieve what’s possible through the power of technology. At CapTech, we’re passionate about the work we do a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Data Engineer Contractor

    Data Engineer Contractor

    VirtualVocationsWashington, District of Columbia, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer (Python / Databricks) Contractor.Key Responsibilities Design, develop, and deploy enhancements to existing Python-based batch ETL scripts Utilize Databrick...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Data Engineer (Python / PySpark / AWS)

    Data Engineer (Python / PySpark / AWS)

    Infinitive IncAshburn, VA, US
    serp_jobs.job_card.full_time
    Candidates must be local to the Washington D.Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer (Databricks)

    Senior Data Engineer (Databricks)

    Infinitive IncAshburn, VA, US
    serp_jobs.job_card.full_time
    Candidates must possess work authorization which does not require sponsorship by the employer for a visa.Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    IntelliTech LLCArlington, VA, US
    serp_jobs.job_card.temporary
    IntelliTech is seeking experienced Data Engineers to join our team.As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our client's data infrastructure an...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Staff Data Engineer

    Staff Data Engineer

    VirtualVocationsBaltimore, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Staff Data Engineer to design and maintain data systems that drive analytics and product innovation. Key Responsibilities Lead the design, development, and maintenance o...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer IV

    Data Engineer IV

    VirtualVocationsBaltimore, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer IV to develop and optimize data platforms.Key Responsibilities Develop and build interconnected data capabilities and products within the Data Fabric fram...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer

    Senior Data Engineer

    VirtualVocationsRockville, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Engineer.Key Responsibilities : Ensure accurate data flow from creation to presentation layers Enhance the Data Engineering stack through containerization, ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer (Python / PySpark / AWS)

    Senior Data Engineer (Python / PySpark / AWS)

    Infinitive IncAshburn, VA, US
    serp_jobs.job_card.full_time
    Candidates must be local to the Washington D.Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Java Big Data Engineer

    Java Big Data Engineer

    VirtualVocationsAlexandria, Virginia, United States
    serp_jobs.job_card.full_time
    A company is looking for a Java / Big Data Engineer.Key Responsibilities Develop and manage API services on AWS using Java, Scala, or Kotlin Implement and optimize AWS services including Pyspark, ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    ConnsciGaithersburg, MD, US
    serp_jobs.job_card.full_time
    Connsci is seeking a skilled Data Engineer to support the building of an anomaly detection platform for a large government customer. This role will design, build, and manage data pipelines that inte...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer (AI Platforms)

    Data Engineer (AI Platforms)

    VirtualVocationsRockville, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer (AI Platforms) to contribute to building and optimizing data solutions for AI applications. Key Responsibilities Design, build, and optimize scalable data ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Cloud Data Engineer

    Cloud Data Engineer

    VirtualVocationsWashington, District of Columbia, United States
    serp_jobs.job_card.full_time
    A company is looking for a Cloud Data Engineer to design and implement data pipeline services for Azure cloud-based data solutions. Key Responsibilities Maintain and operate legacy ETL processes a...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    BI Lead Data Analytics Engineer

    BI Lead Data Analytics Engineer

    VirtualVocationsAlexandria, Virginia, United States
    serp_jobs.job_card.full_time
    A company is looking for a BI Lead - Data Analytics Engineer.Key Responsibilities Architect and build a centralized data lake to unify various data types Develop pipelines for advanced multi-tou...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Lead Data Engineer

    Lead Data Engineer

    VirtualVocationsAlexandria, Virginia, United States
    serp_jobs.job_card.full_time
    A company is looking for a Lead Data Engineer to lead the development and operationalization of data pipelines for improved health outcomes. Key Responsibilities Lead the design and implementation...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Senior Data Discovery Engineer

    Senior Data Discovery Engineer

    VirtualVocationsBaltimore, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Engineer - Data Discovery.Key Responsibilities Design, architect, and deploy data discovery and inventory platform Administer tools and processes to implement P...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Data Engineer III

    Data Engineer III

    VirtualVocationsRockville, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer III.Key Responsibilities Drive cloud solutions utilizing Azure DevOps and Data Bricks Lead data stewardship workgroups and champion metadata and data qua...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Analytics Engineer

    Analytics Engineer

    VirtualVocationsBaltimore, Maryland, United States
    serp_jobs.job_card.full_time
    A company is looking for an Analytics Engineer to design and maintain product data pipelines.Key Responsibilities Design, build, and optimize scalable data models using SQL, dbt, and cloud-based ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30