Talent.com
Senior Data Engineer

Senior Data Engineer

People Data LabsSan Francisco, CA, US
job_description.job_card.30_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.permanent
job_description.job_card.job_description

Job Description

Job Description

Note for all engineering roles : with the rise of fake applicants and AI-enabled candidate fraud, we have built in additional measures throughout the process to identify such candidates and remove them.

About Us

People Data Labs (PDL) is the provider of people and company data. We do the heavy lifting of data collection and standardization so our customers can focus on building and scaling innovative, compliant data solutions. Our sole focus is on building the best data available by integrating thousands of compliantly sourced datasets into a single, developer-friendly source of truth. Leading companies across the world use PDL's workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more.

We are looking for individuals who can balance extreme ownership with a "one-team, one-dream" mindset. Our customers are trying to solve complex problems, and we only help them achieve their goals as a team. Our Data Engineering Team is the secret sauce behind all that we do and we are looking for the best of the best.

If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it.

What You Get to Do

  • Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks
  • Building an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets.
  • Developing CI / CD pipelines and anomaly detection systems capable of continuously improving the quality of data we're pushing into production.
  • Dreaming up solutions to largely undefined data engineering and data science problems.

The Technical Chops You'll Need

  • 5-7+ years of industry experience with clear examples of strategic technical problem-solving and implementation
  • Strong software development fundamentals.
  • Experience with Python
  • Expertise with Apache Spark (Java, Scala, and / or Python-based)
  • Experience with SQL
  • Experience building scalable data processing systems (e.g., cleaning, transformation) from the ground up.
  • Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar)
  • Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
  • Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.)
  • Experience with cloud computing services (AWS (preferred), GCP, Azure or similar)
  • Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar)
  • Understanding of modern data storage formats and tools (e.g., parquet, ORC, Avro, Delta Lake)
  • People Thrive Here Who Can

  • Balance high ownership and autonomy with a strong ability to collaborate
  • Work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities)
  • Demonstrate strong written communication skills on Slack / Chat and in documents
  • Exhibt experience in writing data design docs (pipeline design, dataflow, schema design)
  • Scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders
  • Some Nice To Haves

  • Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering
  • Experience working with entity data (entity resolution / record linkage)
  • Experience working with data acquisition / data integration
  • Expertise with Python and the Python data stack (e.g., numpy, pandas)
  • Experience with streaming platforms (e.g., Kafka)
  • Experience evaluating data quality and maintaining consistently high data standards across new feature releases (e.g., consistency, accuracy, validity, completeness)
  • Our Benefits

  • Stock
  • Competitive Salaries
  • Unlimited paid time off
  • Medical, dental, & vision insurance
  • Health, fitness, and office stipends
  • The permanent ability to work wherever and however you want
  • Comp : $190K - $210K

    People Data Labs does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.

    Qualified Applicants with arrest or conviction records will be considered for Employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.

    Personal Privacy Policy for California Residents

    https : / / www.peopledatalabs.com / pdf / privacy -policy-and-notice.pdf

    serp_jobs.job_alerts.create_a_job

    Senior Data Engineer • San Francisco, CA, US

    Job_description.internal_linking.related_jobs
    • serp_jobs.job_card.promoted
    Senior / Lead Data Solution Engineer

    Senior / Lead Data Solution Engineer

    MeltwaterSan Francisco, CA, United States
    serp_jobs.job_card.full_time
    We're thrilled to embark on the search for a seasoned.Senior / Lead Data Solution Engineer.This pivotal role offers an exciting opportunity to shape the future of technology within our organization.A...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Lead Data Engineer

    Lead Data Engineer

    VirtualVocationsConcord, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Lead Data Engineer to lead the development and operationalization of data pipelines for improved health outcomes. Key Responsibilities Lead the design and implementation...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Analytics Engineer

    Senior Analytics Engineer

    VirtualVocationsSanta Clara, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Analytics Engineer.Key Responsibilities Design and maintain data models and transformation logic for analytics and reporting Own the semantic layer, managing sh...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer

    Senior Data Engineer

    VirtualVocationsOakland, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Data Engineer.Key Responsibilities : Ensure accurate data flow from creation to presentation layers Enhance the Data Engineering stack through containerization, ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer II

    Data Engineer II

    VirtualVocationsHayward, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer II to develop and operationalize data pipelines for improved health outcomes.Key Responsibilities Design and implement standardized data management proced...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data & Analytics Engineer

    Senior Data & Analytics Engineer

    MeshySunnyvale, CA, US
    serp_jobs.job_card.full_time
    Headquartered in the Silicon Valley, Meshy is the leading 3D generative AI company on a mission to.Meshy makes it effortless for both professional artists and hobbyists to create unique 3D assets&m...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_days
    • serp_jobs.job_card.promoted
    Data Engineer IV

    Data Engineer IV

    VirtualVocationsHayward, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer IV to develop and optimize data platforms.Key Responsibilities Develop and build interconnected data capabilities and products within the Data Fabric fram...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer (AI Platforms)

    Data Engineer (AI Platforms)

    VirtualVocationsFremont, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer (AI Platforms) to contribute to building and optimizing data solutions for AI applications. Key Responsibilities Design, build, and optimize scalable data ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer

    Senior Data Engineer

    VisaFoster City, CA, United States
    serp_jobs.job_card.full_time
    Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more t...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Software Engineer

    Senior Data Software Engineer

    PsiQuantumPalo Alto, CA, United States
    serp_jobs.job_card.full_time
    Quantum computing holds the promise of humanity's mastery over the natural world, but only if we can build a.PsiQuantum is on a mission to build the first real, useful quantum computers, capable of...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Data Engineer

    Senior Data Engineer

    Plum IncSan Francisco, CA, US
    serp_jobs.job_card.full_time
    PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior Databricks Engineer

    Senior Databricks Engineer

    VirtualVocationsOakland, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Databricks Platform Engineer.Key Responsibilities Design and configure Databricks Unity Catalog and medallion-tiered environments for scalable data engineering ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Senior AI Engineer

    Senior AI Engineer

    VirtualVocationsHayward, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior AI Engineer II.Key Responsibilities Own end-to-end implementation of AI-powered product features, from prototypes to production Handle prompt engineering, model...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Staff Data Engineer

    Staff Data Engineer

    VirtualVocationsConcord, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Staff Data Engineer to design and maintain data systems that drive analytics and product innovation. Key Responsibilities Lead the design, development, and maintenance o...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    • serp_jobs.job_card.new
    Senior Data Discovery Engineer

    Senior Data Discovery Engineer

    VirtualVocationsOakland, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Senior Engineer - Data Discovery.Key Responsibilities Design, architect, and deploy data discovery and inventory platform Administer tools and processes to implement P...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_variable_hours
    • serp_jobs.job_card.promoted
    Java Big Data Engineer

    Java Big Data Engineer

    VirtualVocationsHayward, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Java / Big Data Engineer.Key Responsibilities Develop and manage API services on AWS using Java, Scala, or Kotlin Implement and optimize AWS services including Pyspark, ...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_1_day
    • serp_jobs.job_card.promoted
    Data Engineer III

    Data Engineer III

    VirtualVocationsFremont, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer III.Key Responsibilities Drive cloud solutions utilizing Azure DevOps and Data Bricks Lead data stewardship workgroups and champion metadata and data qua...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30
    • serp_jobs.job_card.promoted
    Data Engineer

    Data Engineer

    VirtualVocationsFremont, California, United States
    serp_jobs.job_card.full_time
    A company is looking for a Data Engineer, DEVX.Key Responsibilities Integrate diverse data sources and vendor products for analytical and operational use Automate ETL deployment tasks to enhance...serp_jobs.internal_linking.show_moreserp_jobs.last_updated.last_updated_30