Data Engineer (Houston)

Data Engineer - Databricks Performance & Optimization

Location : Houston, TX

Compensation : Base salary range : $130,000$150,000 plus 5% bonus. Benefits package available (details to be discussed).

Qualifications : Build and operate Lakehouse pipelines on Databricks (Bronze / Silver / Gold) using Delta Lake, Delta Live Tables (DLT), and / or Jobs.

Optimize ingestion patterns (Autoloader, CDC, streaming).

Model data, implement quality checks, and performance optimization.

Profile and tune Spark / SQL workloads : partitioning, clustering, constraints, liquid clustering.

Job Description : Engineer Delta tables for speed and cost : partitioning, Z-Ordering / clustering, constraints, file sizing; manage table health with Auto Optimize, OPTIMIZE, and VACUUM.

Implement incremental processing (MERGE with Change Data Feed, APPLY CHANGES INTO) with idempotency and exactly-once delivery.

Deliver reliable, well-documented datasets with clear SLAs.

Design and implement dashboards and reports using Power BI and other visualization tools.

Collaborate with business units to gather requirements and deliver technical solutions.

Integrate data from multiple sources, including real-time field equipment and sensors.

Educate and support stakeholders on data tools and best practices.

Engage in continuous improvement and adoption of new data management technologies.