Senior Data Engineer
Sr. Data Engineer
Direct Hire no C2C eligible to work without sponsorship
onsite in West Houston
We have the following Senior Data Engineer role that plays a crucial role in architecting, implementing, and managing robust, scalable data infrastructures.
This position demands a blend of systems engineering, data integration, and data analytics skills to enhance a company’s data capabilities, supporting advanced analytics, machine learning projects, and real-time data processing needs.
ESSENTIAL FUNCTIONS
- Design and implement scalable and reliable data pipelines to ingest, process, and store diverse data at scale, using technologies such as Apache Spark, Hadoop, and Kafka.
- Work within cloud environments like AWS or Azure to leverage services including but not limited to EC2, RDS, S3, Lambda, and Azure Data Lake for efficient data handling and processing.
- Develop and optimize data models and storage solutions (SQL, NoSQL, Data Lakes) to support operational and analytical applications, ensuring data quality and accessibility.
- Utilize ETL tools and frameworks (e.g., Apache Airflow, Talend) to automate data workflows, ensuring efficient data integration and timely availability of data for analytics.
- Collaborate closely with data scientists, providing the data infrastructure and tools needed for complex analytical models, leveraging Python or R for data processing scripts.
- Ensure compliance with data governance and security policies, implementing best practices in data encryption, masking, and access controls within a cloud environment.
QUALIFICATIONS
- Bachelor's degree in computer science, MIS, or other business discipline or equivalent combination of education and / or experience.
- 8+ years of experience in data engineering, with a proven track record in designing and operating large-scale data pipelines and architectures.
- Strong experience in AWS and Azure cloud services, with hands-on experience in integrating cloud storage and compute services with Databricks.
- Expertise in developing ETL / ELT workflows.
- Comprehensive knowledge of platforms and services like Databricks, Dataiku, and AWS native
- Hands on RDBMS experience (data modeling, analysis, programming, stored procedures)
Senior Data Engineer Platform Engineering
A company is looking for a Senior Data Engineer, Platform Engineering. ...
GCP Data Engineer Architect/Senior Architect
GCP’s Data Storage and Processing services such as BigQuery, Dataflow, Bigtable, Dataform, Data fusion, cloud spanner, Cloud SQL. Architect Data Engineering Google Cloud Platform (GCP) is responsible to develop and deliver effective cloud solutions for clients. Experience in complex migrations from ...
Senior Data Platform Engineer
A company is looking for a Senior Data Platform Engineer. ...
Senior Data Engineer
Data Integration and ETL: Develop and maintain advanced Data Engineering processes and patterns using Spark, Python, Scala, Databricks, Azure Data Factory and other relevant tools to integrate data from various sources. Reporting to the Data Engineering Manager, the Sr Data Engineer is a software en...
Senior Data Engineer - ADF Databricks Snowflake
A company is looking for a Senior Data Engineer with expertise in ADF, Databricks, and Snowflake. ...
Senior Data Engineer
If you are a Senior Data Scientist with a passion for solving complex business problems using machine learning and data science, we want to hear from you! Join our team and help us build innovative solutions that drive our business forward. We are seeking a highly skilled Senior Data Scientist to jo...
Senior Data Engineer (MedInsight)
We are seeking a seasoned Senior Data Engineer proficient in Spark and SQL to spearhead the design and enhancement of our data infrastructure. Minimum of 5 years of proven experience as a Data Engineer, with strong command of Databricks/Spark. Develop and uphold scalable data architectures utilizing...
Senior Data Engineer
Data Modeling structure data stored in databases for efficient processing, manage the flow from databases through the database management system and structure the space required for the data to flow without any problem. TASKS & RESPONSIBILITIES</b></p> <ul> <li style="t...
Senior Data Engineer
Scope key business challenges, identify the right data and provide direction to data analysts, data analytics product managers, data scientists, data engineers and business stakeholders. As a Senior Data Engineer, you be responsible for the design and development of data pipelines, automation, and v...
Software Engineer Senior (Database/ Data Modeling)
As the Senior Software Engineer for (Database/ Data Modeling)in Houston, Texas for ENGIE FlexGen (Flexible Generation including Storage, Thermal, and Hydrogen) North America you will be the lead software developer focused on design, development and implementation of the database architecture for the...