As a Senior Data Engineer, you will play a pivotal role in designing, developing, and maintaining large-scale, high-performance data processing systems.
You will work with cross-functional teams to understand data requirements and implement solutions that are scalable, secure, and reliable.
Your role will involve building data pipelines, managing data storage systems, and ensuring data quality and accessibility across the organization.
Responsibilities :
- Design and develop scalable data architectures and pipelines to support data integration, transformation, and storage.
- Collaborate with Data Scientists, Analysts, and Engineers to define Data Requirements and translate them into technical solutions.
- Manage and optimize data storage systems (e.g., data lakes, data warehouses) for high availability and performance.
- Ensure data integrity, security, and governance practices are adhered to.
- Automate manual data processing tasks and implement ETL processes.
- Maintain and improve data workflows to ensure they are efficient and scalable.
- Monitor and troubleshoot data pipelines to ensure smooth operation and timely data delivery.
- Document technical solutions, best practices, and procedures.
- Stay current with industry trends and emerging technologies in data engineering and analytics.
Required Skills and Qualifications :
- Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
- 10+ years of experience in Data Engineering or related fields.
- Strong proficiency in programming languages such as Python, Java, or Scala.
- Hands-on experience with cloud platforms like AWS, Azure, or Google Cloud.
- Expertise in big data technologies such as Hadoop, Spark, Kafka, and Hive.
- Extensive experience with data warehousing solutions (e.g., Redshift, BigQuery, Snowflake).
- Strong experience in SQL and database design (relational and NoSQL databases).
- Proficiency in designing and implementing ETL pipelines.
- Experience with data governance, security, and compliance frameworks.
- Familiarity with CI / CD tools and DevOps practices.
- Excellent communication and collaboration skills.
Preferred Skills :
- Knowledge of Kubernetes and container orchestration.
- Experience with real-time data streaming and event-driven architectures.
- Familiarity with data visualization and BI tools like Tableau, Power BI
Sr Lead Data Platform Engineer, Shopping (Remote-Eligible)
What You’ll Do: Collaborate with and across Agile teams to design, develop, itdetest, implement, and support technical solutions in full-stack development tools and technologies Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems Util...
Sr. Lab Data Engineer
As a Senior Lab Data Engineer, you will leverage your extensive experience managing, optimizing, and integrating Laboratory Information Management Systems (LIMS) databases. Design, build, and optimize data pipelines to integrate and transform data from various LIMS databases into centralized reposit...
Sr Lead Data Platform Engineer, Shopping (Remote-Eligible)
Center 1 (19052), United States of America, McLean, VirginiaSr Lead Data Platform Engineer, Shopping (Remote-Eligible)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environme...
Sr. Data Engineer
Performance and Innovation: Monitoring and optimizing the Firm’s data platforms (Mostly, but not limited to Microsoft SQL Server), data engineering processes, identifying areas for improvement, and driving innovation to enhance data processing efficiency, data quality, and overall system performance...
Sr Lead Data Platform Engineer, Shopping (Remote-Eligible)
Sr Lead Data Platform Engineer, Shopping (Remote-Eligible). As a Capital One Senior Lead Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transformation within Capital One. Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL data...
Sr. Functional Data Engineer - Investments
Functional Data Engineer (Investments)</b></p> <p><b>Location: Boston, MA (Onsite)</b></p> <p><b>Type: Contract</b></p> <p><b>Description:</b></p> <ul> <li>Strong understanding of Investment - Data manage...
Sr. Data Engineer, Business Intelligence
Has knowledge of data engineering best practices (data discovery, data documentation, infrastructure development/deployment, and information security). Work with a team of data engineers and analysts to build a world-class, self-service data environment that everyone at the company can use. Experien...
Sr. Cloud Engineer (Google Cloud Platform, Azure, Tool Access & Authentication) - Remote
Cloud Engineer will be a Technical Subject Matter Expert / Individual Contributor accountable for expertise and solution concept creation in the areas of cloud identity access management and authentication methods. This role will contribute to the Enterprise Cloud Tool Enablement and Strategy team w...
Sr. AWS Data Engineer
Our mission is clear: to grow our clients' businesses by driving transformative data applications, with a commitment to transparency, best-in-class data analytics, technology, and innovation. By leading with innovation, you'll help shape new product offerings and streamline billions of daily data ev...
Sr. Data Engineer
Data Engineer (AWS, Pyspark/Snowflake/Glue, Tableau or MySQL). Data engineer with extensive working experience using Pyspark, Advanced SQL, Snowflake, complex understanding of SQL and Skills, Spark, Snowflake and Glue. Data Governance (Knowledge about meta data mgmt. Experienced on Big Data platform...