Job Summary : We are seeking a skilled AWS Data Engineer with experience in managing and optimizing EMR clusters to join our dynamic team.
The ideal candidate will have a strong background in data engineering, cloud computing, and big data technologies.
Key Responsibilities :
- Design, implement, and manage data pipelines using AWS services, particularly Amazon EMR.
- Optimize EMR cluster configuration for performance and cost efficiency.
- Develop ETL processes to extract, transform, and load data from various sources into data lakes or warehouses.
- Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
- Monitor and troubleshoot EMR clusters to ensure high availability and reliability.
- Implement data governance and security best practices.
- Create and maintain documentation for data engineering processes, workflows, and systems.
- Stay updated with the latest AWS technologies and best practices in data engineering.
Qualifications :
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Data Engineer, with a focus on AWS and EMR.
- Strong knowledge of AWS services (S3, Redshift, Glue, Lambda, etc.).
- Proficiency in programming languages such as Python, Java, or Scala.
- Experience with big data technologies (Hadoop, Spark, etc.).
- Familiarity with SQL and NoSQL databases.
- Excellent problem-solving skills and attention to detail.
- Strong communication skills and the ability to work collaboratively in a team environment.
Preferred Qualifications :
- AWS certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect).
- Experience with data visualization tools (Tableau, Power BI, etc.).
- Knowledge of data warehousing concepts and architectures.
4 days ago