AWS Data Engineer with EMR clusters

Accord Tecnologies.Inc
Piscataway, NJ, United States of America
Part-time

AWS Data Engineer with EMR clusters Piscataway, NJ 12 months Visa : Any visa independent. Data Pipeline Development : Design and implement robust ETL processes to extract, transform, and load data from various sources into data lakes and warehouses.

AWS EMR Clusters : Configure, manage, and optimize Amazon EMR clusters for big data processing using Apache Spark, Hive, or Presto.

Container Orchestration : Utilize Kubernetes for deploying, scaling, and managing containerized applications and services.

CI / CD Implementation : Develop and maintain CI / CD pipelines for automated deployment of data applications and services using tools like Jenkins, GitLab CI, or AWS CodePipeline.

SQL Development : Write complex SQL queries for data manipulation and retrieval, ensuring high performance and scalability.

Data Quality and Governance : Implement data quality checks, monitoring, and logging mechanisms to ensure data reliability and compliance.

Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

Documentation : Maintain comprehensive documentation of data architecture, processes, and workflows. Qualifications : Education : Bachelor'

s degree in Computer Science, Data Science, Information Technology, or a related field. Experience : 3+ years of experience in data engineering or a related role, with a focus on AWS technologies.

AWS Expertise : Proficient in AWS services such as EMR, S3, RDS, Redshift, Lambda, and CloudFormation. Kubernetes Knowledge : Experience with Kubernetes for container orchestration and microservices architecture.

CI / CD Tools : Familiarity with CI / CD tools and practices, including version control using Git. SQL Proficiency : Strong knowledge of SQL, with experience in relational databases (e.

g., MySQL, PostgreSQL) and data warehousing solutions. Programming Skills : Proficiency in programming languages such as Python, Java, or Scala for data processing tasks.

Problem-Solving Skills : Ability to troubleshoot complex data issues and optimize performance. Communication : Excellent communication skills with the ability to work collaboratively in a team environment.

8 days ago
Related jobs
Accord Tecnologies.Inc
Piscataway Township, New Jersey

AWS Data Engineer with EMR clusters Piscataway, NJ 12 months Visa: Any visa independent. AWS EMR Clusters: Configure, manage, and optimize Amazon EMR clusters for big data processing using Apache Spark, Hive, or Presto. Experience: 3+ years of experience in data engineering or a related role, with...

Synechron
Piscataway, New Jersey

We are seeking a skilled AWS Data Engineer with experience in managing and optimizing EMR clusters to join our dynamic team. Proven experience as a Data Engineer, with a focus on AWS and EMR. Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datas...

HAN
Woodbridge Township, New Jersey

Configure, manage, and optimize Amazon EMR clusters for big data processing using Apache Spark, Hive, or Presto. Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. Strong knowledge of SQL, with experienc...

Phaxis LLC
Edison, New Jersey

Our client, a financial services firm based in NJ is looking for a Senior Level Engineer with solid experience in developing and optimizing cloud database platforms. Experience with cloud platforms like AWS or Azure, and the expertise in deploying and managing databases on these platforms. Optimize ...

Synechron
Piscataway, New Jersey

Proven experience with AWS EMR, including configuration and management of EMR clusters. We are seeking an AWS DevOps Engineer [EMR Cluster] who is passionate about data processing and infrastructure automation. Design, implement, and manage AWS EMR clusters for large-scale data processing tasks. AWS...

Radiant Digital
Bedminster, New Jersey

Job Title: AWS and On-Prem DevOps EngineerJob Description:The AWS and On-Prem DevOps Engineer is responsible for designing, implementing, and maintaining AWS and on-premises infrastructure, including RedHat OpenShift Container Platform. The ideal candidate will have a strong understanding of AWS and...

HAN
Woodbridge Township, New Jersey

...

Compunnel Inc.
NJ, United States

Senior Data Engineering ETL Developer who will be responsible for designing and implementing robust ETL pipelines, integrating AWS services such as Redshift and S3, and developing telemetry dashboards that provide insightful analytics on developer activities. At least 10 years of experience in data ...

Accord Tecnologies.Inc
Warren, New Jersey

Required Skills & Experience: AWS (the dataset from the Cloud Engineer and Snowflake data lake), Snowflake, SQL, Python and PowerBI Responsibilities: They will be designing, developing, and deploying a portfolio of Data & Analytics technology assets and platforms leveraging cloud-based tools...

Vanator Technology Pvt. Ltd
NJ, United States

Amazon Web Service frameworks·         3 years of experience with Cybersecurity·         Experience with writing system security plans·         Experience working in environments that leverage continuous deployment with tools, including Jenkins·         Ability to write and implement continuous moni...