AWS Data Engineer

Hiring AWS Data Engineer Need to work on our W2

Position : AWS Data Engineer

Location : San Francisco, CA and Jersey City, NJ - Hybrid

Job Description :

As an AWS Data Engineer, you will be instrumental in designing, developing, and managing our data infrastructure. You will be tasked with integrating various data sources into our data lake, developing robust data pipelines, and ensuring data quality and governance.

Your knowledge of AWS services and best practices in data engineering will be key in advancing our data initiatives.

Key Responsibilities :

Data Onboarding : Integrate diverse data sources into the data lake, ensuring smooth integration and data consistency.

Data Pipeline Development : Design, build, and maintain scalable, efficient data pipelines using AWS services such as Lambda, Step Functions, and EMR.

Data Registration : Manage metadata and register data sources to enhance data discoverability and accessibility.

Data Quality Management : Implement data validation checks and transformations to guarantee data accuracy and reliability.

Data Governance : Adhere to data governance standards to ensure data security, privacy, and regulatory compliance.

Infrastructure as Code : Use Terraform scripts to automate and manage AWS infrastructure.

Data Processing : Utilize Spark and other big data technologies for large-scale data processing and analysis.

Orchestration : Employ tools like Airflow and Step Functions to orchestrate complex data workflows.

Data Modeling : Design and optimize data storage solutions using Snowflake, Iceberg table formats, and other data modeling tools.

Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver quality solutions.

Required Skills and Qualifications :

AWS Expertise : Proficient in AWS Lake Formation, Step Functions, Lambda (serverless), EC2, EMR, and EKS.

Programming and Scripting : Strong experience with Python and Terraform scripting.

Data Tools : Skilled in using Jupyter Notebook, RDS, Snowflake, and Iceberg table formats.

Big Data Technologies : Expertise in Spark and orchestration tools like Airflow and dbt for managing data pipelines.

Data Engineering Fundamentals : Solid understanding of ETL processes, data warehousing, and data modeling techniques.

Data Governance : Knowledgeable in data governance standards and best practices.

Problem-Solving Skills : Strong analytical skills with the ability to troubleshoot and resolve data-related issues.

Communication : Excellent communication skills with a proven ability to collaborate effectively with cross-functional teams.

Preferred Qualifications :

Certifications : AWS Certified Data Engineer or Analytics Specialty, AWS Certified Solutions Architect, or similar certifications.

Experience : Previous experience in a fast-paced, data-centric environment in a similar role.