AWS Data Engineer
Position : AWS Data Engineer
Employment Type : Contract to Hire
Location : Irving, TX (Onsite)
Job Responsibilities :
Looking for an AWS Data Engineer that has AI understanding and wants to build on this knowledge!
Design, build & maintain scalable data pipelines in AWS to ensure their compliance with up-time, security, and privacy.
Actively contribute to both data & application governance initiatives by designing and implementing policies, procedures, and best practices
Implement API-based integrations to support continuing increases in data volume and complexity
Collaborate with report developers / analysts and business teams to improve data models that feed BI tools and increase data accessibility.
Build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Python and SQL
Assist in the design of processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
Troubleshoot data pipeline related issues and oversee their resolution.
Identify ways to improve data reliability, efficiency, and quality.
Collaborate with all business units and development teams to develop strategy for long term data platform architecture.
Familiarity with one or more DevOps tools (Git, Jenkins, CI / CD, Jira)
Identify and integrate external data sources from various source formats.
Assist and oversee other projects as assigned
Job Requirements :
AI experience or understanding
Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
Experience building and optimizing big data data pipelines, architectures, and data sets.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Strong analytic skills related to working with structured and unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable data stores.
We are looking for a candidate with 5 years of experience in a Data Engineer role, who has attained a degree in Computer Science, Statistics, Information Systems, or another quantitative field.
Experience with data pipeline and workflow management tools.
Experience with AWS cloud services : EC2, Lake Formation, Athena, Glue, Redshift Spectrum, S3 Storage, IAM, and KMS
AWS #data #AI
Data Engineer - Python, SQL, & AWS
This platform, built on AWS, will utilize advanced data technologies to integrate data from various sources for real-time analytics in their application. The Data Engineer will also optimize the SaaS application and establish connections with technology partners in the automotive data field. Develop...
AWS Data Engineer
Looking for an AWS Data Engineer that has AI understanding and wants to build on this knowledge!. Experience building and optimizing big data data pipelines, architectures, and data sets. Build processes supporting data transformation, data structures, metadata, dependency, and workload management. ...
Remote Lead Data Engineer (Databricks/AWS)
In this role, the Data Engineer will be a member of the RISC team and will be a design, develop, and maintain scalable and efficient data pipelines for extracting, transforming, and loading (ETL) data from various sources into our data warehouse leveraging AWS, Databricks, Python and SQL technologie...
SR. DATA ENGINEER with DATABRICKS, AWS, PYSPARK/ SPARK, PYTHON
Must have skills: DATABRICKS, AWS, PYSPARK/ SPARK, PYTHON. Must have good understanding to create complex data pipelines and use of data ingestion tools like nifi. Must have worked on AWS and should have working experience of AWS services like S3, Lambda, DynamoDB, SQS, CloudWatch. Data engineering ...
Data Engineer II ( Python, AWS, Snowflake, Databricks )
Incorporate core data management competencies including data governance, data security and data quality. Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Data Engineer, you will play a key role in growing and trans...
AWS Data Engineer
CGI has an immediate need for a AWS Data Engineer to join our team in Reston, VA, Plano, TX or Lafayette, LA. Working with Architecture team for design and implementation of Big & Fast Data Infrastructure on AWS cloud. Design and develop various on-going projects for migrating on-premises applicatio...
SR. DATA ENGINEER with DATABRICKS, AWS, PYSPARK/ SPARK, PYTHON
Must have skill: DATABRICKS, AWS, PYSPARK/ SPARK, PYTHON. Must have good understanding to create complex data pipelines and use of data ingestion tools like nifi. Must have worked on AWS and should have working experience of AWS services like S3, Lambda, DynamoDB, SQS, CloudWatch. Data engineering b...
Lead Software Engineer (Pyspark/Data/Snowflake/AWS Lead)
As a Lead Software Engineer at JPMorgan Chase within the Home Lending and Auto Technology, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Leads communities of practice across Softwa...
Data Engineer II ( Python, AWS, Snowflake, Databricks )
Incorporate core data management competencies including data governance, data security and data quality. Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Data Engineer, you will play a key role in growing and trans...
SR. DATA ENGINEER with DATABRICKS, AWS, PYSPARK/ SPARK, PYTHON
Must have experience implementing atleast 2 end to end projects with Databricks which includes components like</p> <p>- Delta lake</p> <p>- Unity Catalog</p> <p>- Databricks Connect</p> <p>- DataBricks API</p> <p>- Databricks workflow Orche...