job summary : undefined
undefined
responsibilities :
The person on this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.
- Data Ingestion : Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.
- Data Transformation : Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.
- Data Storage : Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.
- Data Processing : Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.
- Data Monitoring and Optimization : Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.
- Integration : Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.
- Documentation : Maintain thorough documentation of data engineering processes, data flows, and system configurations.
- Scalability : Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.
- Cost Management : Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.
- Troubleshooting : Diagnose and resolve AWS-related issues to minimize downtime and disruptions.
- Qualifications
qualifications :
- Experience level : Experienced
- Minimum 9 years of experience
- Education : Bachelors
skills :
- AWS Cloud Formation
- Python
AWS Python Data Engineer- Onsite
Our clients need flexible data structures and a streamlined data architecture that quickly turns data resources into informative, meaningful intelligence. The ideal candidate will have extensive experience in AWS AWS Glue and Python. Develop and maintain high-quality software solutions using AWS AWS...
SQL Server Lead / Azure Cloud / Big Data Engineer
The Lead Engineer will possess knowledge and experience in applications software development principles and methods sufficient to participate in the design, development, testing and implementation of new or modified applications software; operating systems installation and configuration procedures; ...
AWS Data Engineer
Brooksource is searching for an AWS Data Engineer with expertise in data warehousing using AWS Redshift to join our Fortune 500 Energy & Utilities client in Charlotte, NC. Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Le. AWS...
AWS Big Data Engineer
Data Ingestion: Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs. Data Processing: Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets. Data Storage: Manage and optimize da...
Senior Software Engineer Big Data Tooling & API Development
Wells Fargo is seeking a Senior Big Data Engineer in the role of a Big Data Tooling & API Development Software Engineer within its Risk Development organization. The Risk Data management Services group is seeking a Senior Big Data Engineer (Sr. Specialty Software Engineer) to work on building and su...
AWS Data Engineer
AWS Data Engineer, Snowflake Data Engineer. AWS Data Engineer needs 2 to 4 years of IT and business/industry development experience. Experience with AWS product categories like Analytics, Application Integration, Compute, Database, Serverless;. Experience with One or more data Integration Technologi...
SQL Server Lead / Azure Cloud / Big Data Engineer
Expert on data warehouse architecture principles, such as data staging areas, data marts, data lakes, and the overall data flow. Expert in data integration techniques and data quality processes to ensure data accuracy, consistency, and reliability in the data warehouse. Database management principle...
Snowflake Data Engineer - AWS
Title - Snowflake Data Engineer - AWS. Collaborate with data scientists and analysts to understand data requirements and implement solutions Optimize data workflows for performance scalability and reliability. DBT Experience with DBT Data Build Tool for modeling data and creating data transformation...
Senior Big Data Engineer
In partnership with other business, platform, technology, and analytic teams across the enterprise, design, build and maintain well-engineered data solutions in a variety of environments, including traditional data warehouses, Big Data solutions, and cloud-oriented platforms. Align with security, da...
Sr. Cloud Engineer (TS or TS/SCI required)
This engineer will be working to restore and enhance cloud capabilities in support of the warfighter. Proven skills, experience and career path focused on Cloud Services and Solutions (AWS, Azure, GCP, Oracle Cloud). Terraform, ARM Templates, Cloud Formation). ...