Our client is looking Big Data Engineer for Long term project in Multi Location (USA) below is the detailed requirements.
Job Title : Big Data Engineer
Location : Multi Location (USA)
Duration : Contract / Full-Time
Primary Skill :Job Description :
- Using AWS cloud services.
- Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.
- Write high-quality, maintainable code in Python to support data operations.
- Integrate various internal and external data sources (including SharePoint, One Drive, One Note, to create a unified data platform.
- Build tools for document ingestion, dynamic file previewing, and tabular data extraction.
- Manage graph database systems and support large text corpus management.
- Implement security controls to maintain data integrity and compliance.
- Use Large Language Models for keyword extraction and data analysis to gain insights.
Requirements
- Proven experience with Python programming and AWS services (EC2, RDS, S3, Lambda, .
- Proficient in managing and orchestrating workflows with Airflow.
- Solid understanding of CI / CD tools and practices.
- Demonstrated experience with data engineering and familiarity with graph database management.
- Expertise in data processing, corpus management, and keyword extraction.
- Strong problem-solving abilities, attention to detail, and organizational skills.
- Excellent communication and teamwork skills to collaborate with cross-functional teams.
Big Data Engineer
A company is looking for a Big Data Engineer to support internal business stakeholders with their big data capabilities. ...
Big Data Engineer
Demonstrated experience with data engineering and familiarity with graph database management. Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG. Integrate various internal and external data sources (including SharePoint, One Drive, One N...
Engineer - Big Data - LUMI
Consumer Services and Enterprise Digital & Data Technology Team brings together foundational strategic technology capabilities in digital experience engineering (Mobile and Web), AI/ML, marketing technology, enterprise communications, travel and lifestyle, and automation, grounded in our data techno...
Engineer - Java / python / Big Data / SQL/ NoSQL - Machine Learning
Consumer Services and Enterprise Digital & Data Technology Team brings together foundational strategic technology capabilities in digital experience engineering (Mobile and Web), AI/ML, marketing technology, enterprise communications, travel and lifestyle, and automation, grounded in our data techno...
Big Data Engineer
This role requires a seasoned developer having + years of experience in Big Data, Java. The ideal candidate will be having sound knowledge of big data ecosystem esp. Spark and handling massive data sets. ...
Automation QA Engineer / Big Data / ETL Tester
Understanding of streaming data pipelines and how they differ from batch systems understanding of ETL and ELT and ETL/ELT tools such as Data Migration Service etcFamiliarity and/or expertise with Great Expectations or other data quality/data validation frameworks a bonusAdvanced level SQL experience...
Big Data Engineer with Java
Big Data Engineer with Java Phoenix, AZ Onsite(Hybrid) Contract W2 24+ Months 07+ Years Must-Have: Minimum 7 years of experience as a Big Data Engineer or Java Developer, with a strong focus on big data technologies. Familiarity with data modeling techniques and experience in designing...
Senior Engineer- Big Data
Looking for a self-driven experienced Sr Engineer with good years of experience in big data ecosystem. Part of the ODS Hollow-out and Finance modernization initiatives, he/she would be expected to work on modernizing the data pipelines, leveraging FALCON provided data and using that to create Financ...
Senior Big Data Engineer
In partnership with other business, platform, technology, and analytic teams across the enterprise, design, build and maintain well-engineered data solutions in a variety of environments, including traditional data warehouses, Big Data solutions, and cloud-oriented platforms. Align with security, da...
Big Data Engineer -- 1 year contract -- Phoenix, AZ OR New York, NY (Day 1 onsite)
Location: Phoenix, AZ OR New York, NY (Day 1 onsite) .Should be strong in spark, AWS or GCP .Should be very well experienced in Spark Streaming and Scala.Working experience in java and python is good to have....