Our client is looking Big Data Engineer for Long term project in Multi Location (USA) below is the detailed requirements.
Job Title : Big Data Engineer
Location : Multi Location (USA)
Duration : Contract / Full-Time
Primary Skill :Job Description :
- Using AWS cloud services.
- Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.
- Write high-quality, maintainable code in Python to support data operations.
- Integrate various internal and external data sources (including SharePoint, One Drive, One Note, to create a unified data platform.
- Build tools for document ingestion, dynamic file previewing, and tabular data extraction.
- Manage graph database systems and support large text corpus management.
- Implement security controls to maintain data integrity and compliance.
- Use Large Language Models for keyword extraction and data analysis to gain insights.
Requirements
- Proven experience with Python programming and AWS services (EC2, RDS, S3, Lambda, .
- Proficient in managing and orchestrating workflows with Airflow.
- Solid understanding of CI / CD tools and practices.
- Demonstrated experience with data engineering and familiarity with graph database management.
- Expertise in data processing, corpus management, and keyword extraction.
- Strong problem-solving abilities, attention to detail, and organizational skills.
- Excellent communication and teamwork skills to collaborate with cross-functional teams.
Big Data Engineer with Java - Contract W2
Minimum 7 years of experience as a Big Data Engineer or Java Developer, with a strong focus on big data technologies. Familiarity with data modeling techniques and experience in designing efficient data storage solutions, including relational databases, NoSQL databases, and data lakes. We are seekin...
Data Engineer - Java, Python, GCP, SQL - Next Generation Big Data
Utilize GCP's big data technologies such as BigQuery, Dataflow, Dataprep, and Pub/Sub to implement effective data processing solutions. Consumer Services and Enterprise Digital & Data Technology Team brings together foundational strategic technology capabilities in digital experience engineering...
Data Engineer - Java, Python, Gcp, Sql - Next Generation Big Data
Utilize GCP's big data technologies such as BigQuery, Dataflow, Dataprep, and Pub/Sub to implement effective data processing solutions. Consumer Services and Enterprise Digital & Data Technology Team brings together foundational strategic technology capabilities in digital experience engineering (Mo...
Big Data Engineer -- 1 year contract -- Phoenix, AZ OR New York, NY (Day 1 onsite)
Location: Phoenix, AZ OR New York, NY (Day 1 onsite) .Should be strong in spark, AWS or GCP .Should be very well experienced in Spark Streaming and Scala.Working experience in java and python is good to have....
Engineer - Java, Python, Big Data, NoSQL - Machine Learning
Consumer Services and Enterprise Digital & Data Technology Team brings together foundational strategic technology capabilities in digital experience engineering (Mobile and Web), AI/ML, marketing technology, enterprise communications, travel and lifestyle, and automation, grounded in our data te...
Big Data Engineer with Java
Big Data Engineer with Java Phoenix, AZ Onsite(Hybrid) Contract W2 24+ Months 07+ Years Must-Have: Minimum 7 years of experience as a Big Data Engineer or Java Developer, with a strong focus on big data technologies. Familiarity with data modeling techniques and experience in designing...
Automation QA Engineer / Big Data / ETL Tester
Understanding of streaming data pipelines and how they differ from batch systems understanding of ETL and ELT and ETL/ELT tools such as Data Migration Service etcFamiliarity and/or expertise with Great Expectations or other data quality/data validation frameworks a bonusAdvanced level SQL experience...
Senior Engineer- Big Data
Looking for a self-driven experienced Sr Engineer with good years of experience in big data ecosystem. Part of the ODS Hollow-out and Finance modernization initiatives, he/she would be expected to work on modernizing the data pipelines, leveraging FALCON provided data and using that to create Financ...
Big Data Engineer
SoftTek Solutions Inc is seeking a talented Big Data Engineer to join our team. You will work closely with cross-functional teams to design, implement, and optimize data processing and analytics systems utilizing Hadoop, Spark, and other big data technologies. In this role, you will be responsible f...
Big Data Engineer with GCP
We are hiring an experienced Big Data Engineer with a strong focus on GCP to join our team in Phoenix, AZ. Ability to design, develop, and deploy big data solutions in a GCP environment. Experience with migrating data systems to GCP. ...