Big Data Developer

Big Data Developer

Location : St. Louis, MO. ONSITE

Job Description :

Follows company software development lifecycle to design, code, configure, test, debug, and document system and application programs. Assists in preparing technical design specifications based on functional requirements and analysis documents. Reviews functional requirements, analysis and design documents and provides feedback. Collaborates with other development staff to achieve quality and consistency. Participates in architecture, design and code reviews. . Develops and maintains operational and system level documentation.

Required

5 - 10 years of IT experience.

6+ years experience in a building robust data pipelines / ETL and data processing

5+ years hands-on experience and proficiency in SQL a must - knowledge of SQL and programming languages in order to optimize data processes and retrieval.

3+ years hands-on experience with PySpark / Spark SQL

3+ years hands-on experience with Linux shell scripting.

3+ years of Apache Spark design and development experience using Scala.

3+ years of experience with Big Data ecosystem like Mapreduce, Hive, iceberg.

3+ years of experience with Data Frames and Resilient Distributed Datasets (RDDs).

3+ years of experience in building and scaling the distributed infrastructure.

3+ years of experience in production cloud infrastructure with AWS services especially S3, Lambda, RDS , EKS, GLUE, EMR services.

3+ years of working with messaging / data streaming services especially Kafka.

3+ years of experience in programing languages such as Java, Python, etc.

3+ Strong expertise with Linux environment preferred

2+ years of Agile scrum experience and JIRA.

2+ years of CI / CD / CT for software deployment and integration.

Experience building API's using REST / GraphQL is plus.

Ability to manage tasks independently and take ownership of responsibilities.

Demonstrates critical thinking skills with ability to communicate across functional departments to achieve desired outcomes.

Excellent interpersonal skills with proven ability to influence with impact across functions and disciplines.

Ability to manage multiple projects / deadlines, identifying the necessary steps and moving forward through completion