Senior Data Software Engineer - Spark - Databricks

Description

DESCRIPTION

Join EPAM as a Senior Data Software Engineer.

In this role, you'll manage and maintain data infrastructure, implement and optimize data processing workflows, and collaborate with cross-functional teams to understand data requirements.

If you have a BA / BS Degree in a related field, 6-8 years of related work experience, and a strong working knowledge of Python and PySpark, we'd love to hear from you.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities.

We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow.

No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Responsibilities

• Manage and maintain data infrastructure to ensure secure, reliable, and accurate access for stakeholders
• Implement and optimize data processing workflows using Spark, PySpark, and Spark SQL
• Collaborate with cross-functional teams to understand data requirements and deliver solutions
• Ensure fault-tolerant infrastructure designs using regions and availability zones
• Extract entities and relationships from unstructured data and handle data indexing

Requirements

• BA / BS Degree (4-year) in a related field and 6-8 years of related work experience
• Experience on a cloud-based Data Engineering Team and strong working knowledge of Python and PySpark
• Advanced knowledge in entity and relationship extraction from unstructured data
• Proficient in data handling, analysis, and interpretation
• Experience with SQL technologies : Databricks (Spark), Azure Synapse and SQL Server, Google BigQuery, and Firebase

Nice to have

Knowledge of C# or other object-oriented languages

Technologies

• Spark, PySpark, Spark SQL
• Azure SQL Server, Azure Synapse
• Databricks, Google BigQuery, Firebase
• Kafka, Stream Analytics
• Event Hubs
• Azure functions
• Microsoft SQL, MySQL

We Offer

• Career plan and real growth opportunities
• Unlimited access to LinkedIn learning solutions
• International Mobility Plan within 25 countries
• Constant training, mentoring, online corporate courses, eLearning and more
• English classes with a certified teacher
• Support for employees initiatives (Algorithms club, toastmasters, agile club and more)
• Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)
• Flexible work schedule and dress code
• Collaborate in a multicultural environment and share best practices from around the globe
• Hired directly by EPAM & % under payroll
• Law benefits (IMSS, INFONAVIT, 25% vacation bonus)
• Major medical expenses insurance : Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)
• 13 % employee savings fund, capped to the law limit
• Grocery coupons
• 30 days December bonus
• Employee Stock Purchase Plan
• 12 vacations days plus 4 floating days
• Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)
• Relocation bonus : transportation, 2 weeks of accommodation for you and your family and more
• Monthly non-taxable amount for the electricity and internet bills
30+ days ago