Responsibilities
- Build and maintain scalable ETL pipelines effectively transform data from disparate data sources into a canonical data store
- Design data architecture and partner with our infrastructure team to execute the build out of the data foundation
- Operate current data ingest services while building the next generation of ETL pipelines
- Define a common data vocabulary across the organization
- Develop audit reports to provide insights into overall system health and resiliency
- Contribute to the technology roadmap
Requirements :
- Bachelor’s degree in Information Technology, Computer Science, or related field
- 5+ years as a Data Engineer working on large scale ETL systems
- AWS experience developing Integrations and analytical services
- Demonstrated proficiency in Python and SQL Spark / PySpark
- Experience on distributed system & No SQL databases
- Experience with one or more commercial or open-source technologies like Databricks, Informatica, Talend, Airflow, Kafka is essential
- Experience collaborating with data scientists and data analyst on projects related to forecasting, dashboards, and reports
- Passionate about mentoring other team members and foster a culture of open feedback and communication
- Able to communicate effectively with a technical and non-technical audience
- Experience building web-based reporting solutions with data visualization technologies like Power BI, Tableau etc. is a plus
- Able to contribute to prototypes and architecture by contributing code and designs
- Experience working with Product Management in an Agile environment and break down complex features into manageable user stories
- Demonstrated track record building data solutions based on a data lake is a plus
- AWS, Azure, or GCP cloud certification
- Databricks certification
- Prior experience with object-oriented programming languages
- Experience on search engines (Elasticsearch / Algolia)
Compensation :
- $160,000 to $180,000, depending on experience
- 10% bonus
30+ days ago