Job Summary :
- Role : Sr. Engineer, Analytics (Streaming)
- Location : Remote
- Duration : Full-time
- Involved in designing, building, and maintaining real-time analytics infrastructure
- Works with data scientists, product managers, and other engineers to develop and deploy data pipelines
Responsibilities :
- Design, build, and maintain large-scale data pipelines using technologies such as Apache Beam, Kafka, Kinesis, Google Cloud Pub / Sub, and DataFlow
- Develop and implement streaming data processing jobs using Java or Python
- Collaborate with data scientists and product managers to develop and deploy real-time analytics applications
- Collaborate with other engineers to integrate data pipelines with data warehouse and data lake
- Work with cloud providers such as Google Cloud Platform (GCP), Amazon Web Services (AWS), or Microsoft Azure
- Monitor and troubleshoot streaming pipelines to ensure high availability and performance
- Implement DevOps principles for efficient and reliable deployment of batch processing systems
- Knowledge of containerization technologies such as Docker and Kubernetes
Requirements :
- Bachelor's degree in computer science or a related field
- Minimum of 5 years of experience in software engineering with focus on data pipelines and analytics
- Proficient in Java or Python programming
- Experience with a public cloud provider, preferably GCP
- Experience with messaging / stream processing systems such as Apache Kafka, Apache Kinesis, Google Cloud Pub / Sub, and Google Cloud DataFlow
- Experience with data warehousing and data lake technologies
- Strong understanding of data modeling, data governance, and data security
- Excellent problem-solving skills and can work independently and collaboratively
- Excellent communication skills, ability to explain complex technical concepts to non-technical stakeholders
- Experience with machine learning and data science technologies is a plus
- Certification in a public cloud provider or a relevant technology is a plus.
3 days ago