Talent.com
MRC Confluent Kafka Senior Architect

MRC Confluent Kafka Senior Architect

Ness Digital EngineeringChicago, IL, US
job_description.job_card.30_days_ago
serp_jobs.job_preview.job_type
  • serp_jobs.job_card.full_time
  • serp_jobs.filters.remote
job_description.job_card.job_description

Description

Ness is seeking an DevOps Architect with Confluent Kafka (MRC / CFK), Kubernetes expertise. Customer is required by regulators to demonstrate the ability to run all clearing and risk systems through internal datacenter ("On-Prem") and to be able to switch all production functionality running through On-Prem within the predefined SLA. The newly developed clearing and risk systems takes advantage of scalability, ease of provision, and the reduced cost offered by cloud services (AWS) and the cloud offered Multi-Region as Disaster Recovery (DC) strategy.

Responsibilities :

  • As a DevOps Architect, setup CI / CD pipeline for application deployment in Kubernetes
  • Setup Confluent Kafka (MRC or CFK) across all the regions in hybrid environment
  • Setup Rancher and downstream Kubernetes clusters in multiple environments and regions
  • Setup IaC pipelines using TFE and Ansible to automate the Kafka cluster deployments
  • Setup IaC pipelines using TFE and Ansible to automate the Rancher and downstream K8 cluster deployments
  • Work in the DevOps team, to build new shared infrastructure services for on premises failover environment : S3, Confluent Kafka, Data Store
  • Work with DevOps team to establish connectivity between new failover shared services and existing shared services : Secret, identity, LDA, DNS, Artifactory, Jenkins, Splunk services
  • Work with DevOps team to automate deployment DR strategy Automation of data replication between Cloud and failover environment required between all applications and shared services CI / CD deployment tools
  • Continuously improve the processes and the DevOps team using thoughtful, calculated approaches to identify opportunities, and challenge those around you to strive for perfection.
  • Ideate solutions to complex technical challenges; code, test, troubleshoot, debug, and document the solutions you develop. Use agile software development model to produce well-designed programs, scripts, and tools required to provision, configure, and monitor new shared infrastructure services for on premises failover environment

Must have :

  • We need someone who's good at setting up CI / CD pipelines using DevOps tools like Jenkins, Artifactory, Confluent CFK, Kubernetes, Vault, SonarQube, GitHub, Terraform, Ansible, Rancher, and Harness. It's also important that DevOps Architect has experience with deploying the applications on Kubernetes using Rancher, Helm Charts and Harness. Plus, he / she should know about deploying Kafka MRC or CFK and the use of monitoring tools like Logic Monitor and Splunk. And it'd be great if he / she has experience with AWS services and on-prem as hybrid environment, especially setting up Confluent CFK or MRC, and know how to do S3 replication with Cloudian.
  • Kubernetes Administration :
  • Helm
  • CRDs
  • K8 Operators
  • All general things in K8 including PVCs
  • CFK Administration :
  • Kafka
  • Kraft / Zookeeper
  • Kafka Connect
  • Kafka Monitoring like consumer lag, broker health, topics growth, etc.
  • Kafka Cluster Management (Topics, Partitions, Brokers, etc.)
  • Kafka Backup Solution and Support
  • Pilot Kafka data replication between AWS and On-Prem
  • Experience with Rancher RKE2 K8s clusters and orchestration platform
  • K8S cluster management, Monitoring, operational tools integration
  • K8s cluster and secrets management integration
  • K8s CSI plug-ins with object, file and block storage vendors
  • Kubernetes backup solution and support
  • S3 / Cloudian object storage for shared infrastructure
  • Specializing in Cloud Infrastructure Modernization, virtualization, data center setup, DR & BC Strategies, and DevOps
  • Experience in on-premises data center operations, AWS hosted data center and operations management
  • Hands on experience with migrations tools
  • Experience with continuous deployment tools, techniques, and automation frameworks - especially Terraform Enterprise and Ansible.
  • Hands-on experience writing testable scripts using Python or other languages.
  • Experience managing helm charts and deploying into Kubernetes (K8s)
  • Expertise with monitoring related tools and frameworks like Splunk, LogicMonitor, SignalFX, and Prometheus.
  • Worked on projects involving deployment and management of micro services, and hybrid cloud / on-prem infrastructure
  • Intermediate working knowledge of development tools like Maven / Gradle, Java, and distributed message brokers like Kafka, RabbitMQ, ActiveMQ, Amazon Kinesis, etc.
  • Good to Have
  • Knowledge of setting up CI / CD pipelines for streaming Flink based applications is an added advantage
    • serp_jobs.job_alerts.create_a_job

    Senior Architect • Chicago, IL, US