Scientific Systems-HPC Engineer - PACC (CPMI)

• Job Type : Officer of Administration
• Regular / Temporary : Regular
• Hours Per Week : 35
• Salary Range : $150,000 - $175,000

The salary of the finalist selected for this role will be set based on a variety of factors, including but not limited to departmental budgets, qualifications, experience, education, licenses, specialty, and training.

The above hiring range represents the University's good faith and reasonable estimate of the range of possible compensation at the time of posting.

Position Summary

The Scientific Systems-HPC Engineer, reporting to the Director of Bioinformatics and the Director of the Columbia Precision Medicine Initiative, is responsible for supporting the computational needs of genomics analysis and bioinformatics research.

This role involves maintaining analysis pipelines, deploying and operating High-Performance Computing (HPC) environments, and managing associated applications.

Responsibilities

• Collaborate with the Director of Bioinformatics, Software Engineers, Data Scientists, InfoSec, and Application Administrators to design, deploy, and support research systems within a dynamic HIPAA / PHI environment.
• Partner with stakeholders to design, build, configure, and support bioinformatics systems, HPC clusters, and batch compute workflows in our cloud environment.
• Ensure the performance, scalability, and security of analysis pipelines and HPC clusters in collaboration with business application owners and technical resources.
• Diagnose and resolve system, software, and storage issues in coordination with CUIMC and vendor teams.
• Monitor system performance, analyze log files and data outputs, and manage issue triage using a ticketing system.
• Utilize automation tools to deploy infrastructure and applications.
• Implement patching solutions, backup, and disaster recovery plans, ensuring secure data handling and storage.
• Propose and implement cost optimization strategies for application and system deployments.
• Consider cloud infrastructure, application design, and monitoring when designing cloud solutions.
• Establish security policies and best practices in collaboration with InfoSec.
• Participate in code reviews and design discussions to better understand deployed systems.
• Conduct disaster recovery readiness assessments, including periodic tabletop exercises.
• Evaluate existing bioinformatics and HPC infrastructure, proposing enhanced computing architectures and conducting proof-of-concept tests.
• Perform design reviews and risk assessments for new applications integrating with core services.
• Guide product engineering teams in adopting security standards within the software development lifecycle.
• Create and maintain documentation for new and existing processes and deployments.
• Perform other duties as assigned.

Minimum Qualifications

• Bachelor's degree or equivalent in education and experience, plus four years of related experience
• Experience in an HPC infrastructure environment.
• Experience with cloud-native solutions across an enterprise.
• Professional certifications (e.g., AWS Solution Architect, Google Cloud Architect, Red Hat Linux).
• Ability to work effectively in a team as well as independently on projects and tasks.
• Strong experience in supporting genomic analysis pipelines and HPC clusters.
• Experience with architecture, migration, and deployment of on-prem applications in AWS / GCP and hybrid environments.
• Experience in migrating on-prem applications to cloud-native solutions.
• Excellent written and verbal communication skills.
• Ability to work in a fast-paced, deadline-driven environment with changing priorities and multiple projects.
• Precision and attention to detail are essential.
• Ability to work with minimal supervision and mentor junior engineers.
• Willingness to work weekends and off-hours as needed.

Preferred Qualifications

• Experience in stakeholder management within complex organizations, with a consultant mindset to ensure client satisfaction and timely delivery of bioinformatics / HPC solutions.
• In-depth knowledge of Linux, system administration tools, and system security.
• Proficiency in at least one scripting or programming language (e.g., Bash, Java, Python, Go).
• Knowledge of programming languages such as C++ is a plus.
• Knowledge of orchestration tools (e.g., Ansible, Chef, Puppet) and the ability to automate processes and workflows.
• Experience managing HPC environments and using tools like Slurm, SGE, MemVerge, or AWS Parallel Cluster.
• Experience deploying workloads in cloud environments such as AWS or GCP.
• Familiarity with storage solutions like AWS EFS or NetApp, and associated protocols (e.g., CIFS / SMB, Lustre, NFS).
• Experience with database systems such as MySQL, PostgreSQL, and NoSQL options like DynamoDB, MongoDB / DocumentDB.
• Proficiency with Infrastructure as Code tools such as AWS CloudFormation and Terraform.
• Experience using source control tools like AWS CodeCommit, GitHub, or GitLab.
• Familiarity with container solutions such as Docker and Kubernetes.
• Experience working in diverse research and healthcare research environments.
• Ability to conduct Total Cost of Ownership (TCO) analysis and consider security in architecture design and deployment.
• Understanding of cloud deployment pipelines and open-source technologies.
• 8+ years of experience in infrastructure engineering and Linux support.

Other Requirements

Successful completion of applicable compliance and systems training requirements

Equal Opportunity Employer / Disability / Veteran

Columbia University is committed to the hiring of qualified local residents.