Senior HPC engineer, Research infrastructure
Palo Alto, California
Help Luma build some of the biggest & fastest AI supercomputing clusters in the world! As a High-Performance Computing engineer, you’ll work at the intersection of hardware and software, designing systems that deliver the maximum possible performance for running large-scale AI models. We work at the very cutting edge of speed and scale, combining the traditions of High-Performance Computing (HPC) in a modern cloud environment.
For this role, it’s important you understand how to combine CPU’s, GPU’s, and network devices into systems that are then deployed at a large scale to peak efficiency. You understand the lowest levels of the software platforms that sit on top of this hardware, including how to best optimize the Linux kernel and user-space code. You are capable of writing code to automate the monitoring and healing of these systems, commanding a large number of servers with few people.
For this role, it’s important you understand how to combine CPU’s, GPU’s, and network devices into systems that are then deployed at a large scale to peak efficiency. You understand the lowest levels of the software platforms that sit on top of this hardware, including how to best optimize the Linux kernel and user-space code. You are capable of writing code to automate the monitoring and healing of these systems, commanding a large number of servers with few people.
Responsibilities
- In this role, you will work closely with and directly accelerate machine learning researchers, but don't need to be a machine learning expert yourself.
- We value people who can quickly obtain a deep technical understanding of new domains and enjoy being self-directed and identifying the most important problems to solve.
- You’ll be managing training HPC clusters at Luma from provisioning to performance tuning.
- Areas of work will include observability, distributed job tracing, GPU diagnostics, software environment management and additional tooling plus work on the actual code to enable necessary features.
- We believe that increasing compute is a huge lever to AI progress. You will have a direct impact on our ability to grow to an unprecedented scale and likewise produce unprecedented results.
Experience
- 8+ years experience as infrastructure engineer or Devops in large and complex distributed systems.
- Deep understanding of networking, bonus points for experience in HPC networking.
- Experience developing high-quality software in a general-purpose programming language, preferably including Python.
- Excellent problem-solving skills and attention to detail.
- Experience with GPUs in large scale clusters is strongly preferred.
- Strong knowledge of observability and monitoring in distributed systems.
- Tenacious at troubleshooting hardware and network topology failures in distributed systemsIndependently driven and able to own problems and build solutions from end-to-end.
- Experience with large scale data center operations, proficiency in cloud orchestration and system tools.
- Please note this role is not meant for recent grads.
Compensation
- In addition to cash base pay, you'll also receive a sizable grant of Luma's equity.
- The pay range for this position is $180000- 220000/yr for Bay Area. Base pay offered will vary depending on job-related knowledge, skills, candidate location, and experience.
Job stats:
1
0
0
Categories:
Engineering Jobs
Research Jobs
Tags: DevOps Distributed Systems GPU HPC Linux Machine Learning Python Research
Perks/benefits: Career development Equity / stock options Salary bonus
Region:
North America
Country:
United States
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Principal Data Scientist jobsBI Developer jobsStaff Data Scientist jobsPrincipal Data Engineer jobsData Scientist II jobsData Manager jobsJunior Data Analyst jobsData Science Manager jobsResearch Scientist jobsBusiness Data Analyst jobsLead Data Analyst jobsSenior AI Engineer jobsSr. Data Scientist jobsData Engineer III jobsData Science Intern jobsData Specialist jobsJunior Data Engineer jobsSenior Data Scientist, Performance Marketing jobsBI Analyst jobsSoftware Engineer, Machine Learning jobsSr Data Engineer jobsData Analyst Intern jobsData Analyst II jobsSenior Artificial Intelligence/Machine Learning Engineer - Remote, Latin America jobsJunior Data Scientist jobs
Snowflake jobsEconomics jobsLinux jobsHadoop jobsOpen Source jobsJavaScript jobsPhysics jobsComputer Vision jobsAirflow jobsKafka jobsMLOps jobsRDBMS jobsBanking jobsData Warehousing jobsNoSQL jobsScala jobsGoogle Cloud jobsData warehouse jobsKPIs jobsR&D jobsPostgreSQL jobsOracle jobsGitHub jobsSAS jobsCX jobs
Classification jobsStreaming jobsTerraform jobsScikit-learn jobsLooker jobsScrum jobsDistributed Systems jobsPandas jobsData Mining jobsBigQuery jobsPySpark jobsRobotics jobsJenkins jobsJira jobsIndustrial jobsRedshift jobsdbt jobsReact jobsUnstructured data jobsMicroservices jobsMySQL jobsData strategy jobsE-commerce jobsGPU jobsNumPy jobs