Senior GenAI Infrastructure Engineer - JFrog ML

At JFrog, we’re reinventing DevOps and MLOps to help the world’s greatest companies innovate -- and we want you along for the ride. This is a special place with a unique combination of brilliance, spirit and just all-around great people. Here, if you’re willing to do more, your career can take off. And since software plays a central role in everyone’s lives, you’ll be part of an important mission. Thousands of customers, including the majority of the Fortune 100, trust JFrog to manage, accelerate, and secure their software delivery from code to production - a concept we call “liquid software.”

Wouldn't it be amazing if you could join us in our journey?

About the Team

JFrog ML empowers AI and data science teams at every stage of the model lifecycle, from training to deployment, offering advanced model security and seamless operations across cloud and self-hosted environments.

As we expand, we're looking for a GenAI Infrastructure Engineer to help shape the future of our GenAI platform, trusted by the world's leading companies to power their next-generation GenAI solutions.

In this role, you’ll build cutting-edge AI infrastructure, design and develop core components such as model serving, prompt management, guardrails, gateways, and evaluation frameworks. You’ll help provide organizations with the precise tools to integrate and maintain GenAI into their workflows, ensuring scalability, compliance, and an excellent developer experience.

You’ll collaborate with platform, security, AI/ML, and data teams to optimize deployment, monitoring, and governance of GenAI applications at scale. We’re looking for data-driven problem-solvers who thrive in fast-paced environments and are passionate about building AI-driven solutions.

If you want to build the infrastructure of the future from concept to production, we’d love to hear from you!

As a GenAI Infrastructure Engineer at JFrog ML you will...

• Design and develop the infrastructure of our new GenAI solution.
• Develop and manage components such as prompts, guardrails, and model gateways, tracing and evaluation frameworks.
• Collaborate with cross-functional teams on shaping how AI applications are structured and managed.
• Stay up to date with the latest advancements in GenAI infrastructure, tooling, and frameworks, integrating industry best practices.

To be a GenAI Infrastructure Engineer at JFrog ML you need...

• 5+ years of experience as a Software Engineer
• Experienced in designing, developing, and debugging complex, distributed systems (microservices, event-driven)
• Proven hands-on experience in containerized environments, microservices and Kubernetes
• Experienced with at least one of the main cloud provider platforms (e.g. AWS, GCP)
• Experience with designing, structuring, and maintaining Python SDKs
• Familiarity with AI application development frameworks and tools (e.g. LangChain, OpenAI, Hugging Face, MLflow)
• Experienced in Java and Python

Bonus Points

• Experience working with AI Agents and LLM-based application frameworks
• Background in data engineering, feature stores, or vector databases for AI workloads

Previous experience working in developer-first companies or on AI-powered solutions