Senior Product Manager - Inference Team (f/m/d)
Berlin
Aleph Alpha
Pioneering sovereign, European AI technology to transform human-machine interaction that can find solutions for the challenges of tomorrow.Overview:
We're looking for a technically strong Product Manager to join our PhariaAI Inference Team and help shape the future of our advanced inference platform. This role is ideal for someone with product management experience, a deep understanding of AI infrastructure, and a strong grasp of the performance and economics of large language model deployment.
Your responsibilities:
Shape the product strategy and roadmap for our inference platform in close collaboration with engineering and research, aligning OKRs with business goals and user needs
Provide clarity on goals and constraints, enabling the team to explore and deliver the best solutions.
Work closely with engineering to prioritize and deliver high-impact features, ensuring a fast, reliable, and scalable inference stack
Define clear, actionable requirements and success criteria that balance technical feasibility with user and business impact
Continuously learn from real-world usage, incorporating performance metrics, user feedback, and experimentation results into iterative improvements
Stay informed about the latest in inference technologies, optimization techniques, and the broader LLM landscape to inform product direction
Partner with customer-facing teams to articulate the value and differentiation of our inference capabilities in a fast-moving competitive environment
Your profile:
Experience in product management for software products, ideally with exposure to developer tools, AI/ML systems, or technical platforms
Familiarity with modern product discovery and agile delivery practices
Strong technical curiosity fluency and willingness to learn about AI inference technologies
Strong communication skills, especially when distilling technical complexity for non-technical audiences
Strong analytical skills to evaluate market trends and competitive offerings
A customer-obsessed mindset and the ability to deeply understand user needs—even when those users are internal AI teams
Ability to thrive in a fast-paced environment and manage multiple priorities
Bonus (but not required):
Basic understanding of inference optimization techniques such as quantization, LoRA adapters, function calling, structured outputs, and batch processing
Familiarity with the economics of LLM inference, including GPU utilization, token economics, and performance trade-offs
Exposure to inference engines such as vLLM, SGLang, TGI, or similar technologies
Experience with retrieval-augmented generation (RAG) pipelines, embeddings, and multi-modal systems
Understanding of the challenges in long-context handling and advanced sampling methods
Experience with multi-modal AI systems
What you can expect from us:
Access to a variety of fitness & wellness offerings via Wellhub
Mental health support through nilo.health
Substantially subsidized company pension plan for your future security
Subsidized Germany-wide transportation ticket
Budget for additional technical equipment
Regular team events to stay connected
Flexible working hours for better work-life balance
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Economics Engineering GPU LLMs LoRA Machine Learning ML infrastructure Model deployment OKR Pipelines RAG Research Security vLLM
Perks/benefits: Flex hours Team events
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.