Software Engineer - AI Infrastructure
New York City, NY
Full Time Senior-level / Expert USD 135K - 280K
Assembled
Assembled is a support operations platform that combines modern workforce management and AI-powered issue resolution to scale exceptional customer support. Leading brands use Assembled to optimize omnichannel staffing, gain visibility into...About Assembled
Assembled builds the infrastructure that underpins exceptional customer support, empowering companies like CashApp, Etsy, and Robinhood to deliver faster, better service at scale. With solutions for workforce management, BPO collaboration, and AI-powered issue resolution, Assembled simplifies the complexities of modern support operations by uniting in-house, outsourced, and AI-powered agents in a single operating system. Backed by $70M in funding from NEA, Emergence Capital, and Stripe, and driven by a team of experts passionate about problem-solving, weâre at the forefront of support operations technology.
The Role
Weâre looking for a software engineer to join our Infrastructure teamâbuilding and operating the core systems that power Assist, our rapidly growing AI agent platform for customer support. Assist automates support workflows across email, chat, and voice, and has grown from $0 to $1M in ARR in just 3 months. As adoption accelerates, weâre investing deeply in scaling its infrastructure to meet increasing demand and security expectations from enterprise customers.
As part of the AI Infrastructure team, youâll be responsible for the systems that enable Assist to be fast, reliable, and secure. Youâll work on foundational platform components that power real-time LLM usage at scale, while also exploring how AI can be leveraged internally to make our engineering team more productive. This team is highly cross-functional, working closely with the AI, security, and product engineering teams.
This is a high-ownership role for someone whoâs excited by 0-to-1 building and shaping the infrastructure backbone of our AI products.
Some projects owned by the Infrastructure team
Agent service reliability and scaling: We manage and scale the infrastructure that serves LLM-powered agents across chat, email, and voice. This includes selecting inference strategies, integrating with model providers (e.g. OpenAI, Anthropic), and dynamically routing traffic for performance and cost efficiency.
Prompt and embedding storage systems: Assist relies heavily on dynamically generated prompts and semantic search across support content. The team owns highly-available, fast-access storage and indexing layers optimized for real-time AI interactions.
Privacy and security: Enterprises expect strict guardrails around AI use. Weâre building systems like network-level intrusion detection (IDS/IPS), audit logging, and LLM usage policy enforcement to meet these expectations and unlock new sales channels.
Observability and usage analytics: We operate systems that surface key metricsâtoken usage, latency, cost per response, and quality signalsâso the Assist team can continuously improve Assistâs performance and accuracy.
AI-powered developer tools: We are beginning to explore and evangelize the use of AI to accelerate internal engineering workflowsâthrough internal chat agents, pair programming tools, and intelligent automation for deployment, debugging, and on-call. Our goal is to empower engineers across the company to build faster and more confidently with AI.
You may be a good fit if you:
Have 6+ years of engineering experience, with past ownership of high-scale, production-critical infrastructure
Have experience with distributed systems and container orchestration (especially Kubernetes)
Have worked with AI/ML platforms or are excited to build foundational infrastructure for LLM-based applications
Thrive in fast-paced environments with shifting requirements and ambiguous problem spaces
Are motivated by impact, enjoy deep technical challenges, and want to work cross-functionally across security, AI, and product
Have strong familiarity with one or more parts of our tech stack:
Cloud provider: AWS
Orchestration: Kubernetes + Karpenter
LLM integration: Experience with OpenAI, Anthropic, or open-source model serving (e.g., vLLM, HuggingFace TGI, Ray Serve)
Prompt & embedding infrastructure: Vector databases (e.g., Pinecone, Weaviate, PGVector), semantic search, prompt templating systems
Datastores: Postgres + PgBouncer, Snowflake, Redis
Languages: Go and Python
Monitoring & CI/CD: Datadog, Mezmo, CloudWatch, Buildkite, CircleCI
Our U.S. benefits
Generous medical, dental, and vision benefitsÂ
Paid company holidays, sick time, and unlimited time off
Monthly credits to spend on each: professional development, general wellness, Assembled customers, and commuting
Paid parental leaveÂ
Hybrid work model with catered lunches everyday (M-F), snacks, and beverages in our SF & NY offices
401(k) plan enrollment
Tags: Anthropic AWS CI/CD Distributed Systems Engineering HuggingFace Kubernetes LLMs Machine Learning ML infrastructure OpenAI Open Source Pinecone PostgreSQL Privacy Python Security Snowflake vLLM Weaviate
Perks/benefits: Flex vacation Health care Medical leave Parental leave Snacks / Drinks Unlimited paid time off
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.