Director, Engineering - Inference Serving Engine
Tasks
- Collaborate with product management and stakeholders
- Define and enforce security and isolation best practices
- Define technical roadmap for high throughput scheduling
- Deploy topology aware scheduling
- Engineer GPU utilization optimizations
- Enhance cluster performance and reliability
- Ensure production health stability and on call rotation
- Guide system design for distributed inference platform
- Implement and manage disaggregated AI inference pipelines
- Institutionalize benchmarking observability and auto tuning
- Orchestrate model weight distribution and fault tolerance
- Own project execution and delivery
- Recruit mentor and coach engineers
Perks/Benefits
- Employee assistance program
- Flexible time off
- LinkedIn Learning access
- Local Employee Meetups
- Training and education reimbursement
Skills/Tech-stack
Auto Scaling | Benchmarking | CRIU | CUDA | Checkpoint Restore | Distributed Systems | Fault Tolerance | Fractional GPU allocation | GPU Allocation | GPU Programming | GPU resource management | GVisor | Kata Containers | Kubernetes | LLM Inference | Memory Optimization | MicroVMs | NUMA | NVIDIA CUDA | NVIDIA Grove | NVIDIA cuda checkpoint | NVLink | OCI Image Volumes | Observability | PCIe | Precision Management | ROCm | Resource Management | SGLang | SLO Management | Scheduling | Time Based Fairshare | VLLM
Education
N/A
Roles
Related jobs
-
Director of AI/ML Engineering (EDA & Semiconductor Design) INR 2400K-6000KArtificial Intelligence | Cloud Platforms | Compute Infrastructure | Data Pipelines | Data labelingEmployee resource groups | Flexible work environment | Remote workExecutive-level Full TimeHyderabad, India12h ago
-
Manager, Machine Learning Engineering INR 1000K-1800KAWS | Automatic Speech Recognition | Cloud platform | Computer Vision | Distributed SystemsChallenging ML problems | Equal opportunity employer | Fast track growth opportunitiesMid-level Full TimeBangalore - Embassy Tech Village, India22h ago
-
Lead Software Engineer (Cloud Native | Microservices | AWS) INR 3000K-4000K.NET | API | AWS | AWS CDK | Behavior-Driven DevelopmentEnhanced medical benefits | Hybrid work | Paid time off | Wellbeing benefits | Work-life balanceSenior-level Full TimeHyderabad, India1d ago
-
Senior Director of Engineering – Storage Products INR 2755K-3500KAmazon S3 | Block Storage | Capacity Management | Capacity Planning | CephConference reimbursement | Education reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning accessSenior-level Full TimeBengaluru1d ago
-
Engineering Manager INR 2200K-3000KAI-assisted coding | API Gateway | AWS | Angular | Assisted codingMid-level Full TimeHyderabad, India1d ago
-
Manager, Data Engineering INR 1600K-2400KArtificial Intelligence | Azure | Azure Data | Azure Data Lake | Azure Data Lake StorageAccidental insurance | Maternity leave | Medical insurance | Paid leave | Paternity leaveMid-level Full TimeRemote - India R2d ago
-
Director, Engineering - Serverless Inference INR 1962K-6000KAPI Gateway | Capacity Planning | Cloud Native | Distributed Systems | Fault ToleranceEmployee assistance program | Employee stock purchase program | Flexible time off | LinkedIn Learning access | Local Employee MeetupsExecutive-level Full TimeBengaluru2d ago
-
Data Scientist Director INR 4000K-5199KArtificial Intelligence | Attribution measurement | Data Governance | Data publishing | Data readinessExecutive-level Full TimeBengaluru, Karnataka, India2d ago
-
Engineering Manager – AI Platforms INR 3000K-5000KAWS Bedrock | AWS SageMaker | Agentic Workflows | Agile | Azure AICareer growth | Collaborative team environment | Leadership development | Training sessionsSenior-level Full TimeMumbai, MH, India2d ago
-
Senior-level Full TimeMumbai, Maharashtra, India2d ago
-
Director, Engineering - Forward Deployed Engineering INR 1500K-5199KAI infrastructure | AI orchestration | Agentic Systems | Agents SDK | AutomationConference reimbursement | Employee assistance program | Employee stock purchase program | Flexible time off | LinkedIn Learning accessExecutive-level Full TimeBengaluru2d ago
-
Director of Software Engineering - GenAI, Python INR 3000K-6000KA/B | A/B Testing | AWS | Agentic Workflows | AnthropicExecutive-level Full TimeBengaluru, Karnataka, India2d ago
-
Engineering Lead /Sr. Associate Director, Technology Management INR 1300K-2000KApache Airflow | Apache Beam | Apache Flink | BigQuery | Cloud platformCareer development | Health and wellbeing benefits | Performance-based bonus | Training opportunitiesMid-level Full TimePune, Maharashtra, India R2d ago
-
Senior-level Full TimeIndia - Bangalore2d ago
-
Director- Software Engineering Lead INR 3584K-5000KAWS | Angular | Azure | Bias Testing | By DesignSenior-level Full TimeIndia - Bangalore2d ago
-
Director of Software Engineering – AI/ML INR 2400K-6000KArtificial Intelligence | Cloud Computing | Data Pipelines | Data labeling | DeploymentEmployee resource groups | Flexible work environment | Remote work optionExecutive-level Full TimeHyderabad, India3d ago
-
Associate Practice Manager - IT Development INR 1800K-3300KAPI Management | AWS | Agile | Algorithms | AuditabilityMid-level Full TimeHyderabad, Telangana, India3d ago
-
Director, Software Engineering - Big Data Platform & Storage INR 3000K-4000KAWS | Airflow | Apache Hive | Apache Spark | AzureExecutive-level Full TimeIndia - Hyderabad3d ago
-
Senior Director of Engineering, Managed MySQL INR 2475K-3380KAutomated failover | Connection Routing | Database Management | Distributed Systems | High AvailabilityConference reimbursement | Employee assistance program | Employee meetups | Flexible time off | LinkedIn Learning accessSenior-level Full TimeBengaluru4d ago
-
AI/ML Silicon Validation Lead, Google Cloud, TPU INR 3200K-5000KAXI | Board Schematic | Board Schematic Analysis | Bring-up | DDRSenior-level Full TimeBengaluru, Karnataka, India6d ago
-
Engineering Manager INR 5000K-8000KAgile | Application development | Automated testing | CI/CD | Cloud NativeMid-level Full TimeBengaluru, Karnataka, India6d ago
-
Amazon Web Services | Apache Spark | Azure Data | Azure Data Engineering | Code reviewExecutive-level Full TimeBengaluru Millenia, India6d ago
-
Cloud Native | Cloud Native Architecture | Distributed Storage | Distributed Systems | High AvailabilityCareer growth | Equity | Flexible time off | Health, dental, vision coverage | Learning and developmentSenior-level Full TimeIndia7d ago
-
Senior Engineering Manager, Data Platform INR 2817K-4132KA/B | A/B Testing | API Design | AWS | Agentic WorkflowsSenior-level Full TimeBengaluru Office, India7d ago
-
Senior Engineering Manager - Data Platform INR 2500K-4800KCloud Computing | Contract-driven Development | Data Catalog | Data Governance | Data IngestionEquity compensation | Flexible time off | Health & wellness benefits | Learning and development | Performance-based bonusSenior-level Full TimeIndia R8d ago