Director, Engineering - Inference Serving Engine
Tasks
- Collaborate with product management and engineering stakeholders
- Coordinate multicomponent inference deployments
- Define security best practices for AI workloads
- Define technical roadmap for high throughput scheduling
- Deploy topology aware scheduling
- Enforce isolation using container sandboxing
- Engineer GPU utilization optimizations
- Ensure production health stability and on call rotation
- Guide architecture of distributed inference platform
- Implement disaggregated AI inference pipelines
- Implement fault tolerance with checkpoint restore
- Implement multi scheduler and batch dispatching
- Implement startup ordering and multilevel autoscaling
- Improve cluster performance and scheduling latency
- Institutionalize benchmarking observability and auto tuning
- Optimize placement for NVLink PCIe and NUMA
- Orchestrate model weight distribution and storage
- Own project execution and delivery
- Recruit mentor and coach engineers
- Translate business goals into technical roadmaps
Perks/Benefits
Skills/Tech-stack
AMD GPUs | Auto-tuning | Benchmarking | CRIU | CUDA | Checkpoint Restore | Container Security | Distributed Systems | Fractional GPU allocation | GPU Allocation | GPU scheduling | GVisor | Helm | Kata Containers | Kubernetes | LLM Inference | MicroVM | NUMA | NVIDIA CUDA | NVIDIA GPUs | NVIDIA cuda checkpoint | NVLink | OCI Image Volumes | Observability | PCIe | Performance Tuning | ROCm | Topology Aware Scheduling
Education
N/A
Roles
Related jobs
-
Associate Director, IT Security-NHI & Agentic AI lead INR 2000K-2700KAKS | API Key | API key management | API key rotation | AWSMid-level Full TimeIND-Pune-Smartworks, India22h ago
-
Senior-level Full TimePrimary location: Bengaluru, India22h ago
-
Senior Manager INR 2500K-5000KAI architecture | Agentic AI | Artificial Intelligence | Autonomous Agents | Cloud ComputingSenior-level Full TimeIndia2d ago
-
Senior Engineering Manager, Data Science INR 2771K-4064KAutomated Monitoring | BigQuery | Data Governance | Data Privacy | DockerCreche allowance | Internal mobility | Lab days | Leadership programs | Learning and development allowanceSenior-level Full TimeBengaluru2d ago
-
Engineering Manager- Data Platform INR 2500K-4000KAWS | Apache Airflow | Apache Druid | Apache Flink | Apache HudiSenior-level Full TimeBangalore, Karnataka2d ago
-
Associate Director, RDU IT Data Engineering INR 1800K-2700KAWS | AWS Glue | Anomaly Detection | Apache Airflow | CI/CDMid-level Full TimeLocation: Bengaluru, India2d ago
-
VP AI ML Engineering INR 3000K-6000KAI Deployment | API Integration | Agentic AI | CI/CD | ContainerizationCareer development opportunities | Comprehensive benefits | Continuous learning | Inclusion-oriented cultureExecutive-level Full TimePrimary location: Hyderabad, India2d ago
-
Associate Director, RDU IT Data Engineering INR 1800K-2700KAWS | AWS Glue | Anomaly Detection | Apache Airflow | CI/CDMid-level Full TimeIndia - Bangalore2d ago
-
Executive-level Full TimeBengaluru, India2d ago
-
Associate Director, RDU IT Data Engineering INR 1800K-2700KAWS | Amazon Glue | Anomaly Detection | Apache Airflow | CI/CDMid-level Full TimeIndia - Bangalore2d ago
-
Director - Data & Analytics Integration & Eventing INR 4000K-5199KAPI Management | AWS | AWS Glue | Automation | AxwayExecutive-level Full TimeHyderabad (Office), India4d ago
-
Director of Data Engineering INR 3000K-4000KAWS | Apache Kafka | Apache Spark | Azure | Azure DevOpsExecutive-level Full TimeBengaluru Luxor North Tower, India4d ago
-
Senior Engineering Manager INR 2000K-3000KAPI Design | Access Control | Asynchronous Workflows | Auditing | Best practicesSenior-level Full TimeHyderabad, Telangana, India4d ago
-
Engineering Manager – AI Platforms INR 3000K-5000KAWS Bedrock | AWS SageMaker | Agentic Workflows | Agile Scrum | Azure AIDynamic collaborative team environment | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India5d ago
-
Engineering Manager, Data Foundations INR 2200K-2700KApache Airflow | Apache Hudi | Cloud Providers | Cost Optimization | Data GovernanceMid-level Full TimeIndia Remote R5d ago
-
Mid-level Full TimeMumbai, India6d ago
-
Director – Enterprise Data Engineering Lead INR 3845K-5000KAWS Glue | AWS Redshift | AWS S3 | Apache Airflow | Apache IcebergSenior-level Full TimeIndia - Chennai6d ago
-
Director – Enterprise Data Engineering Lead INR 3845K-5000KAWS Glue | AWS Redshift | AWS S3 | Apache Airflow | Apache IcebergFlexible work schedule | In person work minimum three days per weekSenior-level Full TimeIndia - Chennai6d ago
-
Software Engineer - Front Line Manager INR 1000K-1820KApplication Architecture | Availability Monitoring | BDD | Cloud Cost Optimization | Code SmellsCareer growth | Guided career tracks | Health insurance | Hybrid work | Online learning platformMid-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India6d ago
-
Senior Engineering Manager INR 4000K-6000KCareer development | Data Ingestion | Debt management | Distributed Systems | Embedding-based retrievalSenior-level Full TimeBengaluru, Karnataka, India7d ago
-
Senior Director - Data & AI Engineering INR 3242K-4600KAWS | Apache Spark | Artificial Intelligence | Azure | Data ArchitectureSenior-level Full TimePune, Maharashtra, India7d ago
-
Director - Database Administration (DBaaS) INR 8500K-11000KAs-a-Service | Automation | Backup and Recovery | Capacity Planning | Cause analysisExecutive-level Full TimeBengaluru, Karnataka, India7d ago
-
Manager, Full-Stack Engineering, SMAI INR 3000K-4000KAIOps | APIs | AWS | Agent systems | Agentic AIMid-level Full TimeHyderabad - Phoenix Aquila, India7d ago
-
Director, Data Architecture/Engineering INR 3584K-5000KAutomated testing | CI/CD | Data Preprocessing | Docker | EmbeddingsFlexible time off | Flexible-hybrid work | Healthcare | Retirement plan | Tuition reimbursementSenior-level Full TimeM11 - Commers III, 43, 44 … R7d ago
-
AI Engineering Lead/Architect INR 2500K-6000K.NET | AI Assisted Development | Architecture governance | Automated testing | CI/CDSenior-level Full TimePune, Maharashtra, India8d ago