Director, Engineering - Inference Serving Engine
Tasks
- Collaborate with product management and engineering stakeholders
- Coordinate multicomponent inference deployments
- Define security best practices for AI workloads
- Define technical roadmap for high throughput scheduling
- Deploy topology aware scheduling
- Enforce isolation using container sandboxing
- Engineer GPU utilization optimizations
- Ensure production health stability and on call rotation
- Guide architecture of distributed inference platform
- Implement disaggregated AI inference pipelines
- Implement fault tolerance with checkpoint restore
- Implement multi scheduler and batch dispatching
- Implement startup ordering and multilevel autoscaling
- Improve cluster performance and scheduling latency
- Institutionalize benchmarking observability and auto tuning
- Optimize placement for NVLink PCIe and NUMA
- Orchestrate model weight distribution and storage
- Own project execution and delivery
- Recruit mentor and coach engineers
- Translate business goals into technical roadmaps
Perks/Benefits
Skills/Tech-stack
AMD GPUs | Auto-tuning | Benchmarking | CRIU | CUDA | Checkpoint Restore | Container Security | Distributed Systems | Fractional GPU allocation | GPU Allocation | GPU scheduling | GVisor | Helm | Kata Containers | Kubernetes | LLM Inference | MicroVM | NUMA | NVIDIA CUDA | NVIDIA GPUs | NVIDIA cuda checkpoint | NVLink | OCI Image Volumes | Observability | PCIe | Performance Tuning | ROCm | Topology Aware Scheduling
Education
N/A
Roles
Related jobs
-
Software Engineer - Front Line Manager INR 1800K-2525KAirflow | Angular | Apache Beam | BigQuery | BigtableGuided career tracks | Healthcare packages | Hybrid work setting | Online learning platform | Organizational growth potentialMid-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India R19h ago
-
Engineering Lead (AI & Automation Products) INR 4000K-5000KAPI Design | Access Control | Automation workflows | Azure Key Vault | Background JobsSenior-level Full TimeDGS India - Bengaluru - Manyata …1d ago
-
Principal Software Engineer INR 1500K-3000KAWS Database | AWS Database Migration Service | AWS DynamoDB | AWS EC2 | AWS GlueSenior-level Full TimeBangalore, India1d ago
-
Senior-level Full TimeBangalore, India1d ago
-
Engineering Manager – AI Platforms INR 3000K-5000KAWS | AWS Bedrock | Agile Scrum | Amazon SageMaker | AzureCollaborative team environment | Diverse projects | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India2d ago
-
Senior-level Full TimeMumbai, Maharashtra, India2d ago
-
Director, Data Architecture/Engineering INR 3584K-5000KAPIs | Automated testing | CI/CD | Data Pipelines | Data PreprocessingComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Tuition reimbursementSenior-level Full TimeM11 - Commers III, 43, 44 … R2d ago
-
Vice President – Data Engineering Delivery Lead INR 2000K-3000KAWS Athena | AWS Glue | AWS Lambda | Agile | Amazon S3Senior-level Full TimeChennai, DLF IT Park, India2d ago
-
Senior /Staff Data & Cloud Platform Engineer INR 1492K-2900KAWS Glue | AWS IAM | AWS Lambda | AWS S3 | Amazon RedshiftSenior-level Full TimeHyderabad - Phoenix Aquila, India2d ago
-
AI Architect- Associate Director INR 1800K-4000KAI Search | AKS | AWS | Agentic AI | Application InsightsMid-level Full TimeBangalore, Karnataka, India5d ago
-
APIs | Aerospike | Agile | Application Security | AutomationRelocation assistanceMid-level Full TimeGurugram5d ago
-
Director - Data Engineering INR 4111K-6000KAI Assisted Development | Apache Spark | Azure | CI/CD | CM360Executive-level Full TimeDGS India - Chennai - Anna …5d ago
-
Architecture | Availability | Cloud Computing | Cost Optimization | Data PipelinesEqual opportunity employer | Fast track growth opportunities | Great place to workSenior-level Full TimeBangalore - Embassy Tech Village, India5d ago
-
Engineering & Delivery Lead - Data Services INR 3000K-5000KAWS Athena | Apache Drill | Apache Flink | Apache Kafka | Apache RangerSenior-level Full TimeLocation(s): Pune, Maharashtra, India5d ago
-
Associate Director, RDU IT Data Engineering INR 1500K-2500KAWS | AWS Glue | Anomaly Detection | Apache Airflow | CI/CDHybrid work minimum three days per week in office | Immediate joiner preferred | Local candidates preferredMid-level Full TimeIndia - Bangalore6d ago
-
Associate Director, RDU IT Data Engineering INR 1500K-2500KAWS | AWS Glue | Anomaly Detection | Apache Airflow | Batch ProcessingMid-level Full TimeLocation: Bengaluru, India6d ago
-
Associate Director, RDU IT Data Engineering INR 1500K-2500KAWS | AWS Glue | Anomaly Detection | Apache Airflow | CI/CDMid-level Full TimeIndia - Bangalore6d ago
-
Director, Machine Learning Engineering, AI Studio INR 4000K-6000KAWS | AWS Bedrock | Agents | Agile methodologies | AzureExecutive-level Full TimeIndia - Hyderabad7d ago
-
Executive-level Full TimeIN - HYDERABAD ORION, India8d ago
-
Engineering Leader – Investment Technology, Vice President II INR 3715K-5449KAWS | Apache Spark | Automation | Azure | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeBangalore, India8d ago
-
Engineering Leader – Investment Technology, Vice President II INR 3715K-5449KAWS | Apache Spark | Azure | CI/CD | Cloud platformEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeBangalore, India8d ago
-
Tech S and T-Cloud AI Platform Architect-ISR-Manager-GDSF02 INR 3000K-5000KAI Pipelines | ARM | AWS | Access Control | Agentic AISenior-level Full TimeNoida, UP, IN, 2013018d ago
-
Executive-level Full TimeGurugram, Haryana, India9d ago
-
Senior Data Engineer, Director INR 1500K-3000KAWS EMR | Amazon Athena | Amazon DynamoDB | Amazon Kinesis | Amazon QuickSightSenior-level Full TimeGurugram, India9d ago
-
Senior Engineering Manager INR 6500K-9000KAI orchestration | API Design | AWS | Automated testing | AzureSenior-level Full TimeBengaluru, Karnataka, India9d ago