Director, Engineering - Inference Serving Engine
Tasks
- Collaborate with product management and stakeholders
- Define and enforce security and isolation best practices
- Define technical roadmap for high throughput scheduling
- Deploy topology aware scheduling
- Engineer GPU utilization optimizations
- Enhance cluster performance and reliability
- Ensure production health stability and on call rotation
- Guide system design for distributed inference platform
- Implement and manage disaggregated AI inference pipelines
- Institutionalize benchmarking observability and auto tuning
- Orchestrate model weight distribution and fault tolerance
- Own project execution and delivery
- Recruit mentor and coach engineers
Perks/Benefits
- Employee assistance program
- Flexible time off
- LinkedIn Learning access
- Local Employee Meetups
- Training and education reimbursement
Skills/Tech-stack
Auto Scaling | Benchmarking | CRIU | CUDA | Checkpoint Restore | Distributed Systems | Fault Tolerance | Fractional GPU allocation | GPU Allocation | GPU Programming | GPU resource management | GVisor | Kata Containers | Kubernetes | LLM Inference | Memory Optimization | MicroVMs | NUMA | NVIDIA CUDA | NVIDIA Grove | NVIDIA cuda checkpoint | NVLink | OCI Image Volumes | Observability | PCIe | Precision Management | ROCm | Resource Management | SGLang | SLO Management | Scheduling | Time Based Fairshare | VLLM
Education
N/A
Roles
Related jobs
-
Engineering Lead,Microservices,Java Springboot /Sr. Associate Director, Technology Management INR 1300K-2200KApache Airflow | Apache Beam | Apache Flink | Best practices | BigQueryMid-level Full TimePune, Maharashtra, India R4d ago
-
Head of Engineering Applied AI INR 3000K-4000KAgentic SDLC | Architectural Principles | Artificial Intelligence | Automated testing | Cloud ArchitectureExecutive-level Full TimeHyderabad RDT, India4d ago
-
Senior-level Full TimeBangalore, IN4d ago
-
Software Engineering Manager, Enterprise Data Platform INR 3845K-5500KArtificial Intelligence | C# | C++ | Data Architecture | Data EngineeringSenior-level Full TimeHyderabad, Telangana, India5d ago
-
Engineering Manager – AI Platforms INR 3000K-5000KAWS Bedrock | AWS SageMaker | Agile | Azure AI | CI/CDCollaborative team environment | Global consulting experience | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India5d ago
-
Technical Program Manager INR 3584K-5000KAI Agents | APIs | AWS | Analytics | AzureProfessional growthSenior-level Full TimeBengaluru, Pune5d ago
-
Data Engineer II INR 1000K-2000KAmazon S3 | Apache Kafka | Apache Spark | CDC | CI/CDAutonomy | Inclusive and diverse environment | Learning stipend | Mentorship | Remote environmentMid-level Full TimeIndia - Remote R5d ago
-
AD- Engineering INR 3715K-6000KAgile methodology | Architectural patterns | Automated testing | CI/CD | DevOpsSenior-level Full TimeBellandur, Karnataka, India5d ago
-
GCP BigQuery,DevOps,Credit Card Domain/Associate Director, Data and Analytics Specialist INR 650K-900KApache Airflow | Artificial Intelligence | BigQuery | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, Telangana, India R5d ago
-
API | AWS | Access Control | Audit Logging | AzureGlobal workforce collaboration | Remote workExecutive-level Full TimeIndia-Contractors R6d ago
-
Senior-level Full TimeBengaluru, India7d ago
-
Generative AI / Associate Director, Software Engineering INR 1200K-2500KAccess Control | Active Directory | Agile | Alerts | AnsibleMid-level Full TimePune, Maharashtra, India R7d ago
-
Director Software Engineering, ITC INR 3000K-6000KAWS | Amazon Kinesis | Apache Airflow | Apache Kafka | Apache SparkExecutive-level Full TimeINDIA TECHNOLOGY CENTER, India7d ago
-
Mid-level Full TimeBangalore, India Office (BANGALORE)10d ago
-
Director - Data Engineering INR 4000K-5199KAWS | Amazon Web Services | Cloud Computing | Cloud platform | ContainerizationBonuses for company approved content | Broadband reimbursement | Medical insurance | Paid certifications | Professional development allowanceExecutive-level Full TimeIndia - Remote R10d ago
-
Senior Manager I - Software Engineering INR 3000K-4800KAWS | Architectural patterns | Azure | Deep learning | DockerSenior-level Full TimeINDIA, Bangalore11d ago
-
AI Evaluation | APIs | AWS | Agent Orchestration | Anomaly DetectionCharity matching | Corporate volunteering | Dental insurance | Employee discounts | Medical insuranceExecutive-level Full TimeBengaluru, KA, IN12d ago
-
Director of Engineering INR 2100K-6000KAI Governance | AWS Fargate | Anomaly Detection | Apache Kafka | Cause analysisExecutive-level Full TimeBangalore, India12d ago
-
Manager, Machine Learning Engineering INR 1000K-1500KAWS | Automatic Speech Recognition | Cloud Computing | Cloud platform | Computer VisionEqual opportunity employer | Fast track growth opportunitiesMid-level Full TimeBangalore - Embassy Tech Village, India13d ago
-
Senior Manager, AI Engineering INR 3584K-5000KAI Agents | AI Observability | AWS | Agile | Automated testingSenior-level Full TimeHyderabad, India; Hyderabad Office13d ago
-
Executive-level Full TimePune, India14d ago
-
Senior-level Full TimeHyderabad, Telangana, India18d ago
-
Senior Engineering Manager (AI) INR 3264K-5876KAI-assisted coding | API Design | AWS | Assisted coding | CI/CDSenior-level Full TimeHyderabad, India18d ago
-
Director Of Engineering - Java, React, RAG & GenAI INR 3000K-6000KAI Governance | AI Observability | Azure | Azure OpenAI | CI/CDExecutive-level Full TimeHyderabad, India18d ago
-
Director AI/ML Engineering INR 3000K-4000KAI Governance | AWS | Agent Frameworks | Azure | Cloud NativeExecutive-level Full TimePrimary location: Hyderabad, Telangana18d ago