Staff/Principal DevOps Engineer, AI Inference
Tasks
- Build CI CD pipelines for model artifacts and benchmarking
- Build intelligent request routing and load balancing
- Configure autoscaling for inference compute capacity
- Create infrastructure as code for EKS and GPU clusters
- Deploy inference pipelines with canary rollouts A B testing and rollback
- Design GPU accelerator infrastructure on Kubernetes
- Implement model serving platforms
- Implement observability for latency and GPU utilization
- Optimize inference performance and token throughput
- Optimize request batching and caching
Perks/Benefits
- Commuter benefits
- Company Subsidized Lunch Program
- Company holidays
- Dental insurance
- Educational assistance
- Flexible time off
- Life and disability insurance
- Medical insurance
- Paid parental leave
- Vision insurance
Skills/Tech-stack
AWS IAM | AWS S3 | Amazon EC2 | Amazon EKS | Amazon VPC | Autoscaling | CI/CD | CUDA | CUDA Driver | Continuous batching | Docker | EFA | Helm | Inference Server | Infrastructure as Code | KV cache | Kubernetes | Latency profiling | Load Balancing | Monitoring | NCCL | NVIDIA GPUs | Network interconnect | Pipeline parallelism | PrivateLink | Python | Quantization | SLI | SLO | Speculative decoding | Tensor Parallelism | Terraform | Token Throughput | Triton Inference | Triton Inference Server | VLLM | “as-code”
Education
N/A
Roles
DevOps | DevOps Engineer | Engineer | Principal | Principal DevOps Engineer | Staff DevOps | Staff DevOps Engineer
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R19d ago
-
Senior Data & ML Ops Engineer USD 137K-206KAWS | Airflow | Amazon SageMaker | Automated retraining | Automated testingHealth insurance | Paid time off | Retirement contributionsSenior-level Full TimeBoston, Massachusetts, US, 022105h ago
-
Accessibility API | Algorithms | Analytics | App SDK | Asynchronous programmingSenior-level Full TimeKansas City, MO, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Flexible remote work | Laid-back atmosphereMid-level Full TimeSeattle, WA, USA6h ago
-
Bash | Cloud infrastructure | Cloud platform | Data Ingestion | Data ProcessingAsynchronous culture | Diverse and inclusive workplace | Portfolio support | Remote-friendly environmentMid-level Full TimeChicago, IL, USA6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Career growth | Competitive compensation | Remote-friendly cultureMid-level Full TimeLong Beach, CA, USA6h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Friendly work environment | Remote workMid-level Full TimeSalt Lake City, UT, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Flexible management approachMid-level Full TimeMountain View, CA, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Laid-back atmosphere | Remote workMid-level Full TimeSunnyvale, CA, USA6h ago
-
AI Data | Bash | Cloud infrastructure | Data Ingestion | Data Ingestion PipelinesAsynchronous culture | Flexible team environment | Remote-friendlyMid-level Full TimeBerkeley, CA, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Collaborative team environment | Flexible, hands-off management | Remote-friendly work environmentMid-level Full TimeFremont, CA, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Laid back environment | Remote-friendlyMid-level Full TimeSanta Monica, CA, USA6h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous work culture | Career growth | Flexible management | Impactful missionMid-level Full TimeIrvine, CA, USA6h ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous culture | Flexible management approach | Remote/distributed work | Supportive teamMid-level Full TimeHuntsville, AL, USA6h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous work culture | Friendly work environment | Remote-first environmentMid-level Full TimeTampa, FL, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Equity | Friendly work environment | Portfolio supportMid-level Full TimeNewark, NJ, USA6h ago
-
Bash | Cloud platform | Data Processing | Data Processing Pipelines | Dataset EngineeringMid-level Full TimePhiladelphia, PA, USA6h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous work culture | Friendly work environment | Portfolio supportMid-level Full TimeNew Haven, CT, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Entrepreneurial team | Friendly laid-back atmosphere | Portfolio and LinkedIn sharing | Remote friendly distributed settingMid-level Full TimeTacoma, WA, USA6h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible work environment | Impactful product mission | Remote workMid-level Full TimeColorado Springs, CO, USA6h ago
-
Bash | Data Processing | Data Processing Pipelines | Docker | GCPAsynchronous culture | Bonus | Equity | Flexible management | Remote workMid-level Full TimeHonolulu, HI, USA6h ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous work culture | Bonus opportunity | Friendly work environment | Remote-friendlyMid-level Full TimeMilwaukee, WI, USA6h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Friendly work environment | Remote friendly distributed workMid-level Full TimeBloomington, IN, USA6h ago
-
Bash | Cloud platform | Docker | Google Cloud | Google Cloud PlatformAsynchronous culture | Flexible work environmentMid-level Full TimeDes Moines, IA, USA6h ago
-
Bash | Cloud infrastructure | Cloud platform | Data Ingestion | Data Ingestion PipelinesAsynchronous culture | Bonus | Distributed remote work | Equity | Friendly laid-back atmosphereMid-level Full TimeBuffalo, NY, USA6h ago