ML Systems Engineer
Tasks
- Build autoscaling and capacity management systems
- Collaborate with ML and product teams for model releases
- Design and operate model serving platforms
- Develop deployment workflows with canary releases shadow testing automated rollback
- Document operational procedures and tuning guidance
- Drive end-to-end observability
- Implement caching prompt deduplication and response reuse
- Implement multi tenant routing rate limiting and quality of service
- Implement security controls request signing content filtering and abuse detection
- Integrate model serving with API gateways identity systems and observability
- Operate incident response for high availability services
- Optimize inference performance
- Tune GPU utilization memory management and KV cache strategies
Perks/Benefits
- N/A
Skills/Tech-stack
API Gateway | Abuse detection | Automated rollback | Autoscaling | C++ | Caching | Canary Releases | Capacity Planning | Content Filtering | Deployment Automation | Distributed Systems | FinOps | GPU Utilization | Go | Incident Response | KV cache | Kubernetes | LLM Inference | Latency Histograms | Memory Management | Metrics | Metrics latency histograms | Model Compression | Model Distillation | Multi-tenant | Observability | Performance Engineering | Python | Quality of Service | Quantization | Rate Limiting | Request Routing | Request Signing | Rust | Security controls | Shadow testing | Structured Logging | TensorRT-LLM | Tracing | VLLM
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R17d ago
-
AI Center | AI orchestration | Action Center | Agile | AuthenticationMid-level ContractNashville, United States R1d ago
-
Applied AI Engineer, Investments USD 134K-183KAPIs | Cloud Computing | Data Pipelines | Data Quality | Data ValidationEmployer 401k match | Family forming benefits funding | Hybrid work onsite at least 60 percent of the time | Paid time off | Relocation supportNone Full TimeRedwood City, CA (Hybrid) R1d ago
-
Senior Machine Learning Engineer, ML Efficiency USD 216K-303KBenchmarking | Cost Optimization | Distributed Training | GPU Training | Knowledge DistillationCaregiving support | Employer 401k match | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R1d ago
-
Edge Computing AI Engineer USD 100K-150KC++ | DSP Toolchains | Device optimization | Device security | Edge inferenceMid-level Full TimeUnited States - Remote R1d ago
-
Generative AI Engineer USD 100K-150KBenchmarks | Dataset Construction | Direct Preference Optimization | Distributed Training | EvaluationCareer growth | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
Big Data Engineer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Hudi | Apache IcebergRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Streaming Data Engineer USD 100K-150KACLs | AWS MSK | Ansible | Apache Flink | Apache KafkaRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Machine Learning Data Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Code review | Data LineageRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Machine Learning Infrastructure Engineer USD 100K-150KAutoscaling | Batching | C++ | Caching | Capacity PlanningSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Robotics Engineer USD 100K-150KC++ | Concurrent programming | Control | Gazebo | Isaac SimCareer growth | Mentorship | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
AI Optimization Engineer USD 100K-150KC plus plus | CUDA | Compiler optimization | Cutlass | Deep learningRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior DataOps Engineer USD 150K-180K800-171 | AWS CloudFormation | AWS Glue | AWS GovCloud | AWS IAMRemote position | US government clearance requiredSenior-level Full TimeRemote - United States R1d ago
-
Senior Data Science Engineer I, Claims USD 170K-190KAWS | Apache Airflow | Automated testing | ClickHouse | Data Modeling401k matching | Disability and life insurance | Flexible working hours | Fully remote work | HSA & FSA optionsSenior-level Full TimeRemote R2d ago
-
Senior AI Engineer, Real-World Data USD 137K-175KAI orchestration | AWS | AWS Fargate | AWS Lambda | AWS S3401k benefits | Commuter benefits | Employee referral program | Fertility care benefits | Free testingSenior-level Full TimeUS Remote R2d ago
-
Senior AI Engineer USD 188K-258KAgentic Systems | Agile | Cloud Architecture | Code review | Configuration ManagementCell phone allowance | Comprehensive health coverage | Equity for every employee | Growth-focused environment | Home office setup allowanceSenior-level Full TimeRemote - US R2d ago
-
Amazon Redshift | Amazon SageMaker | Amazon Web Services | Azure Machine Learning | Azure SynapseDental insurance | Flexible spending account | Health insurance | Paid time off | Stock optionsEntry-level Full TimeUnited States - Remote R2d ago
-
Staff Data Engineer USD 237K-270KAmazon EMR | Amazon Redshift | Apache Airflow | Apache Flink | Apache SparkFamily planning support | Flexible time off | Lifestyle stipend | Mental health support | Paid parental leaveSenior-level Full TimeSan Francisco, CA R2d ago
-
Senior Performance Engineer, Data Products USD 172K-195KAPI | Benchmarking | ClickHouse | ClickHouse Cloud | Concurrency401k matching | Co hanging stipend | Company sponsored disability insurance | Company-sponsored life insurance | DCFSASenior-level Full TimeRemote R2d ago
-
Senior DataOps Engineer ( Remote US) USD 113K-163KAWS | AWS CloudFormation | Amazon EMR | Ansible | Apache Airflow401K employee contribution | Employee assistance program | Flexible Time Away From Work | Health & medical benefits | Learning platformSenior-level Full TimeRemote US R2d ago
-
Senior Software Engineer II - Ingestion AI Engineer USD 105K-155KAWS | Agile | Alerting | Azure | Causal InferenceMid-level Full TimeRemote, United States R2d ago
-
Senior Software Engineer, Data Engineering Platform USD 186K-218KAPIs | Bitcoin | Blockchain Protocols | Blockchain infrastructure | Crypto infrastructureSenior-level Full TimeRemote - USA R2d ago
-
Senior Software Engineer (Storage) - remote in the US USD 100K-150KArgoCD | CSI | Cluster API | Flux | GPUDirect StorageCompany outings happy hours | Conferences and working groups | Hackathons and tech talks | Professional development and trainingSenior-level Full TimeRemote, USA, United States R2d ago
-
Senior DevOps Engineer (Storage) - remote in the US USD 200K-250KAir-gapped | Alerting | Argo CD | ArgoCD | Bare MetalAttend conferences and working groups | Professional development and trainingExecutive-level Full TimeRemote, USA, United States R2d ago