Machine Learning Infrastructure Engineer
Tasks
- Build autoscaling and capacity management
- Design model serving platforms
- Develop deployment workflows
- Document operational procedures and tuning guidance
- Drive end-to-end observability
- Implement caching and response reuse
- Implement request routing and batching
- Implement security controls at serving layer
- Integrate with API gateways identity and observability
- Operate incident response for high availability
- Optimize inference performance
- Support model releases with ML teams
- Translate AI serving research into production
- Tune GPU utilization and memory management
Perks/Benefits
- N/A
Skills/Tech-stack
API Gateway | Abuse detection | Automated rollback | Autoscaling | C++ | Caching | Canary Release | Capacity Planning | Cloud Platforms | Content Filtering | Continuous batching | Distributed Systems | Error Tracking | GPU Architecture | GPU Utilization | GPU memory | GPU memory hierarchy | Go | Incident Response | Inference Optimization | KV cache | Kubernetes | Memory Management | Memory hierarchy | Metrics | Multi-region | Multi-region deployment | Multi-tenant | Multi-tenant routing | Observability | Paged Attention | Performance Engineering | Prompt Deduplication | Python | Quality of Service | Rate Limiting | Region deployment | Request Multiplexing | Request Signing | Rust | Shadow testing | Speculative decoding | Structured Logging | Tenant Routing | TensorRT-LLM | Tracing | VLLM
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Senior Data Engineer-JT0224 USD 115K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R9h ago
-
Senior Machine Learning Engineer USD 140K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Dental insurance | Flexible paid time off | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R9h ago
-
Asynchronous training | Containerization | Curriculum learning | Distributed Systems | Distributed TrainingSenior-level Full TimeSF Bay Area, CA, Remote, US, … R13h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R15h ago
-
Sr. Software Engineer (Data)-0336 USD 132K-203KAWS | Apache Spark | ClickHouse | Databricks | GitHubEmotional support | Financial benefits | Health benefits | Remote work days per weekSenior-level Full TimeVA - Reston, 11951 Freedom Dr … R1d ago
-
Sr Staff Software Engineer- Data Platform USD 220K-255KAWS EMR | AWS Lambda | AWS S3 | Access Control | AlgorithmsDental insurance | Disability insurance | Flexible spending account | Health insurance | Health savings accountSenior-level Full TimeCalifornia - Remote Office, United States R1d ago
-
Senior SW Engineer, Data Recording USD 129K-198KBackpressure | Bazel | Build systems | C plus plus | C#Senior-level Full TimeWork From Home - United States, … R1d ago
-
Lead Data Engineer USD 75KApache Airflow | Apache Beam | Autoscaling | BigQuery | CI/CD401k matching | Adoption reimbursement | Commuter benefits | Disability benefits | Discounts and savingsSenior-level Full Time112265-NJ-MetroPark, Iselin, United States R1d ago
-
Principal Embedded Software Engineer USD 150K-175K4G | Battery Management | Battery Management Systems | Board Bring-up | Bring-upSenior-level Full TimeMASSACHUSETTS - VIRTUAL - MA01, United … R1d ago
-
Mid-level Full TimeMASSACHUSETTS - VIRTUAL - MA01, United … R1d ago
-
Fullstack Engineer USD 85K-134KETL | Hadoop | Hive | Java | Linux401k plan | Continuing Education/Professional Development | Dental insurance | Employee assistance program | Life insuranceEntry-level Full TimeRemote, CA, United States R1d ago
-
Big Data Engineer USD 100K-150KAWS EMR | Airflow | Alerting | Apache Spark | Azure HDInsightRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Streaming Data Engineer USD 100K-150KAccess Control | Access Control Lists | Ansible | Apache Kafka | Apache Kafka ConnectSenior-level Full TimeUnited States - Remote R1d ago
-
AI Scientist USD 100K-150KAccelerator hardware | Computer Vision | Deep learning | Distributed Training | Experiment designRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Machine Learning Data Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Caching | Code reviewSenior-level Full TimeUnited States - Remote R1d ago
-
Generative AI Engineer USD 100K-150KAdapters | Benchmarking | DPO | Deep learning | Distributed TrainingSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
LLM Engineer USD 100K-150KAdapter based Fine Tuning | Attention Optimization | Cluster management | DPO | Distributed TrainingSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Director, Machine Learning & AI (BPD) USD 203K-304KAWS | Agentic Systems | Azure | Bias Testing | Cloud ComputingSenior-level Full TimeUS - Gaithersburg - MD, United … R1d ago
-
AI Optimization Engineer USD 100K-150KBenchmarking | C++ | CUDA | Continuous batching | CutlassRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Prompt Engineer USD 100K-150KAgentic Workflows | Chunking | Embeddings | Evaluation | IndexingMentorship opportunities | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Robotics Engineer USD 100K-150KBehavior Trees | C++ | Cameras | Concurrent programming | Control SystemsSenior-level Full TimeUnited States - Remote R1d ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R1d ago