Software Engineer, Inference (AI Data Engineering)
Tasks
- Architect scalable distributed model serving infrastructure
- Benchmark and fine tune inference engines
- Build CI CD infrastructure for endpoint deployment
- Build high concurrency serving systems with uptime and observability
- Collaborate with teams to integrate inference capabilities
- Develop high reliability high throughput inference systems
- Develop tracing and replay tools
- Implement request routing and rate limiting
- Optimize inference latency and throughput
Perks/Benefits
- 401k
- Dental insurance
- Life insurance
- Long-term disability insurance
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid vacation
- Short-term Disability Insurance
- Vision insurance
Skills/Tech-stack
Auto Scaling | Batch scheduling | C plus plus | CI/CD | ClickHouse | Continuous Delivery | Continuous batching | Continuous integration | Distributed Systems | Docker | GPU Kernel | GRPC | Global KV Cache | Go | KV cache | Kubernetes | Latency optimization | Load Balancing | Model Serving | MongoDB | Observability | PostgreSQL | Profiling | Python | Quantization | REST | Rate Limiting | Request Routing | Rust | SDK development | SGLang | Speculative decoding | TensorRT-LLM | Throughput Optimization | Triton | VLLM
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Software Engineer, Data Platform Team USD 152K-179KApache Airflow | Data APIs | Data Catalog | Data Lineage | Data Modeling401k matching | Medical, dental, and vision | Quarterly in person surges | Remote-first work environmentMid-level Full TimeRemote - USA R6h ago
-
Senior Machine Learning Engineer USD 229K-360KAB Testing | APIs | Airflow | Algorithm Development | Amazon S3Disability benefits | Health insurance | Life insurance | Paid Equity Awards | Paid time offSenior-level Full TimeNew York, New York R6h ago
-
Senior Machine Learning Engineer USD 229K-360KA/B | A/B Testing | Amazon S3 | Amazon SageMaker | Apache AirflowCommuter benefits | Disability benefits | Equity awards | Financial wellness support | Health insuranceSenior-level Full TimeSan Jose, California R6h ago
-
Senior AI Engineer – AI Platform USD 140K-175KAWS | Drift Detection | Embeddings | Evaluation Pipelines | Function CallingDaily group classes | Dry cleaning | Free onsite fitness center | Free shuttle transportation | Gourmet cafeteriaSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States R6h ago
-
AI Engineer USD 220K-260KAPI Development | Agent Orchestration | Data Generation | Debugging | Fine Tuning401k matching | Dental insurance | Fertility and family building stipend | Flexible PTO | Health insuranceMid-level Full TimeSan Francisco R8h ago
-
Forward Deployed Engineer, Agentic AI USD 180K-180KAI Security | AWS Bedrock | Access Control | Agent Development | Anthropic ClaudeMid-level Full TimeUS - Remote R8h ago
-
Agents | Golang | Information Retrieval | Language Models | Language Processing401k company match | ESPP | Family leave programs | Flexible spending accounts | Flexible time offSenior-level Full TimeMountain View, CALIFORNIA, United States R10h ago
-
Director, AI & Agentic Engineering CAD 200KAI Foundry | AWS Bedrock | Agent Orchestration | Agentic Systems | Artificial IntelligenceHybrid workExecutive-level Full Time135 W 26th Street, New York, … R10h ago
-
Staff Machine Learning Engineer USD 176K-308KAlgorithms | Code Tuning | Design Patterns | High Performance | High-Performance Computing401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R10h ago
-
GTM AI Engineer USD 112K-140KAPI | Chunking | Compute cost management | Context windows | Cost Management401k matching | Dental insurance | Health insurance | Life insurance | Long-term disabilityMid-level Full TimeUnited States - Remote R11h ago
-
Mid-level Full TimeSan Francisco R11h ago
-
Senior Backend Engineer, Data Ingestion USD 145K-230KAWS | Amazon SQS | Celery | Data Pipelines | Database Query401k | Commuter benefits | Daily catered lunch | Dental insurance | Flexible paid time offSenior-level Full TimeRemote R12h ago
-
Sr Data Engineer USD 109K-156KAPI Management | Access Control | Airflow | Amazon Redshift | Apache SparkSenior-level Full TimeRemote, United States R13h ago
-
Embedded Software Engineer, Onboard Platform USD 100K-220KAutomotive networking | C++ | CAN | Communication Protocols | DO-178CSenior-level Full TimeSan Francisco, CA R13h ago
-
Operations Engineer, Applied AI (Spaceport Facilities) USD 125K-220KAI integration | Agent systems | Enterprise AI | Enterprise AI integration | LLMOnsite work | Travel opportunitiesEntry-level Full TimeStarbase, TX R13h ago
-
Senior Applied AI Engineer USD 197K-266KAlgorithms | Artificial Intelligence | Data Analysis | Data Structures | Deep learningFlexible time off | Mental health support | Paid parental leave | Private health insurance | Work from homeSenior-level Full TimeSan Francisco, CA R13h ago
-
Senior Machine Learning Engineer, Vision Models USD 311K-370KAuto-labeling | Benchmarking | Computer Vision | Data Curation | Deep learningHybrid work policy | Inclusive work environment | Mentorship | Work from home flexibilitySenior-level Full TimeSunnyvale R14h ago
-
Practice Architect AI/ML USD 136K-204KAI RMF | AWS | Agent Orchestration | Agentic Workflows | Autogen401k | Health insurance | Paid time off | Remote workSenior-level Full TimeDallas, Texas, USA R14h ago
-
Active Learning | C++ | Computer Vision | Confidence Scores | Data BalancingSenior-level Full TimeMountain View, CA, USA; San Francisco, … R15h ago
-
Lead Machine Learning Engineer USD 220K-270KC C plus plus | C plus plus | C/C# | CUDA | Camera processingDental insurance | Disability insurance | Flexible spending account | Flexible vacation | Health savings accountSenior-level Full TimeUSA - Remote R15h ago
-
Staff Machine Learning Engineer USD 176K-308KAlgorithms | Artificial Intelligence | Data Structures | Design Patterns | Java401k match | ESPP | Family leave | Flexible time off | Health plansSenior-level Full TimeSanta Clara, California, United States R15h ago
-
Senior ML/AI Modeler, Risk Automation Machine Learning USD 228K-343KAWS | Apache Airflow | Artificial Intelligence | Data orchestration | GCPFamily planning | Flexible time off | Medical insurance | Remote work | Retirement savings plansSenior-level Full TimeBay Area, CA, United States of … R16h ago
-
Senior-level Full TimeRemote or VA, VA, United States R16h ago
-
Senior Data Engineer USD 135K-180KAWS | Azure | Data Export | Data Ingestion | Data Lakes401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeDallas, TX - Hybrid (3x in … R16h ago
-
Staff Data Engineer USD 200K-240KAPI Design | AWS Bedrock | AWS Fargate | AWS Lambda | Amazon RDS401k | Commuter benefits | Company holidays | Dental insurance | Disability insuranceSenior-level Full TimeSan Francisco, CA R17h ago