Senior Inference Engineer, AGI
Tasks
- Apply efficiency techniques including quantization and operator fusion
- Build and tune continuous batching scheduling and preemption
- Build offline evaluation inference for real time conversation quality metrics
- Build offline training inference and reward model serving systems
- Co design model architecture for low latency inference
- Customize inference serving frameworks for streaming generative models
- Develop high performance GPU kernels for critical operations
- Ensure train serve consistency for RL and evaluation
- Establish latency throughput and cost benchmarking
- Implement multi GPU tensor parallelism and collective communication
- Optimize inference path for multimodal transformers
- Own real time streaming serving path with subsecond latency
- Profile and optimize end to end inference performance
Perks/Benefits
Skills/Tech-stack
Attention Mechanisms | Autoregressive decoding | BF16 | C++ | CUDA | Cache Management | Cutlass | Deep learning | Distributed Training | FP8 | FlashAttention | GPU Inference | GPU Performance | GPU Performance Optimization | INT8 | Java | KV cache | KV-cache management | Machine Learning | Multi-GPU | Multi-GPU inference | NCCL | NVLink | Network training | Neural Network | Neural network training | Operator fusion | Paged KV Cache | Performance optimization | Python | Quantization | RLAIF | RLHF | Reinforcement Learning | Speculative decoding | Tensor Parallelism | TensorRT-LLM | Transformers | Triton | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | AutomationMultiple project extensions | Paid on time | Remote work optionSenior-level Full TimeJuno Beach, FL R9h ago
-
AI Agents | API Development | Azure | CI/CD | LLMOpsCollaborative cross-functional team | Fully remoteMid-level Full TimeCanada, Ontario; Toronto, None; Toronto, Ontario; … R10h ago
-
API Integration | AWS Bedrock | Auditing | Authentication | AuthorizationMentoring | MentorshipMid-level Full TimeMenlo Park, CA, United States10h ago
-
Lead Palantir AI and Data Science Engineer II USD 159K-265KAI workflows | Analytics | Cloud Computing | Dashboards | Data GovernanceProfessional development | Travel opportunitiesSenior-level Full TimeRosslyn, Virginia, United States13h ago
-
Analytics and Insights Engineer III USD 131K-218KApache Spark | Data Governance | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeRichmond, Virginia, United States13h ago
-
3D Rendering | A/B | A/B Testing | Agentic AI | AlgorithmsSenior-level Full TimeSan Jose, California, United States13h ago
-
C++ | Computer Vision | Data Processing | Deep learning | Domain AdaptationEntry-level Full TimeSan Jose, California, United States13h ago
-
Staff Software Engineer, AI and Infrastructure USD 207K-300KC++ | Cluster management | Distributed Systems | Kernel | LinuxSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-300KAgentic AI | Artificial Intelligence | Cloud Security | Cloud platform | Data ProcessingSenior-level Full TimeReston, VA, USA; New York, NY, …14h ago
-
Senior Software Engineer, Google Cloud Dataproc USD 174K-252KApache Hive | Apache Spark | C++ | Data Warehouse | Data Warehouse InternalsSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Lead Firmware Engineer, Robotics USD 207K-300KBrushless DC | C# | C++ | Closed Loop | Closed loop controlSenior-level Full TimeMountain View, CA, USA14h ago
-
Agentic Orchestration | C++ | Data Onboarding | Data Pipeline Engineering | Data pipelineSenior-level Full TimeSan Jose, CA, USA14h ago
-
Software Engineer III, Infrastructure, Infra Spanner USD 147K-210KC++ | Concurrency | Data Corruption | Data corruption diagnosis | Database performanceSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Senior Software Engineer, AI/ML, Core USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA14h ago
-
Staff Software Engineer, Search Local Ads Data Platform USD 207K-300KC plus plus | CSS | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA14h ago
-
Principal Hardware Robotics Engineer, Data Centers USD 275K-382KActuator control | Embedded Systems | FPGA | Haptic feedback | Human-in-the-loopSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Embedded Hardware Systems Engineer USD 130K-150KAltium Designer | BLE | Bluetooth Low Energy | Bring-up | C++401k matching | Corporate discounts | Dental insurance | Disability insurance | Employee assistance programMid-level Full TimeSan Jose, CA, US R19h ago
-
Analytics Data Engineer USD 94K-130KAzure Data | Azure Data Factory | CI/CD | Data Analysis | Data FactoryRemote workMid-level Full TimeFlagstaff, AZ, US20h ago
-
AI Application & Cloud Operations Engineer (Mid level) USD 108K-117KAWS | AWS Bedrock | AWS CloudTrail | AWS CloudWatch | AWS PrivateLinkMid-level Contract Full TimePlano, TX, United States21h ago
-
Benchmarking | C plus plus | CUDA | Computational Engineering | Distributed Computing401k match | Generous paid time off | Medical, dental & vision coverage | Stock options | WFH Hardware ReimbursementSenior-level Full TimeSunnyvale, CA R21h ago
-
Software Engineer, Applied AI USD 173K-250KABAC | Agentic Frameworks | Apache Airflow | Audit trails | BIMDental insurance | Generous PTO | Health insurance | Retirement or Pension Plan | Vision insuranceMid-level Full TimeSan Francisco, CA21h ago
-
Staff Data Engineer USD 147K-184KAmazon Kinesis | Amazon Web Services | Apache Airflow | Apache Iceberg | Apache SparkDental insurance coverage | Health insurance coverage | Paid parental leave | Paid vacation | Relocation assistanceSenior-level Full TimeBerkeley, CA23h ago
-
Associate, Client Intelligence (Data Scientist) USD 98K-132KAWS | Apache Spark | Azure | Big Data | Cloud platformEmployee resource groups | Matching gift program | Professional development | Sustainability and Climate Change Initiatives | Volunteerism programMid-level Full TimeNew York, NY, US, 100011d ago
-
Apple Neural Engine | C++ | CPU | Compilers | Computer ArchitectureMid-level Full TimeSeattle1d ago
-
On-Device ML Infrastructure Engineer (CoreML Runtime), Graphics, Games and Machine Learning USD 170K-296KC++ | Core ML | GPU Programming | JAX | LLVMMid-level Full TimeCupertino1d ago