Member of Technical Staff - Inference

Tasks

• Build autoscaling routing load balancing
• Build observability tools
• Build open superintelligence infrastructure
• Design scheduling algorithms
• Develop LLM serving platform
• Develop performance suites
• Document architectures and collaborate
• Embed inference into RL stack
• Establish CI/CD pipelines
• Implement resilience and failover
• Integrate and develop inference frameworks
• Optimize Inference Systems
• Optimize model distribution
• Profile and tune parallelism

Perks/Benefits

• Competitive compensation
• Conference attendance
• Equity incentives
• Flexible work
• Professional development
• Relocation support
• Remote option
• Team off-sites

Skills/Tech-stack

AWS | Ansible | Benchmarks | C++ | CUDA | Disaggregated serving | GCP | GRPC | Grafana | Heterogeneous scheduling | Infiniband | KVCache | Kafka | Kubernetes | Linux | NCCL | Nvidia Dynamo | OpenTelemetry | Performance Profiling | Prometheus | Protobuf | PubSub | PyTorch | Python | Quantization | Redis | Rust | Speculative decoding | TensorRT | Terraform