Staff Software Engineer, Inference
Tasks
- Benchmark inference performance
- Build low latency high throughput inference systems
- Conduct post-incident analysis
- Define inference platform architecture
- Design request routing and scheduling
- Develop speculative decoding
- Drive traffic management and failure mitigation
- Enable KV cache reuse and memory optimization
- Implement micro batching and dynamic scheduling
- Improve P95 P99 latency and throughput
- Improve testing and observability
- Lead capacity planning and autoscaling
- Lead cross-team design reviews
- Lead multi-service cross team design
- Manage GPU resources
- Mentor engineers and elevate engineering standards
- Optimize cost per token
- Own SLIs SLOs reliability
Perks/Benefits
- Critical illness cover
- Employee assistance programme
- Family dental insurance
- Family medical insurance
- Generous pension contribution
- Life assurance
- Living wage accredited employer
- Tuition reimbursement
Skills/Tech-stack
Autoscaling | BF16 | Benchmarking | C++ | CUDA | Capacity Planning | Cloud platform | Distributed Systems | Dynamic Scheduling | FP8 | Failure mitigation | Go | KV cache | Kubernetes | Latency optimization | Memory Optimization | Micro Batching | Mixed Precision | NCCL | NUMA | Observability | Performance optimization | Python | RDMA | SLI | SLO | Speculative decoding | Streaming token delivery | System design | Throughput Optimization | Traffic Management
Education
Roles
Related jobs
-
Technology Specialist - Databricks, Azure PLN 276K-276KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2English communication | Hybrid workEntry-level Full TimeKraków, PL19h ago
-
Senior Data Analytics/Solutions Engineer PLN 240K-328KAzure | Azure DevOps | CI/CD | Data Contracts | Data GovernanceHybrid workSenior-level Full TimeKraków, PL19h ago
-
Data Quality Engineer PLN 276K-276KADLS Gen2 | Agile | Asset bundles | Ataccama | AzureHybrid workMid-level Full TimeKraków, PL19h ago
-
Data Engineering Lead PLN 250K-360KApache Kafka | Azure | Azure Data | Azure Data Factory | Azure DatabricksHybrid workSenior-level Full TimeKraków, PL19h ago
-
Analytics Engineer PLN 153K-211KApache Airflow | Apache Plotly | DBT | Data Documentation | Data ModelingMid-level Full TimeWarsaw1d ago
-
Mid-level Full TimeWarsaw, Poland, PL2d ago
-
Mid-level Full TimeWrocław, Województwo dolnośląskie, PL2d ago
-
Mid-level Full TimeKraków, Województwo małopolskie, PL2d ago
-
Senior-level Full TimeWarsaw, Poland, PL2d ago
-
Software Engineer III, AI/ML Cloud PLN 280K-287KC++ | Data Processing | Debugging | Distributed Computing | Information RetrievalSenior-level Full TimeWarsaw, Poland2d ago
-
Principal Data Engineer USD 180K-220KAWS | Apache Airflow | Artificial Intelligence | CI/CD | Confluence401k | Autonomy | Company paid life insurance | Disability coverage | FSASenior-level Full TimeRemote - US, Canada and Poland R2d ago
-
C++ Machine Learning Engineer, Models Training PLN 292K-470KC++ | Compiler optimization | Computer Architecture | Kernel development | Machine LearningHybrid work | Remote within PolandSenior-level Full TimeGdańsk, Pomeranian Voivodeship, Poland; Warszawa, Masovian …2d ago
-
Analytics Engineer (DBT, BigQuery) PLN 153K-228KAirflow | BigQuery | CI/CD | Code review | DBTCommunity focus | Hybrid work 3 days per week | In-person collaborationMid-level Full TimeWarsaw, Poland2d ago
-
Auto Scaling | CI/CD | CUDA | Container Orchestration | Distributed SystemsEnglish language support | On-call support | Remote workSenior-level Full TimeUkraine R2d ago
-
Associate Data Scientist (AI) PLN 163K-229KAWS | Agentic Applications | Azure | Calculus | DockerHybrid or remote flexibility | International projects | Medical healthcare | Ongoing learning reimbursement | Team eventsMid-level Full TimeBulgaria, Poland2d ago
-
Forward Deployed Engineer, Applied AI PLN 221K-328KAWS | Azure | Data Engineering | Data Modeling | Data ValidationSenior-level Full TimePL-Warsaw3d ago
-
Principal Gen AI Software Engineer PLN 241K-400KAWS | AWS Bedrock | AWS IAM | AWS Lambda | Amazon EC2Employee capital plans | Flexible working hours | Hybrid work model | Life insurance | Medical careSenior-level Full TimeKrakow, Malopolskie, Poland3d ago
-
BigQuery | Concept drift | Concept drift detection | Data Annotation | DatabricksSenior-level Full TimeWarszawa, Poland3d ago
-
Machine Learning Engineer PLN 213K-328KAirflow | Azure | Data Processing | Distributed data | Distributed data processingFriendly atmosphere | Good working conditions | Self-development opportunities | Trust and autonomySenior-level Full TimeKraków, Małopolskie, Poland3d ago
-
Applied AI Engineer (Poland) PLN 189K-229KAutomation | Customer Relationship Management | Customer relationship | Document processing | Fintech domainEnglish communication support | Remote workMid-level Full TimePoland3d ago
-
Mid-level Full TimeWrocław, Województwo dolnośląskie, PL3d ago
-
Mid-level Full TimeWarsaw, Poland, PL3d ago
-
Mid-level Full TimeKraków, Województwo małopolskie, PL3d ago
-
Docker | FastAPI | Functional testing | Integration Testing | JavaScriptRemote workSenior-level Full TimePoland - Remote R3d ago
-
Senior Data Analytics, Senior Vice President PLN 330K-562KConfluence | Counterparty Credit | Counterparty Credit Risk | Credit Risk | Data GovernanceDiscretionary annual performance bonus | Employee assistance program | Holidays allowance | Hybrid working model | Learning and development resourcesSenior-level Full TimeLocation(s): Warsaw, Mazovia, Poland R3d ago