Research Engineer – LLM Evaluation Systems (Seed Infra)
Tasks
- Build distributed evaluation execution on multi-host multi-GPU clusters
- Collaborate with research and engineering teams
- Cover full model lifecycle from training inference evaluation
- Design develop and optimize large-scale LLM evaluation frameworks
- Implement scalable evaluation job orchestration
- Understand and validate LLM inference pipelines
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | Distributed Systems | GPU Computing | High Performance | High-Performance Computing | Kubernetes | Language Models | Large Language Models | Model Evaluation | Model verification | Performance Computing | Ray
Education
N/A
Roles
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R11d ago
-
Senior AI Software Engineer USD 160K-198KA2A | ADK | API | Containers | EnvoyNo visa sponsorship | Remote workSenior-level Full TimeRemote US R9h ago
-
Deployed Engineer (Seattle) USD 150K-250KAWS | Agent Orchestration | Containers | Demo Delivery | Failure handling401k plan | Dental insurance | Flexible vacation | Health insurance | Life insuranceMid-level Full TimeSeattle, WA10h ago
-
Senior Developer, Data & IT - AI Solutions USD 120K-142KAI Agents | API Integration | AWS | AWS Bedrock | AWS SageMakerDental insurance | Dependent Care Account | Health insurance | Health savings account | Mental health counseling supportSenior-level Full TimeNew York, NY, United States11h ago
-
AI Research Engineer USD 190K-280KAgentic AI | Clinical data | Data Pipelines | Data integration | Deep learningDiversity and inclusion initiatives | Flexible work environment | Friendly work environment | Professional developmentMid-level Full TimeSeattle, Washington, United States; South San …12h ago
-
Principal AI Engineer - Nexus Black USD 135K-160KCI/CD | Cloud Native | Cloud Native Architecture | Distributed Systems | EvaluationHybrid workSenior-level Full TimeItasca, United States13h ago
-
Generative AI Inference Engineer USD 152K-287KAWS | CUDA | Cloud platform | Diffusion Models | DockerSenior-level Full TimeUnited States13h ago
-
Distinguished Software Engineer, Data Infrastructure USD 248K-406KAI Inference | AI Training | Batch Processing | Compliance | Data InfrastructureExecutive-level Full TimeMountain View, CA, United States13h ago
-
Manager, Data Engineering USD 130K-166KAWS | Access Controls | Apache Airflow | Audit Logging | AzureCollaborative team culture | Remote work | Work-life balanceSenior-level Full TimeRemote, United States R15h ago
-
Senior Software Engineer - Data Platform USD 150K-195KAPIs | BigQuery | C++ | CI/CD | Cloud Storage401k company contribution | Disability insurance | Fertility and infertility benefits | Industry competitive PTO | Learning and development opportunitiesSenior-level Full TimeUnited States R15h ago
-
AI Search | API Development | AWS | AWS Bedrock | Azure401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States16h ago
-
Machine Learning Engineer USD 155K-228KAirflow | Amazon SageMaker | Apache Spark | Argo CD | Artificial IntelligenceHealthcare insurance | Paid parental leave | Paid personal time off | Paid sick time | Paid time offMid-level Full TimeRemote - US R17h ago
-
Staff Software Engineer, Data USD 270KApache Flink | Apache Kafka | Apache Spark | Cloud Data | Cloud data warehousing401k matching | ADND Insurance | Company holidays | Extended parental leave | Flexible spending accountSenior-level Full TimeUSA, Palo Alto18h ago
-
Software Engineer, Data Security - USDS USD 118K-237KData Security | Distributed Systems | Language Models | Large Language Models | Performance optimizationEntry-level Full TimeSan Jose, California, United States19h ago
-
Software Developer, Scaled Ops AI Acceleration Team USD 147K-203KAI infrastructure | Data Mining | Fine Tuning | Hack | JavaScriptSenior-level Full TimeSunnyvale, CA | Austin, TX | …20h ago
-
Software Engineer - Language (Technical Leadership) USD 213K-293KASR | Benchmarking | C# | C++ | Conversational AISenior-level Full TimeMenlo Park, CA | Seattle, WA …20h ago
-
Code review | Contamination Checking | Data Generation | Data Pipelines | Data ProcessingEntry-level Full TimeMenlo Park, CA20h ago
-
Business Support Engineer USD 136K-197KCall Support | Cloud Computing | Data Analysis | Data Mining | Docker24x7 on-call rotationEntry-level Full TimeMenlo Park, CA20h ago
-
Business Support Engineer USD 159K-223KCloud Computing | Data Analysis | Data Mining | Distributed Systems | Docker24x7 on-call rotation | Cross-functional team collaboration | Global partner supportSenior-level Full TimeMenlo Park, CA20h ago
-
Research Engineer, Media Data Research - MSL FAIR USD 170K-251KComputer Vision | Data Curation | Data Generation | Data Scaling Laws | Data mixingSenior-level Full TimeMenlo Park, CA20h ago
-
Staff Software Engineer, Torch TPU USD 207K-300KCUDA | Computer Vision | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeSunnyvale, CA, USA20h ago
-
C++ | Compilers | Custom Kernels | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA20h ago
-
Technical Solutions Engineer, Cloud AI, Google Cloud USD 150K-218KAI Model Training | AI model | Apache Beam | Apache Hadoop | Apache SparkSenior-level Full TimeSunnyvale, CA, USA; Austin, TX, USA20h ago
-
Senior Data Scientist - AI/ML Engineer USD 185K-250KAPI Development | AWS | Adapter Training | Agentic AI | Cloud NativeAccess to AI resources | Conference Presenting | Conference attendance | Learning budget | Team offsiteSenior-level Full TimeSunnyvale Office1d ago
-
Data Platform Lead USD 155K-190KAWS | Access Control | Apache Kafka | Apache Spark | Data anonymizationSenior-level Full TimeNew York City1d ago