Staff Applied AI Scientist
Tasks
- Build LLM analysis for performance regression
- Design evaluation monitoring and alerting systems
- Enable teams through reusable evaluation frameworks and tooling
- Lead evaluation enablement for product teams
- Own prompt engineering evaluation and continuous improvement
- Partner with product data science and people science on measured quality
- Recommend prompt or system changes
- Stay current with evaluation observability and LLMOps research
Perks/Benefits
- Coaching
- Extended end of year break
- Home workspace setup budget
- Inclusive parental leave
- Learning programs
- Medical insurance coverage
- Monthly Wellbeing Allowance
- Quarterly Refresh Days
- Social impact days
Skills/Tech-stack
Agentic Systems | Data Labelling | Evaluation datasets | Human-in-the-loop | LLM-as-judge | LLMOps | Language Models | Large Language Models | Machine Learning | Observability | Production Monitoring | Prompt engineering | Prompt management | Python | RAG | Sampling | The Loop | TypeScript
Education
Related jobs
-
Senior Data Scientist - Product, Features & Growth AUD 119K-140KA/B | A/B Testing | B testing | Data Modeling | Data VisualizationFlexible leave options | Hybrid work flexibility | Inclusive parental leave | Wellbeing allowanceSenior-level Full TimeSydney, NSW, Australia R10h ago
-
Applied AI Engineer AUD 70K-97KAgent architecture | CI/CD | Data pipeline | Docker | Infrastructure as CodeFlexible work arrangements | Leadership programs | Learning programs | Paid parental leave | Paid volunteering daysSenior-level Full TimeMelbourne, VIC, AU15h ago
-
Adtech | Distributed Systems | Docker | Embeddings | JavaAccess to technical leadership | Collaborative engineering culture | Exposure to cutting edge generative AI | High autonomy and flexibility | High ownership and real world production impactEntry-level Full TimeAustralia R1d ago
-
Senior Data Scientist AUD 102K-125KA/B | A/B Testing | Anomaly Detection | B testing | Data ContractsConference support | Corporate discounts and vouchers | Home-office allowance | Hybrid work flexibility | In-house trainingSenior-level Full TimeMelbourne, Australia1d ago
-
Lead Data Scientist AUD 150K-180KAccountability | CD4ML | Computer Vision | Data Mining | Data PreprocessingCareer development support | Learning and development programs | MentorshipSenior-level Full TimeMelbourne, Australia1d ago
-
AI Engineer - Melbourne AUD 92K-125KAPI Development | AWS | Automated testing | CI/CD | Cloud platformMid-level Full TimeMelbourne - 727 Collins, Australia R1d ago
-
Mid-level Full TimeSydney - Barangaroo, Australia R1d ago
-
Senior Data Scientist AUD 129K-142KData Engineering | Data Science | Data Transformation | Data Validation | Data extractionAnnual flu vaccinations | Discounted health and fitness memberships | Employee assistance program | Flexible working arrangements | Generous leave entitlementsSenior-level Full TimeSydney, NSW, AU, 20001d ago
-
Senior Manager - Responsible AI, Gen AI AUD 130K-143KAI Governance | Foundational models | Generative AI | Machine Learning | Model EvaluationFlexible working hours | In office collaboration at least half time per month | Job flexibility | Job-sharingSenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …3d ago
-
Staff Data Scientist - International (CJKI) AUD 65K-80KA/B | A/B Testing | B testing | Data Science | Data WarehousingEquity packages | Flexible leave options | Inclusive parental leave | Vibe and Thrive allowanceEntry-level Full TimeSydney, Australia R4d ago
-
Data Scientist, Network Fabric Engineering AUD 100K-126KAI systems | AWS Lambda | Amazon Athena | Amazon S3 | BenchmarkingMid-level Full TimeSydney, New South Wales, AUS4d ago
-
Azure AI Developer Consultant AUD 70K-100K.NET | API Management | App Service | Azure API | Azure API ManagementFlexible work schedule | Onsite customer visits when required | Work from home optionSenior-level Full TimeMelbourne, VIC, AU4d ago
-
Senior-level Full TimePERTH, WESTERN AUSTRALIA, Australia5d ago
-
Entry-level Full TimeMelbourne, Australia5d ago
-
Data Scientist - Personalisation AUD 118K-140KA/B | A/B Testing | Applied statistics | B testing | ClassificationFlexible leave options | Inclusive parental leave | Remote work flexibility | Virtual interviews | Wellbeing allowanceMid-level Full TimeSydney, Australia R5d ago
-
Senior Data Scientist - Design Experience Growth AUD 140K-173KAI | Amazon Redshift | Data Warehousing | Event Tracking | ExperimentationEquity packages | Flexible leave options | Flexible remote work | Inclusive parental leave | Wellbeing allowanceSenior-level Full TimeSydney, NSW, Australia R5d ago
-
Data Scientist AUD 80K-90KAlgorithms | Cloud Computing | Data Modeling | Data Structures | Data VisualizationEntry-level Full TimeAUS - Melbourne, Australia5d ago
-
Senior Data Science (Client-Facing FMCG) AUD 129K-150KBias reduction | Data Aggregation | Data Preparation | Data Quality | Data cleaningEmployee assistance program | Flexible working environment | Hybrid work | LinkedIn Learning | Volunteer time offSenior-level Full TimeSydney, NSW, Australia6d ago
-
AI Data Scientist AUD 80K-120KAWS Glue | AWS Glue Data Catalog | Apache Atlas | Azure Purview | CollibraHybrid working | Learning access | Professional developmentMid-level Full TimeSydney, New South Wales, Australia6d ago
-
Solutions Architect AUD 130K-180KAI SDK | APIs | Agentic AI | Cloud Architecture | Computer VisionMentorship opportunities | Remote work | Training sessionsSenior-level Full TimeRemote - Australia R6d ago
-
Senior Data Scientist - Hawthorn East AUD 135K-142KAttribution | Causal Inference | DBT | Data Engineering | Data ModelingDiscounts | Fitness classes | Free parking | Gym facility | Hybrid working optionsSenior-level Full TimeHawthorn East Melbourne, VIC, AU, 31236d ago
-
Senior Data Scientist (Marketing) AUD 152K-223KA/B | A/B Testing | Attribution Modeling | B testing | Cohort AnalysisDisability insurance | Employee stock options | Employer retirement contribution | Life insurance | Medical/Dental/Vision insuranceSenior-level Full TimeMelbourne, VIC, Sydney, NSW R6d ago
-
Senior AI Agentic AI Engineer AUD 130K-165KAPI Development | Agent Orchestration | Anthropic Claude | Autogen | Automated testingSenior-level Full TimeSydney, New South Wales 2000, Australia7d ago
-
Benchmarking | Dataset curation | Deep learning | Distributed Training | Efficient Fine TuningFlexible working arrangements | Fully remote work | Professional growth opportunities | Publication supportSenior-level Full TimeAustralia R7d ago
-
AI and Data Engineer AUD 118K-127KAPI Development | AWS Bedrock | Azure OpenAI | Azure OpenAI API | Build toolSenior-level Full TimeMelbourne - on the lands of …7d ago