Staff Applied AI Scientist
Tasks
- Build LLM analysis for performance regression
- Design evaluation monitoring and alerting systems
- Enable teams through reusable evaluation frameworks and tooling
- Lead evaluation enablement for product teams
- Own prompt engineering evaluation and continuous improvement
- Partner with product data science and people science on measured quality
- Recommend prompt or system changes
- Stay current with evaluation observability and LLMOps research
Perks/Benefits
- Coaching
- Extended end of year break
- Home workspace setup budget
- Inclusive parental leave
- Learning programs
- Medical insurance coverage
- Monthly Wellbeing Allowance
- Quarterly Refresh Days
- Social impact days
Skills/Tech-stack
Agentic Systems | Data Labelling | Evaluation datasets | Human-in-the-loop | LLM-as-judge | LLMOps | Language Models | Large Language Models | Machine Learning | Observability | Production Monitoring | Prompt engineering | Prompt management | Python | RAG | Sampling | The Loop | TypeScript
Education
Related jobs
-
Signals Intelligence Specialist AUD 150K-180KAgent Orchestration | Artificial Intelligence | Data Infrastructure | Data Modeling | Data PipelinesSenior-level Full TimeSydney, Australia2h ago
-
PhD Graduate Machine Learning Engineer Graduate, Trust and Safety Engineering - 2026 Start AUD 65K-75KAlgorithms | Computer Vision | Data Mining | Data Structures | Language ProcessingEntry-level Full TimeSydney, New South Wales, Australia1d ago
-
Experimental Physics Consultant GBP 30K-34KData Analysis | Data acquisition | Electromagnetic systems | Environmental testing | Experimental DesignAnnual leave | Community charity initiatives | Company pension | Performance bonus | Private healthcareNone Full TimeMelbourn, United Kingdom1d ago
-
Senior Data Scientist AUD 131K-146KAPIs | AWS | Advertising metrics | CI/CD | ClassificationAccessibility support | Birthday leave | Career break leave | Company charity drives | Compassionate leaveSenior-level Full TimeNSW - SHS 2 Holt Street …1d ago
-
Amazon Web Services | Apache Spark | Big Data | Data Analysis | HadoopRemote workSenior-level Full TimeSydney, NSW, Australia R4d ago
-
Staff Data Scientist, Cashalo SGD 161K-245KDeep learning | GitHub Copilot | Language Models | Large Language Models | LightGBMEquity | Flexible work support | Global team collaboration | Health insurance | Health plan subsidiesSenior-level Full TimeRemote - Australia R4d ago
-
AI Solutions Architect – Azure AI Foundry & Agentic AI AUD 150K-180K.NET | AI Agents | AI Foundry | AI Governance | AI SearchSenior-level Full TimeMelbourne, 161 Collins Street, Level 5, …4d ago
-
AURIN Data Scientist AUD 111K-120KCI/CD | Cloud Computing | Containerization | Data Processing | Data ValidationCultural leave | Discounted services | Employee assistance program | Flexible work arrangements | Gender Affirmation LeaveMid-level Full TimeParkville, Australia4d ago
-
AWS | Azure | Cloud platform | DBT | Data VisualizationAdditional paid leave | Corporate partner discounts | Hybrid work | Learning opportunities | Mobile PlanSenior-level Contract Full Time TemporaryBarangaroo, NSW, AU4d ago
-
AI Engineer - Conversational AUD 200K-250KAWS | Agile | Amazon Connect | Amazon Lex | Cloud ComputingEmployee discounts | Flexible work schedule | On-site parkingExecutive-level Full TimeSydney Olympic Park, New South Wales …5d ago
-
Senior Data Scientist - Discovery (12-month contract) AUD 118K-127KAnalytics | Dashboarding | Data Warehousing | Data analytics | Experiment designEquity packages | Flexible leave | Hybrid work | Inclusive parental leave | Office setup supportSenior-level Contract Full TimeSydney, NSW, Australia R5d ago
-
Senior Data Scientist AUD 121K-136KAWS SageMaker | Artificial Intelligence | DBT | Data Pipelines | Decision TreesFlexible working arrangements | Hybrid work options | SuperannuationSenior-level Full TimeCity Campus, Australia5d ago
-
Action Recognition | Computer Vision | Data Analysis | Deep learning | Edge AIHigh-performance computing access | Industry placement | Research commercialisation training | Travel grantEntry-level Full Time TemporaryCampus, Clayton campus5d ago
-
Biosignal analysis | Cross Device Generalization | Cross device | Deep learning | EEGAccess to AI infrastructure | Research funding | Research living allowance | Travel grant | Tuition scholarshipEntry-level Full Time TemporaryCampus, Clayton campus5d ago
-
Forward Deployed Engineer III, Gen AI, Retail AUD 135K-180KAPI Integration | Agent systems | Agentic AI | Cloud Computing | CrewAISenior-level Full TimeSydney NSW, Australia; Melbourne VIC, Australia6d ago
-
Chief Ai Engineer AUD 100K-173KAI guardrails | Agentic Systems | Artificial Intelligence | Evaluation Frameworks | Generative AICulture leave | Employee mortgage discounts | Flexible work arrangements | Learning and development opportunities | Lifestyle leaveExecutive-level Full TimeNSW, Australia6d ago
-
Staff Data Scientist - Growth AUD 140K-161KAI integration | Amazon Redshift | Data Storytelling | Data Warehousing | Google BigQueryEquity package | Flexible leave | Parental leave | Remote work flexibility | Wellbeing allowanceSenior-level Full TimeSydney, Australia R6d ago
-
Data Scientist, Specialist AUD 121K-140KAWS SageMaker | Artificial Intelligence | Azure Machine Learning | CI/CD | Data EngineeringHybrid work modelSenior-level Full TimeAUS - Wesley Place, Australia6d ago
-
Senior Data Scientist AUD 129K-140KAgentic AI | Data Analysis | GenAI | Language Models | Language ProcessingFlexible start and finish times | Flexible work arrangement | Job share | Part-time optionsSenior-level Full TimeSydney, NSW - CBP South, 11 …6d ago
-
Chapter Lead, Data Science (Operations) AUD 140K-150KAWS | Agentic Systems | Azure | CI/CD | Cloud ArchitectureHybrid work flexibility | Inclusion and diversity culture | Learning and development opportunitiesSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …6d ago
-
Technical Architect, AI/ML AUD 156K-200KAPI | AWS SageMaker | Agentic Frameworks | Amazon Web Services | Apache SparkTravel opportunitiesSenior-level Full TimeAU-Sydney7d ago
-
Technical Architect AI/ML AUD 156K-200KAPIs | AWS | Azure | ETL | Feature EngineeringTravel opportunitiesSenior-level Full TimeAU-Victoria-Remote R7d ago
-
API Integration | AWS Cloud | Agile delivery | Amazon Connect | Amazon LexFlexible working options | Flexible workplace collaboration | Learning opportunitiesEntry-level Full TimeSydney, NSW - CBP North, 1 …7d ago
-
AI Architect AUD 130K-157KAI Search | AWS | Amazon Bedrock | Amazon Comprehend | Amazon RekognitionAI Driven Learning Platform | Career Development Programs | Discounted health insurance | Industry and Partner Training | Paid parental leaveSenior-level Full TimeMelbourne, VIC, Australia8d ago
-
AI Architect AUD 130K-157KAI Search | Artificial Intelligence | Azure AI | Azure AI Search | Azure CognitiveAI Driven Learning Platform | Career development | Discounted health insurance | Industry training | Paid parental leaveSenior-level Full TimeSydney, NSW, Australia8d ago