Senior LLMOps Engineer
Tasks
- Build deployment health dashboard
- Create feedback flywheel from Intercom tickets and CSAT
- Design model incident response and evaluation workflows
- Enable session to model lineage
- Filter high value feedback into training data
- Generate execution trace summaries for stakeholders
- Implement incident alerting and root cause analysis
- Integrate observability with ASR and model pipelines
- Measure per model unit economics and optimize model selection
- Monitor post deployment performance and improve models
Perks/Benefits
- Equity
- Fertility support
- Health and wellness allowance
- Home office budget
- Learning and development budget
- Paid parental leave
- Work from anywhere
Skills/Tech-stack
Alerting | Dashboarding | Data Modeling | Data Pipelines | Datadog | Distributed tracing | Evaluation | Feedback Triage | Incident Response | Intercom | LLM Agents | LLMOps | Machine Learning | Model Monitoring | Observability | Session Lineage | Tracing | Unit economics
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Elasticsearch | Experimentation | Information Retrieval | Language Processing | Learning-to-RankEquity packages | Flexible leave options | Inclusive parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeSydney, NSW, Australia R2d ago
-
Elasticsearch | Experimentation | Information Retrieval | Learning-to-Rank | Machine LearningEquity package | Flexible leave | Parental leave | Remote work within Australia | Wellbeing allowanceSenior-level Full TimeMelbourne, Australia R2d ago
-
Senior AI Lead AUD 142K-156KAI Agents | Artificial Intelligence | Conversational AI | Data Analysis | Evaluation FrameworksCounseling access | Employee discounts | Employee micro credentials | Employee networks | Flexible working arrangementsSenior-level Full TimeSydney, Australia R2d ago
-
Lead Data Scientist - AI Technical Solutions AUD 142K-170KAI Pipelines | AWS | Apache Airflow | Artificial Intelligence | AzureContinuous learning | Flexible work arrangements | Remote WorkingSenior-level Full TimeSydney, Melbourne R3d ago
-
Lead AI Solutions Delivery Engineer AUD 160K-200KAPI Integration | Autonomous Agents | C++ | Claude | Cloud ComputingSenior-level Full TimeSydney, NS, Australia R4d ago
-
Senior Analytics Engineer AUD 150K-170KA/B | A/B Testing | B testing | BigQuery | DBTCareer growth | Health impact mission | Supportive work environment | Work from anywhereSenior-level Full TimeAustralia R5d ago
-
Senior / Staff Applied Scientist - AI Products AUD 130K-160KCalculus | Computer Vision | Data Pipelines | Data Processing | Language ModelsCollaborative boost days | Hybrid working model | Office access | Remote work flexibilitySenior-level Full TimeAU: Melbourne: (260 Burwood Rd) R6d ago
-
Digital Analytics Specialist AUD 80K-95KBigQuery | Campaign Manager | Conversion API | Customer Match | Data ModelingMid-level Full TimeMelbourne R9d ago
-
Senior Full Stack AI Engineer AUD 138K-145KArtificial Intelligence | Backend Development | Data Engineering | Data Pipelines | Frontend DevelopmentContinuous learning | Flexible work arrangements | Remote working up to 30 days per yearSenior-level Full TimeSydney R11d ago
-
Lead Data Engineer - Azure AUD 125K-140KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksEducation assistance program | Extra annual leave | Extra insurance | Flexible hybrid work model | Parental leaveSenior-level Full TimeSydney - Darling Park, Australia R11d ago
-
Agentic AI Engineer (AWS) AUD 100K-135KAWS | AWS CDK | Agent systems | Alembic | Amazon BedrockFlexible working options | Market-leading parental leave | Mentoring coaching programs | Paid volunteering days | Professional development reimbursementsMid-level Full TimeSydney, NSW, AU R11d ago
-
Data Engineer AUD 65K-70KAWS Redshift | Apache Spark | Automation | CI/CD | DBTDiscounts to retailers | Fitness passport | Flexible working | Paid parental leave | Professional developmentMid-level Full TimeHead Office- Melbourne, Australia R11d ago
-
Access Management | Azure | Azure Data | Azure Data Factory | Azure DatabricksClear communication | Fair disclosed margins | Hybrid work | Professional respectful experienceSenior-level Full TimeCanberra, Australian Capital Territory, Australia R12d ago
-
Senior AI Engineer AUD 102K-130KAWS | Amazon S3 | Bedrock | CI/CD | DockerConferences | Corporate discounts | Flexible hybrid workplace | Home office support | In-house trainingSenior-level Full TimeMelbourne, Australia R12d ago
-
Solutions Engineer AUD 140K-180K.NET | Apache Flink | Apache Hadoop | Apache NiFi | Apache SparkCareer development | Employee resource groups | Flexible work from home | Paid time off | Paid volunteer timeSenior-level Full TimeAustralia-New South Wales-Sydney R12d ago
-
Computer Vision Engineer (C++) (R4633) AUD 110K-131K3D Reconstruction | C++ | Camera Calibration | Computer Vision | Deep learningCareer development pathway | Equity | Hybrid work arrangementEntry-level Full TimeMelbourne R13d ago
-
Data Engineer AUD 125K-126KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageAdditional annual leave | Education assistance program | Employer funded insurance | Parental leave | Wellbeing leaveMid-level Full TimeSydney - Darling Park, Australia R16d ago
-
Machine Learning Platform Engineer SGD 139K-180KAWS | Azure | CI/CD | Dask | Distributed SystemsHealth and wellbeing resources | Paid volunteer daysMid-level Full TimeSydney - Australia - Sydney, 2000 … R16d ago
-
Data Engineer AUD 120K-126KCode review | Data Modeling | Data Warehousing | Data integration | Data pipelineDiscounted air travel | Discounted lounge membership | Discounts on travel insurance and car hire | Dress for your day | Employee assistance programMid-level Full TimeBrisbane, Melbourne, Sydney R17d ago
-
Enterprise Sales Engineer AUD 80K-100KAPM | AWS | Application Performance Monitoring | Application performance | Cloud platformContinuous professional development | Employee stock purchase plan | Global benefits | Hybrid work | Inclusion programsMid-level Full TimeBrisbane, Australia R17d ago
-
Robotics Software Engineer AUD 401KBenchmarking | C++ | CI/CD | Collision Avoidance | Machine Learning401k matching | Dental insurance | Health insurance | Paid Holidays | Travel supportMid-level Full TimeCharlestown, MA R20d ago
-
Senior Machine Learning Engineer AUD 142K-180KAWS | Algorithms | Anomaly Detection | CI/CD | Cloud platformClassPass membership | Employee resource groups | Extra annual leave | Flexible work hours | LinkedIn LearningSenior-level Full TimeMelbourne, Australia R23d ago
-
AI Engineer AUD 132K-180KAWS | Agent Development | Amazon Bedrock | Artificial Intelligence | AutomationFlexible hours | Free food and snacks | Opportunity to drive projects | Socials and Community Groups | Training and developmentSenior-level Full TimeMelbourne, Victoria, Australia R23d ago
-
Senior Machine Learning Engineer AUD 144K-180KAWS | Algorithms | Anomaly Detection | CI/CD | Cloud platformClassPass membership | Culture Swap Days | Employee resource groups | Extra annual leave | Flexible work hoursSenior-level Full TimeSydney, Australia R24d ago
-
AI Engineering Lead AUD 137K-143KAgile | Amazon Web Services | Docker | IAM | LoggingDiscounted flights | Discounts on accommodation | Discounts on car hire | Discounts on experiences | Discounts on lounge membershipSenior-level Full TimeBrisbane, Sydney R25d ago