LLM Ops Engineer
Tasks
- Build model evaluation tooling for clinical accuracy latency cost and safety
- Build prompt engineering and model data pipelines
- Collaborate on fine tuning workflows model selection prompt versioning and context management
- Define alerting thresholds and build model health monitoring tooling
- Design deploy maintain AWS EKS infrastructure for GPU model workloads
- Design offline evaluation harnesses automated regression tests and dashboards
- Instrument token usage latency P99 GPU memory pressure hallucination rates and error classes
- Manage GPU node pools tune autoscaling for inference
- Manage spot and on demand node strategy
- Optimize GPU utilization quantization request batching and model routing
- Own model serving lifecycle from container builds to production rollouts
- Write and maintain infrastructure as code with Terraform
Perks/Benefits
Skills/Tech-stack
A B Deployment | A/B | AWS | Alerting | Autoscaling | Dashboards | EKS | Evaluation harnesses | Fine Tuning | GPU workload scheduling | IAM | Kubernetes | Latency P99 | Model Registry | Model routing | Observability | On Demand | On Demand Instances | Prompt versioning | Python | Quantization | RAG | RLHF | Regression testing | Request batching | Rollback | Shadow traffic | Spot instances | TGI | TensorFlow Serving | Terraform | VLLM | Workload Scheduling
Education
N/A
Regions
Countries
States
Related jobs
-
ML Engineer AUD 150K-200KAmazon SageMaker | BigQuery | CI/CD | Cloud Run | Cloud platformCareer development opportunities | Community volunteering | Company events and celebrations | Hackathons | Health and wellness programSenior-level Full TimeSouth Melbourne, Victoria13h ago
-
Data Engineer AUD 90K-120KA/B | A/B Testing | Attribution | B testing | BigQueryFlexible work environment | Learning and training budget | Snacks and drinks | Team eventsMid-level Full TimeBarangaroo, NSW, Australia14h ago
-
Advanced Analytics Senior Analyst AUD 129K-142KCompliance testing | Controls Testing | Dashboards | Data Pipelines | Data WarehousingSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …15h ago
-
Staff Data Engineer AUD 130K-142KAWS | AWS Glue | AWS Lambda | Agile | Amazon AthenaEmployee assistance support | Flexible work scheduleSenior-level Full TimeEveleigh, NSW - 1 Locomotive Street, …15h ago
-
Staff Data Engineer (AWS Cloud and AI) AUD 129K-140KAPI Gateway | AWS CloudFormation | AWS DataZone | AWS Glue | AWS Lake FormationFlexible work hours | Hybrid work model | Job-sharing | Part-time optionsSenior-level Full TimeSydney, NSW - CBP South, 11 …15h ago
-
AI Observability | AWS | Azure | CI/CD | Cost ControlCareer advancement | Exposure to emerging technologies | Fully remote | Professional development | Work-life balanceSenior-level Full TimeAustralia R3d ago
-
Forward Deployed Engineer, Gen AI, Google Cloud AUD 110K-130KAPIs | Agent systems | Cloud platform | CrewAI | EvaluationMid-level Full TimeDocklands VIC, Australia; Sydney NSW, Australia3d ago
-
AI Deployment Engineer AUD 130K-180KChatGPT | Cloud Architecture | Generative AI | JavaScript | Machine LearningHybrid work model | Relocation assistanceSenior-level Full TimeSydney, Australia3d ago
-
Senior Data Engineer AUD 135K-155KArangoDB | Azure | CCPA | CI/CD | Conformed DimensionsAnnual leave | Collaborative team culture | Flexible working environment | Generous parental leave | Global work experienceSenior-level Full TimeMelbourne, VIC, Australia3d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Competitive salary | Friendly work environment | Remote/distributed teamMid-level Full TimeSydney, Australia3d ago
-
Data Engineer AUD 122K-135KDimensional Modeling | ELT | ETL | Git | GitHubFlexible schedule | Free breakfast | Free snacks | Hybrid work | Live sportsSenior-level Full TimeBrisbane, Queensland, Australia3d ago
-
Chief Engineer - Engineering Data Platform AUD 100K-190KAPIs | AWS | Automation | CI/CD | Cloud ComputingExecutive-level Full TimeEveleigh, NSW - 1 Locomotive Street, …3d ago
-
Staff Engineer Chapter Lead – Generative AI AUD 140K-150KAI Governance | AI ML system security | AI/ML | AWS | AWS CDKSenior-level Full TimeSydney, NSW - CBP South, 11 …3d ago
-
Principal Data Engineer AUD 149K-173KAutomated testing | CI/CD | Cloud Architecture | Data Engineering | Data GovernanceHybrid workingSenior-level Full TimeSydney, New South Wales 2000, Australia3d ago
-
Staff Machine Learning Engineer (Platform) AUD 130K-180KAWS | Azure | CUDA | Data Quality | Data WarehousingEquity compensation | Flexible work environment | Office snacks | Regular office eventsSenior-level Full TimeSydney4d ago
-
Senior Data Engineer AUD 125K-138KBackup and Recovery | Data Architecture | Data Cleansing | Data Governance | Data ModelingCollaborative environment | Flexible work arrangements | Onsite work | Supportive team cultureSenior-level Full TimeCanberra, Australian Capital Territory 2600, Australia4d ago
-
Senior Data Governance Engineer AUD 150K-170KAccess Control | Data Classification | Data Engineering | Data Governance | Data LineageConvention attendance | Flexible working hours | Focus time | Medical advice | Mental health supportSenior-level Full TimeAlbury, Australia4d ago
-
Machine Learning Data Scientist USD 111K-121KCloud Computing | Computer Vision | Data Engineering | Data Warehousing | Deep learningDiscounted Staff Parking | Flexible work options | Health and wellbeing program | Positive workplace culture | Salary packagingSenior-level Full Time50 Flemington Rd, AU4d ago
-
Data Engineer AUD 64K-69KAWS Redshift | Apache Spark | CI/CD | DBT | Data GovernanceDiscounts | Fitness passport | Flexible working | Paid parental leave | Professional developmentMid-level Full TimeHead Office- Melbourne, Australia4d ago
-
Senior AI & Data Scientist AUD 118K-134KAgentic Workflows | Azure | Data Exploration | Data Governance | Feature EngineeringAdditional leave entitlements | Employee growth and development | Employee share plan | Hybrid working | Novated car leaseSenior-level Full TimeAU PER Perth Office, Australia4d ago
-
Principal Data Engineer AUD 140K-152KAWS | Agile | Change Management | Cloud Computing | Cloud NativeFlexible start and finish times | Flexible work arrangements | Job-sharing | Part-time arrangementsSenior-level Full TimeSydney, NSW - CBP North, 1 …4d ago
-
Senior Data Engineer (Scala/Java, Spark, AWS) AUD 129K-142KAWS | Access Control | Agile | Apache Spark | CI/CDFlexible start and finish times | Flexible work arrangements | Hybrid working | Job share | Part-time arrangementsSenior-level Full TimeEveleigh, NSW - 5-7 Central Ave, …4d ago
-
AWS Glue | AWS Lambda | Amazon EMR | Amazon Redshift | Amazon S3Mentorship | On-call supportEntry-level Full TimeSydney, New South Wales, AUS4d ago
-
Data Engineer AUD 110K-140KApache Kafka | Apache Spark | Azure | Azure Synapse | Cloud DataEmployee discounts | Flexible work arrangements | Learning and development opportunities | Leave options | Volunteering initiativesMid-level Full TimeBarangaroo, NSW, Australia4d ago
-
Senior Data Engineer AUD 129K-140KAI Automation | Apache Kafka | Apache Spark | Azure Synapse | Data ArchitectureEmployee discounts | Flexible work arrangements | Learning and development | Leave options | Volunteering opportunitiesSenior-level Full TimeSurfers Paradise, QLD, Australia4d ago