Senior Software Engineer II - Applied AI and Evaluations (Remote Eligible)
Tasks
- Build regression gates
- Collaborate on agent architecture tradeoffs
- Create golden datasets and scorers
- Design LLM evaluation systems
- Diagnose agent failures
- Establish quality improvement methodology
- Improve prompt and context quality
- Run online evaluation on production traffic
- Tune RAG retrieval performance
- Validate measurable quality gains
Perks/Benefits
- 401k match
- Counseling membership
- Dental insurance
- Flexible time off
- Health insurance
- Life insurance
- Long-term disability
- Monthly stipend
- Paid Holidays
- Parental leave
- Short-term disability
- Teleworking options
- Udemy Courses Access
- Vision insurance
- Volunteer day
Skills/Tech-stack
Agent Orchestration | CI/CD | Chunking Strategies | Context engineering | Databricks | Delta Tables | Embedding Models | MLflow | Machine Learning | Multi-Agent | Multi-agent orchestration | Prompt engineering | Python | RAG evaluation | Retrieval-Augmented Generation
Education
Related jobs
-
AI Gateways | AWS CDK | Chunking | Context engineering | Cost Tracking401k match | Counseling membership | Flexible time away | Life insurance | Long-term disabilityMid-level Full Time-REMOTE, USA- R13h ago
-
Agile | Apache Airflow | Artificial Intelligence | Automated testing | BigQueryCollaborative culture | Flexible working hours | Performance evaluations | Professional development opportunities | Remote workSenior-level Full TimeIdaho R20h ago
-
Agile | Apache Airflow | BigQuery | CI/CD | Cloud StorageCollaborative company culture | Flexible working hours | Professional development opportunities | Remote-first work environmentSenior-level Full TimeMinnesota R20h ago
-
Agile | Airflow | BigQuery | CI/CD | Cloud StorageCollaborative culture | Professional development | Remote-first flexibilitySenior-level Full TimeColorado R20h ago
-
Agile | Airflow | BigQuery | CI/CD | Cloud StorageCollaborative company culture | Flexible working hours | Professional development opportunities | Remote-first environmentSenior-level Full TimeColumbia R20h ago
-
Agile | Airflow | Automated testing | BI tools | BigQueryCollaborative company culture | Professional development | Remote-first flexible hoursSenior-level Full TimeIllinois R20h ago
-
Agile | Airflow | BigQuery | CI/CD | Cloud StorageCollaborative company culture | Professional development opportunities | Remote-first flexible hoursSenior-level Full TimeFlorida R20h ago
-
Agile | Airflow | Automated testing | BI tools | BigQueryCollaborative culture | Flexible working hours | Professional development | Remote-first environmentSenior-level Full TimeCalifornia R20h ago
-
Agile | Apache Airflow | Automated testing | BigQuery | CI/CDCollaborative company culture | Professional development opportunities | Remote-first flexibilitySenior-level Full TimeConnecticut R20h ago
-
Agile | Airflow | BI tools | BigQuery | CI/CDCollaborative & Innovative Culture | Professional development | Remote-first flexible hoursSenior-level Full TimeArizona R20h ago
-
Apache Airflow | Apache Hive | Apache Iceberg | Apache Kafka | Apache SparkFully remote work option | International hiring | Long term contractor optionEntry-level Full TimeUnited States R1d ago
-
Defensive Security AI Scientist USD 240K-260KAccelerate | Artificial Intelligence | CISA KEV | CUDA | CVSS401k plan with company matching | Bereavement | Disability insurance | Employee assistance program | Employee discount programSenior-level Full TimeRemote - Nationwide, United States R1d ago
-
ML Infrastructure Engineer USD 145K-165KAWS | Amazon Elastic Kubernetes Service | Amazon SageMaker | BigQuery | CD pipelinesHealth benefits | Paid time off | Remote work optionMid-level Full TimeBoston, MA R2d ago
-
ML Infrastructure Engineer USD 145K-165KAWS | Amazon SageMaker | BigQuery | CI/CD | CloudFormationBenefits plans | Remote work optionMid-level Full TimeNew York, New York, United States R2d ago
-
ML Infrastructure Engineer USD 145K-165KAWS | Amazon SageMaker | BigQuery | BigQuery datasets | CI/CDCompany benefits plan enrollment | Health benefits | Performance-based bonus | Remote work optionMid-level Full TimeLos Angeles, California, United States R2d ago
-
AI | AWS | DBA | Database systems | DevOpsDental insurance | Flexible working hours | Health insurance | Paid time off | Professional developmentSenior-level Full TimeMinnesota R2d ago
-
AI machine learning | AWS | Cloud platform | DBA operations | Database systemsDental insurance | Flexible working hours | Health insurance | Paid time off | Professional developmentMid-level Full TimeIllinois R2d ago
-
C# | MATLAB | NumPy | Pandas | PythonFlexible hours | Non permanent employment | Part-time project workSenior-level Full TimeNew York, New York, United States … R2d ago
-
Senior Python Developer - Code Migration Specialist USD 160K-160KBash | Coverage.py | Dagger | Docker | GcovFlexible schedule | Freelance project-based collaboration | Fully remote | Supportive global communitySenior-level Full TimeNew York, United States - Remote R2d ago
-
Senior Python Developer - Code Migration Specialist USD 160K-160KBash | Black box testing | Black-box | Box testing | Code CoverageFlexible schedule | Project-based collaboration | Remote work | Supportive global communitySenior-level Full TimeTexas, United States - Remote R2d ago
-
Senior Python Developer - Code Migration Specialist USD 160K-160KBash | Black box testing | Black-box | Box testing | Code CoverageFlexible hours | Fully remote | Project-based collaboration | Supportive communitySenior-level Full TimeMichigan, United States - Remote R2d ago
-
Senior Python Developer - Code Migration Specialist USD 160K-160KBash | Black box testing | Black-box | Box testing | Coverage.pyFlexible schedule | Freelance project-based collaboration | Fully remote | Project based hours | Supportive global communitySenior-level Full TimeFlorida, United States - Remote R2d ago
-
Senior Python Developer - Code Migration Specialist USD 160K-160KBash | Coverage.py | Dagger | Docker | GcovFlexible schedule | Freelance project-based collaboration | Fully remote | Opportunity to contribute to AI projects | Supportive global communitySenior-level Full TimeUnited States - Remote R2d ago
-
Senior AI Data Engineer USD 160K-200KAWS | AWS Athena | AWS Glue | AWS Lambda | Amazon Redshift401k matching | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeSan Diego, California, United States R2d ago
-
Causal Inference | Classification | Clustering | Data Warehousing | Experiment designFlexible PTO | Home office stipend | Learning budget | Paid health, dental, vision | Parental leaveSenior-level Full TimeBoston or Remote R2d ago