AI Model Evaluation Specialist
New York, New York, United States
A USD 150K-211K (estimate) Entry-level Full Time
Tasks
- Build and label evaluation datasets
- Collaborate with data scientists and product managers
- Compare prompt strategies
- Create scoring guidelines and rubrics
- Document scoring patterns and errors
- Score LLM outputs
- Use NLP metrics for automated scoring
Perks/Benefits
- N/A
Skills/Tech-stack
BLEU | Cosine similarity | Jupyter | Label Studio | Language Models | Large Language Models | Machine Learning | NLP | Prodi gy | Prompt engineering | Python | ROUGE
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Senior Full Stack AI Developer USD 145K-180KArtificial Intelligence | Cloud Computing | Continuous Delivery | Continuous integration | Language ModelsRemote workSenior-level Full TimeWork from home, VA, United States R4h ago
-
Senior Embedded Software Engineer USD 142K-182KAMD Xilinx | ARM Cortex | C# | Digital Signal Processing | Embedded Linux401k match | Dental insurance | Employee assistance program | Every other Friday off | FSASenior-level Full TimeCamarillo, CA, United States4h ago
-
Senior Data Engineer USD 130K-175K.Net Core | .Net Framework | Apache Airflow | Azure | Azure Data401k match | Career growth opportunities | Dental insurance | Employee resource groups | Life insuranceSenior-level Full TimeRemote, United States R4h ago
-
Analytics Engineer USD 89K-140KAnalytics | Azure Data | Azure Data Factory | DAX | Data Factory401k match | Dental insurance | Health insurance | Health savings account | Life insuranceMid-level Full TimeMurrysville, PA, United States5h ago
-
AWS Glue | Amazon Redshift | Amazon SageMaker | Data Modeling | Data PipelinesMid-level Full TimeHouston, TX, United States6h ago
-
Data Engineer USD 90K-125KAnomaly Detection | Azure | Data Analysis | Data Migration | Data Modeling401k | Dental insurance | Disability coverage | Employee assistance program | Flexible spending accountsMid-level Full TimeSt. Louis, MO, United States6h ago
-
Agile | Automation | Azure AD | Azure Monitor | Azure SentinelKnowledge transfer opportunities | Long-term contract | Onsite in Deerfield Beach FLSenior-level ContractDeerfield Beach, United States6h ago
-
Machine Learning Engineer, Physical AI USD 150K-200KAWS | CUDA | Computer Vision | Deep learning | FastAIBi-annual offsites | Dental insurance | Flexible PTO | Health insurance | Learning and development budgetMid-level Full TimeSan Francisco, CA, US7h ago
-
Robotics Systems Engineer, Physical AI USD 150K-200KCalibration | Computer Vision | Data pipeline | Deep learning | DockerDental insurance | Flexible PTO | Health insurance | Learning and development budget | Monthly socialsMid-level Full TimeSan Francisco, CA, US7h ago
-
Head of AI USD 150K-300KAPI Integration | AWS | Artificial Intelligence | Automations | ChatGPT401k with company matching | Dental insurance | Fully remote work environment | Health insurance | Vision insuranceExecutive-level Full TimeAustin, Texas, Austin HQ, Dallas, Texas, … R7h ago
-
AWS Data Architect USD 140K-216KAPI Integration | AWS CloudFormation | AWS Glue | AWS Lake Formation | AWS LambdaOnsite daily at officeSenior-level Full TimeFremont, United States7h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | AWS SageMaker Studio | Apache Airflow | AvroFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeTampa, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | AWS SageMaker Studio | Airflow | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Office workSenior-level Full TimeFort Lauderdale, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | AWS SageMaker Studio | Airflow | Apache SparkEducation budget | Fitness budget | Flextime with remote and office options | Mentorship | Personalized growth roadmapsSenior-level Full TimeJacksonville, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Amazon SageMaker | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeOrlando, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Personalized growth roadmaps | Remote and office options | TechtalksSenior-level Full TimeTallahassee, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Apache Spark | Avro | DuckDBFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote optionsSenior-level Full TimeBaltimore, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Apache Spark | Avro | Data LakeEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeRichmond, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS SageMaker | Amazon Web Services | Apache Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Remote and office options | TechtalksSenior-level Full TimeMiami, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | Airflow | Apache Spark | AvroEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeAustin, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Apache Spark | Avro | Big DataFlexible schedule | Mentorship | Personalized growth roadmaps | Remote work options | TechtalksSenior-level Full TimeHouston, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Apache Spark | Avro | Data LakehouseFlextime | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeBlacksburg, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | Airflow | Avro | Data Lake | Data LakehouseFlextime | Mentorship | Office work options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeWest Palm Beach, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | AWS SageMaker Studio | Apache Airflow | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeSan Francisco, United States8h ago
-
Data Engineer ID50062 USD 148K-164KAWS | AWS SageMaker | Airflow | Avro | Big DataEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeNew York, United States8h ago