Gen AI Tester
Tasks
- Automate testing with scripts and test tools
- Collaborate with cross functional AI teams in Agile DevOps
- Design and execute test cases for non deterministic AI systems
- Detect hallucinations and bias
- Develop testing strategies for generative AI systems
- Perform exploratory negative and adversarial testing
- Test AI model endpoints using APIs
- Test chatbot and NLP applications
- Validate accuracy relevance consistency and factuality
- Validate data quality and AI performance metrics
Perks/Benefits
- N/A
Skills/Tech-stack
AI Agents | API Testing | AWS Bedrock | Adversarial Testing | Azure OpenAI | Bias detection | Cloud Platforms | Explainability | Exploratory Testing | Fairness | Generative AI | Google Vertex | Google Vertex AI | Hallucination detection | JSON | LLM | Machine Learning | NLP | Negative Testing | Prompt engineering | Python | RAG | REST | Regression testing | Statistics | Test automation | Transparency | Vertex AI
Education
N/A
Roles
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)23d ago
-
Forward Deployed Engineer USD 120K-180KCloud infrastructure | Language Processing | Machine Learning | Natural Language | Natural Language ProcessingCustomer collaboration | Frontier technology | Technical mentorshipEntry-level Full TimeSan Francisco, CA, US8h ago
-
Senior-level Full TimeHerdon, VA, US9h ago
-
Data Engineer – Senior USD 140K-180KABAC | AWS | AWS Glue | AWS IAM | AWS Lake FormationActive MBI required | IRS contract experience preferred | U.S. citizenship requiredSenior-level Full TimeHyattsville, MD, US9h ago
-
AI Research Scientist (Robot Learning) USD 175K-251KData Flywheel | Deep learning | Distributed Training | Fine Tuning | Generative ModelsFree gym | Sports subscription | Team activities | Team tripsMid-level Full TimeSan Francisco13h ago
-
AI Research Engineer (Robot Learning) USD 175K-250KData Flywheel | Deep learning | Distributed Training | Fine Tuning | Generative ModelsFree gym membership | Sports subscription | Stock options | Team meals | Team tripsMid-level Full TimeSan Francisco13h ago
-
AI/ML Engineer II USD 93K-179KAmazon EC2 | Amazon S3 | Apache Airflow | CI/CD | CSS401k | Adoption Assistance | Career path planning | Continuing education | Employee referral programMid-level Full TimePhoenix Campus (Main), United States23h ago
-
Senior Software Engineer, Agentic Engineering USD 184K-356KAgent Orchestration | Autonomous Search | CI/CD | CUDA | Code generationEquity | Health benefits | Paid time off | Relocation assistanceSenior-level Full TimeUS, CA, Santa Clara, United States23h ago
-
MES Software Engineer/Data Analyst USD 109K-175K.NET | AWS | Artificial Intelligence | Azure | Cloud Platforms401k match | Employee assistance program | Employee stock purchase program | Medical/Dental/Vision | Paid HolidaysSenior-level Full TimeUSA-Pennsylvania-Breinigsville-9999 Hamilton Blvd, United States23h ago
-
Senior Systems Engineer, Storage - DGX Cloud USD 208K-414KAlerting | Algorithms | Ansible | Argo CD | CI/CDSenior-level Full TimeUS, CA, Remote, United States R23h ago
-
Director, Precision Medicine Data Science & AI USD 194K-361KAWS | Apache Spark | Azure | CPT | Claims data401-k match | Comprehensive benefits package | Hybrid work | Paid time off | Travel 10 percentExecutive-level Full TimeEast Hanover, United States23h ago
-
Senior Data Engineer USD 105K-149KAWS | AWS Lambda | Agile | Amazon Redshift | CI/CD401k matching | Dental insurance | Family and Medical Leave FMLA | Life insurance | Medical insuranceSenior-level Full TimeKS Overland Park, United States23h ago
-
Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Data Factory401k match | Disability insurance | Education benefit | Employee stock purchase plan | Life insuranceSenior-level Full TimePrudential Tower, 655 Broad Street, Newark, … R23h ago
-
Data Scientist I - Card Fraud USD 73K-110KData Cleansing | Data Preparation | Data Visualization | Econometrics | Hadoop401k | Health insurance | Paid Holidays | Paid time off | Sick leaveNone Full TimeAtlanta GA - 303 Peachtree Center …23h ago
-
Sr. Electricity Market Optimization Software Engineer USD 127K-191KAgile | CI | Development Lifecycle | Economic Dispatch | Extreme programmingDisability benefits | Employee assistance program | Health Coach | Life insurance | Medical Dental Vision Prescription CoverageSenior-level Full TimeBellevue WA US, United States23h ago
-
Quantitative Developer USD 140K-155KAPIs | Agentic AI | Batch Processing | CI/CD | ContainerizationSenior-level Full TimeBoston, United States23h ago
-
CatBoost | Cross-validation | Feature Selection | Hyperparameter Tuning | LightGBMComprehensive benefitsMid-level Full TimeNew York, New York, United States …23h ago
-
Apache Arrow | Backtesting | Cost modeling | Cross-validation | Data QualityMid-level Full TimeNew York, New York, United States …23h ago
-
Lead Applied AI Engineer USD 305KArtificial Intelligence | Data integration | Foundation Models | Guardrails | Language Processing401k plan | Disability benefits | Health benefits | Life insurance | Paid time offSenior-level Full Time111432-TX-Las Colinas Bldg A, Irving Campus, …23h ago
-
Principal AI Engineer USD 305KAWS | Agentic AI | Agile | Artificial Intelligence | Cloud Architecture401-k plan | Commuter benefits | Disability benefits | Discounts and savings | Hybrid work scheduleSenior-level Full Time102557-CA-333 Market, San Francisco, United States23h ago
-
Principal MarTech SWE / AI Engineer USD 145K-235KAPIs | AWS | Agile Development | Asynchronous processing | AzureSenior-level Full TimeSanta Clara, CA23h ago
-
API Keys | Arize | Async Programming | Docker | Event Driven401k plan | Commuter benefits | Disability benefits | Hybrid work schedule | Life insuranceSenior-level Full Time112265-NJ-MetroPark, Iselin, United States23h ago
-
Market Data Platform Engineer USD 168KAutomation | Bash | Failover | Grafana | High Availability401k plan | Commuter benefits | Disability accommodations | Disability benefits | Health benefitsMid-level Full Time142019-NC-300 South Brevard, Charlotte, United States23h ago
-
Director, Precision Medicine Data Science & AI USD 194K-361KAI Governance | AWS | Azure | CDS frameworks | CPTExecutive-level Full TimeEast Hanover, United States23h ago
-
Machine Learning Engineer, AI Solutions Hub (AISH) USD 76K-107KA/B | A/B Testing | AWS | Agile | AlertingCommuting and transportation | Dental insurance | Medical insurance | Paid time off | Retirement benefitsMid-level Full TimePortland, United States23h ago