Generative AI Tester
India - Remote
Applications have closed
Zealogics
Digital transformation and Engineering services company working with Fortune 500 customers across Industries for a Sustainable Tomorrow.
- 7+years of Hands-on experience in testing Generative AI models including text, image, and other content generation outputs.
- Expertise in creating and executing test strategies, test plans and cases specific to generative AI models, including language models, image generators, and other AI applications that address the unique challenges of Generative AI systems.
- Strong understanding of AI/ML concepts, including model training, validation, deployment, and continuous monitoring.
- Proficiency in testing large language models (LLMs) such as GPT, BERT, and similar, focusing on output accuracy, context retention, and consistency.
- Expert-level knowledge in natural language processing (NLP) techniques and the ability to test NLP-driven applications.
- Experience with AI/ML testing frameworks and tools such as TensorFlow, PyTorch, Hugging Face or custom AI testing frameworks.
- Strong familiarity with data validation and testing, ensuring that datasets used for training and testing are accurate, relevant, and unbiased.
- Test for potential biases within AI models by analysing model output across different demographics and data segments.
-Proficient in defining KPIs and metrics for generative AI model testing.
-Conduct model evaluation using relevant performance metrics, such as BLEU, ROUGE, and perplexity for language models.
-Validate model output for accuracy, coherence, and relevance, ensuring that the models align with business and user expectations.
- Expertise in query optimization and data processing to validate AI model performance and output efficiency.
- Perform functional, load, and stress tests on models to validate their accuracy, scalability, and responsiveness under varying conditions.
- Understanding of cloud-based AI/ML deployment and experience in testing AI models deployed on cloud platforms like AWS, Azure, or GCP.
- Proficiency in API testing for AI/ML applications, ensuring seamless integration and accurate data flow between components.
- Experience in using test automation tools for AI/ML testing, including custom automation scripts tailored to AI models.
- Proficiency in programming languages like Python or Java for developing and executing automated test scripts for AI models.
- Knowledge of Continuous Integration/Continuous Deployment (CI/CD) pipelines in the context of AI/ML model deployment and testing.
-Perform continuous testing for model performance and drift post-deployment, identifying areas where model retraining may be required.
- Strong stakeholder management and communication skills to effectively collaborate with cross-functional teams, including data scientists, developers, ML Engineers, Model Ops team and product managers to provide feedback on model performance and potential improvements.
-Familiarity with Model Ops tools and practices for production-level AI testing.
- Strong analytical, problem-solving, and reporting skills with the ability to identify and resolve issues related to AI model performance and integration.
-Excellent communication skills, able to present test results and model evaluations to technical and non-technical stakeholders.
- Exposure and experience in test management tools like JIRA, TestRail, or similar for end-to-end test management.
- Expertise in creating and executing test strategies, test plans and cases specific to generative AI models, including language models, image generators, and other AI applications that address the unique challenges of Generative AI systems.
- Strong understanding of AI/ML concepts, including model training, validation, deployment, and continuous monitoring.
- Proficiency in testing large language models (LLMs) such as GPT, BERT, and similar, focusing on output accuracy, context retention, and consistency.
- Expert-level knowledge in natural language processing (NLP) techniques and the ability to test NLP-driven applications.
- Experience with AI/ML testing frameworks and tools such as TensorFlow, PyTorch, Hugging Face or custom AI testing frameworks.
- Strong familiarity with data validation and testing, ensuring that datasets used for training and testing are accurate, relevant, and unbiased.
- Test for potential biases within AI models by analysing model output across different demographics and data segments.
-Proficient in defining KPIs and metrics for generative AI model testing.
-Conduct model evaluation using relevant performance metrics, such as BLEU, ROUGE, and perplexity for language models.
-Validate model output for accuracy, coherence, and relevance, ensuring that the models align with business and user expectations.
- Expertise in query optimization and data processing to validate AI model performance and output efficiency.
- Perform functional, load, and stress tests on models to validate their accuracy, scalability, and responsiveness under varying conditions.
- Understanding of cloud-based AI/ML deployment and experience in testing AI models deployed on cloud platforms like AWS, Azure, or GCP.
- Proficiency in API testing for AI/ML applications, ensuring seamless integration and accurate data flow between components.
- Experience in using test automation tools for AI/ML testing, including custom automation scripts tailored to AI models.
- Proficiency in programming languages like Python or Java for developing and executing automated test scripts for AI models.
- Knowledge of Continuous Integration/Continuous Deployment (CI/CD) pipelines in the context of AI/ML model deployment and testing.
-Perform continuous testing for model performance and drift post-deployment, identifying areas where model retraining may be required.
- Strong stakeholder management and communication skills to effectively collaborate with cross-functional teams, including data scientists, developers, ML Engineers, Model Ops team and product managers to provide feedback on model performance and potential improvements.
-Familiarity with Model Ops tools and practices for production-level AI testing.
- Strong analytical, problem-solving, and reporting skills with the ability to identify and resolve issues related to AI model performance and integration.
-Excellent communication skills, able to present test results and model evaluations to technical and non-technical stakeholders.
- Exposure and experience in test management tools like JIRA, TestRail, or similar for end-to-end test management.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
5
0
0
Categories:
Deep Learning Jobs
Generative AI Jobs
Tags: APIs AWS Azure BERT CI/CD GCP Generative AI GPT Java Jira KPIs LLMs Machine Learning Model deployment Model training NLP Pipelines Python PyTorch TensorFlow Testing
Regions:
Remote/Anywhere
Asia/Pacific
Country:
India
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Principal Data Engineer jobsPower BI Developer jobsSr. Data Engineer jobsBI Developer jobsStaff Data Scientist jobsStaff Machine Learning Engineer jobsPrincipal Software Engineer jobsData Science Intern jobsDevOps Engineer jobsJunior Data Analyst jobsData Science Manager jobsSoftware Engineer II jobsData Manager jobsData Analyst Intern jobsLead Data Analyst jobsStaff Software Engineer jobsBusiness Data Analyst jobsAccount Executive jobsData Specialist jobsAI/ML Engineer jobsSr. Data Scientist jobsData Governance Analyst jobsBusiness Intelligence Analyst jobsSenior Backend Engineer jobsData Engineer III jobs
Consulting jobsMLOps jobsAirflow jobsOpen Source jobsLinux jobsEconomics jobsKPIs jobsJavaScript jobsKafka jobsTerraform jobsGitHub jobsData Warehousing jobsPostgreSQL jobsRDBMS jobsScikit-learn jobsNoSQL jobsComputer Vision jobsStreaming jobsClassification jobsBanking jobsPhysics jobsGoogle Cloud jobsPrompt engineering jobsRAG jobsOracle jobs
Pandas jobsHadoop jobsR&D jobsBigQuery jobsData warehouse jobsScala jobsdbt jobsLooker jobsGPT jobsReact jobsScrum jobsPySpark jobsDistributed Systems jobsLangChain jobsCX jobsELT jobsIndustrial jobsMicroservices jobsSAS jobsJira jobsRedshift jobsOpenAI jobsJenkins jobsModel training jobsRobotics jobs