Generative AI Tester
India - Remote
Applications have closed
Zealogics
Digital transformation and Engineering services company working with Fortune 500 customers across Industries for a Sustainable Tomorrow.
- 7+years of Hands-on experience in testing Generative AI models including text, image, and other content generation outputs.
- Expertise in creating and executing test strategies, test plans and cases specific to generative AI models, including language models, image generators, and other AI applications that address the unique challenges of Generative AI systems.
- Strong understanding of AI/ML concepts, including model training, validation, deployment, and continuous monitoring.
- Proficiency in testing large language models (LLMs) such as GPT, BERT, and similar, focusing on output accuracy, context retention, and consistency.
- Expert-level knowledge in natural language processing (NLP) techniques and the ability to test NLP-driven applications.
- Experience with AI/ML testing frameworks and tools such as TensorFlow, PyTorch, Hugging Face or custom AI testing frameworks.
- Strong familiarity with data validation and testing, ensuring that datasets used for training and testing are accurate, relevant, and unbiased.
- Test for potential biases within AI models by analysing model output across different demographics and data segments.
-Proficient in defining KPIs and metrics for generative AI model testing.
-Conduct model evaluation using relevant performance metrics, such as BLEU, ROUGE, and perplexity for language models.
-Validate model output for accuracy, coherence, and relevance, ensuring that the models align with business and user expectations.
- Expertise in query optimization and data processing to validate AI model performance and output efficiency.
- Perform functional, load, and stress tests on models to validate their accuracy, scalability, and responsiveness under varying conditions.
- Understanding of cloud-based AI/ML deployment and experience in testing AI models deployed on cloud platforms like AWS, Azure, or GCP.
- Proficiency in API testing for AI/ML applications, ensuring seamless integration and accurate data flow between components.
- Experience in using test automation tools for AI/ML testing, including custom automation scripts tailored to AI models.
- Proficiency in programming languages like Python or Java for developing and executing automated test scripts for AI models.
- Knowledge of Continuous Integration/Continuous Deployment (CI/CD) pipelines in the context of AI/ML model deployment and testing.
-Perform continuous testing for model performance and drift post-deployment, identifying areas where model retraining may be required.
- Strong stakeholder management and communication skills to effectively collaborate with cross-functional teams, including data scientists, developers, ML Engineers, Model Ops team and product managers to provide feedback on model performance and potential improvements.
-Familiarity with Model Ops tools and practices for production-level AI testing.
- Strong analytical, problem-solving, and reporting skills with the ability to identify and resolve issues related to AI model performance and integration.
-Excellent communication skills, able to present test results and model evaluations to technical and non-technical stakeholders.
- Exposure and experience in test management tools like JIRA, TestRail, or similar for end-to-end test management.
- Expertise in creating and executing test strategies, test plans and cases specific to generative AI models, including language models, image generators, and other AI applications that address the unique challenges of Generative AI systems.
- Strong understanding of AI/ML concepts, including model training, validation, deployment, and continuous monitoring.
- Proficiency in testing large language models (LLMs) such as GPT, BERT, and similar, focusing on output accuracy, context retention, and consistency.
- Expert-level knowledge in natural language processing (NLP) techniques and the ability to test NLP-driven applications.
- Experience with AI/ML testing frameworks and tools such as TensorFlow, PyTorch, Hugging Face or custom AI testing frameworks.
- Strong familiarity with data validation and testing, ensuring that datasets used for training and testing are accurate, relevant, and unbiased.
- Test for potential biases within AI models by analysing model output across different demographics and data segments.
-Proficient in defining KPIs and metrics for generative AI model testing.
-Conduct model evaluation using relevant performance metrics, such as BLEU, ROUGE, and perplexity for language models.
-Validate model output for accuracy, coherence, and relevance, ensuring that the models align with business and user expectations.
- Expertise in query optimization and data processing to validate AI model performance and output efficiency.
- Perform functional, load, and stress tests on models to validate their accuracy, scalability, and responsiveness under varying conditions.
- Understanding of cloud-based AI/ML deployment and experience in testing AI models deployed on cloud platforms like AWS, Azure, or GCP.
- Proficiency in API testing for AI/ML applications, ensuring seamless integration and accurate data flow between components.
- Experience in using test automation tools for AI/ML testing, including custom automation scripts tailored to AI models.
- Proficiency in programming languages like Python or Java for developing and executing automated test scripts for AI models.
- Knowledge of Continuous Integration/Continuous Deployment (CI/CD) pipelines in the context of AI/ML model deployment and testing.
-Perform continuous testing for model performance and drift post-deployment, identifying areas where model retraining may be required.
- Strong stakeholder management and communication skills to effectively collaborate with cross-functional teams, including data scientists, developers, ML Engineers, Model Ops team and product managers to provide feedback on model performance and potential improvements.
-Familiarity with Model Ops tools and practices for production-level AI testing.
- Strong analytical, problem-solving, and reporting skills with the ability to identify and resolve issues related to AI model performance and integration.
-Excellent communication skills, able to present test results and model evaluations to technical and non-technical stakeholders.
- Exposure and experience in test management tools like JIRA, TestRail, or similar for end-to-end test management.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
5
0
0
Categories:
Deep Learning Jobs
Generative AI Jobs
Tags: APIs AWS Azure BERT CI/CD GCP Generative AI GPT Java Jira KPIs LLMs Machine Learning Model deployment Model training NLP Pipelines Python PyTorch TensorFlow Testing
Regions:
Remote/Anywhere
Asia/Pacific
Country:
India
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Senior AI Engineer jobsSr. Data Engineer jobsData Engineer II jobsBI Developer jobsPrincipal Data Engineer jobsStaff Data Scientist jobsStaff Machine Learning Engineer jobsData Science Manager jobsData Manager jobsData Science Intern jobsPrincipal Software Engineer jobsBusiness Data Analyst jobsJunior Data Analyst jobsData Specialist jobsResearch Scientist jobsData Analyst Intern jobsSoftware Engineer II jobsLead Data Analyst jobsSr. Data Scientist jobsDevOps Engineer jobsData Engineer III jobsAI/ML Engineer jobsStaff Software Engineer jobsJunior Data Engineer jobsData Engineering Manager jobs
Git jobsLinux jobsAirflow jobsEconomics jobsOpen Source jobsKafka jobsPhysics jobsComputer Vision jobsGoogle Cloud jobsJavaScript jobsMLOps jobsNoSQL jobsData Warehousing jobsKPIs jobsHadoop jobsRDBMS jobsPostgreSQL jobsTerraform jobsScala jobsScikit-learn jobsBanking jobsGitHub jobsPandas jobsData warehouse jobsStreaming jobs
R&D jobsClassification jobsOracle jobsDistributed Systems jobsBigQuery jobsPySpark jobsCX jobsScrum jobsdbt jobsLooker jobsRAG jobsReact jobsRobotics jobsJira jobsMicroservices jobsRedshift jobsData Mining jobsSAS jobsIndustrial jobsMySQL jobsJenkins jobsNumPy jobsGPT jobsE-commerce jobsUnstructured data jobs