Program Analyst Sr – Generative AI, Prompt Engineering & Validation
Tasks
- Assess search relevance context quality source completeness
- Coordinate cross functional testing timelines
- Design prompt test cases and test prompts
- Evaluate prompt variants and document impact
- Evaluate response quality factual correctness relevance completeness clarity format
- Execute functional sanity regression negative exploratory testing
- Log defects with reproduction steps and expected output
- Maintain dashboards trackers action items risk registers
- Maintain prompt libraries test suites validation evidence
- Perform root cause analysis for prompt data retrieval model defects
- Prepare weekly validation summaries executive reports
- Support batch prompt execution comparison reporting
- Track GenAI validation plans
- Validate end to end RAG workflow outputs
Perks/Benefits
- N/A
Skills/Tech-stack
Case design | Cause analysis | Confluence | Data Validation | Data freshness | Excel | Exploratory Testing | Generative AI | Hallucination detection | Information Retrieval | Jira | Keyword search | LLM | Power BI | Prompt engineering | Prompt testing | Python | Quality evaluation | RAG | Regression testing | Root Cause Analysis | Root cause | SQL | Scripting | Semantic Search | SharePoint | Test Case | Test Case Design | Validation
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology | High School Diploma
Roles
Related jobs
-
Senior Data Analyst INR 1971K-2800KDashboard creation | Data Analysis | Data Modeling | Data Quality | Data VisualizationSenior-level Full TimeBangalore - Karnataka - India8h ago
-
Automation | Data Mining | Data Quality | Data Wrangling | JavaMid-level Full TimeBengaluru, Karnataka, India12h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India13h ago
-
Associate Manager Analytics (Hyd) INR 1200K-2000KAmazon Web Services | Jupyter | Linux | Machine Learning | Microsoft SQLMid-level Full TimeHyderabad, Andhra Pradesh, India15h ago
-
Senior-level Full TimeMumbai, Maharashtra, India16h ago
-
Data Analytics Senior Specialist INR 2771K-5000KAlteryx | Anomaly Detection | Azure Data | Azure Data Lake | Azure SynapseSenior-level Full TimeIndia16h ago
-
Global Process Optimization Analyst INR 1500K-3550KArtificial Intelligence | Business Process | Business Process Management | Data Analysis | Digital TransformationMid-level Full TimeChennai, Tamil Nadu, India18h ago
-
Specialist - Spend Analytics INR 1716K-2700KAPriori | Anomaly Detection | Cost engineering | Data Automation | Data VisualizationEmployee assistance programme | Health insurance | Professional developmentSenior-level Full TimeCapital Cyberscape, 2nd Floor, Ullahwas, Sector …21h ago
-
Lead Analyst, Channel Analytics INR 2520K-3880KAd Serving | Ad tracking | Analytics | Artificial Intelligence | Channel attributionSenior-level Full TimeBangalore, India R21h ago
-
Lead Business Analyst – Analytics Platform INR 3000K-4000KAnalytics Cloud | Backlog Management | Data analytics | Enterprise Resource Planning | Impact AssessmentSenior-level Full TimeIndia - Chennai21h ago
-
Analyst - DACI Analytics INR 2500K-4720KBusiness Intelligence | Data Visualization | Predictive Modeling | Python | SQLCollaborative workspaces | Cutting-edge technology | On-site clinic | On-site gymSenior-level Full TimeBengaluru, India21h ago
-
Data Analyst (Business Intelligence, Python, Machine Learning, Power BI) - Bangalore INR 2800K-4064KClassification | Clustering | DAX | Ensemble Methods | Machine LearningCareer growth | Collaborative environment | High-impact projects | Innovation | Learning opportunitiesSenior-level Full TimeIND BNGL FL2-3 TWR 3, India21h ago
-
Senior-level Full TimeAPAC Home Office Hyderabad IN, India21h ago
-
Consultant, Financial Risk Analytics (CCAR, CECL, Python) INR 3000K-4000KCCAR | CECL | Python | SAS | SAS ViyaSenior-level Full TimeBangalore, India21h ago
-
Performance & Analytics Associate INR 500K-700KData Integrity | Data Validation | Equity return calculation | Fixed Income | Foreign ExchangeEntry-level Full Time11073 Ameriprise India - Noida - …21h ago
-
Data Analyst INR 1000K-2205KAnalytical interpretation | Automation tools | Capital Markets | Corporate Finance | Data ValidationContinuing education program | Continuous learning | Family-friendly perks | Flexible time off | Health care coverageNone Full TimeIN - AHMEDABAD, India21h ago
-
Data Analyst / ETL Dev INR 1500K-2300KAlteryx | Data Cleansing | Data Transformation | Data Validation | ETLMid-level Full TimeShaikpet, India1d ago
-
Fraud Strategy and Analytics Senior Associate INR 2817K-5000KArtificial Intelligence | Excel | GitHub | Looker | Machine LearningSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Analyst, Business Intelligence INR 3242K-5000KAgile | Anomaly Detection | Azure OpenAI | CI/CD | Data ValidationEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Data Analysis | Data Visualization | Looker | NumPy | PandasMid-level Full TimeGurugram, HR, India1d ago
-
Data Analyst - Security INR 600K-800KCapital Markets | Cash Flow | Cause analysis | Company analysis | Comparable company analysisEntry-level Full TimeAhmedabad, Gujarat, India1d ago
-
Business Systems Analyst (Clarity PPM, Jaspersoft reports ,Tableau and PowerBI based dashboards) with 3 to 5 years of experience INR 1000K-1700KClarity PPM | Database Design | Gel Script | Jaspersoft | OracleMid-level Full TimeBengaluru, KA, India1d ago
-
Analyst-Data Science INR 1960K-3370KClustering | Data Analysis | Digital analytics | Experimentation | GenAIEntry-level Full TimeGurugram, HR, India1d ago
-
Business Intelligence (Data Analytics & AI) - Associate INR 2000K-4064KAPIs | Alteryx | Data Governance | Data Modeling | Data QualitySenior-level Full TimeMumbai, Maharashtra, India1d ago
-
Data Visualization & Advanced Analytics - Associate INR 1000K-2000KAgile | Alteryx | Data Cleansing | Data Governance | Data QualityMid-level Full TimeMumbai, Maharashtra, India1d ago