Program Analyst Sr – Generative AI, Prompt Engineering & Validation
Tasks
- Design and execute functional sanity regression negative and exploratory tests
- Design and maintain prompt libraries and test prompts
- Develop prompt test scenarios and edge case prompts
- Document prompt libraries test suites and validation evidence
- Evaluate GenAI responses for factual correctness relevance completeness and format quality
- Log defects and perform first level root cause analysis
- Maintain validation dashboards trackers and risk registers
- Prepare weekly validation summaries risk updates and executive status reports
- Support automation for batch prompt execution response capture and reporting
- Track GenAI feature validation plans and deliverables
- Validate data search and RAG workflows using golden datasets
- Validate end to end behavior from prompt to retrieved context to generated output
Perks/Benefits
- N/A
Skills/Tech-stack
Case design | Cause analysis | Confluence | Dashboards | Data Quality | Excel | Exploratory Testing | Generative AI | Information Retrieval | JSON | Jira | Keyword search | LLM Evaluation | Negative Testing | Power BI | Prompt engineering | Python | RAG | Regression testing | Root Cause Analysis | Root cause | SQL | Semantic Search | SharePoint | Test Case | Test Case Design | XML
Roles
Analyst | Data Quality | Data Quality Analyst | Program Analyst | Quality Analyst
Related jobs
-
Mid-level Full TimeGujarat, India11h ago
-
Senior Analyst-Data Analytics (SQL, Python, GenAI) INR 2771K-4064KAudience Segmentation | Clickstream analytics | Experimentation | Funnel Optimization | GenAISenior-level Full TimeGurugram, HR, India11h ago
-
Anomaly Detection | Cluster Analysis | Data Modeling | Data Quality | Data VisualizationSenior-level Full TimeBangalore, INDIA R12h ago
-
Reporting Analyst INR 1860K-3000KArtificial Intelligence | Bias Assessment | Confidence scoring | Content Safety | Data IntegrityMid-level Full TimeKarnataka, India12h ago
-
Account Manager - Business Analytics INR 1000K-1500KBusiness Requirements | Client Management | Data Analysis | Data Modeling | Machine LearningMid-level Full TimeGurugram, Haryana, India12h ago
-
Data Enrichment | Data Governance | Data Management | Data Quality | Data StewardshipFlexible paid time off | Learning and development | Medical benefitsEntry-level Full TimePune, MH, India13h ago
-
Data Analyst ( Data Science ) INR 1000K-2000KCommon table expressions | Data Analysis | Data Quality | Data Transformation | Data ValidationAnnual performance cycle | Awards and bonus | Flexible benefit plan | Flexible leave policy | Hybrid remote workMid-level Full TimeChennai, Tamil Nadu, India R13h ago
-
TA Analytics Specialist INR 300K-600KATS administration | Applicant Tracking System | Applicant tracking | Dashboards | Data VisualizationMid-level Full TimeRemote, India R15h ago
-
Data Analyst INR 800K-2000KAgile | Apache Airflow | Apache Spark | Business Intelligence | ClassificationMid-level Full TimeBangalore,India19h ago
-
Lead Data Analyst INR 2520K-4500KA/B | A/B Testing | Airflow | Apache Spark | Artificial IntelligenceSenior-level Full TimeBangalore,India19h ago
-
Senior Analyst-Data Quality Solution INR 3000K-4000KAWS | Amazon Redshift | Apache Spark | CTE | CollibraSenior-level Full TimeIndia - Chennai19h ago
-
Senior-level Full TimeHyderabad, Telangana, India19h ago
-
Data Analyst INR 1000K-2000KAgile methodology | Business Intelligence | Dashboard | Data Analysis | Data ManagementMid-level Full TimeECP01 - E City Park, (ECP01), …19h ago
-
Senior Analyst-Data Quality Solution INR 3000K-4000KAWS | Amazon Redshift | Apache Spark | CTE | CollibraSenior-level Full TimeIndia - Chennai19h ago
-
Consumer Health Data & Analytics Consultant INR 1500K-2500KBusiness Analytics | Commercial Analytics | Data Analysis | Data Modeling | Data VisualizationMid-level Full TimeBangalore, India19h ago
-
Associate - Operations - Operations - CBS - FIN - FP&A Rptg & Data Analytics Enablement - Gurgaon INR 700K-1100KBusiness Intelligence | Data Modeling | Excel | Financial Analysis | Power AutomateMid-level Full TimeGurugram, HR, IN, 12200319h ago
-
Associate - Operations - Operations - CBS - FIN - FP&A Rptg & Data Analytics Enablement - Gurgaon INR 700K-1100KBusiness Intelligence | Excel | Financial Analysis | Power Automate | Power BICareer development | Training and learning | Wellbeing supportMid-level Full TimeGurugram, HR, IN, 12200319h ago
-
Lead Analyst, Channel Analytics INR 2520K-3380KAI tools | Ad Serving | Ad tracking | Audience targeting | Channel attributionSenior-level Full TimeBangalore, India R19h ago
-
Associate - Brand Analytics INR 1200K-1820KApplied Econometrics | Bayesian Methods | Bayesian analysis | Build of Experiments | D3JSDistributed team collaboration | Flexible working arrangements | In office three days per weekMid-level Full TimeIndia - Bangalore19h ago
-
Data Analyst - Enterprise Relationship Mgmt INR 503K-650KAutomation | Business Intelligence | Dashboards | Data Analysis | Data InterpretationChild day care facility | Educational assistance program | Employee stock investment plan | Life insurance | Medical insuranceMid-level Full TimeIND-HYNB-Hyderabad, India19h ago
-
Collibra | Controls Testing | Data Catalog | Data Governance | Data LineageMid-level Full TimeRMZ Ecoworld Business Park, India19h ago
-
Global Data Quality Assurance Lead INR 2771K-4064KAI | AWS | Compliance | Data Analysis | Data GovernanceSenior-level Full TimeIndia - Chennai19h ago
-
Data Quality Control, Associate, Data Management INR 2771K-3500KAlteryx | Automation RPA | Balance sheet | Balance sheet reporting | Capital reportingSenior-level Full TimeCOMMERZ III, OBEROI GARDEN CITY, India19h ago
-
Senior Quantitative Analyst Market Risk INR 2475K-3000KAdvanced Statistics | Back Testing | Bash | Cash Margining | Cash margining analysisSenior-level Full TimeBangalore RMZ-ECO WORLD, India19h ago
-
Mid-level Full TimePune Cerebrum, India19h ago