IN_Senior Associate_LLM Observability Engineer_Application Technology_Advisory_Bangalore
Bengaluru Millenia, India
INR 2500K-4144K (estimate) Senior-level Full Time
Tasks
- Analyze metrics dashboards
- Capture prompts responses token usage latency and errors
- Collaborate with ML engineers and QA teams
- Design performance test plans
- Detect hallucinations
- Detect toxicity
- Develop test scripts
- Ensure data protection compliance
- Identify performance bottlenecks
- Instrument LLM applications for telemetry
- Monitor performance KPIs
- Monitor security risks prompt injection and sensitive data leakage
- Optimize token usage for cost
- Provide tuning and architecture recommendations
- Run automated evaluation and quality checks
- Score relevance
- Troubleshoot production issues with tracing and logging
Perks/Benefits
- N/A
Skills/Tech-stack
Arize AI | Cost Optimization | Datadog | Gatling | JMeter | Java | Langchain | Langfuse | Load Testing | LoadRunner | Logging | Observability | OpenTelemetry | Performance Testing | Prompt engineering | Python | Token Usage Monitoring | Token usage | Tracing | Usage monitoring
Related jobs
-
Pyspark Bigdata INR 1500K-2500KArchitecture | Big Data | Data Engineering | Data pipeline | Development Life CycleMid-level Full TimeIN-MH-Pune11h ago
-
Sr R&D Engineer (C++, Algorithm, Data Structure) INR 2475K-3500KAlgorithms | C# | C++ | Data Structures | Hardware verificationSenior-level Full TimeBengaluru, India11h ago
-
Cognitive Engineer INR 1500K-3000KAPI Integration | Agent architecture | Application Default Credentials | Asynchronous programming | Automated testingMid-level Full TimeIndia11h ago
-
Software Engineering - Data, Lakehouse and AI Data Platform Engineer-Hyderabad-Vice President INR 3000K-6000KApache Iceberg | Apache Spark | Avro | CI/CD | ContainerizationExecutive-level Full TimeBengaluru, Karnataka, India11h ago
-
Embedded Software Engineer - Delivery INR 2000K-3500KAutomated testing | C# | Code Quality | Coding Standards | Configuration ManagementMid-level Full TimePune, India12h ago
-
Applied AI/ML Senior Associate INR 3799K-5000KApache Hive | Apache Spark | Keras | Language Processing | Machine LearningSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
AI Agents Applied Research/Engineering Lead - Executive Director INR 2500K-5000KA/B | A/B Testing | Adversarial Testing | Agent Orchestration | AuditabilityDiversity and inclusion | Publication opportunities | Research deployment to productionSenior-level Full TimeBengaluru, Karnataka, India13h ago
-
AWS | AWS Lake Formation | AWS Lambda | Airflow | Amazon EMRCareer development opportunities | Employee Assistance Program (EAP) | Financial wellness tools | Flexible working environment | LinkedIn LearningSenior-level Full TimePune, MH, India13h ago
-
Lead Software Engineer - Java, AWS, React INR 2500K-5000KAWS | Agile | Analysis and Design | Asynchronous Communication | Automated testingSenior-level Full TimeHyderabad, Telangana, India14h ago
-
Senior AI Engineer INR 3264K-5000KAI Services | Access Management | Anomaly Detection | Azure | Azure AISenior-level Full TimeIndia14h ago
-
Cognitive Engineer INR 1500K-3000KAPI Integration | Agent architecture | Application Default Credentials | Asynchronous programming | Automated testingMid-level Full TimeIndia14h ago
-
Consultant - Python Developer with Gen AI using GCP INR 2000K-5000KAPI Development | Cloud platform | Gemini Enterprise | Google ADK | Google CloudClient site work 5 days per weekSenior-level Full TimeBangalore, Karnataka, India14h ago
-
Automated testing | Business Intelligence | CI/CD | Dashboards | Data MartMid-level Full TimeHyderabad, Telangana, India14h ago
-
DataOps Engineer INR 1600K-4000KAWS | AWS Direct Connect | AWS PrivateLink | AWS Transit Gateway | AWS VPNSenior-level Full TimeMumbai, India15h ago
-
DataOps Engineer INR 1600K-4000KAWS | AWS Direct Connect | AWS PrivateLink | AWS Transit Gateway | AWS VPNSenior-level Full TimeMumbai, India15h ago
-
Principal Software Engineer- Eng - JAVA + AI Architect INR 3000K-4200KAI-assisted software engineering | APIs | Agentic Automation | Alerting | Audit LoggingComprehensive benefits | Flexible workSenior-level Full TimeNoida,UP,India16h ago
-
Assistant Manager - RNA INR 1500K-2525KAWS | AgentFlow | Asynchronous programming | Autogen | AzureMid-level Full TimeBangalore, KA, India16h ago
-
Senior-level Full TimeIndia16h ago
-
Senior Engineer - AI/ML INR 3500K-5000KArchitecture | Artificial Intelligence | Automation | Code review | DigitalizationSenior-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101316h ago
-
Senior-level Full TimeBangalore, KA, IN,17h ago
-
Mid-level Full TimeTelangana, India17h ago
-
QA Engineer (C1) - Agentic AI INR 1000K-2500KAI Observability | AI Safety | API Testing | AWS | Agent OrchestrationMid-level Full TimeIndia18h ago
-
Associate Consultant - Data Engineer(Python/Pyspark) INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure SynapseMid-level Full TimeBangalore, Karnataka, India18h ago
-
Mid-level Full TimeBengaluru, KA, India19h ago
-
Senior Data Engineer - Remote INR 2520K-3380KAWS Athena | AWS CloudWatch | AWS Glue | AWS IAM | Amazon SQSFlexible schedule | Night shift flexibility | Remote workSenior-level Full TimeChennai, TN, IN R21h ago