IN_Senior Associate_LLM Observability Engineer_Application Technology_Advisory_Bangalore
Bengaluru Millenia, India
INR 2500K-4144K (estimate) Senior-level Full Time
Tasks
- Analyze metrics dashboards
- Capture prompts responses token usage latency and errors
- Collaborate with ML engineers and QA teams
- Design performance test plans
- Detect hallucinations
- Detect toxicity
- Develop test scripts
- Ensure data protection compliance
- Identify performance bottlenecks
- Instrument LLM applications for telemetry
- Monitor performance KPIs
- Monitor security risks prompt injection and sensitive data leakage
- Optimize token usage for cost
- Provide tuning and architecture recommendations
- Run automated evaluation and quality checks
- Score relevance
- Troubleshoot production issues with tracing and logging
Perks/Benefits
- N/A
Skills/Tech-stack
Arize AI | Cost Optimization | Datadog | Gatling | JMeter | Java | Langchain | Langfuse | Load Testing | LoadRunner | Logging | Observability | OpenTelemetry | Performance Testing | Prompt engineering | Python | Token Usage Monitoring | Token usage | Tracing | Usage monitoring
Related jobs
-
Senior Software Engineer INR 2000K-2700KAgile | Cloud services | Code review | Data Modeling | Data WarehousingSenior-level Full TimePune, India2h ago
-
Manager Data and Analytics INR 1400K-2000KAmazon Redshift | Azure | BigQuery | Business Intelligence | Cloud platformMid-level Full TimeBangalore, Karnataka, India R4h ago
-
Senior Artificial Intelligence/Machine Learning Engineer INR 2500K-5000KAPI Integration | AWS SageMaker | Agent systems | Azure ML | CCPACompany paid medical insurance | Company-paid certifications | Financial and legal consultations | Internal workshops and meetups | Language coursesSenior-level Full TimeIndia5h ago
-
Senior AI Engineer INR 3242K-4680KAWS EC2 | AWS ECS | AWS Lambda | AWS RDS | Amazon S3Financial consultation | Legal consultation | Medical insurance | Mental health support | Paid certificationsSenior-level Full TimeIndia5h ago
-
Associate Engineer - AI/ML with Python INR 2500K-4000KAWS | Bitbucket | CI/CD | CSS | CeleryCollaborative work environment | Onsite work 5 days a week | Training and developmentMid-level Full TimeBangalore - Karnataka, India - Kalyani …6h ago
-
Data Scientist – AI & Cybersecurity INR 2500K-4500KAPIs | AWS | Adversarial Testing | Amazon SageMaker | Anomaly DetectionGrowth opportunities | Mentorship programs | Training opportunitiesSenior-level Full TimeBangalore - Karnataka - India6h ago
-
Senior Java Kafka Architect INR 2000K-2700KAmazon Web Services | Apache Kafka | CI/CD | Code Refactoring | Code reviewSenior-level Full TimeIN-AP-Hyderabad6h ago
-
Data Engineer INR 2000K-3000KAPI Design | Asynchronous programming | Caching | Celery | Clean architectureSenior-level Full TimeIN-KA-Bangalore6h ago
-
Azure Data Engineer INR 1500K-2500KApache Spark | Azure Data | Azure Data Lake | Azure Databricks | Data GovernanceMid-level Full TimeIN-KA-Bangalore6h ago
-
Software Engineer III, ML Infrastructure, Google Cloud INR 2500K-5000KBack-end | Back-end development | C++ | Cloud APIs | Data StorageSenior-level Full TimePune, Maharashtra, India8h ago
-
Campus FTE INR 1200K-2500KAgentic AI | Amazon Web Services | Classification | Cloud platform | Data AnalysisMid-level Full TimeBangalore, Karnataka, India8h ago
-
Campus FTE INR 1800K-3000KAgentic AI | Amazon Web Services | Classification | Cloud platform | Data QualityMid-level Full TimeGurgaon, Haryana, India8h ago
-
Mid-level Full TimeIndia, Telangana, Hyderabad; India, Karnataka, Bangalore; …8h ago
-
Algorithms | Amazon S3 | Apache Iceberg | Concurrency | Data ModelingInnovation culture | Mentorship | Remote work flexibilitySenior-level Full TimeHyderabad, India10h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India10h ago
-
Senior-level Full TimeBengaluru, KA, IN10h ago
-
Sr. Data Engineer INR 3000K-4000KApache Kafka | Apache Spark | Data Modeling | Data Pipelines | Data integrationSenior-level Full TimeBangalore, Karnataka, IN10h ago
-
Senior-level Full TimeHyderabad, TS, India11h ago
-
Software Engineer III - Java, Springboot, Cloud INR 2200K-3500KAPI Gateway | Agile | Artificial Intelligence | CI/CD | CassandraSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
GCP, Bigquery, Airflow/Consultant Specialist INR 1500K-2500KAgile | Apache Airflow | Apache Hadoop | Apache Kafka | Apache NiFiContinuous professional development | Flexible working | Inclusive work environment | Opportunity for growthSenior-level Full TimeHyderabad, Telangana, India R12h ago
-
Assistant Director,Data and Analytics INR 3000K-5199KAnomaly Detection | Azure | Cloud platform | Data Visualization | Fraud analyticsExecutive-level Full TimeBangalore, Karnataka, India R12h ago
-
Software Engineer III - ETL Testing ,Pyspark, AWS INR 2000K-3100KAWS CloudFormation | AWS Glue | AWS Glue Data Catalog | AWS Lambda | AWS MSKAgile team environment | Automated testing validation | Peer reviewSenior-level Full TimeMumbai, Maharashtra, India12h ago
-
AI Engineer - FDE (Forward Deployed Engineer) INR 2400K-3900KAWS | Agent systems | Apache Spark | Azure | Deep learningConference participation | Remote work | Travel as neededSenior-level Full TimeRemote - India R13h ago
-
Senior Software Engineer – Java, Cloud & APIs INR 2500K-3500KAPI Integration | Best practices | CI/CD | Cloud Native | Cloud platformSenior-level Full TimeChennai, Tamil Nadu, India13h ago
-
Mid-level Full TimeGurugram, HR, India13h ago