IN_Senior Associate_LLM Observability Engineer_App Tech_Advisory_Pan India (early joiners)
Gurugram 10 C, India
INR 2535K-4500K (estimate) Senior-level Full Time
Tasks
- Analyze performance test results and production metrics
- Collaborate with data scientists ML engineers and QA teams
- Collect telemetry for prompts responses tokens latency errors
- Design performance test plans
- Execute performance test scripts
- Identify and mitigate bottlenecks
- Implement automated model evaluations
- Instrument LLM applications for observability
- Monitor KPIs response time throughput cost per query resource utilization
- Monitor for prompt injection risks
- Provide recommendations for model fine tuning and system architecture
- Track token usage and compute resource cost
- Troubleshoot production issues using tracing and logging
Perks/Benefits
Skills/Tech-stack
Arize AI | Automation | Cost Optimization | Datadog | Gatling | Hallucination detection | JMeter | Java | LLM Observability | Langchain | Langfuse | Load Testing | LoadRunner | Logging | OpenTelemetry | Performance Testing | Prompt injection | Python | RAG | Relevance scoring | SRE Observability | Token Usage Tracking | Token usage | Toxicity classification | Tracing | Usage tracking
Related jobs
-
Senior Data Platform Engineer INR 1500K-2400KAWS Glue | AWS IAM | AWS Lake Formation | AWS Lambda | AWS S3Senior-level Full TimeBangalore, India10h ago
-
Data Platform Engineer INR 1400K-2000KAPI Integration | AWS CloudWatch | AWS CodePipeline | AWS Glue | AWS IAMMid-level Full TimeBangalore, India10h ago
-
AI Search | AKS | API Management | App Service | Azure AISenior-level Full TimeIndia-Noida10h ago
-
Senior-level Full TimeMaharashtra, Pune, India11h ago
-
Senior-level Full TimeIndia, West Bengal, Kolkata11h ago
-
Mid-level Full TimeIndia, West Bengal, Kolkata11h ago
-
Data Engineer INR 1500K-2000KAmazon Web Services | Apache Airflow | Apache Kafka | Apache Spark | Azure DataMid-level Full TimeIndia, West Bengal, Kolkata11h ago
-
Senior Software Engineer - Data License Analytics, Pune INR 3000K-4512KAWS | Apache Airflow | Apache Iceberg | Databricks | Distributed SystemsSenior-level Full TimePune11h ago
-
Business Data Scientist INR 1400K-2400KAI Agents | Data Pipelines | Data Transformation | Database querying | Hypothesis TestingMid-level Full TimeHyderabad, Telangana, India13h ago
-
Senior Databricks Engineer INR 1500K-2040KCI/CD | Databricks | Databricks API | Databricks SDK | ELTSenior-level Full TimeMaharashtra, Pune, India13h ago
-
Senior - Gen AI INR 2500K-3500KDeep learning | Docker | Generative AI | Language Processing | MLOpsSenior-level Full TimeBangalore, Karnataka, India14h ago
-
Mid-level Full TimeBangalore, Karnataka, India14h ago
-
Analytics Solution Associate INR 900K-1475KAWS Glue | AWS Lambda | Agile | Data Governance | Data IngestionMid-level Full TimeMumbai, Maharashtra, India14h ago
-
Software Engineer III - Python & Databricks INR 2050K-3500KAWS S3 | Agile | CI/CD | Cloud Computing | DatabricksSenior-level Full TimeHyderabad, Telangana, India14h ago
-
Senior-level Full TimePune, MH, India15h ago
-
Senior-level Full TimeMumbai, Maharashtra, India17h ago
-
Staff Software Engineer (Python) INR 2500K-3800KAPIs | AWS | Authentication | Batch Processing | Data PipelinesCollaborative team environment | Leadership development | Training programsSenior-level Full TimeMumbai, MH, India17h ago
-
Mid-level Full TimeNoida, UP, IN17h ago
-
Data Engineer INR 2400K-3356KAzure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeHyderabad, TG, IN17h ago
-
Staff Engineer (Python/AWS) INR 2500K-4000KAWS | Aurora MySQL | CI/CD | Code review | DebuggingDevelopment opportunities | Reward and recognition | Volunteering opportunities | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India17h ago
-
Generative AI Engineer INR 1800K-2500KAWS | AWS Bedrock | Amazon SageMaker | Anthropic API | AzureCollaborative team environment | Leadership development programs | Opportunity to lead AI projects | Training sessionsMid-level Full TimeMumbai, Maharashtra, India17h ago
-
Senior-level Full TimeMumbai, Maharashtra, India18h ago
-
Senior Lead Software Engineer - Java, PySpark, and AWS INR 1800K-2300KAWS Glue | AWS Glue Catalog | AWS KMS | AWS Lake Formation | AWS LambdaSenior-level Full TimeHyderabad, Telangana, India18h ago
-
API Design | Automated testing | Elasticsearch | GraphQL | JavaEntry-level Full TimeBengaluru, Karnataka, India18h ago
-
A/B | A/B Testing | Airflow | Apache Spark | B testingCareer growth support | Family support benefits | Flexible work arrangements | Learning and development opportunities | Medical/Dental/Vision insuranceSenior-level Full TimeIndia18h ago