Lead Data Engineer with AI experience
Tasks
- Build agentic data systems with context stores
- Build batch and streaming data pipelines
- Build knowledge graphs and semantic layers
- Configure RBAC and PII handling
- Design and implement retrieval systems
- Develop APIs for autonomous agents
- Develop embedding pipelines
- Evaluate models and track experiments
- Implement Data Governance and Data Quality
- Implement production monitoring
- Implement semantic metadata and data lineage
- Manage dataset curation and feature engineering
- Support machine learning and LLM operations
- Validate schemas and monitor data quality
Perks/Benefits
- Access to modern cloud native technology
- Collaborative engineering culture
- Health insurance
- Hybrid work flexibility
- Learning and development support
- Remote-friendly work
- Wellness benefits
Skills/Tech-stack
AWS | Amazon Bedrock | Apache Kafka | Azure | CI/CD | Chunking | Claude API | Cloud Platforms | Data Lineage | Data Quality | Data quality monitoring | Dataset curation | Delta Lake | Docker | Embeddings | Entity-Mapping | Experiment tracking | FastAPI | Feature Engineering | Hybrid search | Knowledge graphs | Kubernetes | Langchain | Llamaindex | MLOps | MLflow | Metadata Management | Model Evaluation | Ontologies | OpenAI API | PII handling | Production Monitoring | PySpark | Python | Quality monitoring | RBAC | Ranking | Retrieval systems | SQL | Schema Validation | Snowflake | Spark Structured Streaming | Structured Streaming | Vector Databases
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Lead Data Scientist INR 3000K-4000KArtificial Intelligence | Big Data | Classification | Cloud Computing | ClusteringEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, GJ, India7h ago
-
Associate Software Engineer - Spark/Scala INR 1500K-2000KApache Spark | CI/CD | Databricks | DevSecOps | GitHubEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India7h ago
-
Software Engineer (Python, Pyspark, SQL) INR 1200K-2100KAWS | Azure | CI/CD | GCP | GitEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India8h ago
-
Principal Data Engineer INR 2520K-3380KApp Services | Azure | Azure App | Azure App Services | Azure CognitiveSenior-level Full TimeBengaluru Luxor North Tower, India13h ago
-
API Design | AWS | Agentic AI | C++ | Distributed SystemsSenior-level Full TimeHyderabad, Telangana, India13h ago
-
Senior-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior-level Full TimeChennai, Tamil Nadu, India13h ago
-
Associate Engineer INR 1500K-2500KAKS | AWS | App Services | Azure | Azure AppHybrid work model | On-call rotation | Professional office environmentMid-level Full TimeIND-Pune-Smartworks, India13h ago
-
Gen AI Developer - Assistant Vice President INR 2400K-4000KAPI Development | Containerization | Data Privacy | Deep learning | DockerExecutive-level Full TimeIT BUILDING, RAMANUJAN IT SEZ,, India13h ago
-
Data Engineer - People Data, Insights & Analytics INR 1500K-2000KAWS | Apache Spark | Azure | CI/CD | Data ModelingMid-level Full TimePune, Maharashtra, India16h ago
-
Mid-level Full TimeBengaluru21h ago
-
Senior-level Full Timecoimbatore, India21h ago
-
Python Engineer (AI/ML) INR 2500K-3500KCPU | Chroma | Cost Optimization | Data Preprocessing | DockerMid-level Full TimePune, MH, India23h ago
-
Data Analytics AI Specialist INR 15K-20KClassification | Clustering | Data Visualization | Data cleaning | Machine LearningFast-paced startup environment | Growth opportunities | High ownershipEntry-level Full TimeGurugram, HR, IN23h ago
-
Senior-level Full TimePune City, India1d ago
-
Lead Data Engineer INR 3000K-5000KAirflow | Alerting | BigQuery | CI/CD | CloudFormationHybrid work | Paid time off | Reasonable adjustmentsSenior-level Full TimeMumbai1d ago
-
Senior-level Full TimeIN-KA-Bangalore1d ago
-
Gen AI Developer INR 2000K-4500KData Engineering | Distributed Systems | GenAI | Language Models | Large Language ModelsSenior-level Full TimeIN-TN-Chennai1d ago
-
Senior-level Full TimeIN-TN-Chennai1d ago
-
Senior-level Full TimeIN-TN-Chennai1d ago
-
Senior-level Full TimeIN-TN-Chennai1d ago
-
Software Engineer - Azure Data engineer INR 3000K-3500KAccess Control | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCollaborative work environment | In-office 5 days | Professional development | Recognition and rewards | Training opportunitiesSenior-level Full TimeBangalore - Karnataka, India - Kalyani …1d ago
-
Lead Data Engineer - Data Archival - Bangalore INR 1500K-2500KApache Spark | Data Warehousing | ETL | Hadoop | JavaSenior-level Full TimeIndia1d ago
-
Senior-level Full TimePune, Maharashtra, India1d ago
-
Mid-level Full TimeHyderabad, India1d ago