Lead Data Engineer - AI Platforms
INR 2500K-4500K (estimate) Senior-level Full Time
Tasks
- Architect AI first enterprise data platforms
- Build AI ready foundations for feature stores embeddings and vector datasets
- Create resilient fault tolerant data pipelines with idempotency retry checkpointing and recovery
- Define enterprise data strategy and architecture standards
- Design distributed data systems with partitioning sharding clustering and parallel processing
- Design end to end data ingestion processing storage serving and governance
- Design semantic layers and knowledge architectures including semantic APIs and knowledge services
- Enforce compliance for HIPAA GDPR and PII protection
- Establish data governance frameworks including metadata catalog lineage and traceability
- Implement data observability with monitoring logging lineage tracking and alerting
- Implement data quality controls anomaly detection deduplication and SLA monitoring
- Implement knowledge graphs ontologies semantic search and vector search
- Implement lakehouse and data mesh based architectures
- Implement lambda and kappa batch streaming and real time processing
- Implement security controls including RBAC encryption data masking and access auditing
- Lead platform modernization cloud transformation and AI data platform initiatives
- Mentor engineering teams and drive technical excellence
- Optimize storage indexing partitioning query performance and data lifecycle management
Perks/Benefits
Skills/Tech-stack
Access auditing | Agentic AI | Amazon Redshift | Apache Spark | Azure Synapse | Azure Synapse Analytics | BigQuery | Data Governance | Data Lake | Data Lineage | Data Observability | Data Quality | Data Warehouse | Data masking | Data mesh | Databricks | Distributed Systems | Embeddings | Encryption | Fault Tolerance | Feature Store | GDPR | HIPAA | Kafka | Kappa Architecture | Knowledge graphs | Lakehouse | Lambda Architecture | Medallion Architecture | Metadata Management | Ontologies | PII | PySpark | Python | RAG | RBAC | Retrieval-Augmented Generation | SQL | Scalability | Semantic Search | Semantic layers | Snowflake | Synapse Analytics | Vector Databases | Vector Search
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Mid-level Full TimeBangalore, Karnataka, India; Hyderabad, Telangana, India2h ago
-
Partner Forward Deployed Engineer, Generative AI, Google Cloud INR 2000K-5000KAPIs | Agent Development | Agent Development Kit | Agent systems | Cloud platformSenior-level Full TimeMumbai, Maharashtra, India2h ago
-
AI Customer Engineer, Manufacturing, Google Cloud USD 207K-301KAI Agent Development Kit | Agent Development Kit | As-a-Service | Audit Logging | Cloud NetworkingConference attendance | Travel opportunitiesSenior-level Full TimeMumbai, Maharashtra, India2h ago
-
Lead AI/ML Engineer – Security AI & LLMs INR 3000K-5000KAPI Design | AWS SageMaker | Azure ML | Deep learning | DockerSenior-level Full TimePune, Maharashtra, India6h ago
-
Senior-level Full TimeMitratech India7h ago
-
Genetics Data Engineer 1 INR 3000K-4600KAPI Development | AWS | Agentic AI | Azure | BcftoolsCareer development opportunities | Flexible work culture | Inclusive workplaceSenior-level Full TimeBangalore, Karnataka, IN, 5600377h ago
-
APIs | Airflow | Apache Iceberg | Apache Kafka | Apache SparkSenior-level Full TimeBengaluru7h ago
-
Actors | Async/Await | CI/CD | Combine | Computer VisionSenior-level Full TimeBengaluru7h ago
-
Ecommerce Analytics Engineer INR 2500K-4400KAlteryx | Cloud Data | Cloud Data Platforms | DBT | Data GovernanceSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
Software Engineer II INR 2475K-4000KAWS | Alerting | Asynchronous workloads | Azure | Backend ServicesHybrid work environment | Work from homeMid-level Full TimeIndia, Karnataka, Bangalore; India, Telangana, Hyderabad8h ago
-
Senior-level Full TimeHyderabad8h ago
-
Senior-level Full TimeHyderabad, IN, 5000818h ago
-
Senior AI Engineer INR 2500K-5000KA2A | AI Foundry | AI Search | AI guardrailing | Agent architectureCollaboration | Documentation | Startup mindsetSenior-level Full TimeHyderabad8h ago
-
Azure Databricks | Azure DevOps | Data Quality | Data Warehousing | DatabricksCollaborative Agile environment | Fully remote | Hands on cloud data platform exposureMid-level Full TimeIndia R9h ago
-
Data Analytics/Engineer (Python and MS Fabric exp) INR 2500K-4720KAWS | Apache Airflow | Azure Data | Azure Data Factory | Azure SynapseCross-functional collaboration | Mentorship | Professional developmentSenior-level Full TimeIND-Pune-Smartworks, India14h ago
-
AWS S3 | Amazon Redshift | Amazon Web Services | Apache Spark | Azure DataSenior-level Full TimeBengaluru Millenia, India14h ago
-
API | Agentic AI | Anthropic | Bagging | BoostingFlexibility programmes | Hybrid work | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeNoida, India14h ago
-
Technical Lead-Azure Databricks INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Senior-level Full TimeINDIA - BENGALURU - HP, IN14h ago
-
Software Engineer - Python Expert, API Frameworks, RAG, Gen AI, Devops, Frontend Frameworks (5+ Years) INR 2500K-5000KAgentic AI | Alerting | Angular | CI/CD | Cause analysisSenior-level Full TimeIND-BANGALORE, India14h ago
-
Snowflake Data Engineer INR 2000K-2600KAmazon Web Services | Apache Spark | Bitbucket | CI/CD | Cloud platformSenior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBengaluru, KA, IN14h ago
-
EY - GDS Consulting - AI And DATA - Databricks - Senior INR 2400K-4000KAPI | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeTrivandrum, KL, IN, 69558114h ago
-
TTT-.NET-Angular-Full Stack with Gen AI-Staff INR 2000K-3500K.NET | AI Agent | AI Agent Frameworks | AI Search | Agent FrameworkSenior-level Full TimeKolkata, WB, IN, 70009114h ago