Data Engineering Consultant - AI Platform
INR 3000K-4000K (estimate) Senior-level Full Time
Tasks
- Build AI ready datasets and feature pipelines
- Build AI-ready data platforms
- Develop batch streaming and real time processing
- Develop scalable data pipelines
- Enable semantic search and RAG workflows
- Implement RBAC and encryption controls
- Implement automated testing for data pipelines
- Implement data governance and data cataloging
- Implement data lineage and data quality management
- Implement data masking and auditing
- Implement embedding and vector data pipelines
- Implement lakehouse data architecture
- Implement monitoring, logging, and alerting
- Implement resilient data processing patterns
- Optimize storage partitioning and query performance
- Support CI/CD for data pipelines
- Support data observability and DataOps practices
- Troubleshoot production data pipeline issues
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Iceberg | Apache Spark | Automated testing | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Synapse | BigQuery | CI/CD | Cloud Storage | DBT | Data Catalog | Data Engineering | Data Factory | Data Lake | Data Lake Storage | Data Lineage | Data Quality | Data Warehouse | Data masking | DataOps | Databricks | Delta Lake | Distributed Processing | Docker | Embeddings | Encryption | Feature Engineering | Google Cloud | Google Cloud Storage | Kafka | Kubernetes | Lakehouse | Logging | Medallion Architecture | Metadata Management | Monitoring | ORC | Parquet | PySpark | Python | RAG | RBAC | REST APIs | Retrieval-Augmented Generation | S3 | SQL | Semantic Search | Semantic indexing | Snowflake | Vector Databases
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
Senior-level Full TimeIN-MH-Pune7h ago
-
Data Engineer - Python /GenAI INR 2000K-3000KData Engineering | Distributed Systems | Generative AI | Language Models | Large Language ModelsSenior-level Full TimeIN-MH-Pune7h ago
-
Data Engineer - Python /GenAI INR 2000K-3000KData Engineering | Deep learning | Distributed Systems | GenAI | LLMSenior-level Full TimeIN-MH-Pune7h ago
-
Data Engineer - Python /GenAI INR 2000K-3000KData Engineering | Distributed Systems | Generative AI | LLM | Machine LearningSenior-level Full TimeIN-MH-Pune7h ago
-
Data Engineer - Python /GenAI INR 2000K-3000KData Engineering | Distributed Systems | Generative AI | Inference | LLMSenior-level Full TimeIN-MH-Pune7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
Senior-level Full TimeIN-TN-Chennai7h ago
-
PL/SQL Developer INR 1200K-2400KBackup & Recovery | CI/CD | Data Modeling | Data Quality | Database InternalsSenior-level Full TimeIN-TN-Chennai7h ago
-
Data Engineer - Python /GenAI INR 2000K-3000KData Engineering | Deep learning | Distributed Systems | GenAI | LLM-OpsSenior-level Full TimeIN-MH-Pune7h ago
-
Azure Data Engineer INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Functions | Azure SynapseHybrid work modeSenior-level Full TimeNoida, Uttar Pradesh, India8h ago
-
Software Engineer III - (ADLC , Python) INR 3000K-4000KAWS | Agile | Architecture | CI/CD | Data VisualizationSenior-level Full TimeHyderabad, Telangana, India9h ago
-
Data Engineer, Global Affairs INR 1500K-2500KArtificial Intelligence | Big Data | C++ | Cloud platform | Data PipelinesMid-level Full TimeHyderabad, Telangana, India9h ago
-
Principal Engineer, Python fullstack (React+Genai) INR 3000K-5000KAPI Integration | AWS | Agent Orchestration | Azure | ComplianceSenior-level Full TimeRemote, India R9h ago
-
DE&A - Core - Big Data Engineering - Hadoop Ecosystem INR 3000K-4200KAPI | AWS | Apache Hadoop | Apache Spark | AzureSenior-level Full TimePune, Maharashtra, India9h ago
-
AES - DE - Generative AI Prompt Engineers INR 2000K-3500KAnthropic API | Azure OpenAI | CI/CD | Code Analysis | Cost controlsMid-level Full TimePune, Maharashtra, India10h ago
-
Data Engineer INR 704K-2000KAPIs | Batch Processing | CDP | Customer Data | Customer Data PlatformAgile culture | Documentation | Knowledge sharingMid-level Full TimeIndia10h ago
-
AI/ML Engineer INR 2500K-3885KAzure | Deep learning | DeepSpeed | Distributed Systems | DockerCar lease | Certification programs | Corporate pension scheme | Dental & vision coverage | Education reimbursementMid-level Full TimeBangalore, Karnataka, India10h ago
-
Principal Engineer, Machine Learning INR 4000K-5000KArtificial Intelligence | Computer Vision | Deep learning | Docker | Foundation ModelsSenior-level Full TimeRemote, India R10h ago
-
Associate Principal Engineer, Machine Learning INR 1800K-3300KAdversarial Networks | Agent Orchestration | Agent systems | Computer Vision | Deep learningMid-level Full TimeRemote, India R10h ago
-
Data Engineer INR 3000K-4000KAWS | Data Architecture | Data Governance | Data Modeling | Data MonitoringBike to work | Career progression | Educational assistance | Employee assistance program | Employee discountMid-level Full TimeBengaluru, KA, India10h ago
-
Senior-level Full Time Part Timebangalore, India11h ago
-
Mid-level Full TimeIndia11h ago
-
Lead Software Engineer - Java, Spring boot, AWS Cloud & AI/ML (REST APIs, System Design, SDLC) INR 3500K-5000KAI Assisted Development | API Gateway | AWS Athena | AWS Glue | AWS KinesisSenior-level Full TimeBengaluru, Karnataka, India12h ago