Lead Data Engineer (Contract, Full-Time) [HR208]
Tasks
- Architect data architecture
- Build machine learning data pipelines
- Build scalable data pipelines
- Build vector search and retrieval systems
- Collaborate with AI and backend teams
- Define data standards and hiring bar
- Design data ingestion transformation and storage
- Develop batch and real time pipelines
- Ensure data reliability security and governance
- Implement monitoring observability and data quality
- Optimize performance of large datasets and queries
- Translate data capabilities into product decisions
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Spark | Batch data | Batch data pipelines | Cloud infrastructure | Data Governance | Data Modeling | Data Pipelines | Data Processing | Data Quality | Distributed Systems | Elasticsearch | Kafka | Machine Learning | Machine learning data | Machine learning data pipelines | Milvus | MongoDB | MySQL | Observability | OpenSearch | PGVector | Pandas | Polars | PostgreSQL | Python | Qdrant | Real Time | Real-time Data | Real-time data pipelines | Vector Databases | Vector Search
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior Software Engineer, Machine Learning INR 3600K-5400KA/B | A/B Testing | Amazon S3 | Apache Airflow | B testingDental insurance | Financial wellness support | Health insurance | Mental health support | Paid time offSenior-level Full TimeBengaluru, India R4h ago
-
AI Agents | AWS Bedrock | AWS Lambda | Agent Orchestration | Agent systemsContinuous learning | Fixed working hours | Fully remote | Knowledge sharing | Laptop providedMid-level Full TimeIndia R6h ago
-
ABAP | BDC | Business Intelligence | Data Modeling | Data PipelinesCareer development | Employee communities | Global onboarding | Health insurance | Hybrid workSenior-level Full TimeIndia R6h ago
-
API Versioning | Asynchronous processing | Authentication | CI/CD | CeleryCollaborate with backend and frontend teams | Full-time employment | Fully remote | Hands on experience with production LLM and VLM systems | Work on AI infrastructure model routing and microservicesMid-level Full TimeIndia R7h ago
-
Machine Learning Engineer IV INR 4443K-7998KAPIs | Batch Processing | Cost Optimization | Distributed Systems | EmbeddingsBonuses | Disability insurance | Health insurance | Life insurance | Paid parental leaveSenior-level Full TimePune, India R15h ago
-
Staff Software Engineer, Knowledge Catalog, Core Data INR 2500K-5000KDistributed Databases | Distributed Systems | High Throughput | High-Throughput Systems | Indexing pipelinesSenior-level Full TimeBengaluru, Karnataka, India R23h ago
-
AI Agents | AWS Bedrock | Context engineering | Cost Optimization | Evaluation PipelinesCommunity forums | Full medical insurance | Laptop provided | Mentorship | No weekend workMid-level Full TimeIndia R1d ago
-
Senior-level Full TimeIndia, Remote R1d ago
-
MongoDB, Python, MSA/Consultant Specialist INR 1000K-2000KAWS | Aggregation Pipeline | Azure | CI/CD | DockerFlexible working | Inclusive work environment | Professional development opportunitiesSenior-level Full TimeHyderabad, Telangana, India R1d ago
-
AI Engineer - Quality INR 3264K-5876KAPI Testing | Apex | Automation | CI/CD | Data ValidationHybrid work model | Learning platform access | Mental health counseling | Mentorship programs | Paid parental leaveSenior-level Full TimeBangalore, India R1d ago
-
Data Engineer -- Regulatory Reporting & Portfolio Intelligence INR 1000K-1500KAWS | Amazon Redshift | Apache Airflow | Data Quality | GCSEmployee assistance program | Flexible-hybrid work | Generous vacation program | Internal workshops | Learning stipendMid-level Full TimeBangalore R1d ago
-
AWS | Amazon Kinesis | Apache Airflow | Apache Spark | Cloud platformFully remoteMid-level Full TimeIndia R1d ago
-
AWS S3 | AWS storage | AWS storage services | Active Directory | Ansible100 percent remote work | Flexible time off | Holiday allowance | Medical insurance | Shift allowanceMid-level Full TimeIndia R1d ago
-
Databricks Platform Engineer (AWS) INR 1500K-3500KAWS | AWS Cloud | AWS Cloud Database | Amazon Virtual Private Cloud | Asset bundlesFlexible work arrangement | Hybrid working model | Wellness supportSenior-level Full TimeHyderabad, India R1d ago
-
Lead Data Scientist MSAT INR 2475K-4000KArtificial Intelligence | Cause analysis | Data Quality | Data quality control | Deep learningSenior-level Full TimeBengaluru Luxor North Tower, India R1d ago
-
Databricks Data Engineer - Bengaluru INR 1500K-2600KAWS | AWS Glue | Amazon Athena | Amazon EMR | Amazon S3Work from anywhereMid-level Full TimeIndia-Bangalore-Office-Kalyani Vista R1d ago
-
Data Engineer, Vice President INR 3000K-4000KAWS | Azure | Cost Optimization | Data Governance | Data ModelingComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Support for working parentsExecutive-level Full TimeM11 - Commers III, 43, 44 … R1d ago
-
Director, Data Engineering INR 4000K-5000KAPIs | AWS | Agile | Anomaly Detection | AzureComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Support for working parentsExecutive-level Full TimeHA3-Gurugram - DLF Cyber City, India R1d ago
-
Sr Data Engineer INR 2771K-4064KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageFlexible working hours | Paid parental leave | Remote Working | Training and development programmesSenior-level Full TimeBangalore RMZ-ECO WORLD, India R1d ago
-
Computer Scientist II INR 3242K-3584KAI Agents | AIOps | AWS | Alertmanager | ArgoOn-call rotationSenior-level Full TimeBangalore, India R1d ago
-
Senior Computer Scientist INR 3264K-5000KAWS | Agentic AI | Application Insights | Application Security | AzureSenior-level Full TimeBangalore, India R1d ago
-
Computer Scientist 2 INR 3584K-5400KAWS | Azure | CI/CD | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeNoida, India R1d ago
-
Mid-level Full TimeNoida, India R1d ago
-
Software Development Engineer 4 INR 1200K-3000KAI Foundry | API Gateway | AWS | AWS Step Functions | AirflowSenior-level Full TimeBangalore, India R1d ago
-
Machine Learning Associate Advisor - HIH - Evernorth INR 2000K-3000KAWS | AWS Lambda | Amazon EC2 | Amazon EKS | Amazon SageMakerMid-level Full TimeHIH - Hyderabad, India R1d ago