Lead Data Engineer
Tasks
- Apply CI CD integration and code reviews
- Build ML inference pipelines
- Collaborate with engineering, data science, and product teams
- Contribute across software development lifecycle
- Deploy and ensure production readiness
- Design build and operate Databricks data pipelines
- Document and maintain production data solutions
- Drive Databricks performance and cost optimization
- Handle cluster and job management
- Implement security access controls and governance
- Implement test automation
- Manage Databricks workspace configuration and optimization
- Own end-to-end data pipeline lifecycle
- Perform data ingestion, cleansing, and transformation
- Translate business requirements into production pipelines
- Troubleshoot and resolve production incidents
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Agile | Airflow | Amazon S3 | Apache Spark | CI/CD | Databricks | Docker | ETL | Elasticsearch | Git | Grafana | Java | Kafka | Kinesis | Kubernetes | ML inference | Prometheus | Python | Redis | SQL | Scala | Scrum | Terraform
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior Data Engineer INR 3000K-4000KAWS Lambda | Amazon Web Services | Apache Spark | Aurora PostgreSQL | Batch ProcessingBike to work | Educational assistance | Employee assistance programme | Employee discount | Fitness classesSenior-level Full TimeMumbai, MH, India3h ago
-
Industrial Data & Analytics Engineer INR 1200K-2000KExcel | Grafana | Looker | Mqtt | OPC UAContinuous learning | Cross-cultural collaboration | Professional developmentMid-level Full TimeBangalore - Indraprastha, India9h ago
-
Senior Enterprise Engineer (Storage) INR 2000K-3500KAnsible | Automation | Azure | Backup and Recovery | CI/CDHybrid work | On-call rotationSenior-level Full TimeBengaluru, KA, India15h ago
-
Associate Staff Engineer (Python + Agentic Workflows) INR 1800K-2500KAI Agents | Best practices | CI/CD | Cloud Platforms | DockerMid-level Full TimeMumbai, India16h ago
-
Senior Staff Engineer (AI Developer SOC Automation) INR 2500K-4500KAI Search | Analytics rules | Azure AI | Azure AI Search | Azure EventSenior-level Full TimeMumbai, India16h ago
-
Senior Staff Engineer (AI Developer InfraSec Automation) INR 2500K-4500KAI Search | AI vector search | AWS | Access Control | AnthropicSenior-level Full TimeMumbai, India16h ago
-
Senior AI Engineer/Python Developer (Bangalore, India Based) INR 2535K-4225KDjango | FastAPI | Flask | Hugging Face | LangchainHybrid work environment | Learning and growth opportunities | MentorshipSenior-level Full TimeBangalore, India1d ago
-
Mid-level Full TimeHyderabad, India1d ago
-
Staff Software Engineer, AI Garage INR 2000K-4800KAgentic AI | Artificial Intelligence | C# | C++ | Distributed SystemsSenior-level Full TimeHyderabad, Telangana, India1d ago
-
Senior Staff Engineer (AI Developer - AppSec) INR 2000K-4500KAPI Security | API Security Top 10 | API security testing | Agentic Workflows | Azure DevOpsSenior-level Full TimeMumbai, India2d ago
-
Senior Staff Engineer (AI Developer - DevSecOps Tools) INR 3000K-5000KAKS | ARM Templates | AWS EventBridge | Aqua Security | ArgoCDSenior-level Full TimeMumbai, India2d ago
-
Associate Principal, SOX Analytics INR 2940K-4200KAI | API Integration | Audit Methodology | Coso | Data AnalysisBonus incentives | Employee travel credits | Equity programs | Inclusion and belongingMid-level Full TimeIndia2d ago
-
Senior Data Engineer INR 2520K-3380KApache Airflow | Apache Iceberg | Azure Data | Azure Data Lake | Azure DevOpsSenior-level Full TimePune, MH, IN2d ago
-
Principal Engineer Software INR 1500K-3000KAWS | Amazon Redshift | Analytics engineering | Apache Spark | BusinessObjects Data ServicesFlexible work environment | Internal mobility | Volunteering opportunities | Well-being | Work-life balanceSenior-level Full TimeKA Bangalore, India2d ago
-
Mid-level Full TimeGurugram, Haryana, India2d ago
-
Sr. Software Engineer - SnowFlake Job INR 2000K-3000KAccess Control | Data Governance | Data Pipelines | Data Quality | Data SecurityContinuous learning | Flexible work arrangements | Inclusive team environment | Mentorship | Stable employmentSenior-level Full TimeBangalore, KA, IN2d ago
-
Build_MS_AG_CA_AI_1 INR 1500K-2500KAWS Glue | AWS Lambda | AWS Redshift | Amazon S3 | CI/CDFlexible work arrangements | In-person collaboration | Inclusive work environment | Productivity support | Wellbeing supportSenior-level Full TimeIND Noida - A-44/45 Noida Towers …2d ago
-
EY - GDS Consulting - AIA -Generative AI - Manager INR 1850K-3300KAWS | Agentic AI | Agile | Amazon Bedrock | Cloud platformCoaching | Continuous learning | Flexible tools and resourcesMid-level Full TimeKolkata, WB, IN, 7000912d ago
-
Software Engineer II-Continous Learning INR 2000K-3500KAsynchronous programming | Containerization | Distributed Systems | Docker | Event StreamingConference reimbursement | Employee assistance program | Employee stock purchase program | Flexible time off | LinkedIn LearningMid-level Full TimeHyderabad2d ago
-
Software Engineer I-Continuous Learning INR 700K-800KAsynchronous programming | C++ | Containerization | Distributed Systems | DockerConference reimbursement | Employee assistance program | Employee stock purchase program | Equity compensation | Flexible time offEntry-level Full TimeHyderabad2d ago
-
Staff Software Engineer-AI INR 2000K-4500KAsynchronous Architecture | Autogen | Backpressure | CrewAI | Data GenerationEmployee assistance program | Flexible time off | LinkedIn Learning subscription | Local Employee Meetups | Training and education reimbursementSenior-level Full TimeHyderabad2d ago
-
Mid-level Full TimeGurgaon, Haryana, India2d ago
-
Software Engineer, Middle Office Systems - AI/ML INR 1500K-3000KCapacity Planning | Data Processing | Data concepts | Electronic Trading | Event DrivenHealth care benefits | Mental wellness programs | Transportation support | Tuition assistanceMid-level Full TimeBengaluru, India2d ago
-
ETL Test Engineer (3-6 yrs) (Pune/Chennai) - (Must Have- ETL Testing, Python, Pyspark, Azure Databricks) INR 900K-1500KAPI Testing | Apache Airflow | Azure Databricks | Bitbucket | DockerEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai/Pune, India2d ago
-
Senior Data Engineer INR 1500K-2500KACID | AWS S3 | Amazon Redshift | Apache Kafka | Azure DataWork from office 5 days per weekSenior-level Full TimeGurugram, Haryana, India2d ago