Data engineering lead
INR 3000K-4000K (estimate) Senior-level Full Time
Tasks
- Build and optimize scalable data pipelines
- Conduct architecture reviews and code quality governance
- Create engineering standards testing frameworks and release readiness
- Define technical roadmaps and execution plans
- Drive agile delivery reuse automation and operational stability
- Enable analytics BI and AI ML use cases with governed data assets
- Ensure data quality, governance, and lineage
- Implement observability error handling retries and checkpointing
- Lead and mentor data engineering team
- Lead data architecture design and implementation
- Manage cluster configuration autoscaling caching and tuning
- Optimize Spark performance and platform reliability
- Oversee end to end data lifecycle accuracy and timeliness
- Partner with stakeholders to clarify requirements and acceptance criteria
- Redesign pipelines and storage for performance and cost
Perks/Benefits
- N/A
Skills/Tech-stack
AWS DMS | AWS Lambda | AWS Step Functions | Amazon Aurora | Amazon CloudWatch | Amazon EventBridge | Amazon S3 | Apache Spark | Autoscaling | CI/CD | Caching | Checkpointing | Cluster configuration | Data Governance | Data Lakes | Data Lineage | Data Modeling | Data Quality | Data Reconciliation | Data Warehouses | Database Architecture | Databricks | Databricks Unity | Databricks Unity Catalog | Delta Lake | Delta Lakehouse | Delta Sharing | Dimensional data | Dimensional data modeling | ETL | Git | GitHub Actions | Observability | Partitioning | PySpark | Python | Retry Mechanisms | SLAs | SLO | SQL | Spark Structured Streaming | Step Functions | Storage Optimization | Structured Streaming | Unity Catalog | Version control | Z-ordering
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Data Engineering | Data Engineering Lead | Engineer | Engineering Lead | Lead
Related jobs
-
Data Architect INR 1500K-2400KAWS Glue | Amazon S3 | Apache Spark | Asset bundles | Azure ADFCareer progression | Health & wellness benefits | Hybrid working | Retirement benefits | Training budget for Databricks certificationSenior-level Full TimePune, India7h ago
-
Senior-level Full TimeIN-TN-Chennai8h ago
-
Senior-level Full TimeIN-TN-Chennai8h ago
-
Senior-level Full TimeIN-AP-Hyderabad8h ago
-
Senior-level Full TimeIN-AP-Hyderabad8h ago
-
Senior-level Full TimeIndia9h ago
-
Agentic AI | As-a-Service | Audit Logging | Cloud Networking | Distributed tracingConference participation | Customer facing technical collaboration | Travel opportunitiesSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
Data and Analytics Lead, Google Cloud INR 1500K-3000KBigQuery | Cloud Migration | Data Modeling | Data Transformation | Database ManagementSenior-level Full TimeBengaluru, Karnataka, India; Mumbai, Maharashtra, India10h ago
-
Partner Forward Deployed Engineer, Generative AI, Google Cloud INR 2000K-4000KAPIs | Agent systems | Artificial Intelligence | Cloud platform | Data PipelinesSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
Senior-level Full TimeBengaluru, Karnataka, India10h ago
-
CCB Risk Strategic Analytics – GenAI INR 1960K-2800KAgentic AI | Control processes | Credit Risk | Data Analysis | Data EngineeringSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
Snowflake Data Engineer- Consultant INR 1500K-2500KAWS Glue | AWS Lambda | Airflow | Amazon Athena | Amazon EC2Mid-level Full TimeBangalore, Karnataka, India11h ago
-
Senior-level Full TimeBengaluru, Karnataka, India11h ago
-
Senior Expert INR 3000K-3880KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Employee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India11h ago
-
AES - DE - Generative AI Prompt Engineers INR 3000K-4000KCI/CD | Code Analysis | Cypress | DeepEval | DockerSenior-level Full TimePune, Maharashtra, India12h ago
-
Senior Data Engineer INR 2000K-3000KAPI Design | Apache Kafka | Apache Spark | Automated testing | CI/CDSenior-level Full TimeBengaluru, KA, India12h ago
-
Senior Software Engineer 7+ years (Must have- Python, FastAPI (or Flask/Django), Langchain, Mongodb, RestAPI) INR 1960K-2829KAnthropic | Azure OpenAI | CI/CD | Cohere | DjangoEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India12h ago
-
Expert, Analytics Engineering INR 2520K-3880KAWS | Airflow | CI/CD | Cognitive Computing | DBTRelocation assistanceSenior-level Full TimeMumbai, MH, IN13h ago
-
Associate Process Manager INR 800K-1800KBigQueryML | CRM Analytics | Customer Segmentation | Dashboarding | Data ModelingAPAC shift workMid-level Full TimeMumbai, Maharashtra, India13h ago
-
AWS | Alerting | Audit Logging | Azure | CI/CDWork from OfficeSenior-level Full TimePune, MH, India13h ago
-
AES - DE - Generative AI Prompt Engineers INR 2800K-4000KAgent Orchestration | Autogen | Azure DevOps | Azure OpenAI | C SharpSenior-level Full TimeBangalore, Karnataka, India13h ago
-
Head of AI & Data Privacy Risk INR 4000K-5000KAI NIST | Alteryx | Analytics | Artificial Intelligence | AuditExecutive-level Full TimeIndia14h ago
-
Senior-level Full TimeChennai, TN, India15h ago
-
Senior Software Engineer I INR 2500K-3500KAPI Design | Agile Development | Airflow | Alerting | Apache SparkCareer growth opportunities | Health and wellness support | Learning opportunities | Mentoring opportunitiesSenior-level Full TimeChennai, TN, India15h ago
-
KGS - AWS DevOps - Consultant - Bangalore INR 2125K-3500KAWS | Alerting | Ansible | Artifactory | BitbucketSenior-level Full TimeBangalore, Karnataka, India16h ago