Senior Data Engineer
Tasks
- Build lakehouse and data platform solutions on Databricks and AWS
- Collaborate with architects analysts data scientists product teams and DevOps teams
- Conduct architecture and code reviews troubleshoot and perform root cause analysis
- Define data models data contracts and reusable integration patterns
- Develop CI CD pipelines and automated testing for data solutions
- Develop metadata driven data integration frameworks
- Evaluate emerging technologies and recommend solutions
- Implement metadata management lineage cataloging governance RBAC and data security controls
- Implement workflow orchestration monitoring alerting data quality controls and recovery processes
- Integrate structured semi structured and unstructured data from enterprise manufacturing API and third party sources
- Lead batch and real time ETL ELT pipeline design and development
- Mentor engineers and provide technical guidance
- Optimize Spark workloads Databricks compute SQL queries partitioning storage and caching
- Own data solutions from requirements design deployment and production support
- Support estimation sprint planning and technical roadmaps
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Alerting | Apache Spark | Auditability | Automated testing | Batch Processing | CI/CD | Caching | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Security | Data Warehousing | Data integration | Databricks | Delta Lake | DevOps | Distributed Computing | ELT | ETL | Encryption | Git | Lakehouse | Metadata Management | Monitoring | Partitioning | Performance Tuning | PySpark | Python | Query Optimization | RBAC | SQL | SQL Query Optimization | SQL query | Spark SQL | Spark performance tuning | Storage Optimization | Streaming | Workflow Orchestration
Education
Roles
Related jobs
-
Senior-level Full TimeIndia3h ago
-
Lead Integration Engineer INR 1500K-3000KAWS EC2 | AWS IAM | AWS VPC | Amazon CloudWatch | Amazon MSKSenior-level Full TimePune - Maharashtra - India4h ago
-
Mid-level Full TimeBangalore - Karnataka - India4h ago
-
Senior-level Full TimeBangalore - Karnataka - India4h ago
-
Senior-level Full TimeBangalore - Karnataka - India4h ago
-
Senior Software Engineer - (Backend,AI) INR 3242K-5000KAPI Gateway | AWS | AWS Lambda | Agent Orchestration | Amazon APIFamily well-being benefits | Health benefits | Hybrid work | Paid time off | Wellbeing supportSenior-level Full TimeHyderabad, India R4h ago
-
Senior Data Engineer INR 3000K-4000KAI | AWS | Amazon Aurora | Automated reporting | AzureBike to work | Educational assistance | Employee assistance programme | Employee discount | Enhanced maternity/paternity leaveSenior-level Full TimeMumbai, MH, India R7h ago
-
Senior Data Engineer INR 1500K-2700KApache Airflow | Apache Hadoop | Apache Iceberg | Apache Spark | DBTSenior-level Full TimeHyderabad, TS, India8h ago
-
Databricks Engineer INR 1500K-2800KADLS Gen2 | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeHyderabad, TS, India8h ago
-
Sr Generative AI Engineer (Agentic AI & RAG) INR 3264K-5000KAI Agent Builder | Agent Builder | Agent Orchestration | Apex | BigQuerySenior-level Full TimePune, MH, India8h ago
-
Software Engineer, Machine Learning INR 3242K-4580KClassification Models | Data Mining | Deep learning | Distributed Training | Embedding ModelsFlexible work hybrid work arrangementSenior-level Full TimeBengaluru, KA, India R8h ago
-
Robotics Engineer Fresher INR 300K-400KArduino | Artificial Intelligence | Automation | C plus plus | C#Career growth in EdTech and robotics | Exposure to robotics competitions | Mentorship opportunities globallyEntry-level Full TimeBangalore, Karnataka, India8h ago
-
Software Developer (Python, Skywise, Pyspark, AWS) INR 2000K-3500KAPI | AWS Lambda | Agile | Airbus Values | Amazon RDSSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Software Engineer III, Big Data INR 2040K-3465KApache Flink | Apache Hive | Apache Spark | Cloud Composer | Cloud StorageHybrid workSenior-level Full TimeHyderabad, Telangana 500081, India R9h ago
-
AWS | CD tools | CI/CD | CI/CD Tools | Data QualitySenior-level Full TimeMumbai, Maharashtra, India9h ago
-
Manager - Forward Deployed Engineer / AI Engineer INR 1800K-3300KAPIs | Agent Frameworks | Azure | Azure OpenAI | BERTMid-level Full TimeBangalore, Karnataka, India9h ago
-
Associate Engineer INR 1500K-2500KAnalysis Services | Azure Analysis | Azure Analysis Services | Azure Blob | Azure Blob StorageMid-level Full TimeHyderabad, IN10h ago
-
Forward Deployed AI Scientist INR 2500K-4200KAWS Bedrock | Agentic Workflows | Artificial Intelligence | CI/CD | Crew AICommunity outreach programs | Employee Resource Group | Employee stock purchase program | Health and wellness incentivesSenior-level Full TimeBangalore, KA, IN, -10h ago
-
Customer Engineer INR 1436K-1820KAPI Integration | AWS | Algorithms | Cassandra | Cloud platform24/7 on-call rotation | Flexible work arrangements | Weekend shift rotation as neededMid-level Full TimeBangalore10h ago
-
Full Stack Data Engineer INR 1500K-2000KAirflow | Batch Processing | BigQuery | CI/CD | Cloud BuildMid-level Full TimeChennai, Tamil Nadu, India10h ago
-
Senior-level Full TimeBangalore, INDIA11h ago
-
Backend Software Engineer, AI Platform INR 2000K-3000KAPIs | AWS | Agentic AI | Authentication | Auto ScalingMid-level Full TimeBengaluru R12h ago
-
Address Parsing | Data Modeling | Data Normalization | Data Quality | Data StandardizationCareer growth | Continuous learning | Flexible internship schedule | Mentorship | Professional developmentMid-level Full TimeIndia R12h ago
-
Senior Machine Learning Engineer - Recommendations INR 3242K-5000KA/B | A/B Testing | Airflow | Amazon EMR | Amazon S3Senior-level Full TimeBengaluru, India R12h ago
-
Access Control | Agentic Workflows | Apache Airflow | Automated alerting | CI/CDComprehensive benefits | Professional growth opportunities | Remote/office flexibilitySenior-level Full TimeIndia R12h ago