Data Engineer - Assistant Vice President
Location(s): Pune, Maharashtra, India
INR 4000K-6000K (estimate) Executive-level Full Time
Tasks
- Build CI CD pipelines for data engineering artifacts
- Build Spark SQL and Spark jobs
- Build semantic layers for reporting
- Collaborate with data science teams on model operationalization
- Design batch and real-time data pipelines
- Develop and deploy ETL ELT pipelines
- Ensure data governance compliance
- Implement Delta Lake storage optimization
- Implement automated data validation
- Lead code reviews and enforce coding standards
- Manage metadata with Unity Catalog
- Mentor data engineers
- Monitor data quality
- Optimize Spark cluster performance
- Package and deploy dbt Spark jobs and Databricks workflows
- Tune SQL query performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS EC2 | AWS EMR | AWS IAM | Amazon Athena | Amazon Redshift | Amazon S3 | Apache Spark | BigQuery | Bronze Silver Gold | CI/CD | Cloud Storage | Cloud platform | Compute Engine | DBT | Data Quality | Data quality monitoring | Databricks | Databricks Workflows | Dataproc | Delta Lake | Delta Lake ACID | Delta Lake Z Ordering | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | GitHub Actions | GitLab | Google Cloud | Google Cloud Platform | Google Cloud Storage | Google IAM | Jenkins | Medallion Architecture | Partitioning | PySpark | Python | Quality monitoring | SCD | SQL | Slow-changing dimensions | Snowflake Schema | Spark SQL | Star Schema | Unity Catalog | Vacuuming | Z-ordering
Education
N/A
Related jobs
-
Senior Software Engineer (Universal Robots, Bangalore - India) INR 2475K-3500KAgile | C++ | CI/CD | Clean Code | Code reviewExposure to Cutting Edge Robotics Technology | Flexible working conditions | Health insurance | Inclusive work environment | Professional development opportunitiesSenior-level Full TimeBangalore, IN6h ago
-
Senior Software Engineer, Machine Learning INR 3264K-5400KA/B | A/B Testing | Aerospike | B testing | Bayesian analysisCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Flexible remote workSenior-level Full TimeBengaluru, India9h ago
-
Senior Software Engineer (Data Platform) INR 2475K-3500KAWS | Airflow | Apache Flink | Apache Hadoop | Apache HiveCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Healthcare benefitsSenior-level Full TimeBengaluru, India9h ago
-
Senior Cloud Storage Engineer – NetApp CVO on GCP, AVP INR 2755K-3500KAutomation | CI/CD | Cloud Security | Cloud platform | Cloud volumes ONTAPAccident insurance | Certification sponsorship | Childcare assistance reimbursement | Employee assistance program | Gender neutral parental leaveSenior-level Full TimePune - Business Bay, India16h ago
-
Mid-level Full TimePune, India20h ago
-
Senior Software Engineer INR 2000K-2700KAgile | Cloud services | Code review | Data Modeling | Data WarehousingSenior-level Full TimePune, India20h ago
-
Artificial intelligence Developer (Pro Code) INR 2500K-4680KAI Foundry | APIs | Asynchronous processing | Autogen | Azure AISenior-level Full TimeThane, Maharashtra, India21h ago
-
Manager Data and Analytics INR 1400K-2000KAmazon Redshift | Azure | BigQuery | Business Intelligence | Cloud platformMid-level Full TimeBangalore, Karnataka, India R22h ago
-
Senior Artificial Intelligence/Machine Learning Engineer INR 2500K-5000KAPI Integration | AWS SageMaker | Agent systems | Azure ML | CCPACompany paid medical insurance | Company-paid certifications | Financial and legal consultations | Internal workshops and meetups | Language coursesSenior-level Full TimeIndia23h ago
-
Senior AI Engineer INR 3242K-4680KAWS EC2 | AWS ECS | AWS Lambda | AWS RDS | Amazon S3Financial consultation | Legal consultation | Medical insurance | Mental health support | Paid certificationsSenior-level Full TimeIndia23h ago
-
Associate Engineer - AI/ML with Python INR 2500K-4000KAWS | Bitbucket | CI/CD | CSS | CeleryCollaborative work environment | Onsite work 5 days a week | Training and developmentMid-level Full TimeBangalore - Karnataka, India - Kalyani …1d ago
-
Data Scientist – AI & Cybersecurity INR 2500K-4500KAPIs | AWS | Adversarial Testing | Amazon SageMaker | Anomaly DetectionGrowth opportunities | Mentorship programs | Training opportunitiesSenior-level Full TimeBangalore - Karnataka - India1d ago
-
Senior Python GenAI Engineer INR 2000K-4580KAWS | Angular | Anthropic | Azure | Azure OpenAIEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India1d ago
-
Senior Java Kafka Architect INR 2000K-2700KAmazon Web Services | Apache Kafka | CI/CD | Code Refactoring | Code reviewSenior-level Full TimeIN-AP-Hyderabad1d ago
-
Senior-level Full TimeIN-TN-Chennai1d ago
-
Data Engineer INR 2000K-3000KAPI Design | Asynchronous programming | Caching | Celery | Clean architectureSenior-level Full TimeIN-KA-Bangalore1d ago
-
Azure Data Engineer INR 1500K-2500KApache Spark | Azure Data | Azure Data Lake | Azure Databricks | Data GovernanceMid-level Full TimeIN-KA-Bangalore1d ago
-
Mid-level Full TimeIN-KA-Bangalore1d ago
-
Software Engineer III, ML Infrastructure, Google Cloud INR 2500K-5000KBack-end | Back-end development | C++ | Cloud APIs | Data StorageSenior-level Full TimePune, Maharashtra, India1d ago
-
Campus FTE INR 1200K-2500KAgentic AI | Amazon Web Services | Classification | Cloud platform | Data AnalysisMid-level Full TimeBangalore, Karnataka, India1d ago
-
AI Engineer INR 2000K-5000KAWS SageMaker | Azure Machine Learning | C++ | Computer Vision | Deep learningSenior-level Full TimeIndia - Pune1d ago
-
Campus FTE INR 1800K-3000KAgentic AI | Amazon Web Services | Classification | Cloud platform | Data QualityMid-level Full TimeGurgaon, Haryana, India1d ago
-
Mid-level Full TimeIndia, Telangana, Hyderabad; India, Karnataka, Bangalore; …1d ago
-
Senior Data Platform Consultant INR 3000K-4000KAWS architecture | Agile | Amazon Web Services | Azure | Azure ArchitectureHybrid work model | Mentoring and career development | Performance based learning and enablement sessionsSenior-level Full TimePune, India1d ago
-
Associate TSE II PHP 3000KAWS | Active Directory | Authentication | Azure | BenchmarkingHybrid workMid-level Full TimeBengaluru1d ago