Data Engineer
Mumbai, Maharashtra, India
INR 1400K-1800K (estimate) Senior-level Full Time
Tasks
- Build Bronze to Silver transformations cleansing and deduplication
- Build Silver to Gold transformations star schema and fact dimension modelling
- Build and maintain Data Factory pipelines for API ingestion
- Build and optimize Power BI semantic models Direct Lake
- Create Dataflows Gen2 for Excel CSV data cleansing
- Develop Python ETL scripts for extraction and file ingestion
- Develop and maintain Spark Python notebooks and SQL transformations
- Implement data quality frameworks validation and anomaly detection
- Implement incremental loads watermarking and CDC patterns
- Integrate data from Dataverse and enterprise sources into Fabric
- Maintain documentation data dictionaries and architecture decisions
- Manage Fabric capacity pause resume scheduling and scaling
- Monitor pipeline performance platform health and Fabric capacity
- Prepare datasets for reporting and analytics
- Prepare feature datasets for AI ML workloads
Perks/Benefits
Skills/Tech-stack
Anomaly Detection | Apache Spark | Azure DevOps | Azure DevOps Pipelines | CDC | CI/CD | Change Data Capture | DAX | DAX Query | Data Capture | Data Factory | Data Quality | Data Validation | Dataflows Gen2 | Dataverse | DevOps Pipelines | Dimension Modelling | Dimensional Modeling | Direct Lake | ETL | Fact Dimension Modelling | Fact/Dimension | Git | Incremental loading | Lakehouse | Microsoft Data | Microsoft Data Factory | Microsoft Fabric | OData | OneLake | Power BI | Power BI Semantic Model | Power Query | PySpark | Python | REST | SCD | SQL | Semantic Model | Slowly Changing Dimensions | Star Schema | Warehouse | Watermarking
Education
N/A
Roles
Related jobs
-
CI/CD | DBT | Data Analysis | Data Modeling | Data NormalizationCharity contribution match | Corporate Volunteering Hours | Dental insurance | Employee discounts | Medical insuranceMid-level Full TimeBengaluru, KA, IN2h ago
-
Senior Data Engineer INR 2000K-3000KAI integrations | API Integration | Automation workflows | Azure Data | Azure Data FactoryClient-facing work | Fast-paced environment | Independent ownershipSenior-level Full TimeBangalore North, India4h ago
-
Data Science INR 2000K-3300KAgentic AI | Apache Airflow | CrewAI | Data Architecture | Data PipelinesSenior-level Full TimeIN-MH-Pune4h ago
-
Data Engineer with Airflow INR 1500K-2400KAmazon Web Services | Apache Airflow | Data Visualization | Data Warehousing | DockerSenior-level Full TimeIN-AP-Hyderabad4h ago
-
Data Engineer- Airflow INR 1500K-2500KAmazon Web Services | Apache Airflow | Data Visualization | Data Warehousing | DockerSenior-level Full TimeIN-AP-Hyderabad4h ago
-
Data Engineer- Airflow INR 1500K-2500KAmazon Web Services | Apache Airflow | Data Visualization | Data Warehousing | DockerSenior-level Full TimeIN-AP-Hyderabad4h ago
-
Senior-level Full TimeIN-TN-Chennai4h ago
-
DevOps Engineer INR 1500K-2500KAgile | Amazon Web Services | BI dashboarding | CI/CD | Cloud platformInternational collaboration | Occasional travel to EuropeMid-level Full TimePune, MH, India6h ago
-
AI Engineer – Retail/CPG Domain INR 1500K-2500KAWS | Azure | Cloud platform | Data Governance | Data ModelingMid-level Full TimeGurugram, Haryana, India6h ago
-
REF82383U - Data science (ML Algorithms, unsupervised techniques, Databricks, Python, PySpark, Azure,AB testing ) 3- 6 yrs exp INR 1500K-2040KA/B | A/B Testing | B testing | Databricks | Machine LearningSenior-level Full TimeBangalore, KA, India6h ago
-
Specialist - Data Engineering-Data Engineering INR 180K-200KAWS | Apache Kafka | Apache Spark | Data Modeling | Data QualityEntry-level Full TimeMumbai, Maharashtra, India6h ago
-
AWS | Agent Orchestration | Agent systems | Anomaly Detection | AzureCharity match | Corporate volunteering | Dental insurance | Medical insurance | Paid time offSenior-level Full TimeBengaluru, KA, IN7h ago
-
Senior-level Full TimeGurugram, Haryana, India7h ago
-
API Development | AWS | Azure | Cloud platform | Google CloudCharity contribution match | Corporate Volunteering Hours | Dental insurance | Discounts on tickets and travel | Medical insuranceMid-level Full TimeBengaluru, KA, IN8h ago
-
Sr. Staff Engineer (Python,Pyspark,AWS) INR 2800K-4000KAI Agents | API Gateway | AWS | Apache Spark | BedrockEmployee volunteering | Family wellbeing benefits | Health insurance | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India10h ago
-
Senior-level Full TimeIndia - Bengaluru; India - Hyderabad10h ago
-
GCP Data Engineer INR 2000K-3000KApache Airflow | Apache Spark | BigQuery | Cloud Composer | Cloud DataflowSenior-level Full TimeIndia - Bengaluru; India - Hyderabad10h ago
-
Senior Data Engineer INR 3000K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Data FactoryFlexible work environment | Hybrid work model | Learning and development supportSenior-level Full TimeIndia R11h ago
-
AWS | Agentic AI | Apache Spark | Databricks | Generative AICollaborative global engineering culture | Continuous learning | Flexible work environment | Opportunity to work on cutting edge GenAI | Remote-friendly work environmentSenior-level Full TimeIndia11h ago
-
Senior Data Engineer INR 3000K-4200KAWS | Access Management | Amazon Neptune | Apache Airflow | Apache KafkaCareer advancement | Hybrid work options | Professional development programs | Remote work options | Training programsSenior-level Full TimeBengaluru11h ago
-
Data Engineer INR 1500K-2000KAWS Glue | Access Control | Amazon EMR | Amazon Snowflake | Apache AirflowMid-level Full TimeIndia11h ago
-
API | Agile | Apache Airflow | Bitbucket | CI/CDEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, India11h ago
-
Senior-level Full TimeIndia13h ago
-
Lead Software Engineer INR 3000K-4500KAWS | CI/CD | DevOps | Docker | EMRCareer development opportunities | Comprehensive benefits | Continuous learning | Flexible working hours | Work-life balanceSenior-level Full TimeMumbai, India13h ago
-
Staff Software Engineer - Data INR 3200K-4600KAWS Lambda | Airflow | Ansible | Apache Flink | Apache SparkSenior-level Full TimeBengaluru, INDIA, India13h ago