Data Engineer - Databricks & Microsoft FabricData Engineer - Databricks & Microsoft Fabric
Chennai, IN, 600032
INR 1500K-2800K (estimate) Senior-level Full Time
Tasks
- Build Microsoft Fabric Lakehouse solutions
- Build lakehouse Bronze Silver Gold data models
- Create runbooks and deployment guides
- Define data quality rules null duplicate referential cross field
- Design scalable data pipelines
- Develop Azure Databricks notebooks jobs workflows
- Develop Fabric SQL semantic models
- Develop Fabric data pipelines
- Develop batch ingestion pipelines
- Develop incremental ingestion pipelines
- Implement CI CD for notebooks pipelines SQL scripts
- Implement Delta Lake features
- Implement ELT/ETL workflows
- Implement data quality validation reconciliation exception handling
- Implement governance and access controls
- Implement partitioning caching indexing
- Integrate Microsoft Purview
- Integrate Power BI
- Maintain data lineage metadata documentation
- Migrate data platforms to Microsoft Fabric
- Monitor production jobs and resolve incidents
- Optimize Spark jobs
- Orchestrate data pipelines in Fabric
- Prepare support documentation
- Troubleshoot pipeline failures and bottlenecks
- Use Git version control and deployment automation
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | CSV | Data Ingestion | Data Lake Storage | Data Lineage | Data Modeling | Data Quality | Data Reconciliation | Data Transformation | Data Validation | Databricks | Delta | Delta Lake | Dimensional Modeling | Git | JSON | Lakehouse Architecture | Medallion Architecture | Metadata Management | Microsoft Fabric | Microsoft Purview | OneLake | Parquet | Power BI | PySpark | Python | SQL | Semantic models | Spark SQL | Synapse Analytics | XML
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Staff Engineer, Data Engineering INR 1500K-2500KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeBengaluru, KA, India18h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAPI Integration | AWS | Cloud Native | Documentation | GitMid-level Full TimeBengaluru, KA, India18h ago
-
Lead - Data Engineering INR 3000K-4000KData Governance | Data Modeling | Data Quality | Data Visualization | Data WarehousingSenior-level Full TimeBengaluru, Karnataka, India, India19h ago
-
Mid-level Full TimeIN-TN-Chennai19h ago
-
Streamlit & Databricks API INR 1500K-2500KAgile | CI/CD | Databricks | Databricks Apps | Databricks NotebooksSenior-level Full TimeIN-TN-Chennai19h ago
-
Data Engineer with GCP INR 1500K-2500KBatch Processing | BigQuery | Cloud Storage | Cloud platform | Data GovernanceSenior-level Full TimeIN-AP-Hyderabad R19h ago
-
Agentic AI | Alerting | Anomaly Detection | Apache Airflow | Artifact RegistryMid-level Full TimePune, Maharashtra, India R20h ago
-
Agentic AI Data Engineer INR 2000K-5000KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon BedrockSenior-level Full TimePune, Maharashtra, India20h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Airflow | Amazon Aurora | Apache Kafka | Apache SparkBike to work | Career mentorship | Educational assistance | Employee assistance program | Employee discountSenior-level Full TimeMumbai, MH, India R20h ago
-
API Development | Alerting | Batch Processing | Cloud Computing | ComplianceSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Data Engineer, YouTube INR 1000K-2000KApache Flume | Cloud Dataflow | Data Analysis | Data Architecture | Data GovernanceMid-level Full TimeBengaluru, Karnataka, India21h ago
-
Senior Software Engineer, AI/ML, Search INR 4000K-6000KCode review | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Azure Data Engineer INR 1500K-2500KAzure Data | Azure Data Factory | Azure Data Lake | Azure Functions | Azure SynapseSenior-level Full TimeHyderabad, Telangana, India R21h ago
-
Agentic AI- Senior Software Engineer INR 2000K-5000KAWS | Agent systems | Agents SDK | Autogen | AzureSenior-level Full TimeNoida, Uttar Pradesh, India21h ago
-
Software Engineer Data Analytics INR 1500K-2362KAPI Integration | Agile | AppSheet | Apps Script | BigQueryMid-level Full TimeHyderabad, India21h ago
-
Senior Software Engineer - AI Tools INR 3715K-5449KAWS | Amazon EKS | Ansible | Atlassian Jira | BrunoFamily well-being benefits | Health benefits | Paid time off | Work-life balanceSenior-level Full TimeHyderabad, India22h ago
-
Senior Data Quality Engineer INR 3000K-4000KAmazon Redshift | Anomaly Detection | Apache Airflow | BigQuery | CCPACompany-paid certifications | Financial consultations | Language courses | Legal consultations | Medical insuranceSenior-level Full TimeIndia22h ago
-
QA Lead – Data Engineering & ETL INR 1500K-2500KAgile | Apache Airflow | Automation Testing | Azure | CI/CDCareer development | Flexible work hours | Inclusive and collaborative work culture | Insurance | Learning opportunitiesSenior-level Full TimeBangalore, India R22h ago
-
Senior-level Full TimePune, Maharashtra, India R23h ago
-
AI Engineer INR 6000KAPI Integration | Backend Development | Databases | Debugging | DeploymentInternational Internship Certificate | Letter of recommendation | Long-term employment opportunity | MentorshipEntry-level Full Time InternshipMumbai, MH, India1d ago
-
Sr. ML Engineer INR 3000K-4000KAPI Development | CI/CD | Cloud Native | Context engineering | Document ingestionAnnual leave | Employee assistance program | Employee wellbeing platform Unmind | Flexible working | Learning and development opportunitiesSenior-level Full TimeChennai, IN1d ago
-
Manager - Data Analyst INR 700K-1200KAgile | Dataiku | ETL | Forecasting | Language ProcessingAnnual leave | Continuous learning culture | Digital wellbeing platform | Employee assistance program | Flexible workingMid-level Full TimeChennai, IN1d ago
-
Data Engineer - Python and Azure Data Bricks INR 1500K-2500KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeCareer growth | Diversity and inclusion | Inclusive cultureMid-level Full TimeIndia - Bengaluru; India - Chennai …1d ago
-
Specialist, Gen AI Solutions INR 3000K-4000KAML | Anomaly Detection | CI/CD | Classification | ClusteringSenior-level Full TimeBangalore, IN1d ago
-
Python,Pyspark Developer INR 2800K-4000KAgile | Data Engineering | Data Processing | Jira | PMI-ACPAnnual leave | Employee assistance program | Flexible working | Life insurance | Medical insuranceSenior-level Full TimeChennai, IN1d ago