IN_Senior Associate_Data Engineer_Application Technology_Advisory_Kolkata
Kolkata Y-14, India
INR 2040K-3500K (estimate) Senior-level Full Time
Tasks
- Automate Databricks workspace deployment using Databricks Asset Bundles
- Build and optimize ETL/ELT pipelines
- Configure CI/CD pipelines using Azure DevOps and GitHub Actions
- Design Bronze Silver Gold data architecture
- Develop modular Python libraries for data ingestion and transformation
- Enforce unit testing linting and error handling standards
- Ensure audit logging run tracking and change traceability
- Implement data quality checks, validation, and performance tuning
- Implement deployment promotion workflows across dev test and prod
- Integrate with databases and message queues
- Manage Databricks Unity Catalog lineage and access control lists
- Manage secrets key vault integrations and network security
- Package code into custom wheels manage dependencies
Perks/Benefits
Skills/Tech-stack
Asset bundles | Azure Bicep | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure Synapse | Azure Synapse Analytics | Azure Virtual | Azure Virtual Network | CI/CD | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Quality | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks Repos | Delta Lake | Delta Live Tables | Delta Live) | Event Hubs | Git | GitHub Actions | Infrastructure as Code | Key Vault | Linters | Networking Private Link | Private Link | PySpark | Pytest | Python | Schema evolution | Spark | Synapse Analytics | Terraform | Unit Testing | Unity Catalog | Virtual Network | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology | Master of Engineering | Master of Science
Roles
Related jobs
-
Agentic AI | Amazon Web Services | Azure | Cloud Computing | Cloud platformAccess to modern cloud and AI tooling ecosystems | Exposure to cutting-edge AI technologies | Long-term contract | Opportunity to shape technical strategy | Remote work flexibilitySenior-level Full TimeIndia20h ago
-
Airflow | Apache Spark | CI/CD | Contextual Bandits | DatabricksCollaborative environment | Continuous learning | Inclusive culture | Professional growth | Remote-friendlySenior-level Full TimeIndia20h ago
-
Data Engineer ID56374 INR 2000K-2500KAgentic AI | Airflow | Apache Spark | BigQuery | Bitemporal DataEducation budget | Growth budget | Innovative projects | Legal presence in India | Local team connectionSenior-level Full TimeTrivandrum, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | CI/CD | Cloud ComposerEducation budget | Innovative projects | Local connection | Remote work | Wellness budgetSenior-level Full TimeIndore, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | Bitemporal Data | Bitemporal data modeling | CI/CDDedicated budgets for growth | Education budget | Innovative projects | Local connection | Periodic meet upsSenior-level Full TimeKolkata, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KAirflow | Apache Spark | BigQuery | Bitemporal Data | Bitemporal data modelingEducation budget | Growth budget | Innovative projects | Legal presence in India | Local connectionSenior-level Full TimeJalalabad, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | Bitemporal Data | Bitemporal data modelingBudgets for education | Budgets for growth | Innovative projects | Legal presence in India | Local connectionSenior-level Full TimeKochi, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Delta | Apache Delta Lake | Apache Spark | BigQuery cost managementEducation budget | Growth budget | Innovative projects | Legal presence in India | Local connection meetupsSenior-level Full TimePune City, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KAgentic AI | Apache Airflow | Apache Spark | BigQuery | CI/CDDedicated budget for education | Dedicated budget for growth | Dedicated budget for wellness | Local connection | Periodic meetupsSenior-level Full TimeBangalore North, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Growth budget | Innovative projects | Legal compliance in India | Local connectionSenior-level Full TimeMumbai, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KAgentic AI | Apache Airflow | Apache Spark | BigQuery | Bitemporal DataBudgets for growth and education | Innovative projects | Legal presence in India | Local connection meetups | Remote workSenior-level Full TimeAhmedabad, India23h ago
-
Salesforce AI Integration Architect ID68517 INR 2000K-4500KAgent Orchestration | Apex | Authentication | Context Management | Data GovernanceLocal connection | Remote workSenior-level Full TimeIndore, India23h ago
-
Salesforce AI Integration Architect ID68517 INR 2000K-4500KAgent Orchestration | Apex | Authentication | Context Management | Data GovernanceEducation budget | Growth budget | Innovative projects | Local connection meet ups | Local legal complianceSenior-level Full TimeNew Delhi, India23h ago
-
Salesforce AI Integration Architect ID68517 INR 2000K-4500KAgent Orchestration | Apex | Authentication | Context Management | Data GovernanceEducation budget | Growth budget | Innovative projects | Local connection | Periodic meet upsSenior-level Full TimeMumbai, India23h ago
-
Salesforce AI Integration Architect ID68517 INR 2000K-4500KAgent Orchestration | Apex | Authentication | Authorization | Context ManagementDedicated growth education budget | Global clients | Innovative projects | Legal compliance in India | Local connection meet upsSenior-level Full TimeBangalore North, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Growth budget | Innovative projects | Local connection | Periodic meet upsSenior-level Full TimeChennai, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureDedicated budgets for growth | Education budget | Innovative projects | Local connection | Periodic meet upsSenior-level Full TimeGurgaon Kty., India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | Cloud Composer | Cloud Monitoring | Cloud PubSubEducation budget | Growth budget | Innovative projects | Local connection | Periodic meetupsSenior-level Full TimeNew Delhi, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KAirflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureEducation budget | Growth budget | Innovative projects | Legal presence in India | Local team meetupsSenior-level Full TimeNoida, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureDedicated budget for education | Dedicated budget for growth | Innovative projects | Local connection | Periodic meetupsSenior-level Full TimeHyderabad, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KAirflow | Apache Spark | BigQuery | Bitemporal Data | Bitemporal data modelingEducation budget | Legal presence in India | Periodic meetups | Remote work | Wellness budgetSenior-level Full TimeCoimbatore South, India23h ago
-
Data Engineer ID56374 INR 2000K-2500KApache Airflow | Apache Spark | BigQuery | Bitemporal Data | Bitemporal data modelingEducation budget | Growth budget | Innovative projects | Local connection | Periodic meet upsSenior-level Full TimeJaipur, India23h ago
-
Generative AI / Sr. Associate Director, Tech SME INR 1400K-2200KAWS | Active Directory | Ansible | Azure | CI/CDMid-level Full TimePune, Maharashtra, India R1d ago
-
Lead Data Scientist INR 2800K-4000KAWS | Airflow | Azure | BERT | CI/CDHealth insurance | Paid parental leave | Paid sick leave | Work from OfficeSenior-level Full TimeBengaluru1d ago
-
Senior-level Full TimeIN-KA-Bangalore1d ago