Senior Data Engineer - Azure Databricks
Hyderabad, TS, India
INR 300K-500K (estimate) Senior-level Full Time
Tasks
- Build ETL/ELT workflows
- Build and manage Delta Lake tables
- Collaborate with data science teams
- Deploy solutions to production
- Design scalable data pipelines
- Develop SQL transformations
- Develop data ingestion pipelines
- Ensure data quality, validation, and monitoring
- Implement data security access control governance
- Integrate Databricks with Azure Data Lake Storage
- Optimize Spark jobs
- Process structured and semi-structured data
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | CSV | Data Factory | Data Lake Storage Gen2 | Data Modeling | Delta Lake | Distributed Computing | ELT | ETL | Git | JSON | Medallion Architecture | Parquet | PySpark | Python | SQL | Schema evolution | Time Travel
Education
N/A
Roles
Related jobs
-
Assistant Manager Machine Learning INR 1500K-1809KA/B | A/B Testing | AWS Glue | AWS Lambda | AWS SageMakerMid-level Full TimeIndia-Gurugram5h ago
-
Senior-level Full TimeKochi, India5h ago
-
Senior-level Full TimeIN-TN-Chennai5h ago
-
Associate Process Manager INR 1800K-2800KAgentic AI | Agile methodologies | Causal Inference | Cypher | Experimental DesignMid-level Full TimeMumbai, Maharashtra, India7h ago
-
Partner Engineer, Generative AI INR 2500K-3000KAI infrastructure | AWS | Azure | C plus plus | Cloud platformSenior-level Full TimeBangalore, India | Mumbai, India7h ago
-
Entry-level Full TimePune, India7h ago
-
Principal Consultant - Data & AI INR 2500K-3200KAI Services | Agile | Apache Kafka | Apache Spark | Artificial IntelligenceSenior-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN; …7h ago
-
AI Engineer INR 1340K-2400KAI Safety | AWS | AWS Lambda | Adversarial AI | Amazon BedrockDedicated learning week | Employee Share Savings Plan | Hybrid work | Learning and mentoring | Paid volunteer dayMid-level Full TimeBengaluru7h ago
-
Agentic AI/ Prompt Engineer INR 1800K-3000KCloud Platforms | Embeddings | LLMs | Langchain | LlamaindexMid-level Full TimeNoida, Uttar Pradesh, India7h ago
-
AWS | Automated testing | Azure | CI/CD | Data PipelinesMid-level Full TimeMumbai, Maharashtra, India8h ago
-
Senior Data Engineer INR 2500K-2829KApache Airflow | Apache Spark | Azure | Azure Data | Azure Data LakeEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India8h ago
-
Assistant Manager- Python Gen AI with GCP- Bangalore INR 1200K-1700KAI Agents | API Development | Agile methodologies | Application Performance Optimization | Application performanceWork from client office 5 days per weekMid-level Full TimeBangalore, Karnataka, India8h ago
-
Software Engineer III - Data Engineer INR 1200K-2000KAWS | Apache Airflow | Azure | Big Data | Data GovernanceSenior-level Full TimeMumbai, Maharashtra, India8h ago
-
Senior Associate -Applied AI ML -Digital INR 1050K-1496KAlgorithms | Apache Spark | Big Data | Classification | CompilersMid-level Full TimeBengaluru, Karnataka, India9h ago
-
Sr. SW Engineer - SDET - Automation, GenAI tooling, API Testing INR 2156K-3000KAI | AI Agents | API Testing | CI/CD | Cloud NativeHybrid workSenior-level Full TimeBengaluru, INDIA, India9h ago
-
IT Support Specialist 1 (Regional) INR 2000K-2000KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure DevOpsSenior-level Full TimeBengaluru, KA, IN9h ago
-
Staff Software Engineer - Python, AI/ML, Cloud, GenAI INR 2000K-5000KAWS | Agent Orchestration | Azure | CI/CD | ContainerizationSenior-level Full TimeBengaluru, INDIA, India9h ago
-
AI Engineer- Associate Consultant INR 1500K-2000KAWS | Agent Orchestration | Autonomous Agents | Azure | ClassificationCareer growth | Collaborative work environment | Continuous learning | Inclusive work environmentMid-level Full TimeBangalore, Karnataka, India10h ago
-
Data Scientist Consultant INR 2500K-3000KAgent Orchestration | Agentic AI | Azure | Classification | ClusteringSenior-level Full TimeBangalore, Karnataka, India10h ago
-
Software Engineer III - AIML Platform Reliability INR 2500K-3500KAI Operations | AI Ops | AWS | Automation | Cloud ComputingSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Executive-level Full TimeIndia10h ago
-
Software Engineer I - Python, AWS INR 2200K-3600KAWS | AWS IAM | Agile | Amazon CloudWatch | Amazon EMRSenior-level Full TimeMumbai, Maharashtra, India10h ago
-
Senior Database Engineer (3+ years, Aurora, Dynamo, Redis) INR 1244K-2200KAmazon Aurora | Amazon Aurora MySQL | Amazon Aurora PostgreSQL | Amazon Web Services | Aurora MySQL24x7 support rotation | Hybrid workSenior-level Full TimeBengaluru, INDIA, India11h ago
-
AI Senior Engineer INR 2520K-4500KAgent Orchestration | Amazon Web Services | Autogen | Azure | Cloud platformSenior-level Full TimePune, MH, IN11h ago
-
AI/ML Engineer INR 1500K-2000KAzure | Data Pipelines | DeepSpeed | Distributed Systems | DockerCar lease | Career advancement programs | Certification programs | Corporate NPS | Dental coverageMid-level Full TimeBangalore, Karnataka, India12h ago