Sr. Databricks Azure Data Engineer
Tasks
- Build CI/CD automated testing for data pipelines
- Design data solutions on Spark and Flink
- Develop Python SDK for Jupyter Notebooks
- Document solutions and workflows
- Enforce data governance and security practices
- Implement data quality rules
- Maintain CI/CD workflows
- Manage and optimize Azure and Databricks resources
- Monitor and optimize pipelines
- Participate in agile activities
- Transform and prepare data using SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Apache Spark internals | Azure AD | Azure DevOps | CI/CD | Compliance | Data Design | Data Modeling | Data Warehousing | Databricks | Dataframes | Deequ | Delta Lake | Encryption | GitHub | Great Expectations | JUnit | PySpark | Pytest | Python | Spark internals
Education
Roles
Regions
Countries
States
Related jobs
-
Associate Data Engineer USD 100K-131KAgile | Data Pipelines | Databricks | ETL | PySparkEquity | Health coverage | Paid leaveMid-level Full TimeNepal1d ago
-
Data Engineer USD 105K-131KAWS | CI/CD | Data Warehousing | Databricks | ETLEquity | Health coverage | Leave benefitsMid-level Full TimeNepal2d ago
-
Senior Data Engineer USD 152K-247KAWS Glue | AWS Lambda | AWS S3 | AWS SQS | AirflowEquity | Health coverage | Leave benefits | Onsite work arrangement | Professional growth environmentSenior-level Full TimeNepal9d ago
-
Senior Data Engineer USD 139K-247KAWS Glue | AWS IAM | AWS Lambda | AWS S3 | AWS SQSEquity | Health coverage | Paid time offSenior-level Full TimeNepal10d ago
-
Senior Data Engineer - Nepal USD 30K-40KAWS Glue | AWS Lambda | AWS S3 | Airflow | AlertingNight shiftSenior-level Full TimeKathmandu, Bagmati, Nepal (Hybrid) R14d ago
-
Site Reliability Engineer USD 150K-190KAWS | Alerting | CI/CD | Cause analysis | Data PipelinesEquity | Growth-focused environment | Health coverage | Paid leaveMid-level Full TimeNepal22d ago
-
Data Engineer USD 112K-149KCI/CD | Cloud services | Data Warehousing | Databricks | ETLEquity | Growth opportunities | Health coverageMid-level Full TimeNepal1mo ago
-
Senior Data QA Engineer USD 120K-155KAPI Testing | AWS | Athena | Automation frameworks | Big DataEquity | Growth environment | Health coverageSenior-level Full TimeNepal1mo ago