Sr. Databricks Azure Data Engineer
Tasks
- Build CI/CD automated testing for data pipelines
- Design data solutions on Spark and Flink
- Develop Python SDK for Jupyter Notebooks
- Document solutions and workflows
- Enforce data governance and security practices
- Implement data quality rules
- Maintain CI/CD workflows
- Manage and optimize Azure and Databricks resources
- Monitor and optimize pipelines
- Participate in agile activities
- Transform and prepare data using SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Apache Spark internals | Azure AD | Azure DevOps | CI/CD | Compliance | Data Design | Data Modeling | Data Warehousing | Databricks | Dataframes | Deequ | Delta Lake | Encryption | GitHub | Great Expectations | JUnit | PySpark | Pytest | Python | Spark internals
Education
Roles
Regions
Countries
States
Related jobs
-
Site Reliability Engineer USD 150K-190KAWS | Alerting | CI/CD | Cause analysis | Data PipelinesEquity | Growth-focused environment | Health coverage | Paid leaveMid-level Full TimeNepal2d ago
-
Data Engineer USD 112K-149KAWS | Cloud services | Databricks | ETL | GitLabEquity | Growth opportunities | Health coverageMid-level Full TimeNepal11d ago
-
Data Engineer USD 112K-149KCI/CD | Cloud services | Data Warehousing | Databricks | ETLEquity | Growth opportunities | Health coverageMid-level Full TimeNepal12d ago
-
Senior Data Engineer USD 139K-247KAWS | CI/CD | Data Modeling | Databricks | Delta LakeEquity | Growth environment | Health coverageSenior-level Full TimeNepal12d ago
-
Senior Data QA Engineer USD 120K-155KAPI Testing | AWS | Athena | Automation frameworks | Big DataEquity | Growth environment | Health coverageSenior-level Full TimeNepal17d ago
-
Software Engineer II - Data QA USD 91K-130KAWS | Agile | Automation | Data Analysis | Data ValidationEquity | Growth environment | Health coverageMid-level Full TimeNepal23d ago
-
Senior Data Engineer USD 152K-204KAPIs | AWS | Airbyte | Batch Data Processing | Batch dataLife insurance | Medical cover | Office snacks | Professional growth | Remote workSenior-level Full TimeKathmandu, Bagmati Province, Nepal26d ago
-
Data Engineer USD 120K-177KAWS | Airflow | Avro | BigQuery | CachingEquity | Growth environment | Health coverageMid-level Full TimeNepal1mo ago
-
Senior-level Full TimeNepal1mo ago