Azure Databrick with Anaplan
Tasks
- Build Delta Lake medallion architecture bronze silver gold
- Build scalable data ingestion workflows
- Collaborate on data architecture and design patterns
- Conduct code reviews for coding standards and data quality
- Design, develop, and optimize ETL pipelines
- Develop reusable Databricks notebooks and jobs
- Implement data governance security and access control
- Implement data transformation cleansing enrichment validation
- Integrate Databricks with Azure services
- Optimize Spark jobs performance and cost
- Orchestrate pipelines with Azure Data Factory
- Prepare technical documentation and runbooks
- Troubleshoot production issues root cause analysis
Perks/Benefits
- 401k
- Employee assistance program
- Life and disability insurance
- Medical, dental, and vision coverage
- Paid Holidays
- Paid time off
- Retirement savings plan
- Sick leave
Skills/Tech-stack
ADLS Gen2 | Auto Scaling | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Cleansing | Data Enrichment | Data Factory | Data Ingestion | Data Lake Storage Gen2 | Data Processing | Data Validation | Databricks Jobs | Databricks Notebooks | Delta Lake | Distributed data | Distributed data processing | ETL | Event Hubs | Git | Key Vault | Medallion Architecture | PySpark | Spark | Spark SQL | Spark clusters | Synapse Analytics
Education
N/A
Related jobs
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R7h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R7h ago
-
Senior DevOps Engineer ID63545 USD 135K-185KAWS | Apache Airflow | ArgoCD | Azure | BigQueryFlextime | Growth roadmaps | Mentorship | Office work options | Remote work optionsSenior-level Full TimeMiami, United States9h ago
-
BigQuery | Curriculum Development | Data Governance | Data Lakehouse | Data ModelingAsynchronous hiring process | Part-time hoursSenior-level Part TimeBoston, US10h ago
-
Senior Director, AI / Machine Learning Software Engineer USD 136K-300KApache Flink | Apache Spark | CI/CD | Data Lineage | Data PrivacyHealth benefits | Paid leave | Paid volunteer timeSenior-level Full TimeNew York, NY, United States10h ago
-
C++ | Data Analysis | Data Manipulation | Data Processing | Deep learningSenior-level Full TimeMountain View, CA, USA12h ago
-
Software Engineer, Machine Learning USD 207K-300KC++ | Data Processing | Experimentation | Information Retrieval | Just-in-TimeSenior-level Full TimeNew York, NY, USA; Mountain View, …12h ago
-
Customer Engineer, Data Analytics, Google Cloud USD 153K-222KBatch Processing | Big Data | Cloud Architecture | Cloud platform | Customer RequirementsSenior-level Full TimeSunnyvale, CA, USA12h ago
-
C++ | Data Processing | Debugging | Information Retrieval | Language ModelsSenior-level Full TimeMountain View, CA, USA12h ago
-
Cloud Data and AI Engineer, Professional Services USD 127K-183KC++ | Capacity Planning | Cloud Databases | Data Migration | Data PipelinesTravel up to 30 percentMid-level Full TimeReston, VA, USA12h ago
-
Staff Software Engineer, ML Frameworks USD 207K-300KAPIs | Data Processing | Debugging | Fine Tuning | GPU AccelerationSenior-level Full TimeMountain View, CA, USA12h ago
-
Data Scientist USD 67K-150KA/B | A/B Testing | B testing | Clustering | Drift monitoring401k plan | AD and D insurance | Child Life Insurance | Dental insurance | Educational Assistance PlanMid-level Full TimeUnited States14h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeDiversity and Inclusion Commitment | Flexible asynchronous culture | Laid-back atmosphere | Portfolio and LinkedIn supportMid-level Full TimeBellevue, WA, USA17h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management | Remote-friendly, distributed teamMid-level Full TimeDenver, CO, USA17h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudMid-level Full TimePalo Alto, CA, USA18h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible work environment | Remote-friendlyMid-level Full TimeLos Angeles, CA, USA18h ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Flexible remote workMid-level Full TimeKirkland, WA, USA18h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible remote work | Work on impact driven productsMid-level Full TimeSan Jose, CA, USA18h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Friendly work environment | Portfolio and LinkedIn submission flexibility | Remote-friendly environmentMid-level Full TimeCupertino, CA, USA18h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Laid-back atmosphere | Portfolio supportMid-level Full TimeOakland, CA, USA18h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Bonus | Competitive salary | Equity | Laid-back atmosphereMid-level Full TimeSilver Spring, MD, USA18h ago
-
Bash | Cloud infrastructure | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible management | Friendly work environment | Portfolio supportMid-level Full TimeMadison, WI, USA18h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Laid-back atmosphere | Remote/distributed workMid-level Full TimeChapel Hill, NC, USA18h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Opportunity to make impactMid-level Full TimeNashville, TN, USA18h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Portfolio support | Remote-friendlyMid-level Full TimeOrlando, FL, USA18h ago