Azure Databrick with Anaplan
Tasks
- Build Delta Lake medallion architecture bronze silver gold
- Build scalable data ingestion workflows
- Collaborate on data architecture and design patterns
- Conduct code reviews for coding standards and data quality
- Design, develop, and optimize ETL pipelines
- Develop reusable Databricks notebooks and jobs
- Implement data governance security and access control
- Implement data transformation cleansing enrichment validation
- Integrate Databricks with Azure services
- Optimize Spark jobs performance and cost
- Orchestrate pipelines with Azure Data Factory
- Prepare technical documentation and runbooks
- Troubleshoot production issues root cause analysis
Perks/Benefits
- 401k
- Employee assistance program
- Life and disability insurance
- Medical, dental, and vision coverage
- Paid Holidays
- Paid time off
- Retirement savings plan
- Sick leave
Skills/Tech-stack
ADLS Gen2 | Auto Scaling | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Cleansing | Data Enrichment | Data Factory | Data Ingestion | Data Lake Storage Gen2 | Data Processing | Data Validation | Databricks Jobs | Databricks Notebooks | Delta Lake | Distributed data | Distributed data processing | ETL | Event Hubs | Git | Key Vault | Medallion Architecture | PySpark | Spark | Spark SQL | Spark clusters | Synapse Analytics
Education
N/A
Related jobs
-
Software Engineer, ML Data Infrastructure USD 160K-220KAlgorithms | Cloud platform | Data Pipelines | Data Processing | Data StorageAccess to compute resources | Dental insurance | Health insurance | In-office meals | Learning and mentorshipEntry-level Full TimeToronto3h ago
-
Algorithm Engineer, Autonomy Planning & Prediction USD 150K-200KC++ | CI/CD | Code review | Data Engineering | DebuggingMid-level Full TimeHouston, TX or San Francisco Bay …8h ago
-
Amazon Redshift | Data Modeling | Data Pipelines | ETL | ExperimentationMid-level Full TimeBellevue, Washington, USA9h ago
-
Enterprise AI Platform Engineer USD 130K-150KAPI Gateway | AWS | AWS IAM | Audit Logging | Azure401k | Dental insurance | Flexible PTO | Growth opportunities | Medical insuranceSenior-level Full TimeAustin, TX10h ago
-
Software Engineer II, Machine Learning USD 145K-175KAWS | Airflow | CI/CD | Data Preparation | Experimentation401k employer match | Commuter subsidy | Concierge medical membership | Dental insurance | Fitness membership subsidyMid-level Full TimePalo Alto, California11h ago
-
Staff Data Engineer, Finance USD 170K-230KAWS | Asset-based lending | DBT | Data Ingestion | Data Modeling401k matching | Dental insurance | Employee assistance program | Employee recognition | Employee referral programSenior-level Full TimeSeattle, Washington, United States12h ago
-
Junior Data Engineer USD 64K-90KDatabricks | ELT | ETL | Git | JiraActive Secret security clearanceEntry-level Full TimeArlington, VA12h ago
-
Staff Data Engineer USD 180K-200KAWS | Airbyte | Airflow | Apache Iceberg | Automated testing401k match | Dental insurance | Disability insurance | Employee assistance program | Life insuranceSenior-level Full TimeWaltham, MA12h ago
-
Staff Data Engineer USD 180K-200KAPI Development | AWS | Airbyte | Airflow | Amazon Kinesis401k match | Dental insurance | Disability insurance | Employee assistance program | Health and wellness resourcesSenior-level Full TimeNorth Bethesda, MD12h ago
-
Senior Software Engineer, Embedded Systems USD 170K-210KC# | C++ | CI/CD | Docker | Embedded Systems401k | Dental insurance | Disability insurance | Employee stock options | Health insuranceSenior-level Full TimeGlen Cove, NY13h ago
-
Senior Embedded Firmware Engineer USD 120K-153KADC | Bare Metal | C plus plus | C# | Control AlgorithmsOwnership equity | Work-life balanceSenior-level Full TimeWilmington, MA13h ago
-
Senior Data Engineer USD 150K-228KApache Flink | Apache Spark | BigQuery | Data Architecture | Data QualityOn-site work flexibilitySenior-level Full TimeChicago, Illinois13h ago
-
Senior Data Engineer USD 165K-216KAWS | Apache Airflow | BI Integration | CI/CD | Cloud Computing401k match | Company events | Dental insurance | Employee discount | Free swagSenior-level Full TimeBeverly Hills13h ago
-
Staff Machine Learning Engineer USD 200K-220KAWS | Alerting | Auto Scaling | Automated retraining | CI/CD401k match | Dental insurance | Disability insurance | EAP support | Life insuranceSenior-level Full TimeWaltham, MA14h ago
-
Staff Machine Learning Engineer USD 200K-220KAWS | Artificial Intelligence | Automated retraining | CI/CD | Cloud Computing401k match | Bonding Leave | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeNorth Bethesda, MD14h ago
-
Data Engineer USD 81K-97KAWS | AWS Lambda | Apache Airflow | CI/CD | DBT401k | Flexible spending accounts | Health/vision/dental insurance | Hybrid work model | Paid disability benefitsMid-level Full TimeChicago - RAPP R14h ago
-
Senior AI Product Engineer, Frameworks USD 166K-225KAgentic Workflows | Anthropic API | Chunked HTTP | Context Management | Data Processing401k plan | Healthcare concierge services | Hybrid work model | In office Tuesday through Thursday | Life and disability insuranceSenior-level Full TimeHybrid - San Francisco R15h ago
-
Senior-level Full TimeChantilly, VA15h ago
-
AI Engineer USD 109K-140KAgent Orchestration | Computer Vision | Data Validation | Data extraction | Document ClassificationMid-level Full TimeMorristown, NJ, United States15h ago
-
Summer 2026 Data Engineer USD 41K-50KAPIs | Agile | Azure Data | Azure Data Factory | Azure Data LakeExposure to real-world projects | Learning and development opportunities | MentorshipEntry-level InternshipBoston, MA, United States15h ago
-
Summer 2026 Data Engineer USD 41K-50KAPIs | Agile | Azure Data | Azure Data Factory | Azure Data LakeMentorshipEntry-level InternshipBoston, MA, United States15h ago
-
AWS | Agent systems | Apache Spark | Apache Spark SQL | AzurePeriodic onsite work | Remote work option | Travel as neededSenior-level Full TimeMaryland; Virginia; Washington, D.C.15h ago
-
Senior Embedded Software Engineer USD 146K-196KARM Cortex | C# | Digital Signal | Digital Signal Processing | Embedded Linux401k match | Dental insurance | Employee assistance program | FSA | Flexible scheduleSenior-level Full TimeCamarillo, CA, United States15h ago
-
Data Engineer USD 126K-208KAPI Integration | Airflow | Amazon Web Services | BigQuery | CCPADEI initiatives | Dental benefits | Employee rewards program | Medical benefits | Mental health supportMid-level Full TimeRemote, United States R15h ago
-
Alerting | Ansible | Bash | CI/CD | CephRemote workSenior-level Full TimeUnited States, United States R17h ago