Azure Databrick with Anaplan
Tasks
- Build Delta Lake medallion architecture bronze silver gold
- Build scalable data ingestion workflows
- Collaborate on data architecture and design patterns
- Conduct code reviews for coding standards and data quality
- Design, develop, and optimize ETL pipelines
- Develop reusable Databricks notebooks and jobs
- Implement data governance security and access control
- Implement data transformation cleansing enrichment validation
- Integrate Databricks with Azure services
- Optimize Spark jobs performance and cost
- Orchestrate pipelines with Azure Data Factory
- Prepare technical documentation and runbooks
- Troubleshoot production issues root cause analysis
Perks/Benefits
- 401k
- Employee assistance program
- Life and disability insurance
- Medical, dental, and vision coverage
- Paid Holidays
- Paid time off
- Retirement savings plan
- Sick leave
Skills/Tech-stack
ADLS Gen2 | Auto Scaling | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Cleansing | Data Enrichment | Data Factory | Data Ingestion | Data Lake Storage Gen2 | Data Processing | Data Validation | Databricks Jobs | Databricks Notebooks | Delta Lake | Distributed data | Distributed data processing | ETL | Event Hubs | Git | Key Vault | Medallion Architecture | PySpark | Spark | Spark SQL | Spark clusters | Synapse Analytics
Education
N/A
Related jobs
-
Data Engineer- Consumer/Small Business USD 76K-125KAWS | AWS EC2 | AWS Glue | Adobe Analytics | Amazon AthenaMid-level Full TimeMiami Lakes, FL, United States5h ago
-
Senior Data Engineer USD 119K-180KANSI SQL | AWS S3 | Amazon Redshift | Azure Container | Azure Container AppsSenior-level Full TimeOrlando, FL, United States6h ago
-
API Integration | Asynchronous processing | Containerization | Data Pipelines | Deep learningSenior-level ContractAnnapolis, United States R8h ago
-
Data Engineer, Analytics USD 205K-235KData Governance | Data Modeling | Data Quality | Data Security | Data VisualizationSenior-level Full TimeBellevue, WA11h ago
-
Accelerated inference | Algorithms | Android | CPU | ChromeSenior-level Full TimeSunnyvale, CA, USA11h ago
-
ML Engineer, Query Understanding, Search Ads USD 174K-252KC++ | Data Processing | Data Storage | Debugging | Deep learningMid-level Full TimeMountain View, CA, USA11h ago
-
Customer Engineer, Data Analytics, Financial Services USD 153K-222KApache Spark | Batch Processing | Big Data | Cloud Computing | Cloud platformSenior-level Full TimeNew York, NY, USA; Atlanta, GA, …11h ago
-
BigQuery | Cloud platform | Data Processing | Deep learning | Distributed SystemsMid-level Full TimeKirkland, WA, USA11h ago
-
Staff Software Engineer, AI/ML, YouTube USD 207K-300KData Processing | Data Structures | Data Structures and Algorithms | Debugging | Distributed SystemsSenior-level Full TimeSan Bruno, CA, USA; Mountain View, …11h ago
-
Software Engineer III, AI/ML, Google Play USD 147K-211KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA11h ago
-
Senior Software Engineer, AI/ML, Payments Platform USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA11h ago
-
Staff Software Engineer, AI/ML Machine Repairs USD 207K-300KArtificial Intelligence | C++ | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA11h ago
-
Machine Learning Engineer, XLS, Compiler USD 174K-252KAI | C++ | Compiler Construction | Data Processing | Data StructuresMid-level Full TimeChicago, IL, USA11h ago
-
Senior ML/AI Software Engineer, Chrome AI USD 174K-252KAgent Orchestration | Algorithms | Automated testing | C++ | Computer VisionSenior-level Full TimeMountain View, CA, USA11h ago
-
Staff Software Engineer, AI/ML, Agent Assist USD 207K-300KAgentic AI | Cloud Computing | Data Processing | Debugging | Deep learningSenior-level Full TimeSunnyvale, CA, USA11h ago
-
Mid-level Full TimeIndianapolis, IN, 46254, USA12h ago
-
Sr. Data Engineer USD 131K-164KAgentic AI | DBT | Data Governance | Data Quality | Data SecurityConvenient location | Group insurance | Hospitalization insurance | Meal vouchers | Open company cultureSenior-level Full TimeUnited States, FL, US, 56007616h ago
-
Embedded Systems Engineer USD 100K-200KC# | C++ | CAN | Circuit design | Debugging Tools401k | Dental insurance | Disability insurance | Employee stock ownership plan | Employer paid medical insuranceSenior-level Full TimeGoleta, CA, US17h ago
-
Senior AI Engineer USD 170K-200KAWS | Agentic Workflows | Airflow | Data Pipelines | Data TransformationOnsite collaborationSenior-level Full TimeSeattle, Washington, United States19h ago
-
Senior AI Engineer USD 170K-200KAWS | Airflow | Anthropic | Datadog | EmbeddingsOnsite 4 days per weekSenior-level Full TimeLos Angeles, California, United States19h ago
-
Senior AI Engineer USD 170K-200KAWS | Agentic Workflows | Airflow | Anthropic | Data Pipelines401k plan | Bonus plans | Disability coverage | Healthcare benefits | Life insuranceSenior-level Full TimeNew York, New York, United States19h ago
-
Cloud Data Engineer USD 75K-135KAPI Gateway | AWS | AWS Glue | Amazon EC2 | Amazon EKS401k | Life insurance | Medical, dental, and vision | Paid leaveMid-level Full TimeBuilding 400-Whippany Campus, Jefferson Park, United …22h ago
-
Senior-level Full TimeChicago Office, United States22h ago
-
AI Engineer USD 175K-226KAPI | AWS | Asynchronous Communication | Authentication | AuthorizationHealth benefits | Learning and growth | Professional development | Remote work | Travel opportunitiesSenior-level Full TimeUSA - Remote, United States R22h ago
-
Lead Advanced Analytics USD 130K-215KADLS | Autogen | Azure | Azure Data | Azure Data Factory401k plan | Disability insurance | Employee assistance program | Health insurance | Life insuranceSenior-level Full TimeUSA:TX:Dallas / One AT&T Plaza (208 …22h ago