Databricks Reporting Architect
Tasks
- Build PySpark and Spark SQL transformations
- Build monitoring and validation frameworks
- Build self service dashboards and reports
- Coordinate OData and API based integrations
- Create data quality validation pipelines with Delta Live Tables
- Design SAP to Databricks data flow
- Design physical dimensional star schema models
- Develop Gold consumption and reporting layer
- Enforce schema and incremental processing
- Evaluate Databricks MLflow and AutoML capabilities
- Implement Delta Lake ACID data storage
- Implement Medallion Architecture pipelines
- Implement Unity Catalog data governance
- Implement Z Ordering partitioning and liquid clustering
- Leverage AI coding tools for development
- Optimize Delta Lake storage for query performance
- Perform root cause analysis for data discrepancies
- Set up data lineage access control and auditing
- Support SAP and middleware integration pipelines
- Support testing validation and deployment
Perks/Benefits
- 401k
- Dental plans
- Employee assistance program
- Life insurance
- Medical plans
- Paid Company Holidays
- Paid time off
- Parental leave
- Tuition reimbursement
- Vision plan
Skills/Tech-stack
ACID | API Integration | Analytics Cloud | Apache Spark | Automl | Column-Level Security | Data Governance | Data Lineage | Data Quality | Databricks | Databricks SQL | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | Incremental processing | Liquid clustering | Machine Learningflow | OData | Partitioning | Power BI | PySpark | Python | Row Level Security | SAP Analytics | SAP Analytics Cloud | SQL | Schema Enforcement | Star Schema | Tableau | Time Travel | Unity Catalog | Z-ordering
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R27d ago
-
Agent Orchestration | Architecture patterns | Artificial Intelligence | Cloud Architecture | Computer Vision401k program | Car discounts | Cruise discounts | Dental insurance | Flexible spending accountsSenior-level Full TimeFort Worth, TX, US7h ago
-
Manager, Data Engineering USD 140K-185KAzure Data | Azure Data Bricks | Boome | CI/CD | DBT401k match | Dental insurance | Dependent care spending account | EAP | Flexible Friday early log offMid-level Full TimeRemote (United States) R15h ago
-
Staff Data Architect USD 170K-210KAgentic AI | Apache Spark | Automated testing | Big Data | Compliance401k match | Commuter benefits | Dental insurance | Disability insurance | Family planning supportSenior-level Full TimeNew York City16h ago
-
VP, Data Platform Engineering USD 366K-400KAmazon DynamoDB | Amazon Kinesis | Amazon Redshift | Amazon Web Services | Apache Airflow401k plan | Flexible spending accounts | Health savings accounts | Insurance (medical, dental, vision) | Life insuranceExecutive-level Full TimeSan Francisco, CA16h ago
-
VP, Data Platform Engineering USD 330K-360KAWS | Airflow | Amazon DynamoDB | Amazon Kinesis | Amazon Redshift401k plan | Employee life insurance | Flexible spending account | Health savings account | Medical, dental, and vision insuranceExecutive-level Full TimeLos Angeles, CA16h ago
-
Staff Partner Engineer, AI Partnerships USD 150K-206KAI integration | Agent Frameworks | Architecture | Cost Optimization | EvaluationGlobal travelSenior-level Full TimeNew York City, New York; San …16h ago
-
Data Platform Architect USD 137K-287KAccess Control | Azure Synapse | Code repositories | Data Governance | Data LineageSenior-level Full TimeUS-CA-Fremont (1003)17h ago
-
Principal Applied AI Solutions Architect USD 165K-216KAPI Development | AWS | AWS SageMaker | Amazon EMR | Amazon Redshift401k match | Dental insurance | Holidays | Home office & equipment stipend | Learning and development stipendSenior-level Full TimeUS - Remote R22h ago
-
Applied AI Solutions Architect USD 156K-209KAWS Bedrock | AWS SageMaker | Agentic Architectures | Anthropic | Azure AI401k company match | Dental insurance | Home office & equipment stipend | Learning and development stipend | Paid HolidaysSenior-level Full TimeUS - Remote R23h ago
-
Solutions Architect - Real Money Gaming USD 180K-247KAWS | Architecture | Artificial Intelligence | Azure | Cloud ComputingSenior-level Full TimeUnited States23h ago
-
Director, Data Engineering USD 175K-235KAPI | AWS | Apache Airflow | Apache Kafka | Apache Spark401k matching | Accidental death and dismemberment | Company holidays | Dental insurance | Employee stock purchase programExecutive-level Full TimeNeedham, MA, United States1d ago
-
Snowflake Lead / Architect USD 168K-216KAmazon Web Services | Automation | CI/CD | Data Governance | ELTSenior-level Full TimeCumberland, RI, United States1d ago
-
Apache Airflow | Apache Flink | Apache Hive | Apache Kafka | Apache SparkFast-paced work environment | Hands-on project experience | MentorshipEntry-level InternshipSan Jose, California, United States1d ago
-
Senior-level Full TimeNew York City, NY, United States1d ago
-
Data Engineering Leader, Analytics USD 174K-250KBias Mitigation | Business Intelligence | Data Architecture | Data Modeling | Data PipelinesSenior-level Full TimeMenlo Park, CA | New York, …1d ago
-
Cloud Architecture | Cost Optimization | Data Pipelines | Distributed Training | Fine TuningSenior-level Full TimeReston, VA, USA; Boulder, CO, USA1d ago
-
Senior Data Engineering & Analytics Lead USD 133K-170KBusiness Intelligence | Data Governance | Data Modeling | Data Pipelines | Data Quality401k match | Health insurance | Periodic in person team meetings | Profit sharing | Remote work flexibilitySenior-level Full TimePomona, California, United States - Remote R1d ago
-
Engineering Team Lead- Data & AI Platform USD 175K-200KAWS | Amazon EMR | Amazon S3 | Apache Airflow | Apache IcebergSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL1d ago
-
Principal AI Solutions Advisor USD 138KALM | AWS | Access Management | Agent Strategy | Artificial Intelligence401k | Free therapy visits | Hybrid work | Medical insurance | Meditation resourcesSenior-level Full TimeMerrimack, NH, United States1d ago
-
AI Solutions Architect USD 140K-203KAI machine learning | API Design | Access Management | Application Architecture | Architecture DecisionCareer Development Programs | Part-time telework | Relocation assistance | Robust benefitsSenior-level Full TimeReston, VA, US1d ago
-
AWS Glue | AWS Lambda | AWS S3 | Agile | AirflowSenior-level Full TimeTX, Austin, 11501 Alterra Pkwy STE …1d ago
-
Cybersecurity Data Architect USD 132K-195KAccess Control | Cloud Security | Data Architecture | Data Governance | Data LineageMust live in Texas | Remote workSenior-level Full TimeRemote - TX, United States R1d ago
-
Applied AI Specialist USD 171K-200KAPI Integration | Anomaly Detection | Cloud Native | Cloud Native Architecture | Computer VisionContinued Career Development | Employee resource groups | Flexible WFH policy | Generous PTO policy | Mental and physical wellness programsSenior-level Full TimeUS-Washington-Remote, United States R1d ago
-
Block Storage | DBaaS | Data Architecture | Data Backup | Data Governance401k retirement plan | Disability insurance | Life insurance | Medical, dental & vision coverage | Paid HolidaysSenior-level Full TimeLocation(s): Irving, Texas, United States1d ago