Databricks Pipeline Architect
Work from home, VA, United States
R
USD 150K-180K Senior-level Full Time
Tasks
- Advise stakeholders on data engineering
- Build AWS based ETL pipelines
- Create guidance and documentation for internal teams
- Design Databricks architecture for enterprise governance
- Develop Databricks use case applications
- Integrate Databricks with analytics platforms
- Integrate Databricks with metadata solutions
- Lead Databricks pipeline development
- Operate Databricks long term
- Standardize and migrate data to unified data platform
- Unify enterprise data capabilities
Perks/Benefits
Skills/Tech-stack
AWS Glue | AWS Lambda | AWS S3 | Agile | Amazon Web Services | Apache Spark | Data Engineering | Data Governance | Databricks | Delta Lake | Delta Sharing | ETL | Lakehouse Architecture | Medallion Architecture | Python | Web Services
Education
N/A
Roles
Architect | Data Engineer | Databricks Pipeline Architect | Engineer
Related jobs
-
Tier 3 Network Systems Engineer (Remote) USD 80K-101KActive Directory | Ansible | Ansible Playbook | Apache HTTP | Apache HTTP ServerAfter hours availability | Customer support support | On-call rotation | Remote workMid-level Full TimeDallas, TX, US R5h ago
-
Senior Machine Learning Engineer USD 150K-200KDistributed Systems | Feature Engineering | Feature Selection | Language Models | Language Processing401k matching | Cell phone and internet stipend | Employee stock purchase plan | Flexible time off | Learning programsSenior-level Full TimeRemote - USA R16h ago
-
Senior Software Engineer - Data Platform USD 186K-218KAccess Control | Airflow | Apache Kafka | Apache Spark | CachingSenior-level Full TimeRemote - USA R18h ago
-
Senior-level Full TimeHybrid (Salt Lake City, UT, US) R20h ago
-
Lead Data Architect USD 173K-279KAWS Redshift | Apache Flink | Apache Kafka | Apache Spark | Artificial IntelligenceSenior-level Full TimeChicago, Illinois, USA R20h ago
-
Robotics Perception Engineer USD 140K-204KC++ | CI/CD | Camera-based perception | Cameras | Cloud processingCompany-provided laptop | Dental insurance | Health insurance | Paid time off | Remote work optionMid-level Full TimeAtlanta, Georgia, United States - Remote R21h ago
-
Robotics Hardware Integration Engineer USD 120K-158KAWS IoT | AWS IoT Core | AWS IoT Greengrass | C++ | CANDirect impact on shipped products | Early-stage company | Equity | Health, dental, and vision insurance | Permissive time off policyMid-level Full TimeAtlanta, Georgia, United States - Remote R21h ago
-
Robotics Navigation and Control Engineer USD 160K-240KA Star | C++ | CI/CD | EKF | GazeboCompany laptop | Health, dental, vision benefits | Permissive time off | Remote work | Travel to Atlanta facility for hardware integrationSenior-level Full TimeAtlanta, Georgia, United States - Remote R21h ago
-
Snowflake Data Architect (Remote) USD 135K-215KAWS | Access Control | Apache Airflow | Azure | CI/CDPaid adoption leave | Paid parental leave | Paid time off | Professional development | Remote workSenior-level Full TimeUSA CA Remote, United States R21h ago
-
Sr. Full Stack Engineer – AI/ML USD 151K-220KAPI Design | Alerting | Architecture Documentation | Artificial Intelligence | Capacity PlanningSenior-level Full TimeRemote Office - KY, United States R21h ago
-
Agentic AI Engineering Intern USD 48K-86KAgent systems | Autogen | Automated testing | CI/CD | CrewAIEntry-level Full Time Internship6314 Remote/Teleworker US, United States R21h ago
-
Staff Data & Analytics Engineer USD 112K-165KAPI Integration | Data Governance | Data Modeling | Data Quality | Data ValidationAdoption Assistance | Disability insurance | Employee assistance program | Health and wellness programs | Life insuranceSenior-level Full TimeEvendale, United States R21h ago
-
AI Engineer USD 150K-215KAgent Orchestration | Agent systems | AgentOps | Amazon Bedrock | Azure OpenAIMid-level Full TimeRemote United States R1d ago
-
Senior Data Engineer USD 146K-180KAWS | AWS CDK | Amazon Redshift | Apache Airflow | CircleCI401k matching | Disability insurance | Flexible PTO | Free therapy access for employees and one household member | Friends and family discountSenior-level Full TimeNew York, NY (Hybrid) R1d ago
-
Staff Software Engineer, Ads Data Platform USD 221K-279KApache Airflow | Apache Spark | Big Data | DBT | DatabricksAnnual equity refresh grants | Equity grants | Flexible work location | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Software Engineer, Data Platform USD 170K-205KAPIs | AWS | Amazon RDS | Data Integrity | Data Modeling401k matching | Dental insurance | Health & wellness stipend | Health insurance | Hybrid work environmentSenior-level Full TimeSan Francisco, CA (Hybrid), US (Remote) R1d ago
-
Senior Data Engineer USD 143K-229KAnalysis Services Tabular | Automation | Azure Data | Azure Data Factory | Azure DevOpsRemote workSenior-level Full TimeDenver, CO, United States R1d ago
-
Senior Data Engineer USD 143K-229KAnalysis Services Tabular | Azure Analysis | Azure Analysis Services | Azure Analysis Services Tabular | Azure DataMentorship | Remote work | Team collaborationSenior-level Full TimeKansas City, MO, United States R1d ago
-
Senior AI Solution Engineer USD 160KAI workflows | API Integration | Agentic Systems | Automation | Data CurationRemote work | Significant onsite travelSenior-level Full TimeRemote (United States) R1d ago
-
Machine Learning Engineer USD 175K-225KA/B | A/B Testing | B testing | Bias detection | BigQueryCompany holidays | Dental insurance | Flexible paid time off | Home office support | Learning and development budgetSenior-level Full TimeUS Remote R1d ago
-
Data Analyst USD 100K-140KA/B | A/B Testing | AI tools | AWS | Amazon RedshiftCareer growth support | Code review feedback | No on call expectations | Standard business hours | Work-life balanceMid-level Full TimeRemote (United States) R1d ago
-
Staff Software Engineer, Agentic Patient Outreach USD 175K-215KAPIs | AWS | AWS Glue | Apache Airflow | AuthenticationSenior-level Full TimeRemote, United States R1d ago
-
Data Engineer (Job ID #225) USD 117KAWS | CI/CD | Data Cleansing | Data Deduplication | Data Governance401k match | Dental insurance | Health insurance | Life insurance | Paid HolidaysMid-level Full TimeRemote, USA R1d ago
-
Staff Machine Learning Systems Engineer USD 230K-322KApache Beam | Apache Spark | BigQuery | Cloud Storage | Cloud platformSenior-level Full TimeRemote - United States R1d ago
-
Senior Machine Learning Systems Engineer USD 216K-303KApache Beam | Apache Spark | BigQuery | Cloud Storage | Cloud platformSenior-level Full TimeRemote - United States R1d ago