Lead Data Engineer (Databricks)
Tasks
- Architect for scale and reliability
- Coach through code review and pairing
- Create technical documentation and reference architectures
- Design scalable data pipelines
- Drive dataops practices
- Enforce automated testing pipeline observability and CI CD
- Identify delivery blockers
- Implement lakehouse architectures
- Lead client data engagements
- Manage technical scope and timelines
- Mentor data engineers
- Own architecture and build
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS Redshift | Amazon S3 | Apache Spark | Automated testing | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Synapse | Azure Synapse Analytics | CI/CD | Cloud Storage | Data Factory | Data Lake Storage | Data Modeling | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Storage | Google Dataflow | Infrastructure as Code | Lakehouse Architecture | Observability | Python | Scala | Synapse Analytics | Unity Catalog | Version control | “as-code”
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Data Engineer USD 126K-208KAPI Integration | Airflow | Amazon Web Services | BigQuery | CCPADEI initiatives | Dental benefits | Employee rewards program | Medical benefits | Mental health supportMid-level Full TimeRemote, United States R3h ago
-
Alerting | Ansible | Bash | CI/CD | CephRemote workSenior-level Full TimeUnited States, United States R5h ago
-
Ansible | Bash | CI/CD | CentOS | CephContract-to-hire | No sponsorship | Remote workSenior-level Full TimeUnited States, United States R5h ago
-
Machine Learning Engineer USD 131K-178KAWS | Cassandra | Convolutional Neural Networks | Data Lakes | Data PipelinesMid-level Full TimeRemote, NY, US R6h ago
-
Software Engineer, Machine Learning USD 213K-293KAPI Design | Agent Orchestration | Artificial Intelligence | Bias Mitigation | C++Senior-level Full TimeSunnyvale, CA | Remote, US | … R9h ago
-
Senior AI Data Engineer USD 155K-185KApache Airflow | Apache Spark | Azure Synapse | BigQuery | ClickHouseEmployer paid Medical Dental Vision Insurance | Flexible paid time off | Manager check ins | Paid cell phone and service | Paid parental leaveSenior-level Full TimeRemote - United States R18h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS Glue | AWS IAM | Amazon EMR | Amazon S3 | AmundsenDevelopment dollars | Employee stock purchase program | Family-forming benefits | Financial coaching | Flexible time offSenior-level Full TimeRemote, USA R18h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS EMR | AWS Glue | AWS IAM | AWS S3 | Apache AirflowDevelopment dollars | Financial coaching | Flexible remote work | Flexible time off | Free therapy sessionsSenior-level Full TimeRemote, USA R19h ago
-
Staff Machine Learning Engineer USD 189K-389KCalibration | Contextual Bandits | Contextual Decisioning | Data Validation | EmbeddingsEquity eligible | In Office 1 Day Per WeekSenior-level Full TimeSan Francisco, CA, US; Remote, US R20h ago
-
Principal AI/ML Engineer USD 165K-226KC# | C++ | CI/CD | CUDA | Computer Vision401k match | Dental insurance | Health insurance | Life insurance | Paid time offSenior-level Full TimeRemote PA - PA PAR, United … R21h ago
-
APIs | Compliance | Distributed Systems | Enterprise Integration | Generative AIOccasional evening calls | Remote workSenior-level Full TimeRemote - US Based R21h ago
-
AV Safety Engineering Analytics Engineer (GPSSC) USD 160K-246KCI/CD | Dash | Docker | GitHub | JenkinsRemote workMid-level Full TimeWork From Home - United States, … R21h ago
-
Vice President, Internal Audit - Data Analytics USD 162K-215KAWS | Audit Testing | Azure | Azure DevOps | Business IntelligenceFlexible time off | Healthcare | Hybrid work model | Retirement benefits | Tuition reimbursementEntry-level Full TimeNY7 - 50 Hudson Yards, New … R21h ago
-
Agile | C++ | Deep learning | Distributed Computing | GPU ComputingDiscretionary bonus | Flexible time off | Healthcare | Leave benefits | Retirement benefitsExecutive-level Full TimeNY7 - 50 Hudson Yards, New … R21h ago
-
Staff Backend AI Engineer, Remote USD 140K-215KAPI Gateway | AWS CDK | AWS ECS | AWS EKS | AWS Fargate401k matching | Dental insurance | Flexible time off | Flexible work schedule | Medical insuranceSenior-level Full TimeUnited States, UNITED STATES, United States R21h ago
-
Data Engineer USD 160K-210KAPI Integration | AWS | Amazon Kinesis | Artificial Intelligence | CI/CDSenior-level Full TimeUS - Remote R21h ago
-
Senior Machine Learning Engineer, Agentic USD 163K-245KA/B | A/B Testing | B testing | Collaborative Filtering | Content-Based Filtering401-K matching | Fertility benefits | Health insurance | Life and disability insurance | Mental health benefitsSenior-level Full TimeBellevue, WA; Menlo Park, CA R22h ago
-
Data Engineer (Remote) USD 155K-180KAzure DevOps | Bash | Cloud Storage | Cloud platform | Compute EngineFully remote | W2 employmentMid-level Full TimeRochester, MN R23h ago
-
Senior Data Engineer (AI Native) USD 103K-192KAWS EMR | Amazon Kinesis | Amazon MWAA | Amazon S3 | Apache Airflow401k company match | Employee assistance program | Equipment and tools support | Flexible PTO | Learning and developmentSenior-level Full TimeRemote, USA ; Remote, Canada R1d ago
-
Principal Snowflake Data Platform Engineer USD 165K-216KAWS | Agile | Ansible | Backup and Resiliency | BashFlexible hybrid work model | Health and life insurance | Paid time off | Retirement benefitsSenior-level Full TimeTX, United States R1d ago
-
Senior Data Engineer (Remote) USD 108K-160KAkamai | Apigee hybrid | Argo CD | Azure AKS | Batch ProcessingFully remote | W2 Candidates OnlySenior-level Full TimeRochester, MN R1d ago
-
AWS RDS | Agile | Amazon S3 | Dashboards | Data PipelinesMust reside in PST time zone | Remote workMid-level Full TimeSan Jose, CA R1d ago
-
AWS RDS | Agile | Amazon S3 | Jira | MongoDBMust reside in PST time zone | Remote workMid-level Full Timephoenix, AZ R1d ago
-
AWS RDS | Agile | Amazon S3 | Dashboards | Data PipelinesDocumentation support | PST time zone requirement | Remote workMid-level Full Timeremote, CA R1d ago
-
Database Engineer USD 95K-120KAutomation | Bash | Database Design | Database replication | Disaster Recovery401k match | Dental insurance | FSA | HSA | Internal growth opportunitiesMid-level Full TimeRemote, United States R1d ago