Sr. Data Engineer
Tasks
- Apply cluster configuration and job orchestration best practices
- Build and maintain Databricks Spark transformations notebooks and jobs
- Enable data pipelines for AI machine learning and advanced analytics
- Establish data quality checks validations and compliance
- Extract, transform, and load data into data warehouse
- Implement data governance frameworks and monitoring
- Implement medallion architecture bronze/silver/gold
- Integrate data from databases APIs and external systems
- Lead data pipeline design development and optimization
- Optimize data processing workflows and query performance
Perks/Benefits
- 401k match
- Commuter benefits
- Dental insurance
- Life insurance
- Medical insurance
- Paid Holidays
- Paid time off
- Paid time off to volunteer
- Product Discounts
- Rewards and recognition programs
- Student loan repayment
- Tuition assistance
- Vision insurance
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Bash | Cluster configuration | Cost Optimization | Data Cleansing | Data Factory | Data Governance | Data Modeling | Data Quality | Data Validation | Data Warehouse | Data integration | Data pipeline | Databricks | Incremental processing | Java | Job orchestration | Medallion Architecture | NoSQL | Performance Tuning | Powershell | Python | R | Relational databases | SQL | Scala | Synapse Analytics
Education
Roles
Related jobs
-
Senior Analytics Engineer USD 140K-175KBusiness Intelligence | DBT | Data Modeling | Data Pipelines | Data WarehousingDental insurance | Medical insurance | Paid time off | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA12h ago
-
Machine Learning Engineer II USD 46K-54KAWS | Apache Spark | Azure | Business Intelligence | Data Warehousing401k | Career advancement | Dental insurance | Disability insurance | Health insuranceMid-level Full TimeChicago, IL16h ago
-
Principal AI Engineer, Special Programs USD 220K-350KAI Safety | API Development | API Orchestration | AWS | Cloud platform401k retirement plan | Dental insurance | Employee stock purchase plan | Life insurance | Long-term disability insuranceSenior-level Full TimePalo Alto, CA16h ago
-
Software Engineer, Statistical Evaluation and Sampling USD 170K-216KC++ | Data Processing | Data Processing Pipelines | Importance sampling | Machine LearningEntry-level Full TimeMountain View, CA, USA; San Francisco, …17h ago
-
Senior Software Engineer, Data Infrastructure USD 200K-400KAKS | AWS | Airflow | Azure | BigQueryDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical, dental, vision benefitsSenior-level Full TimeNew York City17h ago
-
Senior Software Engineer, Data Infrastructure USD 200K-400KAWS | Airflow | Azure | BigQuery | CDCDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical, dental, and vision benefitsSenior-level Full TimeSan Francisco17h ago
-
Senior Software Engineer, Storage USD 166K-210KAmazon CloudWatch | Amazon EC2 | Backups | Cause analysis | Cloud-basedAnnual equity refresh grants | Equity grants | Remote workSenior-level Full TimeUnited States - Remote R17h ago
-
Senior Software Engineer II, Storage USD 192K-242KAmazon CloudWatch | Amazon EC2 | Amazon RDS | Backups | Cloud platformAnnual refresh grants | Equity grant | Remote workSenior-level Full TimeUnited States - Remote R17h ago
-
Senior Software Engineer, Data Governance & Foundations USD 166K-210KApache Airflow | Apache Flink | Apache Hudi | Apache Iceberg | Apache KafkaAnnual refresh grants | Equity grant | Remote work flexibilitySenior-level Full TimeUnited States - Remote R17h ago
-
Associate Software Engineer, Embedded Development USD 100K-150KAOSP | Android | Bash | Black box testing | Black-box401k match | Dental insurance | Free snacks | Health insurance | Life insuranceMid-level Full TimeRaleigh, NC R18h ago
-
Machine Learning Engineer - Semantic Reasoning (Highway) USD 200K-293KBEV | C++ | Computer Vision | Deep learning | JAXPaid time offSenior-level Full TimeFoster City, CA18h ago
-
IT Controls Data Engineer USD 293K-385KAnalytics engineering | Automation | Data Engineering | Data Lineage | Data ModelingSenior-level Full TimeSan Francisco18h ago
-
Dashboard | Data Visualization | Data pipeline | ETL | Machine LearningOnsite days schedule | Overtime paySenior-level Full TimeSan Mateo, CA, United States R18h ago
-
Sr. Machine Learning Engineer USD 175K-230KAWS | C plus plus | Deep learning | Kubernetes | Language Models401k plan | Cell phone internet reimbursement | Company-Paid Holidays | Flexible paid time off | Health Savings Account employer contributionSenior-level Full TimeRemote - United States R19h ago
-
Senior AI & ML Engineer USD 194K-228KAPIs | Agent Orchestration | Agent routing | Agents SDK | Cloud infrastructureSenior-level Full TimeUnited States - Remote R20h ago
-
Robotics Framework Engineer II USD 143K-195KC++ | CI/CD | Cross-compilation | EtherCAT | Forward KinematicsMid-level Full TimeAustin, Texas, United States20h ago
-
AI Engineer - Operations USD 115K-165KAI Agents | AI coding | AI coding tools | Coding Tools | Data Engineering401(k) plan match | Barista coffee bar | Corporate events | Coworking spaces | Gym reimbursementMid-level Full TimeAlpharetta, GA20h ago
-
Staff ML Systems Engineer, Distributed Systems USD 70K-300KCPU | Concurrency | DDP | Dask | Data LocalitySenior-level Full TimeSeattle, WA21h ago
-
Senior Staff data science engineer USD 141K-206KAzure Machine Learning | CI/CD | Data Science | Databricks | Distributed SystemsDisability insurance | Employee assistance program | Flexible spending account | Health savings account | Life insuranceSenior-level Full TimeMilpitas, CA, United States22h ago
-
Lead Software Engineer, Data Platform USD 220K-250KAPI Design | Auto-labeling | Azure | Computer Vision | Data pipeline100 Percent covered medical dental vision | Citi Bike membership | Commuter benefits | Free lunch | Holiday observancesSenior-level Full TimeNew York, NY22h ago
-
Senior 3D Print Optimization Engineer USD 115K-148KCause analysis | Computer Aided Design | Experiment design | Material characterization | Python401k matching | Culture events | Dental insurance | Equity RSUs | Flexible time offSenior-level Full TimeBoston, MA22h ago
-
Senior 3D Print Optimization Engineer USD 115K-148K3D Printing | CAD | Cause analysis | Chemistry | Electronics401k matching | Culture events | Equity RSUs | Financial advisors | Flexible out of office planSenior-level Full TimeSomerville, MA22h ago
-
3D Print Optimization Engineer USD 85K-125KCause analysis | Computer Aided Design | Electronics | Experiment design | Fluid Dynamics401k matching | Dental insurance | Equity compensation | Financial advisors | Flexible out of office planSenior-level Full TimeBoston, MA22h ago
-
3D Print Optimization Engineer USD 85K-125KCause analysis | Chemistry | Computer Aided Design | Electronics | Fluid Dynamics401k matching | Dental insurance | Flexible time off | Healthcare coverage | On-site parkingSenior-level Full TimeSomerville, MA22h ago
-
Sr. Staff Engineer, Fulfillment Center (FC) Systems USD 159K-299KAWS | Agile | Alerting | Capacity Planning | CassandraSenior-level Full TimeSeattle, USA23h ago