Senior Data Engineer
Tasks
- Automate workflows using Databricks Workflows or Jobs API
- Build test and orchestrate data pipelines in Databricks
- Collaborate with developers and analysts to ensure reliability
- Design and optimize data structures, schemas, and ETL processes
- Diagnose data gaps and quality issues and design validation test cases
- Integrate code into GitHub or GitLab
- Manage database objects in development environments
- Support CI/CD practices
- Translate business use cases into SQL queries, reports, and dashboards
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Apache Airflow | Apache Spark | Azure | CI/CD | Cloud platform | Data Quality | Databricks | Databricks Workflows | ETL | GitHub | GitLab | Google Cloud | Google Cloud Platform | Jobs API | Kafka | SQL
Education
Roles
Regions
Countries
States
Related jobs
-
SOFTWARE ENGINEER - Apache Spark/Big Data - 7+ yrs of Experience - TS/SCI w/Poly clearance is required - ID A USD 234K-241KAnomaly Detection | Apache Airflow | Apache Spark | Bash | Big Data401k retirement plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeFort George G Meade, United States2h ago
-
Data Management Engineer III USD 93K-163KCI/CD | Data Deduplication | Data Pipelines | Data Standardization | Data TransformationPublic Trust Clearance | Travel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States3h ago
-
Partner Engineer, Generative AI USD 159K-223KAWS | Azure | Bias Mitigation | C plus plus | Cloud PlatformsEntry-level Full TimeMenlo Park, CA4h ago
-
Senior Software Engineer, Database Internals AlloyDB USD 174K-252KC plus plus | C# | Code optimization | Compute Technologies | Concurrency ControlEntry-level Full TimeSunnyvale, CA, USA5h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud AI USD 174K-252KC++ | Cloud AI | Computer Vision | Data Processing | Google CloudSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA5h ago
-
Staff Software Engineer, AI-Powered GRC Automation USD 207K-300KCloud Platforms | Cloud platform | Continuous controls monitoring | Controls monitoring | Data PipelinesSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA5h ago
-
Senior Staff Software Engineer, BigQuery Core Analytics USD 262K-365KBigQuery | C plus plus | C++ | CI/CD | Distributed SystemsSenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA5h ago
-
AI Software Engineer (Vehicle Engineering) USD 125K-175K3D Reconstruction | Agent systems | Agentic AI | Anomaly Detection | CI/CD401k retirement plan | Dental insurance | Employee stock purchase plan | Life insurance | Life insurance coverageSenior-level Full TimeHawthorne, CA15h ago
-
Lead AI Engineer (ML Ops) USD 116K-170KAPIs | AWS | Agile Scrum | Azure | CI/CD401k match | Employee assistance program | Employee charity match | Employee stock purchase plan | Health and wellness allowanceSenior-level Full TimeIrving - 6011 Connection, United States16h ago
-
Platform and Databricks DevOps Engineer USD 77K-176KAWS | Azure | Bash | CI/CD | DatabricksDependent care | Paid leave | Professional development | Tuition assistance | Work-life programsMid-level Full TimeUSA, VA, McLean (8283 Greensboro Dr, …16h ago
-
Adaptive Systems | Apache Spark | C++ | CUDA | Data ProcessingBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara, United States16h ago
-
Data AI Solutions Sr Analyst - C12 - TAMPA USD 87K-130KAgentic AI | Artificial Intelligence | Data Catalog | Data Governance | Data LineageSenior-level Full Time3800 CITIGROUP CENTER DRIVE BUILDING F …16h ago
-
Senior Machine Learning Engineer, Computer Vision USD 150K-200KAWS | Agile | Airflow | Azure | CI/CD401-k plan | Healthcare benefits | Life insurance | Long-term disability | On-site collaborationSenior-level Full TimeSeattle, Washington, United States17h ago
-
Performance Engineer, Inference Systems USD 350K-850KAutoscaling | Cause analysis | Data Analysis | Data Modeling | Distributed SystemsSenior-level Full TimeSan Francisco, CA | New York …17h ago
-
Full Stack Engineer, Experimentation USD 145K-235KAWS | Agile | Azure | CI/CD | Design PatternsDental insurance | Health insurance | Mental health benefits | Restricted stock units | Vision insuranceSenior-level Full TimeRemote - US R17h ago
-
Senior Data Engineer, Knowledge & Information USD 153K-238KAWS | Alerting | Apache Airflow | Apache Spark | CI/CD401k company match | Dental insurance | Disability insurance | Flexible time off | Health insuranceSenior-level Full TimeUnited States R17h ago
-
Senior Software Engineer, Data USD 144K-216KAWS | Airflow | Azure | DBT | Data LineageDental insurance | Disability insurance | Family-forming benefits | Life insurance | Medical insuranceSenior-level Full TimeSan Jose, California18h ago
-
Compliance, Data Engineering, Dallas, Associate USD 148K-200KAlgorithms | Apache Spark | Batch Processing | C++ | Cloud ComputingSenior-level Full TimeDallas, TX, United States18h ago
-
AI machine learning | Agile | Application Resiliency | CI/CD | Decision supportBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States18h ago
-
Staff AI Engineer USD 170K-220KAPI Development | API Integration | Anthropic API | Artificial Intelligence | Backend Development401k match | Commuter benefits | Employee assistance program | Flexible spending accounts | Gym Fitness Discount ProgramSenior-level Full TimeRemote- US R18h ago
-
Senior Engineer, Developer Platforms - GenAI USD 140K-160KAWS | Agentic Workflows | Amazon Web Services | Datadog | DockerSenior-level Full TimeNew York, NY18h ago
-
Business Analytics Lead - Predictive Modeling USD 130K-140KA/B | A/B Testing | B testing | Business Intelligence | Dashboard Development401k retirement savings plan | Dental coverage | Equity participation | Flexible spending account | Health savings accountSenior-level Full TimeNew York, NY19h ago
-
Quantitative Engineer USD 140K-155KAI Assistant | API Design | AWS | CI/CD | Credit facility401k | Dental insurance | Fitness fund | Health insurance | Learning and development fundSenior-level Full TimeRemote - USA R19h ago
-
Senior, ML Engineer - Auto Tagger USD 177K-212KAWS | Apache Arrow | Apache Beam | Apache Spark | Cloud platform401k match | Company holiday office closures | Company-paid medical, dental & vision | Disability insurance | Flexible scheduleSenior-level Full TimeAnn Arbor, MI, Remote - US R19h ago
-
Data Science Senior Associate - Digiops Analytics USD 170K-201KA/B | A/B Testing | Adobe Analytics | Advanced Analytics | AlteryxBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersMid-level Full TimeWilmington, DE, United States19h ago