Senior Data Engineer
Tasks
- Automate workflows using Databricks Workflows or Jobs API
- Build test and orchestrate data pipelines in Databricks
- Collaborate with developers and analysts to ensure reliability
- Design and optimize data structures, schemas, and ETL processes
- Diagnose data gaps and quality issues and design validation test cases
- Integrate code into GitHub or GitLab
- Manage database objects in development environments
- Support CI/CD practices
- Translate business use cases into SQL queries, reports, and dashboards
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | Apache Airflow | Apache Spark | Azure | CI/CD | Cloud platform | Data Quality | Databricks | Databricks Workflows | ETL | GitHub | GitLab | Google Cloud | Google Cloud Platform | Jobs API | Kafka | SQL
Education
Roles
Regions
Countries
States
Related jobs
-
SOFTWARE ENGINEER - Apache Spark/Big Data - 7+ yrs of Experience - TS/SCI w/Poly clearance is required - ID A USD 234K-241KAnomaly Detection | Apache Airflow | Apache Spark | Bash | Big Data401k retirement plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeFort George G Meade, United States3h ago
-
Data Management Engineer III USD 93K-163KCI/CD | Data Deduplication | Data Pipelines | Data Standardization | Data TransformationPublic Trust Clearance | Travel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States4h ago
-
Partner Engineer, Generative AI USD 159K-223KAWS | Azure | Bias Mitigation | C plus plus | Cloud PlatformsEntry-level Full TimeMenlo Park, CA5h ago
-
Senior Software Engineer, Database Internals AlloyDB USD 174K-252KC plus plus | C# | Code optimization | Compute Technologies | Concurrency ControlEntry-level Full TimeSunnyvale, CA, USA6h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud AI USD 174K-252KC++ | Cloud AI | Computer Vision | Data Processing | Google CloudSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA6h ago
-
Staff Software Engineer, AI-Powered GRC Automation USD 207K-300KCloud Platforms | Cloud platform | Continuous controls monitoring | Controls monitoring | Data PipelinesSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA6h ago
-
Senior Staff Software Engineer, BigQuery Core Analytics USD 262K-365KBigQuery | C plus plus | C++ | CI/CD | Distributed SystemsSenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA6h ago
-
AI Software Engineer (Vehicle Engineering) USD 125K-175K3D Reconstruction | Agent systems | Agentic AI | Anomaly Detection | CI/CD401k retirement plan | Dental insurance | Employee stock purchase plan | Life insurance | Life insurance coverageSenior-level Full TimeHawthorne, CA16h ago
-
Lead AI Engineer (ML Ops) USD 116K-170KAPIs | AWS | Agile Scrum | Azure | CI/CD401k match | Employee assistance program | Employee charity match | Employee stock purchase plan | Health and wellness allowanceSenior-level Full TimeIrving - 6011 Connection, United States17h ago
-
Platform and Databricks DevOps Engineer USD 77K-176KAWS | Azure | Bash | CI/CD | DatabricksDependent care | Paid leave | Professional development | Tuition assistance | Work-life programsMid-level Full TimeUSA, VA, McLean (8283 Greensboro Dr, …17h ago
-
Adaptive Systems | Apache Spark | C++ | CUDA | Data ProcessingBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara, United States17h ago
-
Data AI Solutions Sr Analyst - C12 - TAMPA USD 87K-130KAgentic AI | Artificial Intelligence | Data Catalog | Data Governance | Data LineageSenior-level Full Time3800 CITIGROUP CENTER DRIVE BUILDING F …17h ago
-
Senior Bioinformatics Data Engineer (Consultant) USD 168K-200KADaM | AWS ECS | AWS ECS Fargate | AWS Glue | AWS Glue CatalogBackground check | Code review feedback loops | Compliance training | Embedded pair working model | VPN AccessSenior-level Full TimeRaleigh, United States17h ago
-
AI assisted code | AI-assisted code generation | Amazon Redshift | Anomaly Detection | BigQueryAI focused engineering culture | Mentorship and growth opportunities | On-site collaborationSenior-level Full TimeBoston, MA17h ago
-
AWS | AWS Redshift | Attribution | Audience Segmentation | BigQueryCompetitive compensation | Growth opportunities | Hands on production experience | Mentorship | On-site collaborationEntry-level Full TimeBoston, MA17h ago
-
AWS | AWS Redshift | Amazon Kinesis | Anomaly Detection | Automated testingSenior-level Full TimeBoston, MA17h ago
-
Senior Machine Learning Engineer, Computer Vision USD 150K-200KAWS | Agile | Airflow | Azure | CI/CD401-k plan | Healthcare benefits | Life insurance | Long-term disability | On-site collaborationSenior-level Full TimeSeattle, Washington, United States18h ago
-
Performance Engineer, Inference Systems USD 350K-850KAutoscaling | Cause analysis | Data Analysis | Data Modeling | Distributed SystemsSenior-level Full TimeSan Francisco, CA | New York …18h ago
-
Full Stack Engineer, Experimentation USD 145K-235KAWS | Agile | Azure | CI/CD | Design PatternsDental insurance | Health insurance | Mental health benefits | Restricted stock units | Vision insuranceSenior-level Full TimeRemote - US R18h ago
-
Senior Data Engineer, Knowledge & Information USD 153K-238KAWS | Alerting | Apache Airflow | Apache Spark | CI/CD401k company match | Dental insurance | Disability insurance | Flexible time off | Health insuranceSenior-level Full TimeUnited States R18h ago
-
Senior Software Engineer, Data USD 144K-216KAWS | Airflow | Azure | DBT | Data LineageDental insurance | Disability insurance | Family-forming benefits | Life insurance | Medical insuranceSenior-level Full TimeSan Jose, California19h ago
-
Compliance, Data Engineering, Dallas, Associate USD 148K-200KAlgorithms | Apache Spark | Batch Processing | C++ | Cloud ComputingSenior-level Full TimeDallas, TX, United States19h ago
-
AI machine learning | Agile | Application Resiliency | CI/CD | Decision supportBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States19h ago
-
Staff AI Engineer USD 170K-220KAPI Development | API Integration | Anthropic API | Artificial Intelligence | Backend Development401k match | Commuter benefits | Employee assistance program | Flexible spending accounts | Gym Fitness Discount ProgramSenior-level Full TimeRemote- US R19h ago
-
Senior Engineer, Developer Platforms - GenAI USD 140K-160KAWS | Agentic Workflows | Amazon Web Services | Datadog | DockerSenior-level Full TimeNew York, NY19h ago