Senior Data Engineer
Tasks
- Build distributed data-processing pipelines
- Collaborate with cross-functional teams
- Conduct code reviews and enforce code standards
- Contribute to product improvements
- Create functional and design specifications
- Design data processing solutions
- Develop scalable database systems
- Identify technical risks and implement mitigation plans
- Implement Spark Structured Streaming solutions
- Lead functional and architectural design
- Optimize Spark workloads
- Perform Spark SQL query performance optimization
- Provide effort estimations and contribute to project planning
- Resolve technical crisis situations
Perks/Benefits
- 401k matching
- Employee assistance program
- Life and disability insurance
- Medical, dental & vision coverage
- Paid time off
Skills/Tech-stack
Apache Spark | Asset bundles | Data Processing | Databricks (asset) | Databricks Asset Bundles | Distributed data | Distributed data processing | Linux | Query Optimization | Spark SQL | Spark Structured Streaming | Structured Streaming | Terraform
Education
N/A
Roles
Related jobs
-
Data Engineer (remote) USD 85K-100KAgile | Apache Hive | Apache Impala | Apache Spark | Artificial Intelligence401k match | Employee assistance program | Flexible schedule | Health insurance | Life insuranceMid-level Full TimeWork From Home, United States R10h ago
-
AWS | Alerting | CI/CD | Compliance | Data LakeCareer growth | Dental insurance | Medical insurance | Paid leave | Retirement plansExecutive-level Full TimeHouston, TX, United States11h ago
-
CLOUD SOFTWARE ENGINEER - Big Data - 12+ yrs of Experience - TS/SCI w/Poly clearance is required - TB A USD 240K-247KApache Lucene | Apache Solr | Apache Zookeeper | Aspect Oriented Design | Aspect-oriented401k retirement plan | Company paid medical premiums | Dental plan | Disability coverage | Federal HolidaysMid-level Full TimeAnnapolis Junction, United States13h ago
-
SOFTWARE ENGINEER - Big Data - 5+ yrs of Experience - TS/SCI w/Poly clearance is required - ID A USD 218K-225KAnomaly Detection | Apache Airflow | Apache Kafka | Apache Spark | Bash401k retirement plan | Dental insurance | Federal Holidays | Floating holidays | Life insuranceMid-level Full TimeFort George G Meade, United States13h ago
-
SOFTWARE ENGINEER - Apache Spark/Big Data - 7+ yrs of Experience - TS/SCI w/Poly clearance is required - ID A USD 234K-241KApache Airflow | Apache Spark | Bash | CSV | Configuration Management401k retirement plan | Dental insurance | Disability coverage | Federal Holidays | Health insuranceMid-level Full TimeFort George G Meade, United States13h ago
-
Computer Vision | Data Processing | Deep learning | Feature Engineering | Language ModelsSenior-level Full TimeSan Jose, California, United States14h ago
-
Robotics Firmware & Electrical Engineer USD 170K-240KC# | C++ | CAN | Electrical layout | Embedded SystemsSenior-level Full TimeFremont, CA15h ago
-
Production Engineer USD 147K-208KAI workflow | AI workflow automation | Agent Orchestration | C++ | CDNSenior-level Full TimeMenlo Park, CA15h ago
-
Software Engineer III, AI/ML, Google Cloud AI USD 147K-211KC++ | Data Processing | Debugging | GenAI | Google CloudSenior-level Full TimeSunnyvale, CA, USA15h ago
-
Staff Software Engineer, On-Device Machine Learning USD 207K-300KAndroid | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeSunnyvale, CA, USA15h ago
-
Staff Software Engineer, Biglakes, BigQuery USD 207K-300KApache Arrow | Apache Flink | Apache Spark | Big Data | Big data analyticsSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA15h ago
-
Software Engineer, Next Generation AI/ML Infrastructure USD 147K-211KC++ | Data Processing | Data Storage | Distributed Processing | Feature StoresMid-level Full TimeSunnyvale, CA, USA15h ago
-
Senior Software Engineer, Connect Sales, CRM, GenAI, Ads USD 174K-252KComputer Vision | Data Processing | Debugging | Distributed Computing | Generative AISenior-level Full TimeMountain View, CA, USA15h ago
-
Staff Software Engineer, Performance and Optimization USD 189K-303KAutomated testing | C# | C++ | EBPF | FtraceBonus | Equity compensation | Health benefits | Hybrid work environmentSenior-level Full TimeSeattle, Washington21h ago
-
Staff Software Engineer, Performance and Optimization USD 171K-273KAutomated testing | C# | C++ | Co-design | EBPFAnnual bonus | Benefits | Equity compensation | Hybrid work environmentSenior-level Full TimePittsburgh, Pennsylvania21h ago
-
Staff Software Engineer, Performance and Optimization USD 189K-303KAutomated testing | C plus plus | C# | EBPF | FtraceAnnual bonus | Benefits | Equity compensation | Hybrid work environmentSenior-level Full TimeMountain View, California21h ago
-
Senior Software Engineer, AI Training & Infrastructure USD 200K-300KAWS | Alerting | Algorithms | Automated testing | AzureSenior-level Full TimeSan Mateo1d ago
-
Senior Data & AI Engineer USD 139K-230KAWS | Agent Orchestration | Agent systems | Apache Spark | Artificial Intelligence401k match | Health insurance | Mental health counseling | Paid time off | Volunteer encouragementSenior-level Full TimeHartford - Tower, United States1d ago
-
Data Engineer II (Snowflake, AWS, DBT) USD 126K-208KAmazon Web Services | DBT | Data Cleansing | Data Governance | Data Mapping401k match | Health insurance | Mental health counseling | Paid Holidays | Paid time offMid-level Full TimeHartford - Tower, United States1d ago
-
ARM | C# | C++ | Continuous integration | DebuggingCompany-Paid Holidays | Disability insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeAZCH02, United States1d ago
-
Software Engineer, Robotics USD 191K-253KAlgorithms | Anomaly Detection | C++ | Code optimization | Computer Vision401k matching | Caregiver leave | Commuter benefits | Dental benefits | Disability insuranceMid-level Full TimeIrvine, California, United States1d ago
-
Senior-level Full TimeSan Mateo1d ago
-
Senior-level Full TimeAlameda, CA1d ago
-
Senior Firmware Engineer, Embedded Platform USD 132K-178KADC | ARM Cortex | ARM Cortex-M | Abstraction layer | Bash401k retirement savings plan | Accident insurance | Basic Life & AD&D Insurance | Comprehensive medical, dental, vision | Critical illness insuranceSenior-level Full TimeSan Diego, CA1d ago
-
Autonomous Driving – Internship in Machine Learning USD 68K-136KAzure Machine Learning | Computer Vision | Convolutional Neural Networks | Data Pipelines | Data PreprocessingEntry-level InternshipSunnyvale, CA, United States1d ago