Senior Big Data Engineer
Tasks
- Author solution design documents
- Automate infrastructure on Kubernetes
- Build and optimize batch and real time data pipelines
- Champion engineering best practices with CI CD and testing
- Collaborate with security and application stakeholders
- Design scalable data processing pipelines
- Ensure SLA availability performance and data quality
- Evaluate new technologies and build proofs of concept
- Implement monitoring, alerting, and runbooks
- Lead big data infrastructure delivery
- Maintain end to end observability with Grafana
- Manage on-call incident response
- Onboard and mentor vendor teams and partners
- Provision and manage cloud infrastructure with Terraform
- Triaging and resolve production incidents
Perks/Benefits
- 401k matching
- Employee assistance program
- Flexible paid time off
- Medical, dental, vision, life, and disability insurance
- Paid Holidays
- Paid parental leave
- Paid sick time
- Work from home
Skills/Tech-stack
AWS | Access Control | Airflow | Alerting | Apache Spark | Azure | CI/CD | Data Auditing | Data Governance | Dataproc | Encryption | GCP | Git | Go | Grafana | Incident Management | Infrastructure as Code | Java | Kafka | Kubernetes | Linux | Machine Learning | Monitoring | NLP | Observability | On-Call | On-call incident management | Python | Redpanda | Reliability | Runbooks | Scala | Scalability | Subversion | Systems Design | Temporal | Terraform | “as-code”
Regions
Countries
States
Related jobs
-
AI Solutions Specialist USD 148K-200KAI Search | Agent Frameworks | Artificial Intelligence | Automation | Azure AIHybrid workMid-level Full TimeKansas City, MO, United States6h ago
-
Data Science Engineer (Shreveport, LA) USD 37K-40KData Historian | Data Visualization | Data analytics | Excel | Machine Learning401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeAtlanta, GA, United States R6h ago
-
AWS Bedrock | CI/CD | CUI SPI | Chunking | Data GovernanceGovernment furnished equipment | No sponsorship | No subs requirement | Remote workEntry-level Full TimeUnited States7h ago
-
Staff Software Engineer, Knowledge Catalog, AI USD 207K-301KBig Data | Computer Vision | Context engineering | Conversational Analytics | Data GovernanceBenefits | Bonus | EquitySenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA11h ago
-
Senior Data Engineer USD 156K-227KApache Flume | Apache Spark | Data Analysis | Data Engineering | Data ModelingSenior-level Full TimeMountain View, CA, USA; Zürich, Switzerland11h ago
-
Lead Analyst Data Engineer USD 129K-170KAWS Glue | AWS Lake Formation | AWS Step Functions | Alation | Amazon AthenaSenior-level Full TimeHouston, TX, 77046, US12h ago
-
Principal Research Data Engineer USD 142K-185KAirflow | Analytical processing | ArcGIS | Avro | CI/CDDental | Health care | PTO | Retirement | Sick leaveSenior-level Full TimeSt. Louis, Missouri, US R15h ago
-
Generative AI Engineer - Ford Pro Intelligence USD 118K-232KAPI Integration | Agent Orchestration | Cloud deployment | Cloud platform | Data OutputsMid-level Full TimeBellevue, WA, United States17h ago
-
Machine Learning Engineer (Active Secret Clearance) USD 160K-190KAgile | Asynchronous programming | CI/CD | Data Engineering | Docker401k plan | FSA | Fully remote work | HSA | Hybrid onsite optionMid-level Full TimeRemote; Tacoma, WA R20h ago
-
AI Engineer USD 200K-250KAgentic Systems | Deep learning | Fine Tuning | Inference Optimization | Language ModelsDental insurance | Flexible PTO | Health insurance | Vision insuranceSenior-level Full TimeSan Francisco Office21h ago
-
Lead ML Inference Engineer, Advertising USD 195K-352KArtificial Intelligence | Auction dynamics | Co-design | Control Optimization | Distributed SystemsSenior-level Full TimeAustin, Texas23h ago
-
HR AI/ML Engineer II USD 90K-135KApache Airflow | Databricks | ETL | Git | Machine LearningDental insurance | Employee assistance program | Employee stock ownership plan | Health insurance | Paid parental leaveSenior-level Full TimeHerndon, VA, United States23h ago
-
Cloud Storage | Compute Orchestration | Computer Vision | Data Lineage | Data PipelinesEnd-to-end responsibility | Fast-paced startup environment | High autonomy | Onsite work | OwnershipMid-level Full TimeSan Mateo, CA; Onsite R23h ago
-
Market Data Platform Engineer USD 168KAccess Control | Automation | Bash | Capacity Planning | Disaster Recovery401k plan | Disability benefits | Life insurance | Paid time off | Parental leaveMid-level Full Time142019-NC-300 South Brevard, Charlotte, United States23h ago
-
Principal Optimization Engineer USD 117K-234KCONOPT | Cloud Computing | Convergence analysis | Discrete Optimization | Fluid modelingHealth care benefits | Hybrid remote option | Paid Holidays | Paid sick days | Paid vacationSenior-level Full TimeCAG10: ALC HQ, 1025 Cobb Place … R23h ago
-
Senior AI/ML Scientist USD 143K-229KArtificial Intelligence | Benchmarking | Data Analysis | Data Science | Deep learning401k match | Annual incentive bonus | Health benefits | Paid time off | Wellness programsSenior-level Full TimeRemote Flex - North Carolina, United … R23h ago
-
Data/AI Engineer USD 94K-135KACID | Artificial Intelligence | Automated testing | Azure DevOps | Bronze Silver Gold401k savings plan | Flexible Spending Accounts FSA | Health Savings Account (HSA) | Healthy Lifestyle Programs | Long-term disabilitySenior-level Full TimeUS-Nationwide-FIELD, United States23h ago
-
Data AI Engineer, Dir, P4 USD 150K-190KAWS | Apache Spark | Azure | Data Architecture | Data GovernanceHybrid workSenior-level Full Time1 New York Plaza, United States23h ago
-
Analytics Engineer USD 118K-177KAWS | Airflow | DBT | Dagster | Data Architecture401k match | Dental insurance | Life insurance | Medical insurance | Tuition reimbursementSenior-level Full TimeWaltham Office, United States23h ago
-
Pathology Computational Scientist USD 93K-136KBioinformatics | Containerization | Data Analysis | Data Visualization | Database systemsCareer advancement opportunities | Comprehensive benefits | Onsite work environmentMid-level Full Time60 Fenwood Road Boston (Hale Building), …23h ago
-
APIs | Agile | Azure | Azure Data | Azure Data FactoryPeriodic travel | Remote work permittedSenior-level Full Time6314 Remote/Teleworker US, United States R23h ago
-
Senior-level Full TimePlano, TX, United States23h ago
-
C++ | Embedded Linux | Hardware-in-the-loop | Hardware-in-the-loop Testing | Loop Testing401k match | Education reimbursement | Health Savings Account contribution | Paid time offMid-level Full TimeCambridge - B3 Crossing, United States23h ago
-
Principal Storage & Disaster Recovery Engineer USD 109K-223KAWS EBS | AWS EFS | AWS FSX | AWS S3 | Azure BlobSenior-level Full TimeBusiness Office - Benson Tower, United …23h ago
-
Robotics Forward Deployed Engineer USD 135K-217KAutomation | Cause analysis | Control Systems | Dashboards | Data VisualizationCross-functional collaboration | On site work experience | Travel to customer sitesSenior-level Full TimeWatertown, MA1d ago