Senior Big Data Engineer
Tasks
- Author solution design documents
- Automate infrastructure on Kubernetes
- Build and optimize batch and real time data pipelines
- Champion engineering best practices with CI CD and testing
- Collaborate with security and application stakeholders
- Design scalable data processing pipelines
- Ensure SLA availability performance and data quality
- Evaluate new technologies and build proofs of concept
- Implement monitoring, alerting, and runbooks
- Lead big data infrastructure delivery
- Maintain end to end observability with Grafana
- Manage on-call incident response
- Onboard and mentor vendor teams and partners
- Provision and manage cloud infrastructure with Terraform
- Triaging and resolve production incidents
Perks/Benefits
- 401k matching
- Employee assistance program
- Flexible paid time off
- Medical, dental, vision, life, and disability insurance
- Paid Holidays
- Paid parental leave
- Paid sick time
- Work from home
Skills/Tech-stack
AWS | Access Control | Airflow | Alerting | Apache Spark | Azure | CI/CD | Data Auditing | Data Governance | Dataproc | Encryption | GCP | Git | Go | Grafana | Incident Management | Infrastructure as Code | Java | Kafka | Kubernetes | Linux | Machine Learning | Monitoring | NLP | Observability | On-Call | On-call incident management | Python | Redpanda | Reliability | Runbooks | Scala | Scalability | Subversion | Systems Design | Temporal | Terraform | “as-code”
Regions
Countries
States
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R10d ago
-
Senior Software Engineer - Backend & Machine Learning USD 180K-250KAWS | Amazon Web Services | Argo | Blue-Green Deployment | Blue/greenSenior-level Full TimeNew York, New York, United States6h ago
-
Senior-level Full TimeMansfield, MA (Remote) R7h ago
-
AI Integration Engineer USD 125K-188KAWS | AWS Lambda | Algorithms | Amazon S3 | Artificial Intelligence401k matching | Commuter benefits | Dental insurance | Free snacks | Health insuranceMid-level Full TimeWashington D.C.11h ago
-
Staff Software Engineer, Data Platform (FedRamp) USD 194K-267KApache Beam | Apache Flink | Apache Hadoop | Apache Spark | Data Observability401k | Dental insurance | Flexible spending account | Health insurance | Paid leaveSenior-level Full TimeSan Francisco, California11h ago
-
Data Engineer USD 125K-188KAWS | Apache Spark | Data Governance | Data Modeling | Data Validation401k company match | Commuter transportation benefit | Dental insurance | Free snacks | Medical insuranceEntry-level Full TimeWashington D.C.11h ago
-
API | AWS Glue | Amazon Redshift | Amazon Web Services | Apache AirflowFully remote | On call production supportMid-level Full TimeOrlando, FL, United States R12h ago
-
Cleared Hybrid Data Engineer (5418) USD 103K-171KAPI Integration | AWS | AWS Glue | AWS Redshift | Access ControlHealth insurance | Learning and development opportunities | Paid leave | RetirementSenior-level Full TimeHanover, MD R12h ago
-
Embedded Software Engineer– Technical Lead USD 150K-175KARM | ARMv7 | ARMv8 | Agile | Bring-up401k match | Health insurance | Minimal travel | Paid time offSenior-level Full TimeWaltham, MA, United States15h ago
-
Senior Data Engineer (P1692) USD 98K-169KAKS | Agile | Apache Spark | Azure | Azure Event401 K | Dental insurance | Health insurance | Health savings account | Maternity leaveSenior-level Full TimeCincinnati, OH; Chicago, IL16h ago
-
Machine Learning Engineer - Computational Drug Discovery USD 100K-130KAI Agents | AWS | CI/CD | Cheminformatics | LLMEquity | Flexible work | Health insurance | Paid time off | Professional developmentEntry-level Full TimeWatertown, MA16h ago
-
AI Solutions Engineer - Professional Services USD 107K-144KAPI Integration | Anthropic Claude | Benefit Analysis | Change Management | Code MigrationRemote work | Work from anywhereMid-level Full TimeUnited States R17h ago
-
Deep Learning Engineer USD 160K-175K3D Modeling | Artificial neural networks | C++ | Computer Graphics | Computer Vision401k | Dental insurance | Employee discounts | Medical insurance | Paid leaveSenior-level Full TimeNew York, NEW YORK, United States17h ago
-
Director, Data Science: Data Science Tools USD 125K-211KAirflow | Bash | Celery | Dash | DatabricksHybrid schedule | Professional development | Remote workExecutive-level Full TimeRemote, Remote, United States R17h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KData Analysis | Data Visualization | Experiment design | Machine Learning | Model DeploymentSenior-level Full TimeSan Jose, California, United States18h ago
-
Brand recognition | Computer Vision | Data Mining | Deep learning | Entity MatchingMid-level Full TimeSeattle, Washington, United States18h ago
-
Click Through Rate | Click Through Rate Prediction | Conversion Rate | Conversion Rate Prediction | Data AnalysisSenior-level Full TimeSeattle, Washington, United States18h ago
-
Computer Vision | Fraud Detection | Language Processing | Machine Learning | Natural LanguageSenior-level Full TimeSeattle, Washington, United States18h ago
-
A/B | A/B Testing | B testing | Data Mining | Distributed ComputingMid-level Full TimeSeattle, Washington, United States18h ago
-
Machine Learning Engineer, Pricing - TikTok E-commerce Global Supply Chain and Logistics USD 129K-194KCausal Inference | Data Mining | Forecasting | Graph Databases | Knowledge graphsMid-level Full TimeSeattle, Washington, United States18h ago
-
Alignment | Chain-of-Thought | Computer Vision | Data Mining | Fine TuningSenior-level Full TimeSeattle, Washington, United States18h ago
-
Machine Learning Engineer, Recommendation - E-Commerce USD 145K-337KClick Through Rate | Click Through Rate Modeling | Computer Vision | Conversion Rate | Conversion rate modelingSenior-level Full TimeSan Jose, California, United States18h ago
-
Machine Learning Engineer, Ecommerce Risk Control USD 175K-266KA/B | A/B Testing | B testing | Data Visualization | Deep learningSenior-level Full TimeSeattle, Washington, United States18h ago
-
Senior Software Development Engineer - Data USD 156K-338KBig Data | Data Engineering | Data Transformation | Reliability | ScalabilitySenior-level Full TimeSan Jose, California, United States18h ago
-
Mid-level Full TimeSan Jose, California, United States18h ago