Senior Data Engineer (Python/PySpark/AWS)
Tasks
- Automate testing and deployment
- Build PySpark batch workflows
- Build PySpark streaming workflows
- Build Python data processing scripts
- Design data architecture solutions
- Develop CI/CD pipelines
- Develop data models and schemas
- Ensure data quality and integrity
- Implement ETL pipelines
- Integrate data from multiple systems
- Monitor data pipeline reliability
- Optimize code for performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Azure | Batch Processing | CI/CD | Data Integrity | Data Modeling | Data Privacy | Data Quality | Data Security | Data Warehousing | Data integration | Docker | ETL | Git | GitHub Actions | Google Cloud | Jenkins | Kubernetes | PySpark | Python | Streaming
Education
N/A
Roles
Related jobs
-
Forward Deployed Machine Learning Engineer USD 180K-300KAPI Design | Cloud Computing | Deep learning | Diffusion Models | Fine TuningIn-person collaboration days | Remote work flexibility | Travel cost coverageSenior-level Full TimeSan Francisco (USA) R7h ago
-
Senior ML Engineer, Fauna USD 184K-250KComputer Vision | Data Quality | Distributed Training | Edge Deployment | Evaluation PipelinesSenior-level Full TimeNew York, New York, USA8h ago
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States8h ago
-
Specialist Solutions Architect - AI/ML USD 180K-247KAI guardrails | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingMentorship | Remote work | Technical training | Travel up to 30 percentSenior-level Full TimeUnited States9h ago
-
Lead Data Engineer USD 115K-130KAgile | Apache Airflow | Azure Data | Azure Data Factory | Cloud Orchestration401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementSenior-level Full TimeUniversal City, CALIFORNIA, United States9h ago
-
Cyber Data Engineer USD 140K-145KAWS | ArcSight | Bash | Cribl | DHCP401k match | Accrued PTO | Health/Dental/Vision | Life insurance | Long-term disabilitySenior-level Full TimeSpringfield, VA11h ago
-
Ansible | ArgoCD | CI/CD | Chef | Configuration ManagementSenior-level Full TimeNew York, NY, United States11h ago
-
Senior Embedded Software Quality Engineer II USD 129K-178KAS9100 | AS9115 | Agile methodologies | Bug Reporting | Cause analysisDental insurance | Disability insurance | FSA | HSA | Health insuranceSenior-level Full TimeWestminster, CO12h ago
-
Senior Engineer Embedded System Engineer USD 94K-125K8D methodology | A3 problem solving | Board Design | Bus Traces | Bus analyzerLife, accident, and disability insurance | Medical/Dental/Vision insurance | Paid sick leave | Paid vacation time | Tax-advantaged flexible spendingSenior-level Full TimeIrvine, CA, United States14h ago
-
A/B | A/B Testing | APIs | Airflow | B testingSenior-level Full TimeUnited States14h ago
-
Senior Systems Analyst – Robotic Algorithms and Control USD 134K-201KC# | C++ | CAD Tools | Classical control | Computer VisionSenior-level Full TimeSunnyvale, CA, United States14h ago
-
Full Stack Software Engineer - Robotics USD 125K-200KAWS | Datadog | Distributed Systems | Edge Computing | Grafana401k | Cell phone reimbursement | DC FSA | Employee assistance program | EquityMid-level Full TimeSan Francisco || Oakland, CA R14h ago
-
Senior Research Engineer, Voice + Speech USD 200K-400KData Pipelines | LLM | Language Processing | Machine Learning | Model EvaluationDaily meals snacks | Disability benefits | Fertility benefits | Life insurance | Medical/Dental/VisionSenior-level Full TimeNew York City15h ago
-
Staff Research Engineer, Voice + Speech USD 200K-400KConversational AI | Data Pipelines | Deep learning | Information Retrieval | LLM DeploymentDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical/Dental/Vision insuranceSenior-level Full TimeNew York City15h ago
-
Senior-level Full TimeCosta Mesa, California, United States15h ago
-
Senior Research Engineer, Voice + Speech USD 200K-400KData Pipelines | Deep learning | Information Retrieval | LLM Deployment | Language ModelsDaily meals | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeSan Francisco15h ago
-
AI Platform Engineer, Training and Inference USD 150K-225KANN indexing | BF16 | DDP | Embeddings | FP8Career growth | Learning opportunitiesSenior-level Full TimeSan Francisco15h ago
-
API Gateway | AWS Glue | Access Control | Apache Airflow | Apache AtlasSenior-level Full TimeFAIRFAX, VA, United States15h ago
-
AI Developer - Model Creation & Full Stack USD 150K-175KAWS | Angular | Azure | CI/CD | D3.jsRemote work | USPS Public Trust Clearance eligibleMid-level Full TimeWork from home, VA, United States R15h ago
-
Enterprise Data Engineering & Operations Lead SME USD 107K-190KAWS GovCloud | Access Control | Apache Kafka | Apache Spark | Azure GovernmentSenior-level Full TimeFAIRFAX, VA, United States15h ago
-
Software/Embedded Systems Engineer USD 135K-158KAgile Development | C++ | Change Control | DoD Systems | Embedded SoftwareTravel 25%Senior-level Full TimeArlington, VA, United States15h ago
-
Data/ML Scientist SME USD 105K-150KAWS GovCloud | Anomaly Detection | Apache Spark | Bayesian Causal Inference | Bayesian MethodsMid-level Full TimeFAIRFAX, VA, United States15h ago
-
AI Data Engineer USD 165K-225KAudit Columns | CI/CD | Cortex Analyst | Cortex Complete | DBT401k company match | Flexible paid time off | Learning and development | Medical benefits | Paid parental leaveSenior-level Full TimePhiladelphia, PA, United States15h ago
-
AWS | Agile | Azure | Cloud platform | DB2401k match | Disability insurance | Life insurance | Medical, dental, and vision insurance | Paid bench timeSenior-level Full TimeCincinnati, Ohio, United States16h ago
-
Lead AI Engineer USD 180K-280KCI/CD | ColBERT | Docker | Faiss | Fine Tuning401k match | Bonus | Childcare benefits | Dental insurance | Disability insuranceSenior-level Full TimeQuincy, MA, United States16h ago