Data Engineer (PySpark)
Tasks
- Advise analysts
- Consult with stakeholders
- Define operating standards
- Design data systems
- Implement testing and deployment
- Monitor software development phases
- Optimize spark performance
- Provide user support
- Recommend security measures
Perks/Benefits
- Employee assistance program
- Life and disability insurance
- Medical, dental & vision coverage
- Paid Holidays
- Paid time off
- Retirement savings plan
- Sick leave
Skills/Tech-stack
Apache Cassandra | Apache Flink | Apache Spark | Apache Zookeeper | Dimension Table | Dimensional Modeling | Fact Table | HBase | HDFS | MapReduce | MongoDB | PySpark | Python | SQL | Spark SQL | Spark Streaming | YARN
Education
Roles
Related jobs
-
Cloud infrastructure | Distributed Systems | GPU | JAX | Machine LearningMid-level Full TimeUnited States R5h ago
-
LLM Application Engineer USD 130K-175KAPIs | Agent Frameworks | Databases | Distributed Systems | EvaluationMid-level Full TimeUnited States R5h ago
-
Senior/Staff - Agent Platform (Cortex Code) USD 236K-309KApache Airflow | DBT | Data Analysis | Data Engineering | Data ModelingSenior-level Full TimeUS-CA-Menlo Park7h ago
-
Machine Leaning Performance Engineer (Inference) USD 200K-300KC++ | CUBLAS | CUDA | CUDA C | CutlassFitness Events | Free meals | Hybrid work options | Paid time off | Volunteer opportunitiesMid-level Full TimeNew York R8h ago
-
Machine Learning Research Engineer USD 200K-300KAI Agents | API Design | Automation | Dask | Data pipelineHybrid work | Paid time off | Paid wellness expenses | Volunteer opportunities | Workshops and continuous learningMid-level Full TimeNew York R8h ago
-
General Pipeline - FullStack Applied AI Engineer USD 150K-250KAWS | Access Control | Asynchronous programming | Authentication | AuthorizationSenior-level Full TimeUnited States R10h ago
-
Senior Software Engineer (Data & AI Solutions) USD 125K-156KAWS | Apache Airflow | Apache Spark | BAM | Bioinformatics401k benefits | Commuter benefits | Employee referral program | Fertility care benefits | Free testingSenior-level Full TimeUS Remote R11h ago
-
Sr. Embedded Controls Engineer – Scanning Systems USD 140K-175KADC | BLDC | Brushed DC | C# | Domain analysisSenior-level Full TimeMountain View, CA11h ago
-
Staff Software Engineer, Backend (Online Storage) USD 230K-290KAWS | Capacity Planning | Database Benchmarking | Disaster Recovery | Distributed SQLDental and vision coverage | Employee stock purchase plan | Flexible spending wallets | Health care coverage | Remote work flexibilitySenior-level Full TimeRemote US R12h ago
-
A/B | A/B Testing | API Integration | Airflow | Attribution ModelingCross-functional collaboration | Remote workSenior-level Full TimeUnited States R12h ago
-
Data Engineer USD 150K-230KAirflow | Anomaly Detection | Apache Spark | CTE | Cause analysis401k | Dental insurance | Equity | Flexible vacation policy | Health insuranceSenior-level Full TimeLos Angeles, CA12h ago
-
Staff Machine Learning Engineer - Moloco Commerce Media USD 172K-244KA/B | A/B Testing | B testing | CTR Prediction | Conversion Rate401k company match | Flexible time off | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeMenlo Park, California, United States12h ago
-
Senior Embedded Engineer- Developer Experience USD 130K-179KBLE | Bluetooth Low Energy | C# | CI/CD | CMakeSenior-level Full TimeRemote, Seattle, Washington, United States R13h ago
-
Lead Data Engineer USD 125K-160KBigQuery | CI/CD | Cloud Composer | Cloud Storage | Cloud platform401k | Commuter benefits | Dental insurance | Disability coverage | Employee assistance programSenior-level Full TimeColumbia, MD, United States13h ago
-
Data Engineer - Databricks USD 172K-237KACID transactions | Airflow | Auto Loader | Azure Data | Azure Data FactoryMid-level Full TimeChicago, IL, United States13h ago
-
Data Engineer USD 170K-209KAWS | Amazon Kinesis | CI/CD | Compliance Frameworks | Container OrchestrationSenior-level Full TimeUS - Remote; Canada - Remote R13h ago
-
Senior Applied ML Engineer – LLMs & NLP USD 140K-175KConvolutional Neural Networks | Human-in-the-loop | Information Extraction | Java | Language ModelingDaily group classes | Free onsite fitness center | Free shuttle transportation | Gourmet cafeteriaSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States14h ago
-
AWS and Azure Data Governance Engineer USD 116K-146K800-53 | AWS CLI | AWS CloudFormation | AWS CloudWatch | AWS Cost ExplorerSenior-level Full TimeWashington, DC14h ago
-
AWS and Azure Data Governance Engineer USD 116K-146KAWS CLI | AWS CloudFormation | AWS CloudWatch | AWS Cost Explorer | AWS EC2Senior-level Full TimeWashington, DC14h ago
-
Senior AI Systems Engineer – LLM & Agentic Systems USD 140K-160KAWS | Agentic AI | Alerting | Drift Detection | EmbeddingsDaily group classes | Dry cleaning benefits | Free onsite fitness center | Free shuttle transportation | Gourmet cafeteriaSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States R14h ago
-
Staff Data Scientist (AI/ML) USD 174K-278KAWS | Agile | Azure | BigQuery | Causal InferenceFlexible work optionsSenior-level Full TimeUS - Boston, MA ; US …14h ago
-
Lead ML Engineer USD 170K-210KAWS | Cloud infrastructure | Cloud platform | Computer Vision | Cost OptimizationExecutive-level Full TimeEnglewood Cliffs, NEW JERSEY, United States15h ago
-
Data Warehouse Software Engineer USD 70K-80KApache Airflow | Cloud Composer | Cloud platform | Git | Google BigQuery401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeSomerville, Massachusetts, United States R15h ago
-
Data Engineer II - Street Data USD 79K-133KAPI | AWS | Apache Airflow | Apache Hadoop | Apache Spark401k | Dental insurance | Health insurance | Life insurance | Paid HolidaysMid-level Full TimeRedlands, CA15h ago
-
Senior Database Engineer USD 114K-195KAWS CloudFormation | AWS RDS | Access Control | Always On | Always On AvailabilitySenior-level Full TimeUS Remote R15h ago