BigData PySpark Consultant
Tasks
- Apply architecture and testing best practices
- Build and deploy data pipelines
- Communicate project status and risks
- Deploy Hadoop solutions
- Design data solutions
- Develop Hive queries
- Develop and deploy ETL workflows
- Ensure on-time delivery of high-quality code
- Execute software development life cycle
- Research and recommend technology improvements
- Write Spark SQL jobs
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Hadoop | Apache Hive | Apache Spark | HiveQL | PySpark | Spark SQL | Unix
Education
N/A
Roles
Related jobs
-
A/B | A/B Testing | Apache Flink | Apache Hive | Apache SparkEntry-level Full TimeSan Jose, California, United States6h ago
-
Senior-level Full TimeUnited States - Remote R18h ago
-
Data Engineer- Activity Group USD 130K-180KAPI Integration | AWS Databricks | AWS MWAA | AWS S3 | Amazon ECSSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL23h ago
-
Data Engineer - Research and Development USD 90K-120KAWS | Apache Airflow | Apache Spark | CI/CD | DBTMid-level Full TimePittsburgh, Pennsylvania, United States1d ago
-
Machine Learning Data Engineer USD 120K-190KAmazon SageMaker | Anomaly Detection | Apache Beam | Apache Spark | Cloud platformMid-level Full TimeNew York City1d ago
-
Forward Deployed Data Engineer USD 90K-198KAPI Development | AWS | Apache Airflow | Apache Spark | AzureCompetitive benefits | Continuous learning | Flexible working models | Health and wellbeing support | Hybrid work modelSenior-level Full TimeNew York, NY, US, 100011d ago
-
Data Platform Engineer-Solutions Delivery USD 125K-174KAPI Integration | API Management | APIs | Azure API | Azure API Management401k matching | Company holidays | Employee purchase program | Medical/Dental/Vision | Paid time offSenior-level Full TimeMerriam, KS, United States2d ago
-
API | Airflow | Amazon Web Services | Apache Spark | AzureLearning and development | Mentorship | Travel opportunitiesEntry-level Full TimeAtlanta, Georgia, United States; Boston, Massachusetts, …2d ago
-
databricks engineer - Delivery Senior Consultant, Data Engineering and Conversion Solutions USD 124K-213KAPI | Agile | Data Governance | Data Modeling | Data QualityHybrid work model | Travel 10 percentSenior-level Full TimeLake Mary, Florida, United States2d ago
-
Senior Engineer, Big Data USD 134K-170KAmazon Web Services | Apache Spark | Automated testing | Azure | CI/CDSenior-level Full TimeUnited States2d ago
-
Senior Data Ops Developer USD 94K-151KApache Spark | Azure | Azure Data | Azure Data Lake | Azure SQLContinuing education | Global mobility support | Health and wellness benefits | LinkedIn learning licenses | Mentorship programSenior-level Full TimeUS, Austin Las Cimas, United States2d ago
-
Staff Software Engineer - Streaming USD 182K-247KApache Spark | Data Storage | Database systems | Distributed Systems | Latency optimizationSenior-level Full TimeSeattle, Washington2d ago
-
AI & Machine Learning Vice President USD 120K-220KAWS | Alteryx | Anomaly Detection | Azure | C#401k | Dental insurance | Disability insurance | Health insurance | Hybrid work modelExecutive-level Full TimeUnited States R2d ago
-
API Design | AWS | AWS CloudWatch | AWS Lambda | AWS Step FunctionsBackup childcare | Financial coaching | Health care coverage | Mental health support | Retirement savings planSenior-level Full TimePlano, TX, United States2d ago
-
Senior Software Engineer, Data Lake USD 196K-230KAWS S3 | Apache Flink | Apache Kafka | Apache Spark | Change Data Capture401k matching | Disability insurance | Fertility benefits | Health insurance | Life insuranceSenior-level Full TimeBellevue, WA2d ago
-
AI Engineer - FDE (Forward Deployed Engineer) USD 152K-210KAWS | Agent systems | Apache Spark | Azure | Databricks Intelligence PlatformRemote work | Travel opportunitiesSenior-level Full TimeUnited States2d ago
-
Data Engineer USD 160K-200KAWS | Apache Spark | Azure | CI/CD | DBT401k match | Continuing education support | Employee assistance program | Flexible spending accounts | Generous paid time offSenior-level Full TimeAuburn Hills, MI, US R2d ago
-
Senior Data Engineer II, AI Native USD 148K-216KAmazon Web Services | Apache Airflow | Apache Spark | Azure | BigQuery401k matching | Dental insurance | Disability insurance | Employee assistance program | Equipment reimbursementSenior-level Full TimeRemote, USA R2d ago
-
API Design | AWS | Access Control | Amazon S3 | Apache AirflowEmployee discounts | Employee equity | Medical, dental & vision coverage | Pet insurance | Unlimited PTOSenior-level Full TimeNashville, TN2d ago
-
API Design | AWS | Apache Airflow | Apache Flink | Apache KafkaDental insurance | Employee discounts | Employee equity | Health insurance | Pet insuranceSenior-level Full TimeRemote - United States R2d ago
-
API Integration | Apache Airflow | Apache Spark | Big Data | Classified EnvironmentsHealth insurance | Paid leave | RetirementSenior-level Full TimeHanover, MD2d ago
-
Senior Data Engineer USD 102K-209KAnalytics Cloud | Analytics Platforms | Apache Airflow | Apache Kafka | Apache Spark401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeNashville, TN, United States2d ago
-
Senior Data Engineer USD 115K-175KAWS Fargate | AWS Glue | AWS IAM | AWS Lambda | AWS MWAACharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R2d ago
-
Desk Apps Developer USD 95K-130KApache Spark | Data pipeline | FastAPI | HTML5 | Hadoop401k company match | Dental | Life insurance | Long-term disability | MedicalMid-level Full TimeHouston, TX, United States3d ago
-
Amazon Web Services | Apache Airflow | Apache Flink | Apache Hive | Apache HudiContinuing education | Dental insurance | FSA | HSA | Health insuranceSenior-level Full TimeRochester, MN, United States R3d ago