Big Data PySpark Lead Engineer - Vice President
Location(s): Jersey City, New Jersey, United States
USD 142K-213K Senior-level Full Time
Tasks
- Automate pipeline scheduling and orchestration
- Build data pipelines using PySpark
- Design and develop Hadoop ecosystem solutions
- Design data models and data architecture patterns
- Develop real-time and batch data workflows
- Identify and resolve technical risks and data issues
- Write SQL queries for data analysis and validation
Perks/Benefits
- 401k
- Accident insurance
- Disability insurance
- Life insurance
- Medical, dental & vision coverage
- Paid Holidays
- Paid time off
- Wellness programs
Skills/Tech-stack
Apache Hadoop | Apache Hive | Apache Impala | Apache Kafka | Apache Spark | Autosys | Data Governance | Data Modeling | Data Quality | Data Warehouse | Dimensional Modeling | Distributed Systems | HDFS | PySpark | SQL | Scala | Shell Scripting | Sqoop | Streaming
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R24d ago
-
Data Engineer USD 91K-114KANSI SQL | API | Amazon Redshift | Apigee | Azure401k with company match | Bereavement leave | Employee assistance program | Employee discount program | Family planning benefitsMid-level Full TimeRemote - Nationwide, United States R23h ago
-
Senior Data Engineer USD 111K-139KAPI Integration | Amazon Redshift | Apigee | Azure | BI Publisher401k match | Bereavement leave | Dental insurance | Employee assistance program | Employee discountsSenior-level Full TimeRemote - Nationwide, United States R23h ago
-
Data Platform Engineer-Solutions Delivery USD 125K-174KAPI Integration | API Management | APIs | Azure API | Azure API Management401k matching | Company holidays | Employee purchase program | Medical/Dental/Vision | Paid time offSenior-level Full TimeMerriam, KS, United States1d ago
-
Software Engineer, Platform USD 165K-225KAWS | Asynchronous processing | CI/CD | Data Pipelines | Debugging401k access | Company-sponsored fitness classes | FSA and HSA Access | Health, dental and vision coverage | Meals providedEntry-level Full TimeSan Francisco1d ago
-
Data Product Operations Lead USD 145K-175KData Pipelines | Data Quality | ETL | Metrics | Production Monitoring401k | Dental insurance | FSA | Gym classes | HSASenior-level Full TimeSan Francisco / New York City1d ago
-
Staff Software Engineer, Platform USD 195K-315KAPI Design | AWS | Asynchronous processing | Backend Development | CI/CD401k | Dental insurance | FSA | Fitness classes | HSASenior-level Full TimeSan Francisco1d ago
-
Access Control | Azure | Azure Backup | Azure CLI | Azure DataHybrid work | Remote workSenior-level ContractSaint Paul, United States1d ago
-
Mid-level Full TimeUS-IL-Chicago1d ago
-
API | Airflow | Amazon Web Services | Apache Spark | AzureLearning and development | Mentorship | Travel opportunitiesEntry-level Full TimeAtlanta, Georgia, United States; Boston, Massachusetts, …1d ago
-
databricks engineer - Delivery Senior Consultant, Data Engineering and Conversion Solutions USD 124K-213KAPI | Agile | Data Governance | Data Modeling | Data QualityHybrid work model | Travel 10 percentSenior-level Full TimeLake Mary, Florida, United States1d ago
-
Mid-level Full TimeJersey City, New Jersey, United States1d ago
-
Cyber Data Scientist, Analytics and Insights USD 102K-171KCloud Computing | Data Modeling | Data Pipelines | Jira | Machine LearningMid-level Full TimeHuntsville, Alabama, United States1d ago
-
Research Engineer, Computer Vision USD 147K-208KAR/VR | Algorithm Development | Artificial Intelligence | C++ | Computer VisionSenior-level Full TimeSunnyvale, CA1d ago
-
Data Engineer, Product Analytics (University Grad) USD 173K-196KCompliance | Data Modeling | Data Quality | Data Security | Data TransformationCareer growth | Skill developmentNone Full TimeMenlo Park, CA | New York, …1d ago
-
Applied Machine Learning | C++ | Concurrency | Distributed Systems | GoSenior-level Full TimeSan Jose, CA, USA1d ago
-
Staff Software Engineer, AI/ML GenAI, Google Cloud USD 207K-300KComputer Vision | Data Processing | Debugging | Distributed Systems | Fine TuningSenior-level Full TimeSunnyvale, CA, USA1d ago
-
Staff Software Engineer, Infrastructure, Google Cloud AI USD 207K-300KCompute | Data Storage | Deployment | Distributed Systems | InfrastructureSenior-level Full TimeSunnyvale, CA, USA1d ago
-
C++ | Data Processing | Data Structures | Data Structures and Algorithms | Data durabilitySenior-level Full TimeKirkland, WA, USA1d ago
-
AI Deployment Lead – Devin AI | Onsite USD 44K-154KAPI Design | Access Management | Agent Orchestration | Agentic Systems | CI/CD401k retirement plan | Medical, dental, and vision benefits | Paid Holidays | Paid time off | Variable pay/incentivesSenior-level Full TimeUnited States1d ago
-
Senior Engineer, Big Data USD 134K-170KAmazon Web Services | Apache Spark | Automated testing | Azure | CI/CDSenior-level Full TimeUnited States1d ago
-
Software Engineer, Data Engineering USD 70K-200KAWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon EC2Mid-level Full TimeIrvine, CA1d ago
-
Senior-level Full TimeHouston, TX, US1d ago
-
APIs | AWS CloudWatch | AWS Kinesis | Agile | Amazon DynamoDB401k savings plan | Adoption benefits | Career development | Disability benefits | Employee assistance programSenior-level Full TimeIrving, Texas, United States1d ago
-
Staff Software Engineer - Streaming USD 182K-247KApache Spark | Data Storage | Database systems | Distributed Systems | Latency optimizationSenior-level Full TimeSeattle, Washington1d ago