Big Data/PySpark Engineering Lead - Vice President
Location(s): Tampa, Florida, United States
USD 113K-170K Senior-level Full Time
Tasks
- Benchmark performance on legacy systems
- Build CI/CD pipelines
- Collaborate with product managers on data availability
- Conduct code reviews
- Design scalable data processing pipelines
- Develop data models and schemas
- Implement data parity testing
- Implement data validation and encryption
- Integrate Big Data tools and frameworks
- Manage risk and compliance issues
- Mentor engineers
- Migrate data from legacy systems to data lakehouse
- Monitor system health
- Optimize SQL queries
- Orchestrate phased migration cutovers
- Partner with management teams on system enhancements
- Perform schema evolution
- Re engineer ETL and stored procedures
- Translate business requirements into technical specifications
- Troubleshoot performance bottlenecks
- Write Python code for data processing
Perks/Benefits
- 401k matching
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- Paid time off
- Vision insurance
- Wellness programs
Skills/Tech-stack
Apache Spark | Automated testing | Avro | Bitbucket | CI/CD | CSV | Code review | Collibra | Data Lakehouse | Data Lineage | Data Modeling | Data Validation | ELT | ETL | Encryption | Flink | Git | HBase | Hadoop | Hive | Iceberg | Impala | Informatica | JSON | Kafka | Latency optimization | MongoDB | NoSQL | Parquet | Performance Benchmarking | Presto | PySpark | Python | SQL | Schema Design | Shell Scripting | Spark SQL | Starburst | Trino | Unix | YARN
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …2h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA2h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …2h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA2h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA2h ago
-
Software Engineering Manager II, AI/ML, Google Cloud USD 207K-301KC# | C++ | Cloud platform | Data Processing | DebuggingSenior-level Full TimeSunnyvale, CA, USA2h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA2h ago
-
Senior Software Engineer, AI/ML, Ads Training USD 174K-253KC++ | Distributed Systems | GPU | JAX | Machine LearningSenior-level Full TimeMountain View, CA, USA2h ago
-
Software Engineer, Product USD 208K-269KAgentic Frameworks | Anthropic | Claude Code | Cursor | DocumentationDental insurance | Health insurance | Paid time off | Retirement plan | Vision insuranceMid-level Full TimeSan Francisco, CA7h ago
-
C++ | Containers | Distributed Systems | Distributed Training | Distributed inferenceSenior-level Full TimeCupertino7h ago
-
Sr. Manager Data Engineering USD 125K-188KAutomated testing | Bitbucket | CI/CD | Cloud Functions | Cloud RunSenior-level Full TimeSouthlake, TX, United States9h ago
-
Senior Data Science Manager USD 160K-200KCausal Inference | Coded Quality | Data Pipelines | Data Visualization | Experiment designSenior-level Full TimeCupertino11h ago
-
Data Analysis | Data Visualization | GenAI | Java | Jupyter NotebookSenior-level Full TimeCupertino12h ago
-
Customer Analytics Manager USD 107K-154KBigQuery | Cross-sell | Cross-sell modeling | Customer Health | Customer health scoringMid-level Full TimeRemote - USA R12h ago
-
Forward Deployed AI Engineer USD 180K-230KAPI | AWS | Azure | CI/CD | Cloud platform5-day workweek | Collaborative work culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States13h ago
-
Principal AI Engineer USD 200K-245KAgent-based | Agent-based architecture | Amazon Web Services | Azure | CI/CD5 days per week | Collaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States13h ago
-
Machine Learning Engineer, Sensor Pipelines USD 175K-215KC++ | Computer Vision | JAX | Machine Learning | PyTorchHybrid work scheduleMid-level Full TimeMountain View, CA, USA13h ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R13h ago
-
AI Engineer USD 105K-115KAPI Cost Modeling | API Integration | API Wrappers | Anthropic API | Conversation DesignCalm health employee membership | Company social events | Comprehensive benefits package | Employee referral incentive program | Employee resource groupsEntry-level Full TimeCincinnati, OH, US13h ago
-
Data Engineer USD 112K-160KAWS | AWS S3 | Airflow | Azure | CI/CD401k match | Family leave | Medical, dental, and vision insurance | PTO | Paid HolidaysMid-level Full TimeSanta Monica, CA, United States13h ago
-
Gen AI - CVS - Dallas USD 36K-126KAPI Gateway | Aiohttp | Amazon Web Services | Asyncio | CI/CD401k retirement plan | Medical, dental, vision benefits | Paid Holidays | Paid time off | Variable pay/incentivesMid-level Full TimeUnited States14h ago
-
Senior Numerical Optimization Engineer USD 191K-253KAPIs | C++ | CI/CD | CPLEX | Cloud ComputingComprehensive benefits | Health insurance | Paid on-call supportSenior-level Full TimeCosta Mesa, California, United States14h ago
-
Sr. Software Engineer, AI Infrastructure USD 144K-236KApache Beam | Apache Flink | Apache Spark | C# | C++Career development | Hybrid work | Stock grantsSenior-level Full TimeSunnyvale, CA, United States15h ago
-
Software Engineer, Algorithms (SDE) USD 150K-220KAI tools | C++ | Concurrency | Digital Signal | Digital Signal ProcessingSenior-level Full TimeCupertino15h ago