Lead Spark Data Engineer
New York, NY
USD 146K-250K (estimate) Senior-level Full Time Found 12d ago
Tasks
- Architect data solutions on Spark and Flink
- Build Query Validator
- Develop Spark Adapter
- Ensure parity between Spark and Flink implementations
- Implement IoT Query Language grammar
- Implement relationships logic within core
- Maintain documentation
- Manage Azure and Databricks resources
- Monitor workloads and pipelines
- Participate in agile activities
- Promote data engineering best practices
- Transform and prepare data using SQL, Python, Java
Perks/Benefits
- N/A
Skills/Tech-stack
ANTLR | Apache Spark | Azure | Azure DevOps | CI/CD | Data Governance | Data Modeling | Data Quality | Data Warehousing | Databricks | Delta Lake | Flink | GitHub | Graph/Ontology | Java | Python | SQL | Security
Education
Roles
Regions
Countries
States
Cities
Language: en |
Views: 0 |
Clicks: 0
Related jobs
-
Senior Machine Learning Engineer - Cybersecurity USD 151K-246KAnomaly Detection | Cybersecurity | Deep learning | Machine Learning | PyTorchSenior-level Full TimeSan Jose, CA, United States22h ago
-
Senior Software Engineer USD 119K-267KAWS | Agile Development | Algorithms | Analytics | Data ProcessingSenior-level ContractAnnapolis, MD, US1d ago
-
Sr. Data Engineer USD 108K-158KAWS | Apache Spark | Azure | CI/CD | Data Modeling401k | Dental | Disability | Employee discounts | Health insuranceSenior-level Full TimeNew York-TONAWANDA1d ago
-
AI/ML Engineer (TS/SCI Poly) USD 107K-179KDashboard Development | Data Pipelines | Data Visualization | ELT workflows | ETL/ELTBroad benefits | Inclusive culture | Professional developmentMid-level Full TimeArlington/Rosslyn, Virginia, United States1d ago
-
Software Engineer, Machine Learning USD 219K-240KAlgorithms | Availability | C++ | Code editors | ConsistencyMid-level Full TimeNew York, NY1d ago
-
Software Engineer, AI Native USD 173K-247KAI Automation | AI Safety | AI orchestration | AI/ML | AI/ML techniquesSenior-level Full TimeMenlo Park, CA1d ago
-
Senior-level Full TimeMenlo Park, CA | Seattle, WA …1d ago
-
Code review | Data Filtering | Data Generation | Data Pipelines | Distributed SystemsSenior-level Full TimeMenlo Park, CA1d ago
-
Application development | C++ | Data Analysis | Distributed Computing | Large Software SystemsBenefits | Bonus | EquityMid-level Full TimeSunnyvale, CA, USA1d ago
-
AI Agents | Algorithms | Automation | C++ | Data StructuresBenefits | Bonus | EquitySenior-level Full TimeNew York, NY, USA1d ago
-
Staff Software Engineer, AI/ML, Google Cloud USD 197K-291KAlgorithms | Artificial Intelligence | Cross-Functional Collaboration | Cross-functional | Data ProcessingBenefits | Bonus | EquitySenior-level Full TimeCambridge, MA, USA; New York, NY, …1d ago
-
Senior Software Engineer, AI/ML GenAI, Google Workspace USD 166K-244KC++ | Computer Vision | Data Processing | Debugging | Distributed ComputingBenefits | Bonus | EquitySenior-level Full TimeNew York, NY, USA1d ago
-
Software Engineer, Infrastructure Security USD 255K-325KApplication Security | Automation | Cloud infrastructure | Communication skills | IAMMid-level Full TimeSan Francisco1d ago
-
Lead Machine Learning Engineer USD 190K-260KAI Coding Assistants | AI coding | AI tools | Apache Spark | Cloud NativeEvents and activities | Flexible PTO | Healthcare coverage | Inclusive environment | Ownership via equitySenior-level Full TimeSeattle, WA1d ago
-
AWS Big Data | Athena | Big Data | Big Data Technologies | Data ModelingMid-level Full TimeSeattle, Washington, USA1d ago
-
Sr. Data Engineer I (6436) USD 128K-138KAsync Programming | Caching Strategies | ETL | Evaluation Frameworks | Functional Programming401k | Dental insurance | Dependent care | Employee assistance | Flexible workSenior-level Full TimeHybrid - Local to D.C. metro … R1d ago
-
Machine Learning Engineer USD 150K-190KAPIs | CI/CD | Cloud Platforms | Containerization | Data EngineeringMid-level Full TimeRemote, United States R1d ago
-
Data Solution Engineer USD 88K-138KAWS | Apache Kafka | Apache Spark | Automation | AzureNot specifiedSenior-level Full TimeJersey City, NJ, United States1d ago
-
AI / ML Engineer USD 150K-230KAPIs | Agent Frameworks | Agent systems | Anthropic SDK | AutoscalingSenior-level Full TimeCambridge, MA1d ago
-
Senior-level Full TimeOakland, CA, United States1d ago
-
Data Engineer (Databricks + Python + Azure) USD 139K-247KAzure | Batch Processing | CI/CD | Cloud Platforms | Data ArchitectureSenior-level Full TimeMissouri2d ago
-
A/B | A/B Testing | Airflow | Algorithms | B testingBenefits | Bonus | Employee travel credits | Equity | Inclusive cultureSenior-level Full TimeRemote-USA R2d ago
-
Senior Principal AI Engineer USD 140K-210KAWS | Azure | Collaboration | Communication | Data PreprocessingSenior-level Full TimeChantilly/Herndon, VA2d ago
-
Data Engineer USD 110K-149KAPIs | AWS | Agile methodologies | Azure | CI/CDComprehensive benefits | Supportive cultureSenior-level Full TimeFort Meade, MD2d ago
-
AI Operations Engineer USD 85K-128KAI infrastructure | Automation | Azure | CI/CD | Cloud PlatformsAdoption Assistance | Dental plan | Gym discount | Insurance | Medical plansMid-level Full TimeMinneapolis, MN, United States2d ago