Lead Software Engineer- Python / Pyspark / Java / BigData / Data Modernization / AI
Jersey City, NJ, United States
USD 176K-200K (estimate) Senior-level Full Time
Tasks
- Build and operate autonomous agents for data engineering
- Collaborate with teams to develop AI first analytics and data product solutions
- Define and enforce AI driven data product lifecycle architecture
- Design analytics platforms for real time and batch analytics
- Detect schema drift and propose data transformations
- Embed governance lineage and data quality by design
- Establish monitoring and alerting for performance scalability and availability
- Industrialize entity resolution with ML LLM solutions
- Lead migration and modernization to strategic data mesh
- Provide technical leadership and guidance
- Triaging data incidents and maintaining governance evidence
Perks/Benefits
- Backup childcare
- Financial coaching
- Health care coverage
- Mental health support
- On-site health and wellness centers
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
Access Control | Agile | Alerting | Apache Spark | Autonomous Agents | Batch Processing | CI/CD | Containerization | Data Engineering | Data Governance | Data Lakehouse | Data Lineage | Data Quality | Data mesh | Databricks | Docker | Entity Resolution | Human-in-the-loop | Iceberg | Java | LLM | Language Models | Language Processing | Large Language Models | Machine Learning | Monitoring | NLP | Natural Language | Natural Language Processing | PySpark | Python | Real Time | Real-time analytics | Schema drift | Secure Software | Secure Software Development | Semantic Reconciliation | Semantic layer | Snowflake | Software development | The Loop
Education
N/A
Roles
Engineer | Lead | Lead Software Engineer | Software Engineer
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R27d ago
-
Technical Deployment Lead, Delta USD 152K-222KAI systems | API Integration | Agent Design | Agent communication | Agent to AgentBenefits | Bonus | Equity | Travel up to 30%Senior-level Full TimeAustin, TX, USA; Atlanta, GA, USA3h ago
-
Data Engineer, gUP Engineering USD 106K-151KApache Flume | Apache Spark | Cloud Dataflow | Data Analysis | Data ModelingMid-level Full TimeBoulder, CO, USA3h ago
-
Senior Staff Software Engineer, App Hub USD 262K-365KAlgorithms | Artificial Intelligence | Back-end systems | Big Data | C++Senior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Staff Software Engineer, AI/ML USD 262K-365KAccelerated inference | Android | Android on device ML | Artificial Intelligence | C++Benefits | Bonus | EquitySenior-level Full TimeSunnyvale, CA, USA3h ago
-
Smart NIC Software Engineer, Embedded USD 147K-211KAI machine learning | C# | C++ | Data center | Data center networkingBonus target | Equity | Health insurance | Paid time off | Retirement benefitsMid-level Full TimeDurham, NC, USA; Raleigh, NC, USA3h ago
-
Computer Vision | Data Processing | Data Structures | Data structures algorithms | Distributed Computing401k match | Health insurance | Paid time offSenior-level Full TimeMountain View, CA, USA3h ago
-
Senior Embedded Software Engineer, Microcontroller OS USD 174K-253KBSP | BSP Development | Bazel | C++ | CMakeSenior-level Full TimeSan Francisco, CA, USA; San Jose, …3h ago
-
Staff Software Developer, AI/ML, Safety and Security USD 207K-301KComputer Vision | Data Processing | Debugging | Fine Tuning | Language ModelsSenior-level Full TimeWaterloo, ON, Canada; New York, NY, …3h ago
-
Data Engineer, GCS Data Science USD 106K-151KBig Data | C plus plus | Data Documentation | Data Modeling | Data ValidationEntry-level Full TimeNew York, NY, USA; Mountain View, …3h ago
-
AOSP | Abstraction layer | Algorithms | Android Hardware Abstraction Layer | Android hardwareSenior-level Full TimeMountain View, CA, USA3h ago
-
Agent Engine | Agentic Development Kit | Agentic development | Cloud platform | Computer VisionSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Staff Software Engineer, Vertex GenAI Evaluation Service USD 207K-301KCloud infrastructure | Computer Vision | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud USD 174K-253KC++ | Computer Vision | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Cloud platform | Computer Vision | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA; New York, NY, …3h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | ClickHouse | Data Contracts | Data Modeling401k | Catered lunches | Commuter benefits | Continued education benefit | Equity compensationSenior-level Full TimeNew York, New York, United States9h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Company wellness day off | Continued education benefitSenior-level Full TimeLos Angeles, California, United States9h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Continued education benefit | FSASenior-level Full TimeSan Francisco, California, United States9h ago
-
AI Engineer USD 113K-180KAngular | Apache Kafka | CI/CD | Context Management | DevOps401k matching | Charitable matching | Community outreach groups | Flexible paid time off | Life insuranceMid-level Full TimeLeawood, KS, US R9h ago
-
Applied AI Engineer USD 80K-120KAPIs | Agent Orchestration | Data Pipelines | Evaluation | FedRAMPDental insurance | Equity compensation | Health insurance | Travel benefits | Vision insuranceMid-level Full TimeSF Bay Area, CA, USA10h ago
-
Azure Data Engineer USD 119K-219KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Autonomy | Career growth | Collaborative team | Community | Supportive cultureMid-level Full TimeIndianapolis, IN, US13h ago
-
Sr. Embedded Software Development Engineer, Ring Team USD 193K-261KC# | C++ | Debugging | Diagnostic tooling | Embedded Linux401k matching | Employee assistance program | Flexible spending accounts | Health insurance | Paid time offSenior-level Full TimeSunnyvale, California, USA15h ago
-
Principal Technical Lead, Software Engineering (Data Infrastructure) - North America Software Center USD 166K-241KAmazon S3 | Apache Airflow | Apache Flink | Apache Hudi | Apache Iceberg12 holidays per year | 401k match | Dental insurance | Fitness Center | Game roomSenior-level Full TimeVancouver, WA, US, 9868315h ago
-
Senior Data Science Engineer USD 116K-198KApache Airflow | CI/CD | DBT | Databricks | Databricks AutoloaderFlexible work optionsSenior-level Full TimeRemote (United States) R15h ago