Lead Data Engineer (New York)
Tasks
- Architect cloud data platforms
- Automate deployments with CI/CD
- Build AI ML infrastructure for feature stores and vector databases
- Collaborate with data scientists and stakeholders
- Conduct architecture reviews
- Deploy containerized data applications
- Design AI enhanced data pipelines
- Drive data platform modernization
- Ensure automated testing and code quality
- Ensure data quality, reliability, and governance
- Establish branching strategies
- Implement NoSQL document modeling
- Implement real-time inference pipelines
- Implement relational data models
- Integrate AI agents into data workflows
- Lead data pipeline design
- Mentor data engineers
- Perform security scanning
Perks/Benefits
- Employee resource groups
- Global parental leave
- Healthcare benefits
- Hybrid work environment
- Matched donation programs
- Mentorship programs
- Paid volunteer days
- Retirement planning
- Training and development
- Tuition reimbursement
Skills/Tech-stack
AWS EKS | AWS EMR | AWS Glue | AWS Lambda | AWS RDS | AWS S3 | Agentic AI | Agile | Amazon EKS | Amazon Kinesis | Amazon RDS | Amazon Web Services | Apache Airflow | ArgoCD | Autogen | Automated testing | CI/CD | Cloud Computing | Code Quality | Confluence | Continuous integration | Data Engineering | Data Pipelines | Databricks | Docker | Feature Stores | GitHub | GitHub Actions | GitOps | Java | Jira | Kubernetes | LLMs | Langchain | Llamaindex | MCP | Model Context Protocol | MongoDB | MongoDB sharding | PostgreSQL | Prompt engineering | RAG | Replication | Security Scanning | Snowflake | Spring Boot | Vector Databases | Web Services
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Principal | Principal Data Engineer
Related jobs
-
ABAC | Access Control | Airflow | Apache Spark | Attribute Based FilteringSenior-level Full TimeHouston, TX, United States4h ago
-
ABAC | Airflow | Apache Spark | Automated testing | Azure DevOpsSenior-level Full TimeHouston, TX, United States4h ago
-
Data Engineer (Senior) USD 170K-225KAmazon Web Services | Apache NiFi | Apache Spark | Data Pipelines | Data VisualizationSenior-level Full TimeHuntsville, United States6h ago
-
Senior Manager / Principal Data Science USD 155K-178KAI Agents | Dashboards | Data Pipelines | Data Visualization | LLMHybrid workSenior-level ContractDeerfield Beach, United States6h ago
-
API Development | AWS | Airflow | BigQuery | Cloud ComputingMid-level Full TimeDearborn, United States6h ago
-
Apache Spark | Cloud Computing | Data Governance | Data Lineage | Data WarehousingSenior-level Full TimeTexas-Austin6h ago
-
GenAI Engineer III USD 110K-218KArtificial Intelligence | Containerization | Data Pipelines | Docker | Generative AISenior-level Full TimeArlington/Rosslyn, Virginia, United States7h ago
-
Generative AI Engineer III USD 110K-218KArtificial Intelligence | Data Pipelines | Docker | Kubernetes | Language ModelsSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …7h ago
-
Agile | Airtable | Anthropic | Artificial Intelligence | CursorSenior-level Full TimePittsburgh, Pennsylvania, United States7h ago
-
Platform Ops & Support Lead - ETL/DBA USD 125K-155KAWS | Ataccama | CI/CD | Change Management | DataikuSenior-level Full TimeJersey City, New Jersey, United States7h ago
-
Autoscaling | Deep learning | Distributed Systems | Elastic Systems | KubernetesMid-level Full TimeSan Jose, California, United States7h ago
-
Senior Software Engineer, Managed Spark, OpenSource USD 174K-253KApache Hudi | Apache Iceberg | Benchmarking | C++ | CloudSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Senior Software Engineer, Storage AI/ML USD 174K-253KAlgorithms | Benchmarking | Cloud Storage | Data Structures | Deep learningSenior-level Full TimeSeattle, WA, USA8h ago
-
C++ | Capacity Planning | Cloud APIs | Cloud Computing | Data StructuresSenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA8h ago
-
Technical Lead Manager, Product Software, Quantum AI USD 207K-301KAPI Design | Client side | Client-side development | Cloud Computing | Cloud SpannerSenior-level Full TimeSeattle, WA, USA; Goleta, CA, USA8h ago
-
Senior Staff Software Engineer, Cloud AI, Full Stack USD 262K-365KC++ | CSS | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Senior Software Engineer, Google Cloud AI, Full Stack USD 174K-253KArtificial Intelligence | C++ | CSS | Data Storage | Data StructuresSenior-level Full TimeNew York, NY, USA8h ago
-
Senior Software Engineer, AI/ML, Creative Intelligence USD 174K-253KAd Creative Optimization | Ad creative | C++ | Creative Optimization | Data ProcessingSenior-level Full TimeMountain View, CA, USA8h ago
-
Databricks Solution Architect USD 180K-247KAWS S3 | Apache Spark | Autoscaling | Azure Data | Azure Data LakeSenior-level Full TimeUnited States R14h ago
-
Agentic Systems | Backend Development | Cloud Data | Cloud data warehousing | Cost OptimizationEquity | Remote workSenior-level Full TimeNew York, NY, United States15h ago
-
Senior Quantitative Developer USD 165K-240KAWS | Alerting | Amazon Kinesis | Cloud platform | Data Transformation401k match | Dental insurance | Hardware provided | Health insurance | Unlimited PTOSenior-level Full TimeNew York18h ago
-
ADA | API Design | Agile | Board Support | Board Support PackageActive US Security Clearance | High impact classified program supportSenior-level Full TimeGrand Rapids, MI, United States18h ago
-
Senior Applied Field Engineering, Analytics USD 165K-216KAnalytics | Data Architecture | Data Modeling | Database technology | Geospatial Analytics401k retirement plan | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountSenior-level Full TimeUS-WA-Bellevue18h ago
-
Senior Analytics Engineer USD 140K-195KAccess Control | Audit trails | Census | Cloud Data | Cloud Data Warehouse401k | Dental insurance | Fitness stipend | Health insurance | Meal stipendSenior-level Full TimeMcLean, VA18h ago
-
C++ | Cloud Computing | Code Reviews | Deployment Automation | Distributed Systems401k match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R19h ago