Senior Data Engineer
New York, NEW YORK, United States
USD 140K-160K Senior-level Full Time
Tasks
- Build CI CD pipelines and orchestration workflows
- Build batch and real-time ingestion pipelines
- Design Lakehouse architecture
- Ensure data quality, observability, and lineage
- Implement data governance and security
- Integrate ML workflows with MLflow and feature stores
- Optimize Spark performance and manage costs
- Provide technical mentorship through code reviews
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Auto Loader | CI/CD | Change Data Capture | Data Capture | Data Governance | Data Security | Databricks | Databricks Auto Loader | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Feature Store | FinOps | Git | Lakehouse Architecture | ML pipelines | MLOps | MLflow | Medallion Architecture | Photon acceleration | PySpark | RBAC | SQL | Schema Enforcement | Spark | Structured Streaming | Unity Catalog
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R27d ago
-
Senior AI Engineer, VideoAI USD 144K-236KA/B | A/B Testing | B testing | Deep learning | Distributed ComputingFlexible work hybrid model | Health and wellness programs | Time away from workSenior-level Full TimeMountain View, CA, United States8h ago
-
Application Insights | Azure | Azure Monitor | BeautifulSoup | CI/CD401k | Dental insurance | Educational assistance | Health insurance | Open to remoteMid-level Full TimeNew York, United States R10h ago
-
Senior Data Platform Engineer USD 160K-220KAnalytics | CDC | Data Classification | Data Governance | Data Lake401k employer matching | Childcare reimbursement | Company events | Company-paid parking | Fertility treatment coverageSenior-level Full TimeSan Diego, CA, United States10h ago
-
Senior-level Full TimeCincinnati, OH, United States12h ago
-
Mechanical Robotics Engineer USD 77K-130KAutomation Controls | CAD design | Cause analysis | Condition Monitoring | Current Signature AnalysisMid-level Full TimeLincolnton, NC, United States12h ago
-
Sr. Systems Engineer - AI USD 141K-209KAlerting | Artificial Intelligence | Auto Scaling | Automation | Backup and RecoverySenior-level Full TimeKansas City, KS, United States12h ago
-
Data Analytics Engineer USD 139K-230KAWS | Azure | Azure Data | Azure Data Lake | Cloud platformDEI initiatives | Dental insurance | Medical insurance | Mental health support | Remote workSenior-level Full TimeRemote, United States R12h ago
-
Data Engineer USD 126K-208KAPI Integration | APIs | AWS | Amazon Redshift | Apache AirflowDEI initiatives | Dental insurance | Employee recognition program | Medical insurance | Mental health supportMid-level Full TimeRemote, United States R12h ago
-
Data Engineer (Healthcare) USD 105K-130KAWS | AWS Redshift | Apache Airflow | Azure | Azure DataKnowledge sharing | Mentoring | Quarterly travel | Remote workMid-level Full TimeUSA, Remote; USA, Remote; USA, Remote R13h ago
-
ETL | Elasticsearch | JSON | Jupyter Notebooks | Kibana401k match | Paid medical | Paid time off | Training supportSenior-level Full TimeColumbia, MD R13h ago
-
Staff Deep Learning Engineer USD 220K-260KArtificial neural networks | C++ | Computer Graphics | Computer Vision | Deep learning401k | Dental insurance | Discounts | Medical insurance | Paid leaveSenior-level Full TimeNew York, NY, United States14h ago
-
Data Science Team Leader USD 155K-165KCI/CD | Cloud platform | Docker | Google BigQuery | Google CloudSenior-level Full TimeDenver, Colorado, United States14h ago
-
Senior Engineer, Data Management USD 140K-150KAPI Development | AWS | Agile | Azure | Cloud platform401k | Dental insurance | Disability insurance | Employee assistance program | Life insuranceSenior-level Full TimeNew York, NY, United States15h ago
-
Lead Data Engineer USD 170K-185KAWS Athena | AWS Data | AWS Data Pipeline | AWS EMR | AWS Glue401k | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeMcLean, VA, United States16h ago
-
Data Engineer III USD 97K-163KApache Spark | Data Modeling | Data Quality | Data integration | DatabricksAbility to work independently | Mentorship | Professional development | Travel opportunitiesSenior-level Full TimeBaltimore, Maryland, United States17h ago
-
Senior-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …17h ago
-
Senior-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …17h ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL19h ago
-
PIIM Data Engineer USD 105K-160KAPIs | Azure Databricks | Data Governance | Data Lake | Data ModelingCareer growth | Relocation assistance | Time to rechargeMid-level Full TimeNew Albany, OH, US1d ago
-
AI Search | Application Insights | Azure AI | Azure AI Search | Azure Container401k | Commuter benefits | Dental insurance | Educational assistance | Health savings accountMid-level Full TimeNew York, United States R1d ago
-
Staff Machine Learning Engineer - Ads USD 164K-282KAirflow | Apache Spark | BigQuery | Cloud Computing | Deep learning401k plan with company match | Flexible spending accounts | Health savings account | Medical/dental/vision/life insurance | Paid parental leaveSenior-level Full TimeMountain View, USA1d ago
-
Staff Machine Learning Engineer - Ads USD 164K-282KAirflow | Amazon SageMaker | Apache Spark | BigQuery | Contrastive Learning401k plan with company match | Electric Car Charging Station | Employee assistance program | Flexible spending accounts | Health savings accountSenior-level Full TimeMountain View, USA1d ago
-
Senior Data Engineer USD 135K-160KAWS | Airflow | Backfills | CI/CD | Data Modeling401k | Central time zone | Dental insurance | Employer Paid Long-term Disability | Employer Paid Short-term DisabilitySenior-level Full TimeChicago, IL, US R1d ago
-
AI Infrastructure Engineer – Agents & ML Systems USD 175K-200KAirflow | Amazon S3 | Backend Development | C++ | Dagster401k matching | Equity package | Global Entry | Health and wellness stipend | Health, dental, vision insuranceMid-level Full TimeRemote R1d ago