Senior Data Engineer (Azure & Databricks)
New York, NY, United States
USD 170K-225K (estimate) Senior-level Full Time
Tasks
- Architect and implement master data management platform
- Architect modern data stack
- Build and maintain data models for analytics and AI
- Build batch and near real time processing pipelines
- Create datasets for machine learning and AI applications
- Define data modeling standards and pipeline best practices
- Design and build scalable data systems
- Develop data pipelines in Databricks
- Ensure scalability, reliability, and security
- Implement data governance lineage and quality frameworks
- Implement monitoring observability and alerting for data systems
- Ingest and transform enterprise data
- Optimize data processing jobs for performance and cost
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Cloud Data | Cloud Data Platforms | Data Governance | Data Lineage | Data Management | Data Modeling | Data Quality | Data platforms | Databricks | Delta Lake | Distributed Systems | ELT | ETL | Lakehouse | Machine Learning | Master Data | Master Data Management | Metadata Management | Python | SQL
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)23d ago
-
Sr Data Engineer USD 115K-145KAWS S3 | Apache Airflow | Azure Blob | Azure Blob Storage | BigQuery401k | Dental insurance | Discounts | Fully remote | Health insuranceSenior-level Full TimeNew York, NEW YORK, United States R6h ago
-
Senior AI Engineer USD 100K-115KAWS | Agile | Cost Optimization | Embeddings | Generative AI401k | Dental insurance | Dry cleaning services | Meal benefits | Medical insuranceSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States9h ago
-
Sr. Engineer - Data Analytics USD 115K-150KApache Airflow | Apache Kafka | Apache Spark | Azure | Azure DataSenior-level Full TimeLas Vegas, NV, United States12h ago
-
Applied AI Engineer USD 90K-175KAWS | Agile Development | Azure | Data Preprocessing | DockerOnsite work 4 days per weekMid-level Full TimeNew York, NY, United States12h ago
-
Forward Deployed Engineer USD 120K-180KCloud infrastructure | Language Processing | Machine Learning | Natural Language | Natural Language ProcessingCustomer collaboration | Frontier technology | Technical mentorshipEntry-level Full TimeSan Francisco, CA, US14h ago
-
Senior-level Full TimeHerdon, VA, US15h ago
-
Data Engineer – Senior USD 140K-180KABAC | AWS | AWS Glue | AWS IAM | AWS Lake FormationActive MBI required | IRS contract experience preferred | U.S. citizenship requiredSenior-level Full TimeHyattsville, MD, US15h ago
-
Staff Software Engineer, Data Cloud Memorystore USD 207K-301KC++ | Cache | Compute Technologies | Data Structures | Data Structures and AlgorithmsBonus | Equity | Health insurance | Learning and development | Paid time offSenior-level Full TimeKirkland, WA, USA18h ago
-
AI Research Scientist (Robot Learning) USD 175K-251KData Flywheel | Deep learning | Distributed Training | Fine Tuning | Generative ModelsFree gym | Sports subscription | Team activities | Team tripsMid-level Full TimeSan Francisco19h ago
-
AI Research Engineer (Robot Learning) USD 175K-250KData Flywheel | Deep learning | Distributed Training | Fine Tuning | Generative ModelsFree gym membership | Sports subscription | Stock options | Team meals | Team tripsMid-level Full TimeSan Francisco19h ago
-
AI/ML Engineer II USD 93K-179KAmazon EC2 | Amazon S3 | Apache Airflow | CI/CD | CSS401k | Adoption Assistance | Career path planning | Continuing education | Employee referral programMid-level Full TimePhoenix Campus (Main), United States1d ago
-
Senior Software Engineer, Agentic Engineering USD 184K-356KAgent Orchestration | Autonomous Search | CI/CD | CUDA | Code generationEquity | Health benefits | Paid time off | Relocation assistanceSenior-level Full TimeUS, CA, Santa Clara, United States1d ago
-
MES Software Engineer/Data Analyst USD 109K-175K.NET | AWS | Artificial Intelligence | Azure | Cloud Platforms401k match | Employee assistance program | Employee stock purchase program | Medical/Dental/Vision | Paid HolidaysSenior-level Full TimeUSA-Pennsylvania-Breinigsville-9999 Hamilton Blvd, United States1d ago
-
Senior Systems Engineer, Storage - DGX Cloud USD 208K-414KAlerting | Algorithms | Ansible | Argo CD | CI/CDSenior-level Full TimeUS, CA, Remote, United States R1d ago
-
Director, Precision Medicine Data Science & AI USD 194K-361KAWS | Apache Spark | Azure | CPT | Claims data401-k match | Comprehensive benefits package | Hybrid work | Paid time off | Travel 10 percentExecutive-level Full TimeEast Hanover, United States1d ago
-
Senior Data Engineer USD 105K-149KAWS | AWS Lambda | Agile | Amazon Redshift | CI/CD401k matching | Dental insurance | Family and Medical Leave FMLA | Life insurance | Medical insuranceSenior-level Full TimeKS Overland Park, United States1d ago
-
Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Data Factory401k match | Disability insurance | Education benefit | Employee stock purchase plan | Life insuranceSenior-level Full TimePrudential Tower, 655 Broad Street, Newark, … R1d ago
-
Data Scientist I - Card Fraud USD 73K-110KData Cleansing | Data Preparation | Data Visualization | Econometrics | Hadoop401k | Health insurance | Paid Holidays | Paid time off | Sick leaveNone Full TimeAtlanta GA - 303 Peachtree Center …1d ago
-
Quantitative Developer USD 140K-155KAPIs | Agentic AI | Batch Processing | CI/CD | ContainerizationSenior-level Full TimeBoston, United States1d ago
-
CatBoost | Cross-validation | Feature Selection | Hyperparameter Tuning | LightGBMComprehensive benefitsMid-level Full TimeNew York, New York, United States …1d ago
-
Apache Arrow | Backtesting | Cost modeling | Cross-validation | Data QualityMid-level Full TimeNew York, New York, United States …1d ago
-
Lead Applied AI Engineer USD 305KArtificial Intelligence | Data integration | Foundation Models | Guardrails | Language Processing401k plan | Disability benefits | Health benefits | Life insurance | Paid time offSenior-level Full Time111432-TX-Las Colinas Bldg A, Irving Campus, …1d ago
-
Principal MarTech SWE / AI Engineer USD 145K-235KAPIs | AWS | Agile Development | Asynchronous processing | AzureSenior-level Full TimeSanta Clara, CA1d ago
-
API Keys | Arize | Async Programming | Docker | Event Driven401k plan | Commuter benefits | Disability benefits | Hybrid work schedule | Life insuranceSenior-level Full Time112265-NJ-MetroPark, Iselin, United States1d ago