Senior Data Engineer
Tasks
- Build automation CI CD frameworks
- Build optimize ETL and streaming pipelines
- Design build maintain scalable data pipelines
- Develop PySpark / Spark SQL transformations
- Develop data models schemas for analytics and reporting
- Implement Delta Lake / Lakehouse patterns
- Implement data quality validation monitoring alerting
- Maintain data governance GDPR compliance
- Optimize joins partitioning caching shuffle
- Troubleshoot Azure Databricks pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Automated testing | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | CI/CD | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Modeling | Data Monitoring | Data Quality | Data Validation | Databricks Notebooks | Databricks SQL | Databricks Workflows | Delta Lake | ELT | ETL | GDPR | Lakehouse | MLflow | Performance Tuning | PySpark | Python | SQL | Scala | Schema Design | Security Scanning | Spark SQL | Streaming | Unity Catalog | Version control
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior-level Full TimeBoston, MA6h ago
-
Staff Engineer, Data Migration USD 151K-297KDebezium | Java | JavaScript | Kafka | MySQL401k | Fertility and adoption assistance | Flexible paid time off | Health benefits | Mental health counselingSenior-level Full TimeCalifornia; Oregon; Washington9h ago
-
Senior-level Full TimeBelmont, CA, US, 9400211h ago
-
AWS S3 | Active IQ | Ansible | Azure Blob | Azure Blob Storage401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States11h ago
-
AI Agents | AI Search | AWS | AWS Bedrock | Agentic Workflows401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States11h ago
-
Senior Data & ML Engineer USD 151K-205KDBT | Data Governance | Data Observability | Data Quality | Data Security401k match | Dental insurance | Equity | Family planning resources | Flexible vacation policySenior-level Full TimeRemote - USA R11h ago
-
Senior Software Engineer, Knowledge Graph USD 196K-230KData Ingestion | Data Processing | Distributed Systems | Flink | Graph DatabasesSenior-level Full TimeUnited States11h ago
-
Senior Machine Learning Engineer USD 198K-287KArtificial Intelligence | Data Engineering | Fine Tuning | Foundation Models | GenAIOccasional travel for offsites | On-call rotation participation | Reasonable accommodation for applicantsSenior-level Full TimeRemote - US R11h ago
-
Staff Machine Learning Engineer USD 141K-210KCI/CD | Computer Vision | Deep learning | Distributed Training | DockerSenior-level Full TimeSunnyvale, CA, United States11h ago
-
Senior Software Engineer, Data Platform USD 196K-245KAWS | Apache Airflow | Apache Flink | Apache Spark | AzureEquity | Inclusion and reasonable accommodations | Relocation assistanceSenior-level Full TimeSan Francisco Bay Area13h ago
-
Senior AI Technologist USD 122K-180KAWS | Apache Airflow | Apache Spark | Azure | CI/CDFully remote option | Hybrid option | Onsite optionSenior-level Full TimeRaleigh, North Carolina, United States; Albuquerque, … R13h ago
-
Senior AI Systems Engineer USD 122K-188KAlerting | Bash | CI/CD | CMMC | Cause analysisFully remote option | Hybrid option | Onsite optionSenior-level Full TimeRaleigh, North Carolina, United States; Albuquerque, … R13h ago
-
Senior Data Engineer USD 150K-220KClustering | Common table expressions | DBT | Dagster | Data LineageIn-office work 4 days per week | On-call rotationSenior-level Full TimeNew York, NY13h ago
-
Senior-level Full TimeNew York, NY13h ago
-
Senior AI Engineer USD 170K-205KAWS | Agent Orchestration | Agentic Workflows | CI/CD | ClickHouseHealth benefits | Parental leave | Stock options | Tuition reimbursement | Unlimited PTOSenior-level Full TimeHybrid (NYC Metro) R14h ago
-
Staff Software Engineer, Leverage USD 53K-119KAPI Integration | API Keys | Access Management | Authentication | Backend Development401k match | Business expense stipend | Dental insurance | FSA | HSASenior-level Full TimeRemote - USA, CAN, MEX R14h ago
-
Senior AI Engineer USD 144K-210KAPI Development | Azure | Azure Machine Learning | Azure OpenAI | Cloud Computing401k retirement plan | Dental insurance | Education Course Access | Employee eyewear discount | Health insuranceSenior-level Full TimeDuluth, GA, United States14h ago
-
Sr. Data Engineer I (Splunk) (6382) USD 170K-239KAgile | Data Normalization | Data Parsing | Forwarder | High Availability401k match | Dependent care | Employee Assistance and Wellness Programs | Flexible work arrangements | Health, dental, vision insuranceSenior-level Full TimeWashington, DC14h ago
-
Senior Machine Learning Engineer, Perception USD 220K-292KComputer Vision | Data Pipelines | Deep learning | Edge Computing | Image SegmentationCommuter benefits | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeWashington, District of Columbia, United States14h ago
-
Digital Health And Behavior Data Scientist USD 115K-130KAWS | Azure | Computational simulation | Data Quality | Data Reconciliation401k match | Dependent Care Assistant Program | Educational benefits | Flexible spending account | Medical, dental & vision coverageMid-level Full TimeBaltimore, MD14h ago
-
People Data Intern USD 42K-60KAzure SQL | Data Modeling | Data Quality | Data Validation | ELTOn-site workEntry-level InternshipJersey City, NJ, United States15h ago
-
Generative AI Integration Engineer USD 191K-253KAgent Tool Calling | Autonomous Vehicles | C# | C++ | CameraCommuter benefits | Dental coverage | Disability insurance | Healthcare benefits | Life insuranceMid-level Full TimeCosta Mesa, California, United States15h ago
-
Senior Robotic Systems Integration Engineer (SLA & SLS) USD 102K-130KC++ | Control Systems | Data Analysis | Embedded firmware | Hardware debugging401k matching | Dental coverage | Flexible out of office plan | Healthcare coverage | Onsite Meals Snacks BeveragesSenior-level Full TimeBoston, MA15h ago
-
Senior Robotic Systems Integration Engineer (SLA & SLS) USD 102K-130KC++ | Control Algorithms | Embedded firmware | Hardware debugging | LabVIEW401k matching | Dental insurance | Equity program | Flexible out of office plan | Healthcare coverageSenior-level Full TimeSomerville, MA15h ago
-
Robotic Systems Integration Engineer (SLA & SLS) USD 102K-130KC++ | Control Algorithms | Electro-mechanical | Electro-mechanical Design | Embedded Systems401k matching | Dental insurance | Flexible time off | Health insurance | On-site meals and snacksMid-level Full TimeBoston, MA15h ago