Data Engineer - AI (Spark, Databricks and Healthcare)
Tasks
- Complete assignments using ticketing system
- Create Spark scripts for data management and validation
- Create and maintain data pipelines
- Ensure data integrity and compliance rules
- Maintain SQL scripts for data management and validation
- Optimize queries for daily task efficiency
- Perform data analysis to identify issues
- Support after hours and weekends as needed
- Troubleshoot environmental and network issues
- Validate task results
Perks/Benefits
- 401k savings plan
- Dental insurance
- Disability insurance
- Flexible work schedule
- Health insurance
- Life insurance
- Paid Holidays
- Paid family leave
- Paid time off
- Remote work
- Vision insurance
Skills/Tech-stack
AWS S3 | Airflow | Amazon Web Services | Apache Hadoop | Apache Kafka | Apache Spark | Cloud platform | Data Pipelines | Data Validation | Databricks | Databricks Workflows | ETL | Google Cloud | Google Cloud Platform | Jira | Microsoft Azure | Microsoft SQL | Microsoft SQL Server | Oracle | PL/SQL | Query Optimization | RDBMS | Ray | SQL | SQL Server | Snowflake | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Staff Software Engineer - Core Ingest USD 191K-224KAgile Development | Apache Kafka | Distributed Systems | Docker | Fault ToleranceHealth insurance | Paid time off | Remote work optionsSenior-level Full TimeUnited States, Remote R5h ago
-
Sr. Data Engineer USD 120K-160KAPI | AWS | AWS Glue | Airflow | Amazon RedshiftComprehensive benefits package | In-person collaboration opportunities | Mentorship and growth opportunities | On-call rotation | Remote-firstSenior-level Full TimeOrlando, FL, United States R12h ago
-
Machine Learning Engineer USD 180K-250KAWS | Azure | CUDA | DDP | Distributed Training401k employer match | Health, dental, vision insurance | Paid time off | Professional development | Work-life balanceMid-level Full TimeEmeryville, California, United States; Hybrid (2-3 … R16h ago
-
Apache Airflow | Data Architecture | Data Engineering | Data Governance | Data ModelingCollaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeMassachusetts R17h ago
-
AI Tooling | Apache Airflow | Data Architecture | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeMinnesota R17h ago
-
Apache Airflow | Data Architecture | Data Compliance | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeIllinois R17h ago
-
Airflow | Analytics engineering | Data Architecture | Data Compliance | Data GovernanceAccess to cutting-edge technologies | Collaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programsSenior-level Full TimeIdaho R17h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data ObservabilityAccess to cutting-edge technologies | Collaborative work environment | Continuous learning | Flexible work hours | Health and wellness programsSenior-level Full TimeColumbia R17h ago
-
Apache Airflow | Data Architecture | Data Compliance | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeColorado R17h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data orchestrationCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeFlorida R17h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data orchestrationCollaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeCalifornia R17h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Dimensional dataCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeArizona R17h ago
-
Apache Airflow | Data Architecture | Data Engineering | Data Governance | Data ModelingAccess to cutting edge technologies and data tools | Collaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programsSenior-level Full TimeConnecticut R17h ago
-
Machine Learning Engineer - Computer Vision USD 220K-250KAmazon Bedrock | Cloud ML | Cloud ML services | Computer Vision | Convolutional Neural NetworksCareer growth mindset | Equity | Meaningful impact on product and users | Remote-first work environmentMid-level Full TimeRemote (U.S.) R17h ago
-
Software Engineer - Platform USD 190K-230KAPI Design | Amazon Web Services | CI/CD | Distributed Systems | GraphQLBenefits | Equity | Remote work flexibilityMid-level Full TimeRemote with offices in San Francisco, … R17h ago
-
Automation | Data Engineering | Data Integrity | Data Reconciliation | Data pipelineDental coverage | Medical coverage | Professional development | Remote work | Stock optionsSenior-level Full TimeIdaho R1d ago
-
Automation | Data Engineering | Data pipeline | Databricks | ETLDental insurance | Medical insurance | Professional development | Stock options | Unlimited PTOSenior-level Full TimeColorado R1d ago
-
Senior Data Engineer AI USD 140K-165KAWS | Amazon Web Services | Apache Airflow | Apache Spark | Data Ingestion401k matching | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R1d ago
-
Commercial Analytics - Senior Associate USD 80K-150KBigQuery | Hadoop | Microsoft Excel | Microsoft PowerPoint | Power BIEmployee benefits | Flexible work environment | Home-based option | Incentive eligibilitySenior-level Full Time127 Public Square, Cleveland, OH, United … R1d ago
-
Global Red Team AI Engineer, Analyst USD 98K-123KAI Foundry | AWS Bedrock | Agentic AI | Amazon SageMaker | Azure AIComprehensive health and wellness benefits | Educational assistance | Income replacement for qualified employees with disabilities | Paid Holidays | Paid maternity and parental bonding leaveMid-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Data & Analytics (ML-Enabled Systems) USD 127K-272KAWS | Data Governance | Data Modeling | Data Quality | Data SecurityCareer development and growth | Employee assistance program | Employee incentive programs | Fitness reimbursement | Flexible vacationSenior-level Full TimeUnited States of America, Eagan, Minnesota R1d ago
-
Cloud Machine Learning Engineer - US remote USD 150K-200KAWS CloudWatch | Accelerate | Amazon EC2 | Amazon S3 | Amazon SageMakerConference reimbursement | Flexible paid time off | Flexible working hours | Health, dental, and vision benefits | Parental leaveMid-level Full TimeUnited States - Remote R1d ago
-
Staff Data Engineer USD 187K-245KAPI Gateway | Alerting | Amazon Redshift | Apache Airflow | BigQueryEquity | Flexible paid time off | Health insurance 100% paid premium | Lifestyle stipend | Parental leaveSenior-level Full TimeRemote, US R2d ago
-
Senior Data Engineer USD 117K-161KAWS | Agile | Avro | Azure | Azure Databricks401k match | Career development opportunities | Caregiver leave | Employee charity matching program | HolidaysSenior-level Full TimeWork at Home - Kentucky, United … R2d ago
-
APIs | AWS | Azure | Batch Processing | Data ModelingAccess to cutting-edge tools and technologies | Flexible working hours | Fully remote work environment | Health and wellness benefits | Professional development opportunitiesSenior-level Full TimeMinnesota R2d ago