Data Engineer - AI (Spark, Databricks and Healthcare)
Tasks
- Complete assignments using ticketing system
- Create Spark scripts for data management and validation
- Create and maintain data pipelines
- Ensure data integrity and compliance rules
- Maintain SQL scripts for data management and validation
- Optimize queries for daily task efficiency
- Perform data analysis to identify issues
- Support after hours and weekends as needed
- Troubleshoot environmental and network issues
- Validate task results
Perks/Benefits
- 401k savings plan
- Dental insurance
- Disability insurance
- Flexible work schedule
- Health insurance
- Life insurance
- Paid Holidays
- Paid family leave
- Paid time off
- Remote work
- Vision insurance
Skills/Tech-stack
AWS S3 | Airflow | Amazon Web Services | Apache Hadoop | Apache Kafka | Apache Spark | Cloud platform | Data Pipelines | Data Validation | Databricks | Databricks Workflows | ETL | Google Cloud | Google Cloud Platform | Jira | Microsoft Azure | Microsoft SQL | Microsoft SQL Server | Oracle | PL/SQL | Query Optimization | RDBMS | Ray | SQL | SQL Server | Snowflake | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Staff Software Engineer - Core Ingest USD 191K-224KAgile Development | Apache Kafka | Distributed Systems | Docker | Fault ToleranceHealth insurance | Paid time off | Remote work optionsSenior-level Full TimeUnited States, Remote R6h ago
-
Senior-level Full TimeRemote - United States R10h ago
-
Sr. Data Engineer USD 120K-160KAPI | AWS | AWS Glue | Airflow | Amazon RedshiftComprehensive benefits package | In-person collaboration opportunities | Mentorship and growth opportunities | On-call rotation | Remote-firstSenior-level Full TimeOrlando, FL, United States R13h ago
-
Data Engineer USD 133K-159KBigQuery | Cloud platform | DBT | Data Governance | Data Modeling401k matching | Employee assistance program | Employee stock purchase program | Flexible vacation | Hybrid work modelMid-level Full TimeSan Francisco, CA R16h ago
-
Machine Learning Engineer USD 180K-250KAWS | Azure | CUDA | DDP | Distributed Training401k employer match | Health, dental, vision insurance | Paid time off | Professional development | Work-life balanceMid-level Full TimeEmeryville, California, United States; Hybrid (2-3 … R17h ago
-
Apache Airflow | Data Architecture | Data Engineering | Data Governance | Data ModelingCollaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeMassachusetts R18h ago
-
AI Tooling | Apache Airflow | Data Architecture | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeMinnesota R18h ago
-
Apache Airflow | Data Architecture | Data Compliance | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeIllinois R18h ago
-
Airflow | Analytics engineering | Data Architecture | Data Compliance | Data GovernanceAccess to cutting-edge technologies | Collaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programsSenior-level Full TimeIdaho R18h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data ObservabilityAccess to cutting-edge technologies | Collaborative work environment | Continuous learning | Flexible work hours | Health and wellness programsSenior-level Full TimeColumbia R18h ago
-
Apache Airflow | Data Architecture | Data Compliance | Data Governance | Data ModelingCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeColorado R18h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data orchestrationCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Work from homeSenior-level Full TimeFlorida R18h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Data orchestrationCollaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeCalifornia R18h ago
-
Apache Airflow | Data Architecture | Data Governance | Data Modeling | Dimensional dataCollaborative work environment | Continuous learning | Flexible work hours | Health and wellness programs | Remote workSenior-level Full TimeArizona R18h ago
-
Apache Airflow | Data Architecture | Data Engineering | Data Governance | Data ModelingAccess to cutting edge technologies and data tools | Collaborative work environment | Continuous learning and development | Flexible work hours | Health and wellness programsSenior-level Full TimeConnecticut R18h ago
-
Machine Learning Engineer - Computer Vision USD 220K-250KAmazon Bedrock | Cloud ML | Cloud ML services | Computer Vision | Convolutional Neural NetworksCareer growth mindset | Equity | Meaningful impact on product and users | Remote-first work environmentMid-level Full TimeRemote (U.S.) R18h ago
-
Software Engineer - Platform USD 190K-230KAPI Design | Amazon Web Services | CI/CD | Distributed Systems | GraphQLBenefits | Equity | Remote work flexibilityMid-level Full TimeRemote with offices in San Francisco, … R18h ago
-
Senior-level Full TimeUSA - Remote R22h ago
-
Automation | Data Engineering | Data Integrity | Data Reconciliation | Data pipelineDental coverage | Medical coverage | Professional development | Remote work | Stock optionsSenior-level Full TimeIdaho R1d ago
-
Automation | Data Engineering | Data pipeline | Databricks | ETLDental insurance | Medical insurance | Professional development | Stock options | Unlimited PTOSenior-level Full TimeColorado R1d ago
-
Adversarial Networks | BERT | Clustering | Convolutional Neural Networks | Data PipelinesEmployer-matched 401k | Exceptional benefits package | Flexible vacation | Hybrid work environment | Paid time offSenior-level Full TimeSanta Monica, CA R1d ago
-
Algorithms | Amazon Kinesis | Amazon Kinesis Data Analytics | Apache Beam | Apache FlinkEmployer-matched 401k | Exceptional benefits package | Flexible paid time off | Hybrid work environmentSenior-level Full TimeSeattle, WA R1d ago
-
Amazon Kinesis | Amazon Kinesis Data Analytics | Apache Beam | Apache Cassandra | Apache Flink401k match | Comprehensive benefits | Flexible vacation | Paid time offSenior-level Full TimeSanta Monica, CA R1d ago
-
Algorithms | Amazon Kinesis | Amazon Kinesis Data Analytics | Apache Beam | Apache FlinkCompetitive compensation | Employer matched 401k plan | Exceptional benefits package | Flexible vacation and paid time off | Hybrid work environmentSenior-level Full TimePalo Alto, CA R1d ago
-
Software Engineer, ML Infrastructure USD 155K-190KCloud Computing | Data Annotation | Data Engineering | Data Mining | EmbeddingsMid-level Full TimeUSA (remote) R1d ago