Data Engineer - AWS/Databricks - Mid Level
Reston, VA, United States
R
USD 121K-170K (estimate) Mid-level Full Time
Tasks
- Build PySpark data pipelines in Databricks
- Design Delta Lake tables
- Develop ETL ELT data transformations with Spark SQL
- Implement AWS cloud data solutions with S3 Glue RDS and IAM
- Implement ETL and ELT workflows
- Implement data governance metadata tagging data quality validation audit logging and lineage
- Manage Git versioning and CI CD deployments
- Monitor data pipeline health and cluster utilization
- Optimize pipeline performance for cost and throughput
- Perform technical discovery of legacy data sources
Perks/Benefits
- Comprehensive benefits
- Mentorship
- Mentorship and personalized development plans
- Training budget
- Work/life balance focus
Skills/Tech-stack
AWS | AWS Glue | Adaptive query execution | Amazon CloudWatch | Amazon RDS | Apache Spark | Audit Logging | Broadcast joins | CI/CD | Caching | Data Governance | Data Modeling | Data Quality | Data Warehousing | Databricks | Dataframes | Delta Lake | Delta Lake Tables | ELT | ETL | Git | GitLab | IAM | Jenkins | Lineage | Metadata | Partitioning | PySpark | Query Execution | S3 | Spark SQL
Education
Roles
Related jobs
-
Machine Learning Scientist, BioML USD 200K-330KAWS | Azure | Bioinformatics | Cloud Computing | Computational Biology401k employer match | Equity participation | Health, dental, vision insurance | Paid time off | Professional developmentMid-level Full TimeEmeryville, California, United States; Hybrid (2-3 … R16h ago
-
Machine Learning Platform Engineer USD 135K-160KAmazon SageMaker | Apache Flink | C++ | CI/CD | Cloud PubSub401k match | Annual bonus | Company equipment provided | Company medical dental vision plans | Disability benefitsMid-level Full TimeAtlanta, GA preferred, Remote R18h ago
-
Senior Developer Advocate - Modern App Development USD 194K-237KAPI Integrations | AWS | Cloud platform | Code Quality | Google CloudCommunity groups | Employee stock purchase plan | Inclusion talks | Mental health benefits | Mentor/Buddy programSenior-level Full TimeCalifornia, USA, Remote; Nevada, USA, Remote; … R18h ago
-
Staff Software Engineer, AI Developer Tools USD 180K-245KAPI Design | Agent systems | CI/CD | Compliance | Data PrivacySenior-level Full TimeDenver, CO;San Francisco, CA;New York, NY;Seattle, … R18h ago
-
Staff Software Engineer, Big Data Storage USD 177K-364KApache Flink | Apache Hive | Apache Iceberg | Apache Spark | Column BackfillSenior-level Full TimePalo Alto, CA, US; Remote, US R19h ago
-
Senior AI Data Engineer USD 160K-200KAWS Glue | AWS Lambda | Amazon Kinesis | Amazon Redshift | Amazon S3401k matching | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeSan Diego, California, United States R19h ago
-
Senior Software Engineer, Data Authoring Platform USD 196K-230KAPI Design | Anomaly Detection | Automated testing | DSL | Data GovernanceEmployee travel credits | Remote eligibleSenior-level Full TimeRemote USA R19h ago
-
AWS | Airflow | Apache Spark | Azure Synapse | Azure Synapse Analytics401k matching | Disability insurance | Employee assistance program | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimeRemote, USA ; Remote, Canada R21h ago
-
Principal Data Engineer/ Technical Lead USD 219K-298KAWS | Access Layer | Aggregation pipelines | Apache Kafka | Apache Spark401k match | Employer paid medical/dental/vision | Flexible spending account | Paid parental leave | Remote first work from homeSenior-level Full TimeUnited States (Remote) R22h ago
-
Senior Software Engineer II - (AI Core Platform) USD 100K-177KAPI Development | API Gateway | AWS | Agile | AlertingMid-level Full TimeRemote, United States R22h ago
-
Senior Software Engineer I - AI/ML USD 145K-190KAPI Development | Agile | Alerting | CI/CD | Data ModelingSenior-level Full TimeRemote, United States R22h ago
-
AI Expert USD 148K-175KAWS | Agile | Batch Processing | Data Mapping | Data ModelingHybrid work | Public Trust Clearance | Remote workSenior-level Full TimeMemphis, TN, United States R22h ago
-
People Analytics AI Engineer USD 146K-221KAPI Integration | AWS | Amazon Redshift | Automation | Data ModelingFlexible working | Health benefits | Parental leave plans | Professional development stipend | Remote ModelSenior-level Full TimeRemote - Seattle R23h ago
-
Senior AI Integration Engineer USD 190K-190KAWS | AgenticAI | AmazonS3 | Bash | BedrockPart-time remote workSenior-level Full TimeNew York, New York, United States R1d ago
-
Sr . SAP Datasphere + Databricks Engineer - Hybrid USD 180K-248KAPI | Access Control | CI/CD | Data Classification | Data GovernanceHybrid workSenior-level ContractDurham or Philadelphia, United States R1d ago
-
VP II AI, Machine Learning|US Remote* USD 202K-210KApache Spark | Azure | Databricks | ETL | Flume401k match | Medical/Dental/Vision insurance | Paid time offMid-level Full TimeRemote, United States R1d ago
-
Principal Machine Learning Engineer USD 205K-230KAWS Lambda | BigQuery | C# | CI/CD | Cloud Functions401k | Dental insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeUnited States of America - Remote … R1d ago
-
Generative AI Engineering Intern (Graduate) USD 70K-70KAWS | Agile | Azure OpenAI | Azure OpenAI Service | CI/CDDedicated mentorship | Flexible scheduling | Networking opportunities | Potential full-time employment | Remote friendly schedulingEntry-level Full Time InternshipUnited States R1d ago
-
AI Research Engineer (Applied AI) USD 100K-150KAblation Studies | Accelerator hardware | Data Quality | Data Validation | Data labelingMid-level Full TimeUnited States - Remote R1d ago
-
AI Data Infrastructure Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Caching | Code reviewCareer growth | Health benefits | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
Quantitative Developer (Fintech) USD 100K-150KAudit Logging | Backtesting | C++ | Cloud Computing | ConcurrencyMid-level Full TimeUnited States - Remote R1d ago
-
AI/ML Implementation Engineer (m/f/d) USD 93K-155K8D | APQP | AWS | AWS Lambda | Amazon Bedrock12 paid holidays | Disability benefits | Employee assistance program | Life insurance | Medical, dental & vision coverageSenior-level Full TimeRemote, United States R1d ago
-
Rust, AI Engineer, Director USD 220K-275KAPI Gateway | AWS | Agile | ArgoCD | AzureFlexible time off | Healthcare | Leave benefits | Retirement benefits | Tuition reimbursementExecutive-level Full TimeNY7 - 50 Hudson Yards, New … R1d ago
-
AI Software Engineer (Hybrid Eligible) USD 103K-188KAI Foundry | AI Services | AWS | AWS Bedrock | Azure401k retirement plan | Dental plan | Disability benefits | Educational assistance | Employee assistance planMid-level Full TimeOak Ridge, TN, US, 37830 R1d ago
-
F-35 Data Engineer Mid (Remote) - 28864 USD 110K-140K800-53 | AWS Glue | AWS GovCloud | AWS Lambda | Amazon AthenaAbility to obtain security clearance | Flexible telework schedule | Remote workSenior-level Full TimeArlington, VA, Virginia, United States R1d ago