Data Engineer - AWS/Databricks - Mid Level
Reston, VA, United States
R
USD 121K-170K (estimate) Mid-level Full Time
Tasks
- Build PySpark data pipelines in Databricks
- Design Delta Lake tables
- Develop ETL ELT data transformations with Spark SQL
- Implement AWS cloud data solutions with S3 Glue RDS and IAM
- Implement ETL and ELT workflows
- Implement data governance metadata tagging data quality validation audit logging and lineage
- Manage Git versioning and CI CD deployments
- Monitor data pipeline health and cluster utilization
- Optimize pipeline performance for cost and throughput
- Perform technical discovery of legacy data sources
Perks/Benefits
- Comprehensive benefits
- Mentorship
- Mentorship and personalized development plans
- Training budget
- Work/life balance focus
Skills/Tech-stack
AWS | AWS Glue | Adaptive query execution | Amazon CloudWatch | Amazon RDS | Apache Spark | Audit Logging | Broadcast joins | CI/CD | Caching | Data Governance | Data Modeling | Data Quality | Data Warehousing | Databricks | Dataframes | Delta Lake | Delta Lake Tables | ELT | ETL | Git | GitLab | IAM | Jenkins | Lineage | Metadata | Partitioning | PySpark | Query Execution | S3 | Spark SQL
Education
Roles
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R3d ago
-
Principal Engineer - Data Platform USD 221K-387KAWS | Airflow | Apache Hive | Apache Iceberg | Apache ImpalaRemote workSenior-level Full TimeSanta Clara, California, United States R10h ago
-
AWS Glue | AWS Lambda | AWS S3 | Access Control | Data GovernanceCareer growth opportunities | Collaborative and inclusive work environment | Diverse and inclusive culture | Flexible work arrangements | Permanent remote working modelSenior-level Full TimeCanada R22h ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Data Quality | Data labeling | Data quality monitoring100 percent remote | Career growth | Full-time employment | W2 employmentMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data QualityMid-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache SparkRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HiveRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KActive Learning | Apache Beam | CI/CD | Caching | Code review100 percent remote | Career growth | Full-time employment | H1B transfer support | W2 employmentMid-level Full TimeUnited States - Remote R1d ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Code review | Data Lineage | Data Modeling100 percent remote | Career growthMid-level Full TimeUnited States - Remote R1d ago
-
Principal Applied AI Engineer, Finance USD 193K-340KAPI Development | AWS | Bias Mitigation | CI/CD | Churn modeling401k matching | Adoption Assistance | Development and career growth opportunities | Fertility treatments | Flexible work schedulesSenior-level Full TimeVirtual Office (Massachusetts), United States R1d ago
-
Data Engineer - 2373625 USD 98K-172KAzure Databricks | Data Modeling | Data Pipeline Awareness | Data Visualization | Data Warehousing401k matching | Employee stock purchase plan | Medical, dental, and vision | TelecommutingEntry-level Full TimePrimary location: Eden Prairie, MN R1d ago
-
Senior Data Engineer - 2373647 USD 146K-172KAWS | Agile Scrum | Azure | Cloud technologies | Docker401k matching | Dental insurance | Employee stock purchase plan | Medical insurance | TelecommutingSenior-level Full TimePrimary location: Eden Prairie, MN R1d ago
-
AWS | Airflow | Apache Spark | Azure Event | Azure Event Hubs401k contribution | Career development opportunities | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programsSenior-level Full TimePrimary location: Eden Prairie, MN R1d ago
-
Data Engineer USD 72K-130KAI/ML | Analytics engineering | Azure DevOps | Bronze Silver Gold | CI/CD401k contribution | Career development opportunities | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programsMid-level Full TimePrimary location: Eden Prairie, MN R1d ago
-
NLP Engineer USD 72K-130KArtifact Repositories | Artifactory | C# | CI/CD | Containerization401k contribution | Career development opportunities | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programsMid-level Full TimePrimary location: San Diego, CA R1d ago
-
Principal, Data Engineer USD 107K-216KAI/ML | Azure | Azure Data | Azure Data Factory | Azure DevOpsSenior-level Full Time11 Keewaydin Dr, Salem NH, United … R1d ago
-
AWS | Application Security | Artificial Intelligence | Azure | Cloud SecurityConference speaking opportunities | Flexible schedule | Health Premium Plan Option | Mentorship | Paid trainingSenior-level Full TimeLos Angeles, California, United States R1d ago
-
Machine Learning Engineer V USD 231K-382KAWS | Agent Orchestration | Automated testing | Azure | CI/CDBonus eligibility | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, United States R2d ago
-
Senior AI Engineer USD 145K-181KAWS | Alerting | Azure | Docker | Embeddings401k match | Commuter benefits | Dental | Healthcare | Remote friendly workplaceSenior-level Full Time3750 Market Street, Philadelphia, PA, United … R3d ago
-
AWS | AWS CDK | Access Control | Airflow | Athena401k plan | Health insurance | Paid Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R3d ago
-
Business Analytics | Dashboard Development | Data Engineering | Data Modeling | Data TransformationEmployer matched 401k plan | Health insurance | Paid Holidays | Paid time off | Remote workSenior-level Full TimeRedmond, WA, United States R3d ago
-
Principal Software Engineer, Data Infrastructure USD 295K-345KAWS | Airflow | Chaos Engineering | Data Catalog | Distributed SystemsEquity compensation | Health benefits | Onsite work flexibilitySenior-level Full TimeSan Mateo, CA, United States R3d ago
-
Data Warehouse Software Engineer I USD 70K-80KApache Airflow | Cloud Composer | Clustering | Data Lakes | Data Marts401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeRemote - United States R3d ago
-
Machine Learning Engineer USD 140K-180KAWS | Alerting | Apache Spark | Azure | CI/CD401k | Dental insurance | Life insurance | Light travel | Medical insuranceSenior-level Full TimeSt. Louis, MO; Boston, MA; New … R3d ago
-
Software Engineer ll, Data Platform USD 110K-120KAPI | Amazon Athena | Amazon EMR | Amazon S3 | Apache Airflow401k match | Company holidays | Disability insurance | Employee assistance program | Flexible spending accountMid-level Full TimeUnited States R3d ago