Data Engineer - AWS/Databricks
Reston, VA, United States
R
USD 122K-210K (estimate) Entry-level Full Time
Tasks
- Build and maintain scalable PySpark data pipelines in Databricks
- Build cloud native data solutions on AWS using S3 Glue RDS and IAM
- Conduct technical discovery and map legacy data sources
- Deploy and version data engineering assets using Git and CI CD
- Design and implement Delta Lake tables
- Develop ETL and ELT workflows for centralized data warehouse
- Implement business rules using Spark SQL and DataFrame APIs
- Implement data governance metadata tagging data quality validation and lineage tracking
- Integrate source systems into query optimized warehouse architecture
- Monitor pipeline health job execution and cluster utilization
- Optimize pipeline performance using partitioning caching and joins
- Support ad hoc data access requests and maintain shared notebooks
Perks/Benefits
Skills/Tech-stack
ACID | AWS Glue | AWS S3 | Adaptive Query | Adaptive query execution | Amazon CloudWatch | Amazon RDS | Apache Spark | Audit Logging | Broadcast Join | CI/CD | Caching | Data Governance | Data Lineage | Data Quality | Data Validation | Databricks | Dataframe API | Delta Lake | ELT | ETL | Git | IAM | Metadata tagging | Partitioning | PySpark | Python | Query Execution | Spark SQL
Education
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R25d ago
-
Lead AI Engineer- GTM Systems USD 131K-176KAWS Bedrock | AWS Lambda | Agent coordination | Async event-driven architecture | Async/event-drivenNo relocation assistance not provided | Remote workSenior-level Full TimeRemote - US R16h ago
-
Lead AI Engineer USD 198K-261KAgentic Frameworks | CI/CD | Cloud Computing | Containers | Fine TuningSenior-level Full TimeUSA, USA R16h ago
-
A/B | A/B Testing | AWS | AWS SageMaker | Apache Spark401k employer match | Commuter benefit | Dental insurance | Dependent Care Flexible Spending Account FSA | Flexible time offSenior-level Full TimeLos Angeles, California, United States; San … R17h ago
-
Senior Data Scientist, Applied ML USD 154K-200KAWS | Airflow | Apache Spark | DVC | Data Transformation401k employer contribution | Employer-paid life insurance | Flexible Remote Friendly Work Options | Generous paid holidays | Health Savings Account employer contributionSenior-level Full TimeAustin, Texas | Remote R18h ago
-
Sr Machine Learning Engineer I USD 150K-241KAutomated retraining | Bayesian Filter | CI/CD | Data Association | Extended KalmanDiscretionary paid time off | Emotional and mental wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R20h ago
-
AI Engineer II ( AI Platform) USD 104K-177K.NET | AWS Bedrock | Asynchronous processing | Audit trails | Azure OpenAIMentorship | Remote work | Soft skill sessions | TrainingMid-level Full TimeUS Remote R1d ago
-
Principal AI Engineer USD 130K-197KAI monitoring | Cloud Computing | Deep learning | Fine Tuning | Language Processing401k match | Employee awards | Hybrid work | Industry certifications | MentorshipSenior-level Full TimeUnited States R1d ago
-
Senior Software Engineer (Storage) - remote in the US USD 100K-155KArgoCD | CSI | Cluster API | Flux | GPU DirectConference attendance | Professional development and trainingSenior-level Full TimeRemote, USA, United States R1d ago
-
Data Engineer USD 133K-159KAWS Glue | AWS Lambda | Amazon Redshift | Amazon S3 | Amazon SNS401k match | Casual dress code | Company paid life insurance | Dental insurance | Health insuranceMid-level Full TimeLehi, UT, US R1d ago
-
SkillBridge – US 2026 Active Duty US Military ONLY USD 110K-176KApplication development | Data Visualization | Databases | Databricks | Machine LearningRemote workEntry-level Full TimeAtlanta, GA - Remote R1d ago
-
AI-Driven DevOps Engineer USD 132K-198KAI Agent | Alerting | Ansible | Artificial Intelligence | BashAdditional paid day off per month | Comprehensive onboarding and training | Conference participation fee coverage | English language classes discount | Fully remote workSenior-level Full TimeCyprus - Remote R1d ago
-
NLP AI Engineer USD 130K-180KAWS | Cloud platform | DeepSpeed ZeRO | Direct Preference Optimization | Distributed Training100 percent remote U.S. | Direct W2Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Applied AI Engineer USD 130K-180KAWS | Agentic AI | CI/CD | Cloud platform | Computer VisionCareer growth | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Embedded Robotics Engineer USD 125K-145KC++ | Concurrent Systems | Control Systems | Debugging | GazeboCareer growth | Code review | Design review | Mentorship | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Embedded Software Engineer USD 130K-195KARM Cortex | ARM Cortex A | ARM Cortex-M | Agile | Azure DevOpsSenior-level Full TimeUnited States - Remote R1d ago
-
Quantitative Engineer USD 150K-185KC++ | Cloud Native | Cloud Native Architecture | Columnar Databases | ConcurrencyCareer growth | Mentorship | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUSA- New York Remote Office, United … R1d ago
-
Senior Data Engineer USD 91K-163KAPIs | Access Control | Batch Processing | Cloud Data | Cloud Data Platforms401k contribution | Comprehensive benefits package | Equity stock purchase | Hybrid work schedule | Incentive and recognition programsSenior-level Full TimeEden Prairie, Minnesota, UNITED STATES; Irvine, … R1d ago
-
Calculation Groups | DAX | Data Governance | Data Modeling | Data Quality401k match | Associate discounts | Community involvement opportunities | Dental insurance | Health insuranceMid-level Full TimeRemote, United States R1d ago
-
Senior Lead Data Engineer (Remote) USD 118K-196KAPI Integration | Apache Spark | Authentication | Azure | Azure DataRemote work flexibility | Travel 10 percentSenior-level Full TimeMichigan, Virtual Address, United States R1d ago
-
Senior Decision Intelligence Engineer (NBA) USD 106K-147KA/B | A/B Testing | Airflow | Artifact versioning | B testingFlexible schedule | Health and wellness benefits | Hybrid work option | Remote work optionSenior-level Full TimeWork at Home - Kentucky, United … R1d ago
-
Principal Engineer, Data USD 101K-175KApache Spark | Artificial Intelligence | Azure | CI/CD | Code reviewSenior-level Full TimeWork at Home - Ohio - … R1d ago
-
Cloud Computing | Data Engineering | Experimentation | Generative AI | Language ModelsFlexible time off | Healthcare | Hybrid work model | Leave benefits | Retirement benefitsExecutive-level Full TimeNY7 - 50 Hudson Yards, New … R1d ago