Data Engineer Data Pipelines and ETL
Tasks
- Architect and build real time dashboards using Superset
- Build ETL and ELT frameworks
- Build and maintain scalable batch and streaming data pipelines
- Build real time data pipelines and event driven architectures
- Design dimensional and hybrid data models for analytics and ML
- Develop reusable transformation and semantic layers
- Enforce data governance privacy and security best practices
- Establish data lineage and metadata standards
- Implement CDC incremental loads and event driven processing
- Implement automated data validation anomaly detection and monitoring
- Optimize pipeline performance reliability scalability and cost
- Perform query optimization indexing partitioning and performance tuning
- Write optimized production SQL for large scale analytics
Perks/Benefits
- 401k plan
- Bonus eligible
- Dental insurance
- Disability benefits
- Life insurance
- Life insurance coverage
- Medical insurance
- Paid time off
- Tuition assistance
- Tuition assistance program
- Vision insurance
Skills/Tech-stack
Airflow | Anomaly Detection | Apache Kafka | Apache Spark | CDC | Cloud Dataflow | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Warehouse | Databricks | Dimensional Modeling | ELT | ETL | Embeddings | Event Driven | Event-driven architecture | Feature Streaming | Google Cloud | Google Cloud Dataflow | Google Pubsub | Indexing | Lakehouse | Monitoring | Partitioning | Performance Tuning | Python | Query Optimization | SQL | Superset | Vector Database | Vertex AI
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R1d ago
-
Lead Infrastructure Engineer USD 135K-199KAWS | Configuration Management | Data Architecture | Data Modeling | Database DesignSenior-level Full TimeAlpharetta, GA, United States9h ago
-
Sr. Data Engineer USD 98K-130KAgile | Airflow | CI/CD | DBT | Data ArchitectureBusiness casual environment | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeLatham, NY, United States10h ago
-
AWS | AWS Lambda | AWS MSK | Alerting | Amazon EKSOn-call support | Production incident troubleshooting opportunitiesSenior-level Full TimeAuburn Hills, MI, United States11h ago
-
Ansible | C# | C++ | CMake | CUDAMentorship | Remote work | Training workshopsMid-level Full TimeBaltimore, MD, United States R13h ago
-
AI and Data Science Engineer III with TS/SCI Polygraph USD 119K-198KData Visualization | Language Models | Language Processing | Large Language Models | Machine LearningSenior-level Full TimeBaltimore, Maryland, United States13h ago
-
Risk Adjustment Analytics Data Scientist USD 122K-240KActuarial analysis | Cohort Segmentation | Data Quality | Data profiling | DatabricksMentorship | Professional developmentSenior-level Full TimeArlington/Rosslyn, Virginia, United States13h ago
-
Director, Data Science - Central Product Platform (CPP) USD 253K-314KA/B | A/B Testing | Agent Orchestration | Apache Hive | Apache SparkExecutive-level Full TimeBellevue, WA | Menlo Park, CA14h ago
-
Software Engineer III, AI/ML, Foundational Lanes USD 147K-211KC++ | Computer Vision | Data Processing | Debugging | Geometric algorithmsSenior-level Full TimeMountain View, CA, USA14h ago
-
Software Engineer, BigQuery Metadata USD 147K-211KBigQuery | C++ | Compute Technologies | Data Warehouse | Data analyticsMid-level Full TimeKirkland, WA, USA14h ago
-
Bash | Cause analysis | Continuous Process | Continuous Process Improvement | Corrective actionsSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Research Engineer, DeepMind USD 262K-365KBigQuery | Data Pipelines | Data analytics | Distributed Systems | FlumeSenior-level Full TimeMountain View, CA, USA14h ago
-
Computer Vision | Data Analysis | Data Visualization | Few-Shot Learning | Few-shotMid-level Full TimeMountain View, CA, USA15h ago
-
Access Management | Algorithms | CI/CD | Cloud platform | ComplianceEquity | Health insurance | Paid time off | Performance bonus | Retirement benefitsSenior-level Full TimeReston, VA, USA15h ago
-
Research Engineer, Human Understanding, DeepMind USD 174K-253KComputer Vision | Deep learning | Experimental Design | Flax | Generative AIMid-level Full TimeMountain View, CA, USA; Los Angeles, …15h ago
-
Software Engineer III, AI Agents USD 147K-211KAgent architecture | Agent systems | C++ | Cloud platform | Computer VisionSenior-level Full TimeSunnyvale, CA, USA15h ago
-
Artificial Intelligence Engineer I USD 99K-176KAWS | Azure | Bioinformatics | Deep learning | DockerMid-level Full TimeHouston, TX, US17h ago
-
Senior Embedded Software Engineer USD 140K-160KC++ | Computer Vision | Coral TPU | Deep learning | Device driverFully-paid health insurance | Paid parental leave | Unlimited PTOSenior-level Full TimeBoston, Massachusetts, United States18h ago
-
VP, Data Engineer - DataBricks USD 143K-190KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageHybrid work model | Work from home optionExecutive-level Full TimeCharlotte, NC, US, 2820219h ago
-
Senior Software Engineer, Data Insights USD 196K-243KAWS VPC | Amazon EKS | Analytics infrastructure | Apache Superset | Data EngineeringSenior-level Full TimeSan Mateo, CA, United States R19h ago
-
Senior Software Engineer, Unsecured Installments USD 160K-223KA/B | A/B Testing | Accessibility | B testing | CI/CD401k match | Dental insurance | Disability insurance | Health insurance | Health savings accountSenior-level Full TimeUnited States | Remote R22h ago
-
Software Engineer, Unsecured Installments USD 136K-196KCI/CD | Cypress | GraphQL | Java | JavaScript401k | Company holidays | Dental insurance | Employee assistance program | Health insuranceSenior-level Full TimeUnited States | Remote R22h ago
-
Senior Machine Learning Engineer, Voice Assistant USD 195K-361KAmazon EMR | Amazon S3 | Apache Airflow | Apache Spark | Artificial IntelligenceFinancial wellness support | Flexible remote work | Mental health support | Paid time offSenior-level Full TimeAustin, Texas22h ago
-
Senior Machine Learning Engineer, Voice Assistant USD 216K-273KAmazon EMR | Amazon S3 | Apache Airflow | Automated Speech Recognition | C++Disability benefits | Equity awards | Health insurance | Hybrid work flexibility | Life insuranceSenior-level Full TimeBoston, Massachusetts22h ago
-
Senior Machine Learning Engineer, Voice Assistant USD 229K-367KAmazon EMR | Amazon S3 | Apache Airflow | Apache Spark | Artificial IntelligenceDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offSenior-level Full TimeSan Jose, California22h ago