Data Engineer Data Pipelines and ETL
Tasks
- Architect and build real time dashboards using Superset
- Build ETL and ELT frameworks
- Build and maintain scalable batch and streaming data pipelines
- Build real time data pipelines and event driven architectures
- Design dimensional and hybrid data models for analytics and ML
- Develop reusable transformation and semantic layers
- Enforce data governance privacy and security best practices
- Establish data lineage and metadata standards
- Implement CDC incremental loads and event driven processing
- Implement automated data validation anomaly detection and monitoring
- Optimize pipeline performance reliability scalability and cost
- Perform query optimization indexing partitioning and performance tuning
- Write optimized production SQL for large scale analytics
Perks/Benefits
- 401k plan
- Bonus eligible
- Dental insurance
- Disability benefits
- Life insurance
- Life insurance coverage
- Medical insurance
- Paid time off
- Tuition assistance
- Tuition assistance program
- Vision insurance
Skills/Tech-stack
Airflow | Anomaly Detection | Apache Kafka | Apache Spark | CDC | Cloud Dataflow | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Warehouse | Databricks | Dimensional Modeling | ELT | ETL | Embeddings | Event Driven | Event-driven architecture | Feature Streaming | Google Cloud | Google Cloud Dataflow | Google Pubsub | Indexing | Lakehouse | Monitoring | Partitioning | Performance Tuning | Python | Query Optimization | SQL | Superset | Vector Database | Vertex AI
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior-level Full TimeCanada R6h ago
-
Senior-level Full TimeUnited States R7h ago
-
Senior-level Full TimeUnited States R7h ago
-
Senior Software Engineer - AI Integrations USD 170K-240KAWS | Alerting | C++ | CSS | Continuous integrationSenior-level Full TimeMountain View, CA11h ago
-
A/B | Asymmetric A B | C# | Cryptography | Delta CompressionSenior-level Full TimeSunnyvale, United States12h ago
-
Lead Data Engineer USD 170K-220KAccess Control | Automation | BI Tooling | Backups | Business Intelligence401k | Accessories allowance | Education stipend | Equity tax advisory service | Financial Wellness WebinarsSenior-level Full TimeNew York, NY12h ago
-
Deep Learning Quality Specialist USD 72K-90KAnnotation Guidelines | Computer Vision | Confluence | Convolutional Neural Networks | Data Annotation401k plan | Commuter benefits | Employee assistance program | Flexible PTO | Fully paid medical/dental/visionMid-level Full TimeSeattle, WA R12h ago
-
Corporate AI Engineer USD 154K-200KAPI Integration | Access Control | Data Quality | Embeddings | Generative AIHybrid work schedule | Volunteer time offMid-level Full TimeAddison, TX (Hybrid); Bellevue, WA (Hybrid); … R13h ago
-
AI Automation Engineer USD 96K-140KAPI Integration | Automation | Confidence Thresholding | Connectwise | DocumentationMid-level Full TimeGrand Rapids, Michigan14h ago
-
Software Robotics Design Engineer USD 120K-150KC Sharp | C plus plus | Commissioning | Digital Twin | HMISenior-level Full TimeBoston, Massachusetts16h ago
-
Embedded Software Engineer USD 110K-145KArtifactory | Atlassian Bamboo | Bash | Bitbucket | C++On-site workMid-level Full TimeLafayette, CO16h ago
-
AI Software Engineer-Senior USD 135K-200KAgile | CI/CD | Confluence | Data Engineering | Data PipelinesSenior-level Full TimeAnnapolis Junction, Maryland, United States16h ago
-
Mid-level Full TimeAnnapolis Junction, Maryland, United States16h ago
-
Analytic Software Developer-Senior USD 132K-190KData integration | Data repositories | Java | Jupyter Notebooks | MapReduceSenior-level Full TimeAnnapolis Junction, Maryland, United States16h ago
-
AI Analytic Software Engineer-Senior USD 150K-195KAWS Amplify | AWS Bedrock | Docker | Elasticsearch | Elasticsearch QuerySecurity ClearanceSenior-level Full TimeAnnapolis Junction, Maryland, United States16h ago
-
Senior-level Full TimeAnnapolis Junction, Maryland, United States16h ago
-
Analytics Engineer USD 120K-160KAPI Development | AWS | Azure | CI/CD | Containerization401k | Dental insurance | Medical insurance | Paid Holidays | Paid time offMid-level Full TimeBrooklyn, New York, United States17h ago
-
FDE Data Engineer- Space USD 123K-166KABAC | Apache Airflow | Apache Flink | Apache Hudi | Apache IcebergHealth benefits | Remote work | Travel for site workMid-level Full TimeUnited States - Remote R18h ago
-
Senior FDE Data Engineer USD 148K-201KABAC | Airflow | Apache Iceberg | Apache Kafka | Argo WorkflowsCompany paid premiums | Health savings account | Medical/Dental/Vision | Remote work | Travel to classified sitesSenior-level Full TimeUnited States - Remote R18h ago
-
DevOps Engineer USD 150K-170KAWS | Bash | CI/CD | DAST | Dependency ScanningDental insurance | Health insurance | LTD insurance | Life insurance | One Medical membershipSenior-level Full TimeUnited States20h ago
-
AI Observability | AWS | Azure | CI/CD | Cost ControlCareer advancement | Fully remote work | Professional development opportunities | Work-life balanceSenior-level Full TimeCanada R20h ago
-
Data Engineer USD 122K-174KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeMontvale, New Jersey, United States21h ago
-
Senior-level Full TimeLawrenceville, Georgia, United States22h ago
-
Machine Learning Engineer USD 90K-140KClassification | Clustering | Machine Learning | Model Evaluation | PyTorch401k | Educational assistance | Profit sharingMid-level Full TimeFranklin, Wisconsin, United States22h ago
-
AWS Glue | AWS Lambda | Amazon Redshift | Amazon Web Services | AnsibleContinuous learning | Flexible working hours | Hybrid work model | Professional development | Retirement pension and savingsSenior-level Full TimeCanada23h ago