Data Engineer II
Tasks
- Automate near real time data workflows
- Build ETL ELT pipelines on AWS
- Build SQL tables and queries
- Create and optimize data models
- Design and publish reports
- Develop Python or JavaScript parsing code
- Develop data acquisition and quality protocols
- Implement alerting and observability
- Implement data governance lineage cataloging and access controls
- Integrate data from APIs and databases
- Lead ETL pipeline operations
- Maintain analytics data architecture
- Mentor and collaborate with squadron personnel
- Monitor pipeline health and data quality
- Support data scientists with curated datasets
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | Access Controls | Alteryx | Amazon Redshift | Amazon S3 | Apache Airflow | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Databricks | CI/CD | DBT | Data Cataloging | Data Factory | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Vault | Data Visualization | Databricks | Docker | Docker Compose | Duplicate Detection | Elasticsearch | Elasticsearch Kibana | Git | Google BigQuery | HTTP | Kafka | Kibana | Kubernetes | Medallion Architecture | Null Checks | Power BI | Prefect | Python | R | REST APIs | SOAP | SQL | Schema Validation | Snowflake | Source to Target | Source-to-Target Mapping | Star Schema | Tableau | Target Mapping | Workflow Orchestration
Education
Roles
Related jobs
-
Senior Analytics Engineer USD 140K-175KBusiness Intelligence | DBT | Data Modeling | Data Pipelines | Data WarehousingDental insurance | Medical insurance | Paid time off | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA11h ago
-
Machine Learning Engineer II USD 46K-54KAWS | Apache Spark | Azure | Business Intelligence | Data Warehousing401k | Career advancement | Dental insurance | Disability insurance | Health insuranceMid-level Full TimeChicago, IL16h ago
-
Principal AI Engineer, Special Programs USD 220K-350KAI Safety | API Development | API Orchestration | AWS | Cloud platform401k retirement plan | Dental insurance | Employee stock purchase plan | Life insurance | Long-term disability insuranceSenior-level Full TimePalo Alto, CA16h ago
-
Software Engineer, Statistical Evaluation and Sampling USD 170K-216KC++ | Data Processing | Data Processing Pipelines | Importance sampling | Machine LearningEntry-level Full TimeMountain View, CA, USA; San Francisco, …16h ago
-
Senior Software Engineer, Data Infrastructure USD 200K-400KAKS | AWS | Airflow | Azure | BigQueryDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical, dental, vision benefitsSenior-level Full TimeNew York City17h ago
-
Senior Software Engineer, Data Infrastructure USD 200K-400KAWS | Airflow | Azure | BigQuery | CDCDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical, dental, and vision benefitsSenior-level Full TimeSan Francisco17h ago
-
Senior Software Engineer, Storage USD 166K-210KAmazon CloudWatch | Amazon EC2 | Backups | Cause analysis | Cloud-basedAnnual equity refresh grants | Equity grants | Remote workSenior-level Full TimeUnited States - Remote R17h ago
-
Senior Software Engineer II, Storage USD 192K-242KAmazon CloudWatch | Amazon EC2 | Amazon RDS | Backups | Cloud platformAnnual refresh grants | Equity grant | Remote workSenior-level Full TimeUnited States - Remote R17h ago
-
Senior Software Engineer, Data Governance & Foundations USD 166K-210KApache Airflow | Apache Flink | Apache Hudi | Apache Iceberg | Apache KafkaAnnual refresh grants | Equity grant | Remote work flexibilitySenior-level Full TimeUnited States - Remote R17h ago
-
Associate Software Engineer, Embedded Development USD 100K-150KAOSP | Android | Bash | Black box testing | Black-box401k match | Dental insurance | Free snacks | Health insurance | Life insuranceMid-level Full TimeRaleigh, NC R17h ago
-
Machine Learning Engineer - Semantic Reasoning (Highway) USD 200K-293KBEV | C++ | Computer Vision | Deep learning | JAXPaid time offSenior-level Full TimeFoster City, CA17h ago
-
AWS | AWS Batch | AWS Bedrock | AWS CDK | AWS CodeBuild401k | Health insurance | Paid time off | Wellness stipendSenior-level Full TimeSan Carlos - Hybrid R18h ago
-
IT Controls Data Engineer USD 293K-385KAnalytics engineering | Automation | Data Engineering | Data Lineage | Data ModelingSenior-level Full TimeSan Francisco18h ago
-
Dashboard | Data Visualization | Data pipeline | ETL | Machine LearningOnsite days schedule | Overtime paySenior-level Full TimeSan Mateo, CA, United States R18h ago
-
Sr. Machine Learning Engineer USD 175K-230KAWS | C plus plus | Deep learning | Kubernetes | Language Models401k plan | Cell phone internet reimbursement | Company-Paid Holidays | Flexible paid time off | Health Savings Account employer contributionSenior-level Full TimeRemote - United States R18h ago
-
Senior Software Engineer - Infrastructure R&D USD 130K-300KAWS | Azure | CI/CD | Cloud infrastructure | Cloud platformCommunity guilds | Hybrid work | Inclusion talks | Mentor and buddy program | Professional developmentSenior-level Full TimeDenver, Colorado, USA; New York, New …20h ago
-
Senior Software Engineer - Environments Accelerator USD 130K-300KAWS | Automation | CI/CD | Cloud infrastructure | Cloud platformCommunity guild access | Continuous professional development | Free mental health benefits | Hybrid workplace | Mentor and buddy programSenior-level Full TimeDenver, Colorado, USA; New York, New …20h ago
-
Senior AI & ML Engineer USD 194K-228KAPIs | Agent Orchestration | Agent routing | Agents SDK | Cloud infrastructureSenior-level Full TimeUnited States - Remote R20h ago
-
Robotics Framework Engineer II USD 143K-195KC++ | CI/CD | Cross-compilation | EtherCAT | Forward KinematicsMid-level Full TimeAustin, Texas, United States20h ago
-
AI Engineer - Operations USD 115K-165KAI Agents | AI coding | AI coding tools | Coding Tools | Data Engineering401(k) plan match | Barista coffee bar | Corporate events | Coworking spaces | Gym reimbursementMid-level Full TimeAlpharetta, GA20h ago
-
Staff ML Systems Engineer, Distributed Systems USD 70K-300KCPU | Concurrency | DDP | Dask | Data LocalitySenior-level Full TimeSeattle, WA21h ago
-
Senior Staff data science engineer USD 141K-206KAzure Machine Learning | CI/CD | Data Science | Databricks | Distributed SystemsDisability insurance | Employee assistance program | Flexible spending account | Health savings account | Life insuranceSenior-level Full TimeMilpitas, CA, United States22h ago
-
Lead Software Engineer, Data Platform USD 220K-250KAPI Design | Auto-labeling | Azure | Computer Vision | Data pipeline100 Percent covered medical dental vision | Citi Bike membership | Commuter benefits | Free lunch | Holiday observancesSenior-level Full TimeNew York, NY22h ago
-
Senior 3D Print Optimization Engineer USD 115K-148KCause analysis | Computer Aided Design | Experiment design | Material characterization | Python401k matching | Culture events | Dental insurance | Equity RSUs | Flexible time offSenior-level Full TimeBoston, MA22h ago
-
Senior 3D Print Optimization Engineer USD 115K-148K3D Printing | CAD | Cause analysis | Chemistry | Electronics401k matching | Culture events | Equity RSUs | Financial advisors | Flexible out of office planSenior-level Full TimeSomerville, MA22h ago