Lead Data Engineer / Mission Data Pipeline
Reston, Virginia, United States
USD 129K-184K (estimate) Senior-level Full Time
Tasks
- Build scalable ETL workflows
- Continuously improve and sustain data pipeline ecosystem
- Create data artifact technical documentation
- Create interface documentation for external mission partners
- Define data product structure semantics lifecycle
- Design data exchange interfaces
- Design production data pipelines automated ingestion dissemination
- Develop custom transformation logic
- Develop message schemas data formats transport mechanisms
- Expose data products via REST APIs
- Identify mission data artifacts
- Implement automated data pipelines ingest transform persist expose data artifacts
- Implement data validation normalization transformation logic
- Optimize pipeline performance
- Produce RMF artifacts data message descriptions
- Support RMF process technical machine to machine interfaces
- Support data interface documentation
- Support security authorization technical artifacts
- Troubleshoot pipeline failures implement monitoring logging recovery
Perks/Benefits
- 401k match
- Company Employee Stock Ownership Plan
- Education Training Professional Development
- Flexible spending accounts
- Health savings account
- Life and AD D Insurance
- Long-term disability
- Medical/Dental/Vision insurance
- Paid Holidays
- Paid time off
- Parental leave
- Short-term disability
Skills/Tech-stack
Apache Airflow | Apache NiFi | CSV | Cloud Data | Cloud Data Architecture | Data Architecture | Data Interoperability | Data Modeling | Data Normalization | Data Pipelines | Data Validation | Data integration | ETL | JSON | Logging | Monitoring | NumPy | Pandas | Python | REST APIs | XML
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior Software Engineer, Data USD 126K-189KAPI Development | AWS | Airflow | Docker | ETL401k plan | Commuting Stipend | Dental insurance | Dependent Care Flexible Spending Account | Employee assistance programSenior-level Full TimeSeattle, WA6h ago
-
Senior-level Full TimeNew York10h ago
-
Staff software engineer, data products USD 202K-255KBackend Development | Data Integrity | Data Modeling | Data Pipelines | Data TransformationIn office 4 days per week | Remote-friendlySenior-level Full TimeSan Francisco10h ago
-
Agentic AI Developer USD 140K-200KAgent Framework | Asynchronous programming | Autonomous Agents | Azure Cognitive | Azure Cognitive SearchMid-level Full TimeSan Francisco, CA, United States10h ago
-
Data Mining | Data labeling | Dataset creation | Deep learning | Knowledge DistillationSenior-level Full TimeFoster City, CA11h ago
-
Senior Data Engineer USD 82K-193KAgile | BigQuery | CI/CD | Data Engineering | Data PipelinesCompany-Paid Holidays | Dental insurance | Disability insurance | Employee assistance program | Life insuranceSenior-level Full TimeBridgewater, NJ, US11h ago
-
Senior Data Engineer USD 155K-190KAmazon Redshift | Apache Airflow | Dagster | Data Compliance | Data GovernanceSenior-level Full TimeNortheast11h ago
-
Senior Data Engineer USD 155K-190KAmazon Redshift | Apache Airflow | Dagster | Data Governance | Data QualitySenior-level Full TimeSan Francisco11h ago
-
Machine Learning Engineer - ETA Team USD 137K-299KAirflow | Apache Spark | Deep learning | Experimentation | Feature Engineering401k plan employer matching | Basic life insurance | Commuter benefits match | Dental insurance | Disability insuranceMid-level Full TimeSunnyvale, CA; San Francisco, CA; Seattle, …12h ago
-
Analytics Engineer, Sentry USD 146K-194KDashboarding | Data Engineering | Data Modeling | Data Pipelines | Data QualityCompetitive benefits | Health insurance | Paid time offMid-level Full TimeIrvine, California, United States12h ago
-
Manufacturing Engineer, Analytics USD 129K-171KLean Manufacturing | Microsoft Excel | PLSQL | PowerBI | PythonHealth benefits | Recovery BenefitsMid-level Full TimeCosta Mesa, California, United States12h ago
-
Senior-level Full TimeSeattle, Washington, United States12h ago
-
Sr Engineer II, Machine Learning USD 139K-183KAWS | Bioinformatics | Cell analysis | Cloud Computing | Computer Vision401k match | Discretionary bonus | Flexible paid time off | Health insurance | Stock optionsSenior-level Full TimeSan Diego - Headquarters13h ago
-
Data Engineer USD 84K-110KAPI Development | AWS | Apache Airflow | Apache Spark | AzureHybrid work scheduleEntry-level Full TimeSan Diego, California13h ago
-
Senior Data Engineer, Solutions Architecture USD 110K-145KAPIs | AWS | Access Control | Amazon S3 | AnthropicAutonomous work | Company-provided meals and events | Hybrid work environmentSenior-level Full TimeSan Francisco, California, United States13h ago
-
Senior Data Engineer, Solutions Architecture USD 110K-145KAPI | AWS | Access Control | Airflow | AzureCompany-provided meals | Events | Hybrid work environmentSenior-level Full TimeSan Diego, California, United States14h ago
-
Senior Data Engineer, Solutions Architecture USD 110K-145KAPI | AWS | Access Control | Airflow | AzureAutonomy | Hybrid work environment | Opportunity for impactSenior-level Full TimeScottsdale, Arizona, United States14h ago
-
Senior Data Engineer USD 128K-161KAPI Integration | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake401k match | Hybrid work schedule | Paid time off | Parental leaveSenior-level Full TimeBloomington, MN, United States14h ago
-
Senior AI Data Engineer USD 160K-200KAWS | AWS Athena | AWS Glue | AWS Lambda | Amazon Redshift401k matching | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeSan Diego, California, United States R14h ago
-
Causal Inference | Classification | Clustering | Data Warehousing | Experiment designFlexible PTO | Home office stipend | Learning budget | Paid health, dental, vision | Parental leaveSenior-level Full TimeBoston or Remote R14h ago
-
Senior Partner Engineer, Databricks Alliance USD 170K-240KBusiness Intelligence | Cloud Data | Cloud Data Platforms | Data Engineering | Data Warehousing401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeSan Francisco, CA15h ago
-
Senior Data Engineer USD 155K-220KAPI Integration | Alerting | Amazon Redshift | Automated testing | CI/CDSenior-level Full TimeDenver, CO;San Francisco, CA;New York, NY;Los … R15h ago
-
Senior Data Platform Engineer USD 150K-180KAWS | Containerization | Data Management | Data Processing | Data StructuresGenerous time away from work | Health care insurance | Life insurance | Recognition program | Retirement savings plansSenior-level Full TimeAustin, TX15h ago
-
Senior Data Engineer USD 180K-200KAirflow | BigQuery | CI/CD | DBT | Data Governance401k | Catered lunches | Dental insurance | Employer matching | HSA/FSASenior-level Full TimeNew York City15h ago
-
Infrastructure Engineer (Storage) USD 180K-200KAutomation | Bare Metal | Bring-up | Capacity Planning | CephComprehensive medical/dental/vision coverage | Flexible work environment | Paid parental leave | Paid time off | Pension contributionMid-level Full TimeNew York, New York, United States; … R15h ago