Data Pipeline Engineer, Senior
0BT CAMP H M SMITH HI, United States
USD 75K-158K Senior-level Full Time
Tasks
- Design automated workflows for data ingestion transformation validation synchronization and delivery
- Design secure enterprise data pipelines supporting logistics and mission support operations
- Develop ETL ELT processes integrating government and commercial data systems
- Develop data quality rules validation procedures monitoring and error handling
- Develop technical documentation data models interface specifications and standard operating procedures
- Ensure compliance with cybersecurity data governance and database management standards
- Integrate data from logistics systems asset visibility and transportation management systems
- Mentor junior database and integration personnel
- Monitor pipeline performance and ensure operational availability
- Optimize pipeline performance database queries storage utilization and processing efficiency
- Support cloud migration and enterprise modernization
- Support near real time data feeds for operational dashboards and analytics
- Troubleshoot integration issues and implement corrective actions
Perks/Benefits
- Continuing education
- Flexible time off
- Healthcare
- Learning and development opportunities
- Retirement benefits
- Wellness
Skills/Tech-stack
API Integration | APIs | AWS GovCloud | Apache Kafka | Apache NiFi | Azure Data | Azure Data Factory | Azure Government | Cloud Computing | Cloud Native | Cloud-native Architectures | Data Factory | Data Governance | Data Pipelines | Data Quality | Data Quality Management | Data Validation | Data Warehousing | Database Performance Optimization | Database performance | Databricks | DevSecOps | ELT | ETL | Microsoft 365 | Oracle Database | Performance optimization | PostgreSQL | Power BI | Python | Quality management | SQL Server | SharePoint | Zero Trust
Related jobs
-
AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | AutomationMultiple project extensions | Paid on time | Remote work optionSenior-level Full TimeJuno Beach, FL R11h ago
-
AI Agents | API Development | Azure | CI/CD | LLMOpsCollaborative cross-functional team | Fully remoteMid-level Full TimeCanada, Ontario; Toronto, None; Toronto, Ontario; … R11h ago
-
API Integration | AWS Bedrock | Auditing | Authentication | AuthorizationMentoring | MentorshipMid-level Full TimeMenlo Park, CA, United States12h ago
-
Senior Software Engineer (Java Full Stack) - Hybrid USD 138K-183KAgile | C plus plus | Concurrency | Data Structures | Embedded SystemsHybrid work | Remote work possibleSenior-level ContractBoston, United States14h ago
-
Lead Palantir AI and Data Science Engineer II USD 159K-265KAI workflows | Analytics | Cloud Computing | Dashboards | Data GovernanceProfessional development | Travel opportunitiesSenior-level Full TimeRosslyn, Virginia, United States14h ago
-
Analytics and Insights Engineer III USD 131K-218KApache Spark | Data Governance | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeRichmond, Virginia, United States14h ago
-
3D Rendering | A/B | A/B Testing | Agentic AI | AlgorithmsSenior-level Full TimeSan Jose, California, United States15h ago
-
C++ | Computer Vision | Data Processing | Deep learning | Domain AdaptationEntry-level Full TimeSan Jose, California, United States15h ago
-
Senior Software Engineer, Google Cloud Dataproc USD 174K-252KApache Hive | Apache Spark | C++ | Data Warehouse | Data Warehouse InternalsSenior-level Full TimeSunnyvale, CA, USA16h ago
-
Agentic Orchestration | C++ | Data Onboarding | Data Pipeline Engineering | Data pipelineSenior-level Full TimeSan Jose, CA, USA16h ago
-
Software Engineer III, Infrastructure, Infra Spanner USD 147K-210KC++ | Concurrency | Data Corruption | Data corruption diagnosis | Database performanceSenior-level Full TimeSunnyvale, CA, USA16h ago
-
Senior Software Engineer, AI/ML, Core USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA16h ago
-
Staff Software Engineer, Search Local Ads Data Platform USD 207K-300KC plus plus | CSS | Data Storage | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA16h ago
-
Analytics Data Engineer USD 94K-130KAzure Data | Azure Data Factory | CI/CD | Data Analysis | Data FactoryRemote workMid-level Full TimeFlagstaff, AZ, US22h ago
-
AI Application & Cloud Operations Engineer (Mid level) USD 108K-117KAWS | AWS Bedrock | AWS CloudTrail | AWS CloudWatch | AWS PrivateLinkMid-level Contract Full TimePlano, TX, United States22h ago
-
Benchmarking | C plus plus | CUDA | Computational Engineering | Distributed Computing401k match | Generous paid time off | Medical, dental & vision coverage | Stock options | WFH Hardware ReimbursementSenior-level Full TimeSunnyvale, CA R22h ago
-
Software Engineer, Applied AI USD 173K-250KABAC | Agentic Frameworks | Apache Airflow | Audit trails | BIMDental insurance | Generous PTO | Health insurance | Retirement or Pension Plan | Vision insuranceMid-level Full TimeSan Francisco, CA23h ago
-
Staff Data Engineer USD 147K-184KAmazon Kinesis | Amazon Web Services | Apache Airflow | Apache Iceberg | Apache SparkDental insurance coverage | Health insurance coverage | Paid parental leave | Paid vacation | Relocation assistanceSenior-level Full TimeBerkeley, CA1d ago
-
Associate, Client Intelligence (Data Scientist) USD 98K-132KAWS | Apache Spark | Azure | Big Data | Cloud platformEmployee resource groups | Matching gift program | Professional development | Sustainability and Climate Change Initiatives | Volunteerism programMid-level Full TimeNew York, NY, US, 100011d ago
-
Apple Neural Engine | C++ | CPU | Compilers | Computer ArchitectureMid-level Full TimeSeattle1d ago
-
On-Device ML Infrastructure Engineer (CoreML Runtime), Graphics, Games and Machine Learning USD 170K-296KC++ | Core ML | GPU Programming | JAX | LLVMMid-level Full TimeCupertino1d ago
-
Apple Neural Engine | C plus plus | CPU performance | Compilers | Computer ArchitectureMid-level Full TimeCupertino1d ago
-
AI Developer - Model Creation & Full Stack USD 130K-145KAWS | Aiohttp | Angular | Asyncio | Azure DatabricksRemote work | USPS Public Trust Clearance supportMid-level Full TimeWork from home, VA, United States1d ago
-
GTM Data Analytics Engineer USD 120K-150KAmplitude | DBT | Hex | Metabase | PythonHealth insurance | Startup equityMid-level Full TimeSan Francisco1d ago
-
API Gateway | AWS Lambda | AWS Step Functions | Amazon SQS | Apache Airflow401k matching | Employee stock options | Health benefits | Hybrid work onsite 2 to 3 days per week | Paid time offExecutive-level Full TimeNew York, United States R1d ago