Data Engineer
Tasks
- Build AI ready data pipelines for multi source data fusion and analytics
- Collaborate with cross functional teams for mission applications and AI governance
- Design distributed data processing pipelines for operational AI
- Develop and optimize streaming architectures for data ingestion and transformation
- Ensure data lineage and metadata management
- Implement and manage enterprise data layer
- Integrate Spark Kafka Trino ClickHouse Iceberg and OpenSearch
- Monitor observability and enable scalable software delivery
- Secure and scale data pipelines using DevSecOps practices
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Iceberg | Apache Kafka | Apache Spark | Batch Processing | ClickHouse | Cybersecurity | Data Fusion | Data Ingestion | Data Lineage | Data Modeling | Data Processing | Data Transformation | DevSecOps | Distributed data | Distributed data processing | Enterprise Data | Enterprise Data Modeling | Metadata Management | Multi Source Data Fusion | Multi-source data | Observability | OpenSearch | Real Time | Real-time analytics | Streaming Data | Streaming data ingestion | Trino
Education
N/A
Roles
Related jobs
-
ABAC | Access Control | Airflow | Apache Spark | Attribute Based FilteringSenior-level Full TimeHouston, TX, United States6h ago
-
ABAC | Airflow | Apache Spark | Automated testing | Azure DevOpsSenior-level Full TimeHouston, TX, United States6h ago
-
Data Engineer (Senior) USD 170K-225KAmazon Web Services | Apache NiFi | Apache Spark | Data Pipelines | Data VisualizationSenior-level Full TimeHuntsville, United States8h ago
-
Agent deployment | Agile | Automation | Azure AD | Azure MonitorKnowledge transfer | Long-term contract | Onsite 3 days per weekSenior-level ContractDeerfield Beach, United States8h ago
-
Apache Spark | Cloud Computing | Data Governance | Data Lineage | Data WarehousingSenior-level Full TimeTexas-Austin8h ago
-
Gen AI Developer Specialist USD 118K-204KAWS | Access Management | Agents | Alerting | Amazon BedrockSenior-level Full TimeUnited States9h ago
-
Senior Software Engineer, Cloud, ML Infrastructure USD 174K-253KC++ | Co-design | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeSunnyvale, CA, USA10h ago
-
Senior Software Engineer, Managed Spark, OpenSource USD 174K-253KApache Hudi | Apache Iceberg | Benchmarking | C++ | CloudSenior-level Full TimeSunnyvale, CA, USA10h ago
-
AI/ML | AI/ML Pipelines | Data Architecture | Data Ingestion | Data LakeSenior-level Full TimeAustin, TX, USA; New York, NY, …10h ago
-
Staff Software Developer, AI/ML, Safety and Security USD 207K-301KClassification | Computer Vision | Data Processing | Debugging | Fine TuningSenior-level Full TimeWaterloo, ON, Canada; New York, NY, …10h ago
-
Senior Software Engineer, AI/ML Computer Vision, XR USD 174K-253KComputer Vision | Computer Vision Systems | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSan Jose, CA, USA10h ago
-
Data Mining | Data Processing | Debugging | Generative AI | Information RetrievalMid-level Full TimeMountain View, CA, USA10h ago
-
Senior Software Engineer, AI/ML, Creative Intelligence USD 174K-253KAd Creative Optimization | Ad creative | C++ | Creative Optimization | Data ProcessingSenior-level Full TimeMountain View, CA, USA10h ago
-
Databricks Solution Architect USD 180K-247KAWS S3 | Apache Spark | Autoscaling | Azure Data | Azure Data LakeSenior-level Full TimeUnited States R17h ago
-
Agentic Systems | Backend Development | Cloud Data | Cloud data warehousing | Cost OptimizationEquity | Remote workSenior-level Full TimeNew York, NY, United States17h ago
-
Lead Analytics Manager USD 120K-150KAffiliate Marketing | Business Intelligence | Dashboard Design | Data Ingestion | Data LiteracySenior-level Full TimeAustin, TX (remote); Dallas, TX (remote); … R18h ago
-
Senior Quantitative Developer USD 165K-240KAWS | Alerting | Amazon Kinesis | Cloud platform | Data Transformation401k match | Dental insurance | Hardware provided | Health insurance | Unlimited PTOSenior-level Full TimeNew York20h ago
-
ADA | API Design | Agile | Board Support | Board Support PackageActive US Security Clearance | High impact classified program supportSenior-level Full TimeGrand Rapids, MI, United States20h ago
-
Senior Applied Field Engineering, Analytics USD 165K-216KAnalytics | Data Architecture | Data Modeling | Database technology | Geospatial Analytics401k retirement plan | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountSenior-level Full TimeUS-WA-Bellevue20h ago
-
Senior Analytics Engineer USD 140K-195KAccess Control | Audit trails | Census | Cloud Data | Cloud Data Warehouse401k | Dental insurance | Fitness stipend | Health insurance | Meal stipendSenior-level Full TimeMcLean, VA20h ago
-
Senior Data Engineer USD 140K-180KAccess Control | Alerting | Audit Logging | Cloud Data | Cloud Data WarehouseDental insurance | Fitness stipend | Health insurance | Meals covered | Paid time offSenior-level Full TimeMcLean, VA20h ago
-
Senior Data Engineer USD 134K-182KApache Airflow | CI/CD | Cloud services | Data Governance | Data ModelingPaid time offSenior-level Full TimeUSA VA Arlington - 2733 Crystal …21h ago
-
Associate Data Engineer (AI/ML) USD 101K-165K.NET | AI Search | AKS | AWS | Azure401k match | Fertility assistance | Generous PTO | Gym reimbursement | Hybrid work optionsMid-level Full TimePlano, United States21h ago
-
APIs | Agentic Workflows | CI/CD | Cost Management | GeminiSenior-level Full TimeRemote - USA, United States R21h ago
-
Analytics Engineer USD 90K-120KAWS | Airflow | CI/CD | Cloud platform | DBT401k plan | ADND Insurance | Healthcare benefits | Life insurance | Paid HolidaysMid-level Full Time71 5th Avenue, United States21h ago