Lead Data Engineer
Tasks
- Automate testing deployment and rollback processes
- Build and maintain CI CD pipelines for data engineering
- Build batch and real-time ingestion pipelines
- Design end to end data engineering architectures
- Develop scalable ETL/ELT pipelines
- Establish data quality standards and validation
- Implement monitoring, alerting, logging, observability
- Manage schema evolution metadata and dependencies
- Optimize data pipelines for performance reliability scalability cost
- Perform root cause analysis for production issues
- Process structured and semi-structured data
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon Redshift | Amazon Web Services | Azure | Azure Synapse | CDC | CI/CD | Change Data Capture | Cloud platform | Data Capture | Data Governance | Data Lakes | Data Modeling | Data Processing | Data Quality | Data Warehousing | Data orchestration | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Platform | Incremental loading | Java | Lakehouse | Logging | Metadata Management | Monitoring | Observability | Pipeline Orchestration | Python | Real Time | Real-time Data | Real-time Data Processing | Scala | Snowflake | Snowflake Schema | Star Schema | Streaming Data | Web Services
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R25d ago
-
Enterprise Analytics Lead USD 119K-222KABAP Managed CDS | Access Control | Analysis Services | Analytics Cloud | Azure Synapse9/80 work schedule | Extra day off every other week | In office schedule in Portsmouth NHSenior-level Full TimePortsmouth, NH, United States2h ago
-
Data Analyst Team Lead USD 105K-144KAgile | Business Intelligence | CTE | Data Modeling | Data WarehouseSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL3h ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL3h ago
-
Business Intelligence | Dashboard Development | Data Architecture | Data Modeling | Data PipelinesSenior-level ContractLansing, United States5h ago
-
Data Modeling | Data Quality | Data Visualization | Data analytics | Database DevelopmentSenior-level Full TimeIowa City6h ago
-
Embedded System Engineer Co-Op USD 56K-60KC# | C++ | Communication Protocols | Electronic Hardware | Electronics designDental insurance | Disability insurance | Educational assistance | Flexible spending account | Health savings accountEntry-level Full TimeNorwich, New York6h ago
-
(General Hire) Machine Learning Engineer Graduate (TikTok Recommendation) - 2027 Start USD 116K-177KAlgorithms | C++ | Computer Vision | Data Structures | Deep learningEntry-level Full TimeSan Jose, California, United States6h ago
-
(General Hire) Machine Learning Scientist Graduate (TikTok Recommendation) - 2027 Start (PhD) USD 119K-177KAlgorithms | C plus plus | C# | Data Structures | Deep learningEntry-level Full TimeSan Jose, California, United States6h ago
-
Algorithms | C++ | Data Mining | Data Pipelines | Data StructuresEntry-level InternshipSan Jose, California, United States6h ago
-
C++ | Computer Vision | Data Mining | Data Pipelines | DatabasesEntry-level Full TimeSan Jose, California, United States6h ago
-
Computer Vision | Data-Driven Strategies | Data-driven | Language Models | Language ProcessingEntry-level Full TimeSeattle, Washington, United States6h ago
-
Machine Learning Engineer Intern (TikTok-Data-Search-Basic Ranking) - 2027 Start (PhD) USD 136K-205KAlgorithms | Data Pipelines | Data Structures | Databases | Distributed SystemsEntry-level InternshipSan Jose, California, United States6h ago
-
Backend Development | C++ | Computer Vision | Data Mining | Data PipelinesEntry-level Full TimeSan Jose, California, United States6h ago
-
Algorithms | Backend Services | C++ | Data Mining | Data PipelinesEntry-level InternshipSan Jose, California, United States6h ago
-
Backend systems | C++ | Computer Vision | Data Mining | Data PipelinesEntry-level Full TimeSan Jose, California, United States6h ago
-
Backend Development | C++ | Computer Vision | Data Mining | Data PipelinesEntry-level Full TimeSan Jose, California, United States6h ago
-
Research Engineer Graduate (Monetization Technology - Business Integrity) - 2027 Start USD 145K-250KAgentic Systems | Computer Vision | Data Pipelines | Debugging | DeepSpeedEntry-level Full TimeSan Jose, California, United States6h ago
-
Backend Development | C++ | Computer Vision | Data Mining | Data PipelinesEntry-level Full TimeSeattle, Washington, United States6h ago
-
Backend Development | C++ | Computer Vision | Data Mining | Data PipelinesEntry-level Full TimeSeattle, Washington, United States6h ago
-
Big Data | C# | C++ | Data Pipelines | Distributed StorageDevelopment workshops | Hands-on experience | Industry exposure | Learning programs | Social eventsEntry-level InternshipSeattle, Washington, United States6h ago
-
Machine Learning Engineer Graduate (Monetization Technology - Ads Core Global) - 2027 Start USD 113K-250KAlgorithms | C++ | Data Structures | Deep learning | Diffusion ModelsEntry-level Full TimeSan Jose, California, United States6h ago
-
A/B | A/B Testing | Apache Flink | Apache Hive | Apache SparkEntry-level Full TimeSan Jose, California, United States6h ago
-
Advertising Technology | Backend Development | C++ | Data Analysis | DebuggingEntry-level Full TimeSan Jose, California, United States6h ago
-
Machine Learning Engineer Graduate (Monetization Technology - Ads Core Global) - 2027 Start (PhD) USD 113K-250KAdvertising | Algorithms | C plus plus | C# | Data StructuresEntry-level Full TimeSan Jose, California, United States6h ago