Data Engineering Team Lead
Tasks
- Aggregate features using Scala
- Build feature store for machine learning
- Build real time bidding data infrastructure
- Conduct code reviews
- Coordinate with infrastructure team on DAG execution
- Create Spark pipelines for ML inference
- Design scalable data pipelines
- Develop data quality observability frameworks
- Enforce data privacy and security policies
- Implement automated data testing and validation
- Lead data engineering teams
- Manage data pipeline lifecycle
- Monitor global production deployments
- Orchestrate workflows with Apache Airflow
- Partner with executives on machine learning roadmaps
- Troubleshoot distributed system incidents
Perks/Benefits
- Dental insurance
- Health insurance
- Learning and training
- Medical benefits
- Travel opportunities
- Vision insurance
Skills/Tech-stack
AWS | Amazon Athena | Amazon EC2 | Amazon EMR | Amazon S3 | Apache Airflow | Apache Spark | Bash | Batch Processing | Cause analysis | Data Governance | Data Observability | Data Privacy | Data Quality | Data Security | Distributed Systems | Docker | Feature Store | Git | Jenkins | Jira | Kubernetes | MySQL | Python | Redis | Root Cause Analysis | Root cause | SQL | Scala | Streaming Data
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R3d ago
-
Featured Feat. Software Engineer USD 40K-150K.NET | APIs | AWS | Angular | AzureContractor hours flexibility | Remote workMid-level ContractRemote R10d ago
-
Featured Feat. Senior Software Engineer USD 80K-170KAlgorithms | C++ | Data Structures | Go | JavaContractor flexibility | Remote workSenior-level ContractRemote R10d ago
-
Manager Software Engineer- ML + NLP+ ppl management INR 3200K-4800KAPIs | BERT | Data Mining | Data Science | Deep learningFlexible work arrangements | Hybrid work environmentSenior-level Full TimeGurgaon - Cyber Park, India8h ago
-
Principal Engineer, Storage Data Science and Analytics INR 3200K-4590KAIOps | Anomaly Detection | Apache Spark | C# | C++Career Development Programs | Flexible work arrangements | Health and wellbeing benefitsSenior-level Full TimeBengaluru, Karnātaka, India8h ago
-
Lead Agentic AI Developer USD 172K-186KAI Development | AI development lifecycle | Agentic AI | Amazon Bedrock | Artificial IntelligenceSenior-level Full TimeSeattle, WA, United States9h ago
-
AWS Bedrock | Agent systems | Anthropic API | Autogen | Azure401k matching program | Adoption Assistance | Development and career growth opportunities | Fertility treatments | Flexible work schedulesSenior-level Contract Full TimeRemote, OR, United States R10h ago
-
Senior Data Engineer USD 90K-110KAWS | Agile | Apache NiFi | Data Architecture | Data ModelingAutonomy | Flexible working hours | Global employee assistance programme | Online training videos | Teambuilding eventsSenior-level Full TimeNew York, United States10h ago
-
Data Engineer USD 74K-133KAgile | Apache Airflow | BigQuery | Cloud Composer | Cloud Data401k retirement plan | Dental insurance | Disability insurance | Flexible time off | Health insuranceMid-level Full TimeLisle, IL, United States R12h ago
-
大模型算法实习生 CNY 36K-37KDeep learning | DeepSpeed | Distributed Training | Java | Machine LearningCollaborative team | Large NLP dataset access | Long term internship support | Technical mentorship | Technical resourcesEntry-level Internship北京、上海12h ago
-
Mid-level Full Time北京13h ago
-
Entry-level Full Time北京、广州、上海13h ago
-
Entry-level Internship深圳13h ago
-
AWS | Airflow | DBT | Fine Tuning | Language ModelsBonuses | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, India R14h ago
-
API Testing | Cypher | Data Quality | DataOps | DevOpsBenefits | Competitive pay | Growth opportunity | Remote work | Travel requiredSenior-level Full TimeReston, VA, United States R14h ago
-
AI/ML engineer - Associate Research Scientist INR 1500K-2500KAIOps | API Development | AWS | Anomaly Detection | AutomationContinuous learning and professional development | Employee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India14h ago
-
Mid-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101314h ago
-
Principal Engineer - Data Platform USD 221K-387KAWS | Airflow | Apache Hive | Apache Iceberg | Apache ImpalaRemote workSenior-level Full TimeSanta Clara, California, United States R14h ago
-
Efs Fcrm 169474 INR 1500K-2400KApache Spark | Azure DevOps | CI/CD | Data Partitioning | Data WarehousingMid-level Full TimeHyderabad, IND-TG, India15h ago
-
Planview Administration 169576 INR 1500K-2500KAlerting | Data Governance | Data Modeling | Data Validation | Data cleaningSenior-level Full TimeHyderabad, IND-TG, India15h ago
-
Data Engineer EUR 33K-40KAWS | Apache Airflow | Azure | CI/CD | Cloud platformInternational travel as needed | Travel within PortugalMid-level Full TimeErmesinde, Porto District, Portugal15h ago
-
Senior Data Engineer GBP 70K-70KAgile | Behavior-Driven Development | CI/CD | Code review | Data EngineeringDiscounted gym membership | Electric vehicle leasing | Experience days | Hybrid work | Life assuranceSenior-level Full TimeLondon, United Kingdom15h ago
-
Data Engineer GBP 57K-57KAgile | Alteryx | Automated testing | Behavior-Driven Development | CI/CDDiscounted gym membership | Electric vehicle leasing | Experience days | Hybrid work | Life assuranceMid-level Full TimeLondon, United Kingdom15h ago
-
AI Engineer H/F - CDI EUR 50K-65KAI Agents | Agent systems | Cloud Computing | Deep learning | Fine TuningCooptation bonus | Equipment bonus | Flexible remote work | Health insurance | Meal vouchersMid-level Full TimeParis, IDF, France R16h ago
-
AWS | Azure | Data Governance | Data Marts | Data ModelingCSE | Career development opportunities | Cooptation bonus | Diversity initiatives | Employee representative councilSenior-level Full TimeAix-en-Provence, Provence-Alpes-Côte d'Azur, France R16h ago