Data Engineering Team Lead
Tasks
- Aggregate features using Scala
- Build feature store for machine learning
- Build real time bidding data infrastructure
- Conduct code reviews
- Coordinate with infrastructure team on DAG execution
- Create Spark pipelines for ML inference
- Design scalable data pipelines
- Develop data quality observability frameworks
- Enforce data privacy and security policies
- Implement automated data testing and validation
- Lead data engineering teams
- Manage data pipeline lifecycle
- Monitor global production deployments
- Orchestrate workflows with Apache Airflow
- Partner with executives on machine learning roadmaps
- Troubleshoot distributed system incidents
Perks/Benefits
- Dental insurance
- Health insurance
- Learning and training
- Medical benefits
- Travel opportunities
- Vision insurance
Skills/Tech-stack
AWS | Amazon Athena | Amazon EC2 | Amazon EMR | Amazon S3 | Apache Airflow | Apache Spark | Bash | Batch Processing | Cause analysis | Data Governance | Data Observability | Data Privacy | Data Quality | Data Security | Distributed Systems | Docker | Feature Store | Git | Jenkins | Jira | Kubernetes | MySQL | Python | Redis | Root Cause Analysis | Root cause | SQL | Scala | Streaming Data
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)24d ago
-
Data Engineer MXN 1040K-1300KAWS Glue | AWS Step Functions | Agile | Amazon Athena | Amazon S3AI learning paths | Company-provided equipment | English lessons | Flexible working options | Learning opportunitiesSenior-level Full TimeGuadalajara, Jal., Mexico6h ago
-
大数据开发（数据挖掘、数据测试、java） CNY 25K-37KApache Kafka | Apache Spark | Data Mining | Data Modeling | Data WarehousingMid-level Full Time保定8h ago
-
Project Management Specialist - AI Products USD 83K-124KComputer Vision | Confluence | Figma | Generative AI | Human-Computer InteractionEntry-level Full Time Internship底特律8h ago
-
Entry-level Full Time广州8h ago
-
Senior-level Full Time上海、北京9h ago
-
None Full Time淄博9h ago
-
None Full Time济南9h ago
-
【27届实习】Ai实习生(可转正) CNY 36K-48KAmazon Web Services | Computer Vision | Deep learning | Docker | KubernetesEntry-level Internship淄博、济南、青岛10h ago
-
Ai Engineer USD 100K-150KAI Agents | API Development | AWS | AWS Bedrock | Agentic Workflows401k | Commuter benefits | Dental insurance | Disability coverage | EAPMid-level Full TimeColumbia, MD, United States10h ago
-
Analytics engineering | Code Refactoring | Data Architecture | Data Governance | Data ModelingAccess to AI learning paths | Career development plans | Company-provided equipment | English lessons | Flexible working optionsSenior-level Full TimeBogotá, Bogota, Colombia10h ago
-
Senior Staff Engineer BRL 180K-210KAWS | Acceptance Testing | Angular | Apache Kafka | Automation TestingSenior-level Full TimeSão Paulo, SP, Brazil R10h ago
-
Mid-level Full Time北京 R11h ago
-
Senior-level Contract Full TimeReston, VA, United States11h ago
-
Miclaw-端云协同调度专家 (Hybrid AI Architect) CNY 240K-360K5G | API Integration | Claude 3.5 | Distributed Systems | GPT-4oHybrid workSenior-level Full Time北京 R11h ago
-
Senior Big Data Engineer SGD 90K-130KAlgorithms | C++ | Data Extraction Transformation Loading | Data Structures | Data extractionSenior-level Full Time Internship新加坡12h ago
-
Java开发工程师（大数据方向） CNY 180K-360KApache Flink | Apache Spark | Data pipeline | Distributed Systems | IO ProgrammingMid-level Full Time武汉12h ago
-
Senior Data Engineer (Remote) USD 155KAgile | Apache Spark | BigQuery | Cassandra | Data Governance401k match | Dental insurance | Employee assistance program | Employee stock purchase plan | Flexible scheduleSenior-level Full TimeWork From Home, United States R13h ago
-
Senior AI Operations Engineer USD 170K-180KAI infrastructure | Azure | CI/CD | Cloud infrastructure | Container Engine for Kubernetes401k match | Employee assistance program | Employee stock purchase plan | Flexible schedule | Flexible spending accountSenior-level Full TimeWork From Home, United States R13h ago
-
Snowflake Senior Data Engineer H/F EUR 47K-55KApache Airflow | Apache Spark | Docker | Git | GitLab CIContinuous training | Learning cultureSenior-level Full TimeNantes, Pays de la Loire, France13h ago
-
Junior AI/ML Engineer CHF 56K-68KAPIs | CI/CD | Deep learning | Docker | Generative AIInternational environmentEntry-level Contract Full TimeGenève, Switzerland13h ago
-
Azure AI Developer Consultant AUD 120K-143K.NET | API Management | App Service | Automated testing | Azure APIFlexible work schedule | Presence on customer site when required | Work-from-home hybridSenior-level Full TimeMelbourne, VIC, AU13h ago
-
Engineering Manager (m/f/d) Machine Learning EUR 42K-51KAWS | Docker | Kubernetes | MLOps | Machine LearningCoverflex allowance | Employee discounts | Flexible work setup | Learning and knowledge exchange | Meal allowanceMid-level Full TimeLisbon, Portugal13h ago
-
Data & Analytics Manager BRL 102K-107KAI | Advanced Analytics | Data Architecture | Data Governance | Data VisualizationDay off on birthday | Health insurance | Home-office allowance | Life insurance | Meal voucherMid-level Full TimeSão Paulo, SP, Brazil14h ago
-
Apache Spark | Azure | Azure Databricks | DBT | Data ModelingCompany-provided equipment | English lessons | Flexible working options | Learning opportunities | Mentorship programsSenior-level Full TimeBogotá, Bogota, Colombia14h ago