Data Engineering Team Lead
Tasks
- Aggregate features using Scala
- Build feature store for machine learning
- Build real time bidding data infrastructure
- Conduct code reviews
- Coordinate with infrastructure team on DAG execution
- Create Spark pipelines for ML inference
- Design scalable data pipelines
- Develop data quality observability frameworks
- Enforce data privacy and security policies
- Implement automated data testing and validation
- Lead data engineering teams
- Manage data pipeline lifecycle
- Monitor global production deployments
- Orchestrate workflows with Apache Airflow
- Partner with executives on machine learning roadmaps
- Troubleshoot distributed system incidents
Perks/Benefits
- Dental insurance
- Health insurance
- Learning and training
- Medical benefits
- Travel opportunities
- Vision insurance
Skills/Tech-stack
AWS | Amazon Athena | Amazon EC2 | Amazon EMR | Amazon S3 | Apache Airflow | Apache Spark | Bash | Batch Processing | Cause analysis | Data Governance | Data Observability | Data Privacy | Data Quality | Data Security | Distributed Systems | Docker | Feature Store | Git | Jenkins | Jira | Kubernetes | MySQL | Python | Redis | Root Cause Analysis | Root cause | SQL | Scala | Streaming Data
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R25d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2h ago
-
Mid-level Full TimeRa'anana, Center District, IL2h ago
-
Senior-level ContractRaleigh, United States3h ago
-
Senior-level Full TimeRemote R3h ago
-
Mid-level Full TimeShaikpet, India3h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES - 032726-2 A USD 136K-140KAI/ML | Agile | Confluence | Jira | LLM401k retirement plan | Dental insurance | Disability insurance | Federal Holidays | Floating holidaysMid-level Full TimeFort George G Meade, United States3h ago
-
Analista de Dados BRL 14K-18KAPI | AWS S3 | Apache Airflow | Apache NiFi | Azure DataDental insurance | Gympass | Health insurance | Life insurance | Meal allowanceMid-level Full TimeBrasília, Brazil4h ago
-
Amazon Redshift | Data Migration | Data Modeling | Data Quality | Data WarehousingFlexible schedule | Remote workSenior-level Full TimeRemote R4h ago
-
Senior Manager, Data Science USD 165K-256KA/B | A/B Testing | Ad Spend | Artificial Intelligence | Attribution401k matching | Disability insurance | Life insurance | Medical/Dental/Vision insurance | Paid Company HolidaysSenior-level Full TimeLocations: Redwood City, California, United States …4h ago
-
Amazon Web Services | Apache Storm | Automation | Bash | Business IntelligenceSenior-level ContractCape Town, South Africa4h ago
-
Ab Initio | Apache Spark | Azure Data | Azure Data Factory | Azure EventEntry-level ContractJohannesburg, South Africa4h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York4h ago
-
Project GenAI Engineer III USD 113K-188KAWS | Bedrock | Generative AI | Machine LearningMentorship | Onsite work | Professional development | Travel up to 25%Senior-level Full TimeArlington/Rosslyn, Virginia, United States4h ago
-
AWS | Agentic AI | Agile | Azure | Business IntelligenceDiscretionary incentive program | Hybrid work model | Mentorship | Professional developmentSenior-level Full TimeAustin, Texas, United States; Dallas, Texas, …4h ago
-
AWS | Artificial Intelligence | Cloud platform | Data Architecture | Data GovernanceTravel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Washington, District …4h ago
-
Project Lead Data Engineer III USD 119K-198KBTEQ | Database Administration | Patch Management | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States4h ago
-
API | AWS | Agile | Amazon Redshift | Apache AirflowFull-time schedule | Hybrid work | Learning opportunitiesMid-level Full TimeLima, Peru4h ago
-
Project Manager - Data / AI CHF 115K-182KAWS | Azure | Consulting | Data & AI | DatabricksCareer development | Internal career paths | Training and coaching | Travel to client sitesMid-level Full TimePully, Switzerland5h ago
-
AWS SageMaker | Azure Machine Learning | CI/CD | Computer Vision | DockerBilingual work environmentMid-level Full TimeMontréal, Canada5h ago
-
Principal AI Engr INR 2500K-5000KAI Foundry | AI Observability | AI Search | AI auditing | Agent systemsSenior-level Full TimeBengaluru, Karnataka, India5h ago
-
Senior-level Full TimeIN-TN-Chennai5h ago
-
Databricks Data Engineer INR 1500K-2200KApache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeIN-KA-Bangalore5h ago
-
Senior-level Full TimeIN-AP-Hyderabad5h ago