Lead BI Engineer
Tasks
- Architect distributed data pipelines with Apache Spark
- Build and optimize Lakehouse architecture with Delta Lake
- Develop automated ingestion frameworks
- Ensure production reliability for data and ML pipelines
- Establish reproducible ML deployment patterns
- Implement cost optimization and cluster tuning
- Lead data quality access control and data cataloging
- Perform troubleshooting and root cause analysis
- Prepare feature ready datasets for machine learning
- Translate business requirements into scalable data solutions
Perks/Benefits
- N/A
Skills/Tech-stack
ACID transactions | Access Control | Amazon Web Services | Apache Kafka | Apache Spark | Audit Ready | Audit Ready Governance | Azure | Azure Event | Azure Event Hubs | CI/CD | Cloud Native | Cloud Native Data | Cloud platform | Cloud-native data platforms | Cluster tuning | Compliance | Data Catalog | Data Governance | Data Lineage | Data Modeling | Data Processing | Data Quality | Data Security | Data platforms | Delta Lake | Distributed data | Distributed data processing | Event Hubs | Experiment tracking | Feature Store | Google Cloud | Google Cloud Platform | Infrastructure as Code | Infrastructure automation | Lakehouse | Machine Learning | Machine Learning Pipelines | Model operationalization | Partitioning | Performance Tuning | Python | SQL | Schema evolution | Web Services | “as-code”
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R1d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R14d ago
-
Sr. Data Engineer USD 98K-130KAgile | Airflow | CI/CD | DBT | Data ArchitectureBusiness casual environment | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeLatham, NY, United States1h ago
-
AI and Data Science Engineer III with TS/SCI Polygraph USD 119K-198KData Visualization | Language Models | Language Processing | Large Language Models | Machine LearningSenior-level Full TimeBaltimore, Maryland, United States5h ago
-
Risk Adjustment Analytics Data Scientist USD 122K-240KActuarial analysis | Cohort Segmentation | Data Quality | Data profiling | DatabricksMentorship | Professional developmentSenior-level Full TimeArlington/Rosslyn, Virginia, United States5h ago
-
Computer Vision Engineer USD 147K-208KComputer Vision | Edge Computing | Gesture Recognition | Hand tracking | Machine LearningSenior-level Full TimeBurlingame, CA6h ago
-
Director, Data Science - Central Product Platform (CPP) USD 253K-314KA/B | A/B Testing | Agent Orchestration | Apache Hive | Apache SparkExecutive-level Full TimeBellevue, WA | Menlo Park, CA6h ago
-
Senior-level Full TimeSunnyvale, CA6h ago
-
Smart NIC Senior Software Engineer, Embedded USD 174K-253KArtificial Intelligence | C# | C++ | Data Center Networks | Data centerSenior-level Full TimeRaleigh, NC, USA; Durham, NC, USA6h ago
-
Software Engineer III, AI/ML, Foundational Lanes USD 147K-211KC++ | Computer Vision | Data Processing | Debugging | Geometric algorithmsSenior-level Full TimeMountain View, CA, USA6h ago
-
Product Policy Lead, Generative AI USD 164K-240KArtificial Intelligence | Content Moderation | Generative AI | Machine Learning | Multimodal Generative AISenior-level Full TimeNew York, NY, USA; Sunnyvale, CA, …6h ago
-
Software Engineer, On-Device Machine Learning USD 174K-253KAndroid | Compiler development | Core ML | Data Processing | DebuggingMid-level Full TimeSunnyvale, CA, USA6h ago
-
Software Engineer, BigQuery Metadata USD 147K-211KBigQuery | C++ | Compute Technologies | Data Warehouse | Data analyticsMid-level Full TimeKirkland, WA, USA6h ago
-
Bash | Cause analysis | Continuous Process | Continuous Process Improvement | Corrective actionsSenior-level Full TimeSunnyvale, CA, USA6h ago
-
Research Engineer, DeepMind USD 262K-365KBigQuery | Data Pipelines | Data analytics | Distributed Systems | FlumeSenior-level Full TimeMountain View, CA, USA6h ago
-
Computer Vision | Data Analysis | Data Visualization | Few-Shot Learning | Few-shotMid-level Full TimeMountain View, CA, USA6h ago
-
Access Management | Algorithms | CI/CD | Cloud platform | ComplianceEquity | Health insurance | Paid time off | Performance bonus | Retirement benefitsSenior-level Full TimeReston, VA, USA6h ago
-
Research Engineer, Human Understanding, DeepMind USD 174K-253KComputer Vision | Deep learning | Experimental Design | Flax | Generative AIMid-level Full TimeMountain View, CA, USA; Los Angeles, …6h ago
-
Software Engineer III, AI Agents USD 147K-211KAgent architecture | Agent systems | C++ | Cloud platform | Computer VisionSenior-level Full TimeSunnyvale, CA, USA6h ago
-
Artificial Intelligence Engineer I USD 99K-176KAWS | Azure | Bioinformatics | Deep learning | DockerMid-level Full TimeHouston, TX, US8h ago
-
Senior Embedded Software Engineer USD 140K-160KC++ | Computer Vision | Coral TPU | Deep learning | Device driverFully-paid health insurance | Paid parental leave | Unlimited PTOSenior-level Full TimeBoston, Massachusetts, United States9h ago
-
Principal Data Scientist, IT Analytics USD 51K-86KAgile | Computer Vision | Data Science | Deep learning | Forecasting401k program | Discounts on hotels cars and cruises | Employee assistance program | Flexible spending accounts | Health dental vision prescription benefitsSenior-level Full TimeFort Worth, TX, US10h ago
-
VP, Data Engineer - DataBricks USD 143K-190KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageHybrid work model | Work from home optionExecutive-level Full TimeCharlotte, NC, US, 2820210h ago
-
Senior Software Engineer, Data Insights USD 196K-243KAWS VPC | Amazon EKS | Analytics infrastructure | Apache Superset | Data EngineeringSenior-level Full TimeSan Mateo, CA, United States R11h ago
-
Senior Software Engineer, Unsecured Installments USD 160K-223KA/B | A/B Testing | Accessibility | B testing | CI/CD401k match | Dental insurance | Disability insurance | Health insurance | Health savings accountSenior-level Full TimeUnited States | Remote R13h ago