Big Data Lead
Tasks
- Automate ETL deployments with CI CD
- Create dimension tables
- Create fact tables
- Design data warehouses
- Develop ETL/ELT pipelines
- Implement data transformations
- Manage ETL version control
- Model star schema
- Monitor scheduled ETL jobs
- Optimize SQL queries
- Set up ETL error handling logging and recovery
- Tune database indexing and partitioning
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | Amazon Redshift | Amazon S3 | Amazon Web Services | Automated Deployments | Azure | Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure DevOps | Blob Storage | Branching Strategies | CI/CD | Data Factory | Data Modeling | Data Recovery | Data Transformation | Data Warehousing | Data pipeline | Database Indexing | Database partitioning | Dimension tables | Dimensional Modeling | ELT | ETL | Error Handling | Fact tables | Git | Jenkins | Job Scheduling | Logging | Oracle | Performance Tuning | Query Optimization | SQL | Snowflake | Star Schema | Talend | TeamCity | Web Services
Education
N/A
Roles
Related jobs
-
Data Engineer (Senior) USD 170K-225KAmazon Web Services | Apache NiFi | Apache Spark | Data Pipelines | Data VisualizationSenior-level Full TimeHuntsville, United States4h ago
-
Senior Manager / Principal Data Science USD 155K-178KAI Agents | Dashboards | Data Pipelines | Data Visualization | LLMHybrid workSenior-level ContractDeerfield Beach, United States5h ago
-
Apache Spark | Cloud Computing | Data Governance | Data Lineage | Data WarehousingSenior-level Full TimeTexas-Austin5h ago
-
Senior Software Engineer, Managed Spark, OpenSource USD 174K-253KApache Hudi | Apache Iceberg | Benchmarking | C++ | CloudSenior-level Full TimeSunnyvale, CA, USA7h ago
-
Databricks Solution Architect USD 180K-247KAWS S3 | Apache Spark | Autoscaling | Azure Data | Azure Data LakeSenior-level Full TimeUnited States R13h ago
-
Lead Analytics Manager USD 120K-150KAffiliate Marketing | Business Intelligence | Dashboard Design | Data Ingestion | Data LiteracySenior-level Full TimeAustin, TX (remote); Dallas, TX (remote); … R15h ago
-
Senior Quantitative Developer USD 165K-240KAWS | Alerting | Amazon Kinesis | Cloud platform | Data Transformation401k match | Dental insurance | Hardware provided | Health insurance | Unlimited PTOSenior-level Full TimeNew York16h ago
-
Senior Applied Field Engineering, Analytics USD 165K-216KAnalytics | Data Architecture | Data Modeling | Database technology | Geospatial Analytics401k retirement plan | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountSenior-level Full TimeUS-WA-Bellevue16h ago
-
Senior Analytics Engineer USD 140K-195KAccess Control | Audit trails | Census | Cloud Data | Cloud Data Warehouse401k | Dental insurance | Fitness stipend | Health insurance | Meal stipendSenior-level Full TimeMcLean, VA17h ago
-
Senior Data Engineer USD 140K-180KAccess Control | Alerting | Audit Logging | Cloud Data | Cloud Data WarehouseDental insurance | Fitness stipend | Health insurance | Meals covered | Paid time offSenior-level Full TimeMcLean, VA17h ago
-
Senior AI/ML Engineer USD 152K-190KAmazon Web Services | CI/CD | Deep learning | Git | Machine LearningCaregiver support | Hybrid work | Work on site in the United StatesSenior-level Full TimeCambridge, MA, United States18h ago
-
Data Scientist VP- Research USD 155K-225KArtificial Intelligence | Databricks | Machine Learning | Power BI | PySparkExecutive-level Full TimeNew York, 745 7th Avenue, United …18h ago
-
Manager, AI Engineering - Analytics USD 197K-267KAgent systems | Artificial Intelligence | Data Modeling | Data Warehousing | EvalsHybrid work flexibility | Professional growth opportunities | Stock equity | Work-life balanceMid-level Full TimeHybrid - San Francisco R18h ago
-
Machine Learning Operations Engineer USD 133K-167KAWS SageMaker | Docker | GitHub Actions | Machine Learning | NumPyCareer development | Communities | Commuting cost coverage | Corporate giving programs | Daily free lunchMid-level Full TimeBoston, Massachusetts, United States R18h ago
-
Senior Analytics Engineer, Client Engagement USD 180K-200KAnalytics engineering | Dashboarding | Data Engineering | Data Modeling | Data PipelinesDental insurance | Health insurance | Healthcare membership | Office lunch | Online medical providersSenior-level Full TimeNew York, New York, United States19h ago
-
Embedded Software Engineer USD 170K-190KC++ | CAN | CI/CD | Cellular | Cloud Storage401k matching | Dental insurance | Health insurance | Paid Holidays | Paid parental leaveMid-level Full TimeHayward, CA19h ago
-
Data Engineer Associate USD 75K-85KASM | Amazon Web Services | Azure | Cloud platform | Container Orchestration401k matching | Dental insurance | Health insurance | Life insurance | On-call supportEntry-level Full TimeRoseville, CA, United States19h ago
-
Senior-level Full TimeRemote - United States R19h ago
-
Analytics Engineer - Music Creator Growth USD 116K-167KAmazon Redshift | BigQuery | Cloud Data | Cloud Data Platforms | DBT401k retirement plan | Flexible holidays | Health insurance | Meal allowance | Paid parental leaveMid-level Full TimeNew York, NY19h ago
-
Senior ML Engineer, Manipulation USD 150K-265KComputer Vision | Data Augmentation | Data pipeline | Dataset curation | Diffusion ModelsOnsite five days a weekSenior-level Full TimeSan Francisco20h ago
-
Go-To-Market AI Engineer USD 115K-170KAI Agents | AI workflows | API Integration | Automation | Clay401k with company match | Company equity | Free product subscription | Medical, dental, and vision | Paid HolidaysMid-level Full TimeSalt Lake City, UT20h ago
-
Data Engineer, Machine Learning USD 170K-240KAirflow | Apache Spark | Coverage monitoring | Dagster | Data Drift401k match | Dental insurance | Employee assistance program | Health insurance | Stock optionsMid-level Full TimeSan Francisco20h ago
-
Anomaly Detection | Automated testing | CI/CD | Confluence | Data ModelingSenior-level Full TimeLos Angeles; New York; San Francisco21h ago
-
AI/ML Software Engineer USD 83K-83KAWS | Angular | Azure | CI/CD | Deep learning401k matching | Flex PTO | Hybrid work | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimeBay Area, CA21h ago
-
NYC Sr. AI Engineer: Assets, Formats & Placements USD 139K-229KA/B | A/B Testing | Apache Lucene | Apache Solr | B testingFlexible work hybrid option | Health and wellness programs | Time away from workSenior-level Full TimeNew York, NY, United States22h ago