Big Data Lead
Tasks
- Automate ETL deployments with CI CD
- Create dimension tables
- Create fact tables
- Design data warehouses
- Develop ETL/ELT pipelines
- Implement data transformations
- Manage ETL version control
- Model star schema
- Monitor scheduled ETL jobs
- Optimize SQL queries
- Set up ETL error handling logging and recovery
- Tune database indexing and partitioning
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | Amazon Redshift | Amazon S3 | Amazon Web Services | Automated Deployments | Azure | Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure DevOps | Blob Storage | Branching Strategies | CI/CD | Data Factory | Data Modeling | Data Recovery | Data Transformation | Data Warehousing | Data pipeline | Database Indexing | Database partitioning | Dimension tables | Dimensional Modeling | ELT | ETL | Error Handling | Fact tables | Git | Jenkins | Job Scheduling | Logging | Oracle | Performance Tuning | Query Optimization | SQL | Snowflake | Star Schema | Talend | TeamCity | Web Services
Education
N/A
Roles
Related jobs
-
ABAC | Access Control | Airflow | Apache Spark | Attribute Based FilteringSenior-level Full TimeHouston, TX, United States4h ago
-
ABAC | Airflow | Apache Spark | Automated testing | Azure DevOpsSenior-level Full TimeHouston, TX, United States4h ago
-
Data Engineer (Senior) USD 170K-225KAmazon Web Services | Apache NiFi | Apache Spark | Data Pipelines | Data VisualizationSenior-level Full TimeHuntsville, United States6h ago
-
Agent deployment | Agile | Automation | Azure AD | Azure MonitorKnowledge transfer | Long-term contract | Onsite 3 days per weekSenior-level ContractDeerfield Beach, United States6h ago
-
Senior Manager / Principal Data Science USD 155K-178KAI Agents | Dashboards | Data Pipelines | Data Visualization | LLMHybrid workSenior-level ContractDeerfield Beach, United States6h ago
-
Apache Spark | Cloud Computing | Data Governance | Data Lineage | Data WarehousingSenior-level Full TimeTexas-Austin7h ago
-
Senior Software Engineer, Managed Spark, OpenSource USD 174K-253KApache Hudi | Apache Iceberg | Benchmarking | C++ | CloudSenior-level Full TimeSunnyvale, CA, USA8h ago
-
Databricks Solution Architect USD 180K-247KAWS S3 | Apache Spark | Autoscaling | Azure Data | Azure Data LakeSenior-level Full TimeUnited States R15h ago
-
Agentic Systems | Backend Development | Cloud Data | Cloud data warehousing | Cost OptimizationEquity | Remote workSenior-level Full TimeNew York, NY, United States15h ago
-
Lead Analytics Manager USD 120K-150KAffiliate Marketing | Business Intelligence | Dashboard Design | Data Ingestion | Data LiteracySenior-level Full TimeAustin, TX (remote); Dallas, TX (remote); … R16h ago
-
Senior Quantitative Developer USD 165K-240KAWS | Alerting | Amazon Kinesis | Cloud platform | Data Transformation401k match | Dental insurance | Hardware provided | Health insurance | Unlimited PTOSenior-level Full TimeNew York18h ago
-
ADA | API Design | Agile | Board Support | Board Support PackageActive US Security Clearance | High impact classified program supportSenior-level Full TimeGrand Rapids, MI, United States18h ago
-
Senior Applied Field Engineering, Analytics USD 165K-216KAnalytics | Data Architecture | Data Modeling | Database technology | Geospatial Analytics401k retirement plan | Dental insurance | Disability insurance | Employee assistance program | Flexible spending accountSenior-level Full TimeUS-WA-Bellevue18h ago
-
Senior Analytics Engineer USD 140K-195KAccess Control | Audit trails | Census | Cloud Data | Cloud Data Warehouse401k | Dental insurance | Fitness stipend | Health insurance | Meal stipendSenior-level Full TimeMcLean, VA18h ago
-
Senior Data Engineer USD 140K-180KAccess Control | Alerting | Audit Logging | Cloud Data | Cloud Data WarehouseDental insurance | Fitness stipend | Health insurance | Meals covered | Paid time offSenior-level Full TimeMcLean, VA18h ago
-
Senior AI/ML Engineer USD 152K-190KAmazon Web Services | CI/CD | Deep learning | Git | Machine LearningCaregiver support | Hybrid work | Work on site in the United StatesSenior-level Full TimeCambridge, MA, United States20h ago
-
Data Scientist VP- Research USD 155K-225KArtificial Intelligence | Databricks | Machine Learning | Power BI | PySparkExecutive-level Full TimeNew York, 745 7th Avenue, United …20h ago
-
Senior Data Engineer USD 134K-182KApache Airflow | CI/CD | Cloud services | Data Governance | Data ModelingPaid time offSenior-level Full TimeUSA VA Arlington - 2733 Crystal …20h ago
-
AWS | Automation | Azure | Cloud Architecture | Cloud Recovery Architecture401k match | Fitness centers | Health and wellness programs | Insurance benefits | Paid time offSenior-level Full TimeSLEEPY HOLLOW, United States20h ago
-
Manager, AI Engineering - Analytics USD 197K-267KAgent systems | Artificial Intelligence | Data Modeling | Data Warehousing | EvalsHybrid work flexibility | Professional growth opportunities | Stock equity | Work-life balanceMid-level Full TimeHybrid - San Francisco R20h ago
-
Senior ML Ops Engineer - Dallas, TX USD 51K-178KApache Spark | Big Data | CI/CD | Containerization | Docker401k retirement plan | Medical, dental, vision benefits | Paid Holidays | Paid time off | Variable pay/incentivesSenior-level Full TimeUnited States20h ago
-
Machine Learning Operations Engineer USD 133K-167KAWS SageMaker | Docker | GitHub Actions | Machine Learning | NumPyCareer development | Communities | Commuting cost coverage | Corporate giving programs | Daily free lunchMid-level Full TimeBoston, Massachusetts, United States R20h ago
-
Senior Analytics Engineer, Client Engagement USD 180K-200KAnalytics engineering | Dashboarding | Data Engineering | Data Modeling | Data PipelinesDental insurance | Health insurance | Healthcare membership | Office lunch | Online medical providersSenior-level Full TimeNew York, New York, United States20h ago
-
Embedded Software Engineer USD 170K-190KC++ | CAN | CI/CD | Cellular | Cloud Storage401k matching | Dental insurance | Health insurance | Paid Holidays | Paid parental leaveMid-level Full TimeHayward, CA20h ago
-
Data Engineer Associate USD 75K-85KASM | Amazon Web Services | Azure | Cloud platform | Container Orchestration401k matching | Dental insurance | Health insurance | Life insurance | On-call supportEntry-level Full TimeRoseville, CA, United States21h ago