Big Data Lead
Tasks
- Design data pipelines
- Develop data integration workflows
- Ensure data consistency and integrity
- Implement data governance controls
- Implement data quality checks
- Load data into data storage systems
- Monitor data systems performance
- Optimize pipeline performance and scalability
- Transform raw data for analytics
- Tune queries with caching and indexing
- Validate and clean data
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Caching | Data Cleansing | Data Modeling | Data Pipelines | Data Quality | Data Transformation | Data Validation | Data Warehousing | Data integration | Database Integration | ELT | ETL | Indexing | Query Optimization | Scalability
Education
N/A
Roles
Related jobs
-
APIs | Agent systems | Cloud platform | Cost Per Request | CrewAIEmployee discounts | Health insurance | Paid time off | Retirement plansMid-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …2h ago
-
Artificial Intelligence | Automated Evaluation | Behavior analytics | C# | C++Senior-level Full TimeMountain View, CA, USA2h ago
-
Forward Deployed Engineer II, Applied AI, Google Cloud USD 147K-211KAPI Integration | APIs | Agent systems | Cloud platform | Evaluation PipelinesTravel opportunitiesMid-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …2h ago
-
Senior Software Engineer, BigQuery Managed Storage USD 174K-252KBigQuery | C++ | Change Data Capture | Cloud platform | Data CaptureSenior-level Full TimeKirkland, WA, USA2h ago
-
Machine Learning Intern USD 80K-90KAWS | Apache Hive | Apache Spark | C++ | Cloud platformFinal evaluation | Mentorship | Performance feedbackEntry-level InternshipMountain View, USA12h ago
-
AWS | Data Governance | Data Lineage | Data Modeling | Data PipelinesMid-level Full TimeHawthorne, California, USA13h ago
-
Senior Data Engineer - Remote - Multiple Levels USD 85K-141KAWS Data | AWS Data Migration | AWS Data Migration Service | AWS Lambda | Airflow401k retirement plan | Dental insurance | Health insurance | Paid Holidays | Parental leaveSenior-level Full TimeHome Office: Tysons, VA, United States R13h ago
-
Senior Machine Learning Engineer, GenAI Data USD 243K-295KAmazon S3 | Batch Processing | C plus plus | Data Pipelines | Data PreprocessingSenior-level Full TimeSan Mateo, CA, United States R14h ago
-
Data Engineer, Human Cohorts USD 191K-195KAWS | Azure | CI/CD | Cloud platform | ComplianceOnsite work requirementSenior-level Full TimeSouth San Francisco, CA14h ago
-
Software Engineer, Cortex AI Infrastructure USD 160K-225KC++ | CI/CD | Cloud Native | Cloud Native Architecture | Data-intensiveMid-level Full TimeUS-CA-Menlo Park14h ago
-
Staff AI Engineer USD 150K-185KAPI Design | API Gateway | API Integration | AWS Lambda | AWS SQSRemote work | Travel opportunitiesSenior-level Full TimeUnited States16h ago
-
Application/Agentic AI Engineer USD 124K-166KAPI Integration | Agentic AI | CI/CD | DevSecOps | Development LifecycleMid-level Full TimeAberdeen, Maryland16h ago
-
Analytics, Finance & Strategy USD 270K-320KAWS | Airflow | Business Intelligence | DBT | DashboardingFlexible working hours | Generous vacation | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA16h ago
-
Edge AI/Model Optimization Engineer USD 115K-175KBatching | Benchmarking | CUDA | Caching | ContainersMid-level Full TimeAberdeen, Maryland16h ago
-
Staff AI Engineer USD 135K-180KAPI Gateway | API Integration | AWS Lambda | AWS Step Functions | Agent OrchestrationSenior-level Full TimeNewark, NJ17h ago
-
Sr. Data Engineer USD 152K-223KAWS | Access Control | CI/CD | Change Data Capture | DBT401k match | Disability insurance | EAP | Health insurance | Hybrid work flexibilitySenior-level Full TimeUtah | Hybrid R17h ago
-
Senior Data Engineer USD 150K-175KAWS CloudFormation | AWS Glue | AWS IAM | Amazon Athena | Amazon CloudWatchContinuing education support | Flexible PTO | Health, dental, and vision insurance | Professional development budget | Work from home optionsSenior-level Full TimeUnited States17h ago
-
Tech Lead Manager, Data Engineer USD 251K-310KBigQuery | C++ | Data Governance | Data Lake | Data ModelingBonus program | Company benefits program | Equity incentive plan | Hybrid work scheduleSenior-level Full TimeMountain View, CA17h ago
-
Senior Staff Data Engineer - Platform Data and Analytics USD 268K-368KAWS | Airflow | Alerting | Apache Spark | Compute OptimizationComprehensive benefits | Equity | Hybrid/Remote flexibilitySenior-level Full TimeSan Francisco, CA R17h ago
-
Consultant, Data Analytics USD 90K-110KAlteryx | Azure Data | Azure Data Factory | Boomi | CI/CD401k match | ADandD insurance | Charitable donation match | Childcare Account | Commuter accountMid-level Full TimeDurham, North Carolina17h ago
-
Data Engineer USD 150K-210KAmazon Redshift | Apache Airflow | CI/CD | DBT | Data GovernanceCompany retreats | Generous PTO | Hybrid work | Lunch snacks and coffee provided | Paid health coverageMid-level Full TimeSan Francisco18h ago
-
Senior Forward Deployed Engineer USD 180K-230KAPI Integration | Agent systems | Distributed Systems | Language Models | Large Language ModelsCustomer facing collaboration | Remote work flexibility | Travel opportunitiesSenior-level Full TimeBellevue18h ago
-
Consultant, Data Analytics USD 90K-110KCI/CD | Data Engineering | Data Monitoring | Data Visualization | Data Warehouse401-k match | Charitable donation match | Childcare Accounts | Commuter benefits | Dental insuranceMid-level Full TimeEmeryville, California18h ago
-
Consultant, Data Analytics USD 90K-110KAzure Data | Azure Data Factory | CI/CD | Data Factory | Data Modeling401k matching | Dental insurance | Disability insurance | Flexible-hybrid work | Health insuranceMid-level Full TimeBoston, Massachusetts18h ago
-
Consultant, Data Analytics USD 90K-110KAlteryx | Azure Data | Azure Data Factory | CI/CD | Data Engineering401k match | AD and D insurance | Charitable donation match | Childcare Accounts | Commuter benefitsMid-level Full TimeRockville, Maryland18h ago