Principal Data Engineer (Python, AWS, Redshift, EMR, Airflow, Databricks, Big data) | Contract to Hire | Irvine, CA (Hybrid) |
Tasks
- Build monitoring and automation tools for servers and clusters
- Conduct root cause analysis validation
- Create data environments and datasets
- Deploy Hadoop big data Spark and storage in AWS
- Design and implement EMR Hadoop big data infrastructure
- Design and operationalize analytics scheduling
- Design develop and deploy data tools with other teams
- Design distributed scalable real time data pipelines
- Investigate evaluate test and recommend data system solutions
- Lead data acquisition and transformation
- Monitor HDFS Hadoop Spark and performance
- Own product data sets end to end
- Perform offline analysis of large data sets
- Prototype big data processing architecture improvements
Perks/Benefits
Skills/Tech-stack
AWS RDS | Airflow | Amazon EMR | Amazon Redshift | Amazon Web Services | Apache HBase | Apache Hive | Apache Spark | Data Modeling | Databricks | DynamoDB | Elasticsearch | HDFS | Hadoop | JSON | Kafka | Kinesis | MapReduce | Mqtt | MySQL | NoSQL | Object Modeling | Python | R | Ruby | SQL | Scala | Web Services | XML | YAML
Education
Roles
Data Engineer | Engineer | Principal | Principal Data Engineer
Related jobs
-
Staff Machine Learning Engineer USD 165K-236KClass imbalance | Docker | Error Analysis | Feature Engineering | Feature Labeling401k | Bonus | Equity | Generous vacation time | Medical/Dental/Vision insuranceSenior-level Full TimeRemote USA R18h ago
-
Staff Data Engineer USD 135K-180KAI coding | AI coding tools | AWS EKS | Alerting | Amazon EC2401k match | Career development | Parental leaveSenior-level Full TimeTempe, Arizona R19h ago
-
Software Engineer - AIM Virtualization USD 157K-230KData Ingestion | Data egress | Database Internals | Functional Programming | Query OptimizationSenior-level Full TimeUS-CA-Menlo Park R20h ago
-
Senior-level Full TimeDenver, CO - Hybrid; New York, … R20h ago
-
Staff+ Software Engineer, Safeguards Data USD 320K-485KAWS | Access Control | Cloud platform | Data Governance | Data LineageFlexible working hours | Generous vacation | Hybrid office policy | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R21h ago
-
Software Engineer, Applied AI USD 170K-300KAWS | Agents | Benchmarking | Datadog | ECSFree coachingSenior-level Full TimeNew York R21h ago
-
Senior Solution Engineer USD 138K-181KAnalytics | Cloud Computing | Data Lake | Data Warehousing | Data fabricTravel opportunitiesSenior-level Full TimeUS-GA-Remote R21h ago
-
Senior Database Engineer USD 130K-150KAWS | AWS CDK | AWS CloudFormation | Access Control | Amazon AuroraSenior-level Full TimeLas Vegas, Nevada, United States; Seattle, … R21h ago
-
Software Engineer, Data Products & Platform USD 170K-300KAWS Glue | AWS Lambda | Amazon Athena | Amazon CloudWatch | Amazon S3Work with coachesSenior-level Full TimeNew York R22h ago
-
Senior Machine Learning Engineer, Trust USD 200K-235KA/B | A/B Testing | Account integrity | Agentic Frameworks | Anomaly DetectionEmployee travel credits | Remote eligibleSenior-level Full TimeSan Francisco, CA R22h ago
-
Machine Learning Engineer USD 150K-250KLanguage Processing | Machine Learning | Natural Language | Natural Language Processing | PyTorchOpen source portfolio | Publications in ML venues | Startup experience | Understanding model failures in the wild | Visa sponsorshipMid-level Full TimeToronto Hybrid R22h ago
-
Data Analyst/BI Engineer USD 85K-110KAmazon S3 | Apps Script | Azure | BigQuery | Data Lineage401k match | Dental insurance | Health insurance | Paid Holidays | Paid time offSenior-level Full TimeUnited States R23h ago
-
Software Engineer, Fulfillment Core Services USD 128K-160KAPI Integration | AWS | Buildkite | CI/CD | CircleCI401k company match | Childcare benefits | Commuter benefits | Dental insurance | Family building benefitsEntry-level Full TimeSeattle, WA R23h ago
-
Senior Analytics Engineer USD 113K-125KChurn analysis | Customer Lifetime Value | DBT | Data Architecture | Data Modeling401k matching | Dental insurance | Employee assistance program | Employer Paid Short-term Disability | Flexible time offSenior-level Full TimeDallas-Fort Worth Metroplex; Denver Metropolitan Area R1d ago
-
Principal Data Analyst, Product USD 205K-251KAmplitude | Automation | Cohort Analysis | DBT | Event instrumentationSenior-level Full TimeUnited States R1d ago
-
Data Engineer II (Remote US) USD 94K-118KAlerting | BigQuery | Clustering | Common table expressions | Cost Management401k matching | Annual learning budget | Dental insurance | Employee resource groups | Health insuranceMid-level Full TimeRemote - U.S. R1d ago
-
Senior Gen AI Software Engineer USD 172K-227KAPI Design | Automated testing | Cost Optimization | Data Modeling | Data PipelinesEquity | Health/Vision/Dental benefits | In person quarterly team gatherings | Remote work | Wellness stipendsSenior-level Full TimeUnited States (Remote) R1d ago
-
Senior MLOps Engineer II (AI Native) USD 148K-216KAWS | Apache Airflow | Apache Ray | Apache Spark | Azure Storage401k match | Employee assistance program | Health insurance | Mental wellness program | Paid parental leaveMid-level Full TimeRemote, USA ; Remote, Canada R1d ago
-
Data Engineer I USD 90K-100KAWS | Access Management | Amazon Redshift | Azure | Azure Synapse401k plan | Education assistance | Health insurance | Paid time offMid-level Full TimeCamden, New Jersey, United States R1d ago
-
Senior Software Engineer, Data Infrastructure USD 243K-295KAWS | Airflow | Data Catalog | Druid | FlinkEquity compensation | Health benefitsSenior-level Full TimeSan Mateo, CA, United States R1d ago
-
Data Engineer (Implementation & Integration) USD 80K-95KData Modeling | Data Transformation | Data Validation | Data Warehousing | Data integration401k match | Dental insurance | FSA | HSA | Internal growth opportunitiesMid-level Full TimeRemote, United States R1d ago
-
AI Engineer USD 100K-176KAWS | CI/CD | CrewAI | Data Governance | EmbeddingsCelebrations | Education | In-person collaborationMid-level Full TimeBoca Raton, FL/Nashville, TN (Preferred) or … R1d ago
-
CoachMePlus - Senior Software Engineer (AI/ML) USD 150K-175KAPI Gateway | AWS | AWS Bedrock | AWS Lambda | Agentic pipelinesFlexible work schedule | Health insurance | Paid time off | Parental leave | Professional development supportSenior-level Full TimeBuffalo, NY R1d ago
-
Software Engineer, Backend Engineer- Credit Coverage USD 156K-235KAWS | GRPC | Grafana | GraphQL | Hadoop401k plan | Medical, dental, and vision benefits | Telecommuting | Wellness stipendsEntry-level Full TimeNew York, NY R1d ago
-
Senior Data Engineer (AWS, Azure, GCP) USD 90K-200KAWS | Amazon Redshift | Aurora | Azure | BigQuery401k matching | Disability insurance | Fertility coverage | Health coverage | Learning and developmentSenior-level Full TimeChicago, IL, United States R1d ago