Principal Data Engineer – (Hadoop/Big Data, AWS, Python, Kinesis) - Irvine, CA - Onsite - Contract to Hire - AS
Tasks
- Build monitoring and automation tools
- Conduct root cause analysis
- Create data environments
- Deploy Hadoop Big Data Spark in AWS
- Design and implement Hadoop EMR clusters
- Design and lead distributed scalable data pipelines
- Design build and deploy data tools for operations
- Ingest and process data at scale and in real time
- Investigate evaluate test recommend solutions
- Lead data acquisition
- Mentor junior engineers
- Monitor HDFS Hadoop Spark and releases
- Operationalize analytics
- Own product data sets end to end
- Perform data transformation
- Perform offline analysis of large data sets
- Prototype big data technologies and solutions
- Schedule analytics
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon Kinesis | Amazon RDS | Amazon Redshift | Amazon Web Services | Apache HBase | Apache Kafka | Apache Spark | Data Modeling | DynamoDB | EMR | Elasticsearch | HDFS | Hadoop | Hive | JSON | MapReduce | Mqtt | MySQL | NoSQL | Object Modeling | Pig | Python | SQL | Scala | Web Services | XML | YAML
Education
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R29d ago
-
Sr. Databricks Consultant - Top Secret Clearance USD 175K-250KAWS S3 | Apache Spark | Azure Data | Azure Data Lake | CI/CDTop Secret clearanceSenior-level Full TimeFAIRFAX, VA, United States2h ago
-
Sr. Databricks Consultant - TS/SCI-FSP USD 175K-250KAWS | AWS S3 | Apache Spark | Azure | Azure DataAdditional funding contingent | Benefits package | Travel as neededSenior-level Full TimeFAIRFAX, VA, United States2h ago
-
Mid-level Full TimeKing George, VA, United States2h ago
-
Sr. Databricks Consultant - DHS Public Trust USD 175K-250KAWS S3 | Apache Spark | Azure Data | Azure Data Lake | CI/CDSenior-level Full TimeWork from home, VA, United States2h ago
-
Data Architect (Hybrid - St. Paul, MN) USD 130K-174KAccess Control | Azure | Azure Backup | Azure CLI | Azure DataHybrid work | W2 employmentSenior-level Full TimeSt Paul, Minnesota R3h ago
-
Principal AI Engineer – Agentic AI / Generative AI | Irvine, CA (Hybrid) | Contract to Hire | AS A USD 100K-150KAI Agents | API | AWS | Agent systems | CI/CDContract-to-hire | Hybrid workSenior-level Full TimeIrvine, CA R3h ago
-
AWS | Apache Iceberg | Apache Kafka | ArgoCD | AzureContract position | On-site workMid-level Contract Full TimeColumbus, United States5h ago
-
Business Intelligence | DBT | Data Integrity | Data Marts | Data SecurityPaid time offSenior-level Full TimeUnited States of America-OHIO-Franklin County-Columbus6h ago
-
Data/Business Analyst USD 115K-145KAI orchestration | API Gateway | AWS AgentCore | AWS Lambda | Amazon APIMid-level Full TimeUS-NY-New York6h ago
-
Data/Business Analyst USD 115K-145KAI Governance | AI orchestration | API Gateway | AWS AgentCore | AWS IAMMid-level Full TimeUS-NY-New York6h ago
-
Lead Principal AI Software Engineer USD 169K-355KAccess Control | AgentOps | Audit Logging | Autogen | Batching401k matching | Disability insurance | Employee stock purchase plan | Flexible spending accounts | Life insuranceSenior-level Full TimeNashville, TN, United States6h ago
-
Data Engineer II USD 92K-176KAWS | Airflow | Azure | Azure DevOps | Azure Synapse401k matching | Dental insurance | Employee resource groups | Health insurance | Onsite fitness centerSenior-level Full TimeUS-Rhode Island-Providence6h ago
-
Large Language Model Training System Engineer Graduate (Applied Machine Learning) - 2027 Start USD 128K-256KData Engineering | Distributed Training | Fine Tuning | Language Models | Large Language ModelsEntry-level Full TimeSan Jose, California, United States6h ago
-
Circuit breaking | Deep learning | Distributed Systems | Language Models | Language ProcessingEntry-level Full TimeSan Jose, California, United States6h ago
-
AIOps | APIs | Anomaly Detection | Automation | Cause analysisEntry-level Full TimeSan Jose, California, United States6h ago
-
Recommendation Architecture AI/ML Infrastructure Engineer Graduate (Data-Arch-TikTok Live) - 2027 Start USD 116K-177KAlgorithms | C++ | CUDA | Computer Systems | Continuous batchingEntry-level Full TimeSan Jose, California, United States7h ago
-
Software Engineer, Databases (Technical Leadership) USD 161K-261KAI Automation | Automated Performance Tuning | Consensus Protocols | Data Integrity | Database InternalsSenior-level Full TimeBellevue, WA | Menlo Park, CA7h ago
-
Machine Learning SoC Architect USD 173K-249KAI accelerators | ASIC Performance Modeling | C# | C++ | Computer ArchitectureSenior-level Full TimeSunnyvale, CA | Austin, TX | …7h ago
-
Staff Software Engineering, GenAI Applications, YouTube USD 207K-300KDeep learning | Dense retrieval | Embedding | Hybrid retrieval | Language ModelsSenior-level Full TimeSan Bruno, CA, USA8h ago
-
Senior Software Engineer, AI/ML GenAI, Search USD 174K-252KC++ | Data Processing | Debugging | Distributed Systems | Information RetrievalSenior-level Full TimeMountain View, CA, USA8h ago
-
Research Engineer, Gemini Retrieval, DeepMind USD 174K-252KC plus plus | Information Retrieval | JAX | Language Models | Language ProcessingMid-level Full TimeMountain View, CA, USA; San Francisco, …8h ago
-
AI Inference Tools Engineer USD 167K-242KAgent Frameworks | Benchmarking | C++ | CUDA | Dashboards401k matching | Flexible paid time off | Health insurance | Hybrid work options | Local meetupsSenior-level Full TimeUnited States, United Kingdom R13h ago
-
Senior, Data Analyst/BI Engineer USD 108K-145KAWS | Airflow | Amazon Redshift | BigQuery | DataStage401k matching | Healthcare dental vision coverage | PTO | Paid Company Holidays | Paid parental leaveSenior-level Full TimeUnited States R15h ago
-
Staff Machine Learning Engineer, Tools and Framework AI USD 175K-300KApplied Machine Learning | C++ | Java | Machine Learning | Machine Learning InfrastructureSenior-level Full TimeCupertino16h ago