Principal Engineer I - Principal Data Platform Engineer
Tasks
- Collaborate on data ingestion processing governance and access patterns
- Create and maintain platform documentation operational runbooks and architecture standards
- Define platform standards for availability performance capacity planning disaster recovery backup monitoring and security
- Drive infrastructure optimization for storage compute workload management and job scheduling
- Establish cluster provisioning configuration management patching upgrades and decommissioning best practices
- Evaluate migration or coexistence strategies for cloud and hybrid architectures
- Lead Hadoop ecosystem administration
- Lead root cause analysis and implement preventive measures
- Mentor engineers and administrators
- Own enterprise Hadoop cluster architecture design and lifecycle management
- Troubleshoot complex distributed system performance and production stability issues
Perks/Benefits
- N/A
Skills/Tech-stack
Container Technologies | DevOps | Disaster Recovery | Django | Docker | Ethernet | FastAPI | Flask | HBase | HDFS | Hadoop | Hive | Impala | JavaScript | Kafka | Kubernetes | Monitoring | MySQL | Network Architecture | Network design | OSI Model | Oozie | PostgreSQL | Python | Routing | Security | Shell Scripting | Spark | Splunk | Storage Compute | Storage Compute Workload Management | Switching | TCP/IP | TypeScript | Workload management | YARN | Zookeeper
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Regions
Countries
States
Related jobs
-
Senior-level Full TimeHuntsville, United States10h ago
-
Senior Data Engineer USD 165K-184KAWS | Apache NiFi | Apache Spark | Data Modeling | Data PipelinesSenior-level Full TimeHuntsville, United States10h ago
-
Agile | Amazon Web Services | Apache Hive | Apache Spark | Apache ZeppelinMid-level ContractPortland, United States10h ago
-
Apache Flink | Apache Hive | Apache Kafka | Apache Spark | Apache ZookeeperSenior-level ContractPortland, United States10h ago
-
Amazon DynamoDB | Apache Hive | Java | NoSQL | Object-OrientedSenior-level ContractBeaverton, United States10h ago
-
Security Data Engineer USD 128K-214KAI | API Integration | API scaling | Anomaly Detection | Automated MonitoringDisability insurance | Health insurance | Holiday pay | Learning and development | Life insuranceSenior-level Full TimeUSA-VA-Herndon10h ago
-
Data Engineer USD 94K-157KAWS | Air-gapped | Air-gapped environments | Bash | CRISP-DMHealth insurance | Holiday pay | Learning and development | Life insurance | Long-term disabilitySenior-level Full TimeUSA-VA-Arlington, USA-Remote Work10h ago
-
Cyber Data Engineer II USD 116K-193KCybersecurity | Gap assessment | Incident Response | Network Architecture | Policy DevelopmentOnsite work 5 days per week | Top secret clearance requiredSenior-level Full TimeTempe, Arizona, United States10h ago
-
Palantir MSS Developer / Engineer USD 84K-140KData Engineering | Data Modeling | Data Pipelines | Data Visualization | ELTMid-level Full TimeBaltimore, Maryland, United States; McLean, Virginia, …10h ago
-
AI and Data Science Engineer III USD 131K-218KAgile | Apache Spark | Artificial Intelligence | Data Modeling | Data PipelinesDiscretionary annual incentive program | Mentorship | Professional development | Travel opportunitySenior-level Full TimeRosslyn, Virginia, United States10h ago
-
AWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon CloudWatchBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeColumbus, OH, United States12h ago
-
Software Engineer, Search, AI/ML USD 147K-210KC++ | Data Governance | Data Governance Framework | Data Management | Data PrivacyMid-level Full TimeCambridge, MA, USA13h ago
-
C++ | Cloud Storage | Compute Technologies | Data Structures | Data Structures and AlgorithmsBonus | Equity | Health insurance | Paid time offSenior-level Full TimeNew York, NY, USA13h ago
-
Software Engineer, AI/ML, Google Research USD 147K-210KData Processing | Data Structures | Data Structures and Algorithms | Debugging | Language ProcessingMid-level Full TimeMountain View, CA, USA13h ago
-
Accessibility | Artificial Intelligence | Code review | Data Storage | Data StructuresSenior-level Full TimeMountain View, CA, USA13h ago
-
C++ | Cloud Computing | Computer Vision | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA13h ago
-
Senior Software Engineer, Map Ads Machine Learning USD 174K-252KAds serving | C++ | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA13h ago
-
Algorithms | Artificial Intelligence | C++ | Code Quality | Data StorageSenior-level Full TimeSunnyvale, CA, USA13h ago
-
API performance | Agent systems | AlloyDB AI | BigQuery | C++Senior-level Full TimeAtlanta, GA, USA; Austin, TX, USA13h ago
-
Systems Yield Engineer, Quantum AI USD 132K-189KCryogenic testing | Data Analysis | Data Visualization | Electrical Noise Analysis | Electrical noiseMid-level Full TimeGoleta, CA, USA13h ago
-
ANSYS | ANSYS APDL | AVL Excite | Computational Fluid Dynamics | Design optimizationMid-level Full TimeColumbus, IN, United States14h ago
-
C# | Git | JTAG | Logic analyzer | MicrocontrollersEntry-level Full TimeHillsboro, OR, US15h ago
-
AI/ML Engineer - Automation/Robotics USD 180K-200KAlgorithms | Azure Machine Learning | Data Structures | Diffusion Models | Fine Tuning401k retirement plan | Flexible spending account | Medical, dental & vision coverage | Paid time off | Parental leaveSenior-level Full TimeRedmond, WA18h ago
-
Data Engineer USD 79K-147KAccess Control | Auditing | Backup and Recovery | Bash | Caching401k match | Dental insurance | Education assistance | Employee assistance program | Flexible spending accountsSenior-level Full TimeHighlands Ranch, CO, US19h ago
-
AWS | Algorithms | Azure | Cloud Native | Data LineageMid-level Full TimeAustin; Sunnyvale19h ago