Data Engineer
Tasks
- Automate data workflow deployments and monitoring
- Create and maintain operational documentation
- Design and implement data lake solutions
- Develop and optimize data pipelines
- Diagnose troubleshoot and resolve job failures and performance bottlenecks
- Manage data governance, quality, and security
- Manage privileged access for cluster administration
- Manage secrets for secure key and secret handling
- Migrate Datastage ETL jobs to Azure
- Optimize SQL queries
- Perform CI CD for data workflows
- Provide on call support and cluster operations
- Tune Hadoop Hive and Spark jobs
- Validate data integrity and performance during migration
Perks/Benefits
Skills/Tech-stack
Ansible | Apache Ranger | Atlas | Automation | Azure | Azure Data | Azure Data Factory | Azure Databricks | Azure Synapse | Azure Synapse Analytics | CI/CD | Cloudera | CyberArk | DNS | Data Factory | Data Governance | Data Lake | Data Quality | Data Security | Databricks | Distributed Systems | HDFS | Hadoop | HashiCorp Vault | Hive | JupyterHub | Kerberos | LDAP | Linux | Network Troubleshooting | PySpark | Python | SQL | Scripting | Shell | Snowflake | Spark | Sqoop | Synapse Analytics | YARN
Education
Roles
Regions
Countries
States
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R10d ago
-
Communication optimization | Data parallelism | Deep learning | Distributed Training | GPU AccelerationMid-level Full TimeSeattle, Washington, United States4h ago
-
Robotics/UAS Engineer II/III USD 89K-176KArduPilot | Autopilot | C++ | CAD | Embedded SystemsCareer Development Training Opportunities | Education assistance | Fitness reimbursement | Flexible work schedules | Paid Time Off N/ASenior-level Full TimeUS-Massachusetts-Wilmington4h ago
-
Product Engineering, Full Sack Engineer USD 85K-142K.NET | .Net Core | AWS | Agile | AngularMentorship | Professional development | Travel opportunitiesNone Full TimeNew York, New York, United States5h ago
-
Software Engineer, Data Security - USDS USD 118K-237KData Security | Distributed Systems | Language Models | Large Language Models | Performance optimizationEntry-level Full TimeSan Jose, California, United States5h ago
-
Partner Engineering GenAI - US USD 133K-203KAPIs | Artificial Intelligence | C plus plus | Claude | Cloud ComputingSenior-level Full TimeMenlo Park, CA | Seattle, WA …6h ago
-
Machine Learning Performance Modeling Architect USD 173K-249KC# | C++ | Data Visualization | Heterogeneous computing | Image qualitySenior-level Full TimeSunnyvale, CA6h ago
-
Mid-level Full TimeSunnyvale, CA | Burlingame, CA6h ago
-
Robotics Engineer - Logistics and Material Flow USD 170K-240KAGV | Automation | Branching | C++ | Computer ScienceSenior-level Full TimeFremont, CA6h ago
-
Software Developer, Scaled Ops AI Acceleration Team USD 147K-203KAI infrastructure | Data Mining | Fine Tuning | Hack | JavaScriptSenior-level Full TimeSunnyvale, CA | Austin, TX | …6h ago
-
Automated testing | C++ | CSS | Debugging | GraphQLSenior-level Full TimeMenlo Park, CA6h ago
-
Robotics Control Engineer - Manipulation USD 170K-240KABB Rapid | AI Motion Planning | Adaptive Control | C++ | Cause analysisSenior-level Full TimeMenlo Park, CA | Fremont, CA6h ago
-
Robotics Manipulation Engineer USD 170K-240KAdaptive Control | Automation | C++ | Deep learning | GazeboSenior-level Full TimeFremont, CA6h ago
-
Software Engineer - Language (Technical Leadership) USD 213K-293KASR | Benchmarking | C# | C++ | Conversational AISenior-level Full TimeMenlo Park, CA | Seattle, WA …6h ago
-
Code review | Contamination Checking | Data Generation | Data Pipelines | Data ProcessingEntry-level Full TimeMenlo Park, CA6h ago
-
Senior-level Full TimeSarpy County, NE | Mesa, AZ …6h ago
-
Business Support Engineer USD 136K-197KCall Support | Cloud Computing | Data Analysis | Data Mining | Docker24x7 on-call rotationEntry-level Full TimeMenlo Park, CA6h ago
-
Business Support Engineer USD 159K-223KCloud Computing | Data Analysis | Data Mining | Distributed Systems | Docker24x7 on-call rotation | Cross-functional team collaboration | Global partner supportSenior-level Full TimeMenlo Park, CA6h ago
-
Senior-level Full TimeMenlo Park, CA | New York, …6h ago
-
Research Engineer, Media Data Research - MSL FAIR USD 170K-251KComputer Vision | Data Curation | Data Generation | Data Scaling Laws | Data mixingSenior-level Full TimeMenlo Park, CA6h ago
-
Staff Software Engineer, Torch TPU USD 207K-300KCUDA | Computer Vision | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeSunnyvale, CA, USA6h ago
-
C++ | Compilers | Custom Kernels | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA6h ago
-
Technical Solutions Engineer, Cloud AI, Google Cloud USD 150K-218KAI Model Training | AI model | Apache Beam | Apache Hadoop | Apache SparkSenior-level Full TimeSunnyvale, CA, USA; Austin, TX, USA6h ago
-
Principal AI Engineer - Core Platform USD 250K-290KAWS | Agents SDK | Anomaly Detection | Automated testing | Classification401k match | Company-provided phone | Health insurance | Hybrid work | PTOSenior-level Full TimeNew York, New York, United States13h ago
-
Senior-level Full TimeDes Moines, IA, United States13h ago