Software Engineer III, Data Pipelines
Tasks
- Analyze datasets using Trino
- Automate data workflows and improve observability
- Build and maintain Python APIs and microservices
- Build and optimize ETL ELT pipelines with dbt
- Build frameworks for model development and deployment
- Containerize and orchestrate workloads with Docker and Kubernetes
- Design data pipeline architecture
- Develop data infrastructure across PostgreSQL Azure Cosmos DB and Azure services
- Implement backend services for real time and batch data access
- Implement data governance with lineage and cataloging
- Integrate data platforms with user facing applications
- Maintain data lakehouse tables with Apache Iceberg
- Manage schema evolution and database performance
- Monitor and troubleshoot data quality issues
- Provide technical mentorship to junior engineers
- Support MLOps workflows with Azure Machine Learning
- Use logging and monitoring for data pipelines
Perks/Benefits
- 401k match
- Company events
- Flexible spending accounts
- Gym membership contribution
- Internal referral program
- Medical, dental, and vision benefits
- Paid Holidays
- Paid sick leave
- Paid vacation
Skills/Tech-stack
Apache Iceberg | Azure | Azure Cosmos | Azure Cosmos DB | Azure Grid | Azure Machine Learning | Cosmos DB | DBT | Data Catalog | Data Governance | Data Lakehouse | Data Lineage | Docker | ELT | ETL | Event Hub | Flask | Kubernetes | Logging and Monitoring | MLOps | Machine Learning | Microservices | Observability | PostgreSQL | Python | SQL | Schema evolution | Trino
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R11d ago
-
Senior Developer, Data & IT - AI Solutions USD 120K-142KAI Agents | API Integration | AWS | AWS Bedrock | AWS SageMakerDental insurance | Dependent Care Account | Health insurance | Health savings account | Mental health counseling supportSenior-level Full TimeNew York, NY, United States9h ago
-
Senior Data Engineer USD 120K-142KApache Airflow | C# | Continuous integration | DBT | Data Modeling401b retirement savings program | Dental insurance | Dependent Care Account | Employer match | Flexible spending accountsSenior-level Full TimeNew York, NY, United States9h ago
-
Principal AI Engineer - Nexus Black USD 135K-160KCI/CD | Cloud Native | Cloud Native Architecture | Distributed Systems | EvaluationHybrid workSenior-level Full TimeItasca, United States11h ago
-
Distinguished Software Engineer, Data Infrastructure USD 248K-406KAI Inference | AI Training | Batch Processing | Compliance | Data InfrastructureExecutive-level Full TimeMountain View, CA, United States11h ago
-
Senior-level Full TimeMorristown, NJ, United States12h ago
-
Staff Data Engineer - Information Security USD 152K-248KAzure | Big Data | C# | C++ | Data PipelinesHealth and wellness programs | Time away from workSenior-level Full TimeSunnyvale, CA, United States13h ago
-
Manager, Data Engineering USD 130K-166KAWS | Access Controls | Apache Airflow | Audit Logging | AzureCollaborative team culture | Remote work | Work-life balanceSenior-level Full TimeRemote, United States R13h ago
-
AI Search | API Development | AWS | AWS Bedrock | Azure401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States13h ago
-
AWS S3 | Access Control | Active IQ | Ansible | Azure Blob Storage401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States13h ago
-
Communication optimization | Data parallelism | Deep learning | Distributed Training | GPU AccelerationMid-level Full TimeSeattle, Washington, United States17h ago
-
Robotics/UAS Engineer II/III USD 89K-176KArduPilot | Autopilot | C++ | CAD | Embedded SystemsCareer Development Training Opportunities | Education assistance | Fitness reimbursement | Flexible work schedules | Paid Time Off N/ASenior-level Full TimeUS-Massachusetts-Wilmington17h ago
-
Product Engineering, Full Sack Engineer USD 85K-142K.NET | .Net Core | AWS | Agile | AngularMentorship | Professional development | Travel opportunitiesNone Full TimeNew York, New York, United States17h ago
-
Content Safety | Data Modeling | Feature Engineering | Graph Databases | Information RetrievalSenior-level Full TimeSan Jose, California, United States17h ago
-
Partner Engineering GenAI - US USD 133K-203KAPIs | Artificial Intelligence | C plus plus | Claude | Cloud ComputingSenior-level Full TimeMenlo Park, CA | Seattle, WA …18h ago
-
Machine Learning Performance Modeling Architect USD 173K-249KC# | C++ | Data Visualization | Heterogeneous computing | Image qualitySenior-level Full TimeSunnyvale, CA18h ago
-
Mid-level Full TimeSunnyvale, CA | Burlingame, CA18h ago
-
Robotics Engineer - Logistics and Material Flow USD 170K-240KAGV | Automation | Branching | C++ | Computer ScienceSenior-level Full TimeFremont, CA18h ago
-
Software Developer, Scaled Ops AI Acceleration Team USD 147K-203KAI infrastructure | Data Mining | Fine Tuning | Hack | JavaScriptSenior-level Full TimeSunnyvale, CA | Austin, TX | …18h ago
-
Automated testing | C++ | CSS | Debugging | GraphQLSenior-level Full TimeMenlo Park, CA18h ago
-
Robotics Control Engineer - Manipulation USD 170K-240KABB Rapid | AI Motion Planning | Adaptive Control | C++ | Cause analysisSenior-level Full TimeMenlo Park, CA | Fremont, CA18h ago
-
Robotics Manipulation Engineer USD 170K-240KAdaptive Control | Automation | C++ | Deep learning | GazeboSenior-level Full TimeFremont, CA18h ago
-
Software Engineer - Language (Technical Leadership) USD 213K-293KASR | Benchmarking | C# | C++ | Conversational AISenior-level Full TimeMenlo Park, CA | Seattle, WA …18h ago
-
Code review | Contamination Checking | Data Generation | Data Pipelines | Data ProcessingEntry-level Full TimeMenlo Park, CA18h ago
-
Business Support Engineer USD 136K-197KCall Support | Cloud Computing | Data Analysis | Data Mining | Docker24x7 on-call rotationEntry-level Full TimeMenlo Park, CA18h ago