Data Engineer
Tasks
- Build and optimize ETL ELT pipelines in AWS
- Build document ingestion indexing embedding workflows
- Collaborate with architecture ML and application teams
- Design RAG pipelines
- Develop structured and unstructured data pipelines
- Ensure data quality lineage governance and security
- Ensure reliability observability and error handling
- Implement ElasticSearch indexing search and retrieval patterns
- Implement NiFi flows for ingestion and transformation
- Monitor system performance and troubleshoot distributed systems
- Orchestrate and schedule workflows in Prefect
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | CI/CD | Data Modeling | Docker | Document chunking | ECS | EKS | ELT | EMR | ETL | Elasticsearch | Embeddings | Glue | IAM | Kubernetes | Lambda | Nifi | OpenSearch | Pandas | Pinecone | Prefect | PySpark | Python | Requests | Retrieval-Augmented Generation | S3 | Step Functions | Vector Databases | Version control | Weaviate
Education
N/A
Roles
Related jobs
-
A/AI Machine Learning Engineering Stf - E4 USD 135K-251KAI | AI RMF | Adversarial Machine Learning | Automation Testing | C#401k match | Dental insurance | EAP | Education assistance | Employee assistance programSenior-level Full TimeShelton, CT, US3h ago
-
PIIM Data Engineer USD 119K-182KAPI Integration | APIs | Azure Databricks | Data Governance | Data LakeFull Time Office Project Telework | Relocation assistanceMid-level Full TimeNew Albany, OH, US3h ago
-
Software Engineer, Data Platform Team USD 152K-179KApache Airflow | Data APIs | Data Catalog | Data Lineage | Data Modeling401k matching | Medical, dental, and vision | Quarterly in person surges | Remote-first work environmentMid-level Full TimeRemote - USA R4h ago
-
Senior Machine Learning Engineer USD 229K-360KAB Testing | APIs | Airflow | Algorithm Development | Amazon S3Disability benefits | Health insurance | Life insurance | Paid Equity Awards | Paid time offSenior-level Full TimeNew York, New York R4h ago
-
Senior Machine Learning Engineer USD 229K-360KA/B | A/B Testing | Amazon S3 | Amazon SageMaker | Apache AirflowCommuter benefits | Disability benefits | Equity awards | Financial wellness support | Health insuranceSenior-level Full TimeSan Jose, California R4h ago
-
Staff Machine Learning Engineer - Ads USD 164K-282KAWS | Airflow | Apache Spark | BigQuery | Cloud Computing401k plan with company match | AD&D insurance | Disability insurance | Electric Car Charging Station | Employee assistance programSenior-level Full TimeMountain View, USA7h ago
-
Agents | Golang | Information Retrieval | Language Models | Language Processing401k company match | ESPP | Family leave programs | Flexible spending accounts | Flexible time offSenior-level Full TimeMountain View, CALIFORNIA, United States R7h ago
-
ASIC | Agile | Automation frameworks | BDD | C#Senior-level Full TimeCupertino, California, USA8h ago
-
Staff Machine Learning Engineer USD 176K-308KAlgorithms | Code Tuning | Design Patterns | High Performance | High-Performance Computing401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R8h ago
-
Principal Machine Learning Engineer - League of Legends USD 292K-437KAI | API Development | Auto-remediation | Automation | Cloud Native401k company match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA9h ago
-
Sr Engineer, Data Engineering USD 32K-46KAmazon Web Services | Apache Spark | Azure | Data Lake | Data WarehouseSenior-level Full TimeKnoxville, TN, United States11h ago
-
Senior Applied AI Engineer USD 197K-266KAlgorithms | Artificial Intelligence | Data Analysis | Data Structures | Deep learningFlexible time off | Mental health support | Paid parental leave | Private health insurance | Work from homeSenior-level Full TimeSan Francisco, CA R11h ago
-
Embedded Software Engineer USD 85K-128KARM | Automated testing | C# | C++ | CAN401k matching | Hands-on training | Mentoring program | Relocation assistance not providedSenior-level Full TimeWaltham, MA, United States12h ago
-
Staff Analyst, Finance Analytics & AI USD 150K-200KArtificial Intelligence | Coding Assistant | DBT | Data Modeling | EmbeddingsSenior-level Full TimeUS-CA-Menlo Park12h ago
-
Senior Data Engineer USD 70K-75KAmazon SageMaker | Apache Airflow | Azure | Azure Compute | Azure Compute VMs401k match | Career development support | Dental insurance | Life and disability insurance | Medical coverageSenior-level Full TimeStafford, TX, United States12h ago
-
Senior-level Full TimeIrvine, CA12h ago
-
Practice Architect AI/ML USD 136K-204KAI RMF | AWS | Agent Orchestration | Agentic Workflows | Autogen401k | Health insurance | Paid time off | Remote workSenior-level Full TimeDallas, Texas, USA R12h ago
-
Active Learning | C++ | Computer Vision | Confidence Scores | Data BalancingSenior-level Full TimeMountain View, CA, USA; San Francisco, … R13h ago
-
Staff Machine Learning Engineer USD 176K-308KAlgorithms | Artificial Intelligence | Data Structures | Design Patterns | Java401k match | ESPP | Family leave | Flexible time off | Health plansSenior-level Full TimeSanta Clara, California, United States R13h ago
-
Senior-level Full TimeRemote or VA, VA, United States R14h ago
-
Senior-level Full TimeVirginia, Virginia, United States14h ago
-
Senior-level Full TimeWashington, DC15h ago
-
Staff Data Engineer USD 200K-240KAPI Design | AWS Bedrock | AWS Fargate | AWS Lambda | Amazon RDS401k | Commuter benefits | Company holidays | Dental insurance | Disability insuranceSenior-level Full TimeSan Francisco, CA R15h ago
-
Software Engineer II - Business Intelligence USD 139K-166KAWS | AWS Glue | Amazon Athena | Amazon QuickSight | DatabricksFlexible location | Remote work optionMid-level Full TimeAnn Arbor, MI; Remote (US) R15h ago
-
Data Engineer - Data Platform USD 185K-210KAmazon RDS | Apache Airflow | CI/CD | DBT | Data ModelingMid-level Full TimeSan Bruno, California15h ago