Data Engineer II
Tasks
- Build and maintain ETL ELT data pipelines in Databricks and Spark
- Build retrieval augmented generation data pipelines with vector integrations
- Design and optimize bulk generative AI data pipelines in Databricks
- Develop and evolve logical and physical data models
- Develop data ingestion, transformation, governance, and monitoring frameworks
- Enable experimentation model training and production deployments with AI teams
- Implement data quality monitoring alerts testing and lineage
- Orchestrate data workflows with Databricks Jobs and DBT
- Translate business needs into data requirements and stable implementations
Perks/Benefits
Skills/Tech-stack
Amazon RDS | Amazon Redshift | Apache Spark | DBT | Data Governance | Data Lineage | Data Modeling | Data Observability | Data Quality | Data pipeline | Data pipeline orchestration | Databricks | ELT | ETL | Embeddings | Gainsight | Generative AI | Gong | Machine Learning | Outreach | Pipeline Orchestration | Python | Retrieval-Augmented Generation | SQL | Salesforce | Snowflake | Vector Databases | Workflow Scheduling
Education
N/A
Roles
Related jobs
-
Principle Data Engineer USD 220K-235KAWS | Airflow | BigQuery | Capacity Planning | Compliance401k | Equity | Essential equipment | Flexible PTO | Fully remoteSenior-level Full TimeCleveland, OH R8h ago
-
Software Engineer, Data Platform USD 105K-132KAPI | AWS | CI/CD | Code review | DBT401k | Baby bonding leave | Commuter benefits | Disability insurance | Employee referral programSenior-level Full TimeUS Remote R10h ago
-
Senior Software Engineer, Data Platform USD 125K-156KAPI Development | AWS | Automation | CI/CD | Data Engineering401k benefits | Baby bonding leave | Commuter benefits | Disability insurance | Employee referral programSenior-level Full TimeUS Remote R10h ago
-
Finance Analytics Engineer USD 185K-221KAnomaly Detection | Apache Airflow | Business Intelligence | Data Modeling | Data MonitoringEquity compensation | Health benefitsMid-level Full TimeSan Mateo, CA, United States R12h ago
-
Staff Machine Learning Engineer, AI Research USD 230K-275KComputer Vision | Feature Engineering | Fine Tuning | Hyperparameter Tuning | Kubeflow401k | Dental insurance | Equity | Fertility treatment benefit | Health insuranceSenior-level Full TimeRemote - United States R13h ago
-
Full-Stack Analytics Engineer II USD 95K-118KAirflow | BigQuery | CI/CD | DBT | Dagster401k match | Dental insurance | Educational reimbursement | Flexible schedule | Health insuranceMid-level Full TimeWashington DC R13h ago
-
A/B | A/B Testing | AWS | Adversarial Testing | Amazon SQSHybrid work | W2 employmentSenior-level Contract Full TimeIrvine, CA, United States R13h ago
-
Principal Engineer, AI Architect USD 190K-210KCloud Architecture | Conversational AI | Enterprise SaaS | GCP | Generative AI401k match | Critical illness insurance | Dedicated WeWork space | Employee assistance program | Employee stock purchase programSenior-level Full TimeRemote - Colorado R13h ago
-
Principal Engineer, AI Architect USD 190K-210KCloud Native | Conversational AI | Enterprise Architecture | GCP | Generative AI401k match | Dental insurance | Development resources | Employee assistance program | Employee stock purchase programSenior-level Full TimeRemote - Colorado R13h ago
-
Senior Machine Learning Engineer USD 180K-200KCloud Computing | Continuous integration | Fine Tuning | Generative AI | Language Models401k match | Dental insurance | Equity package | Health insurance | Hybrid workSenior-level Full TimeRemote - United States R16h ago
-
Senior Machine Learning Engineer, Roblox Assistant USD 196K-243KData Processing | Distributed data | Distributed data processing | Fine Tuning | Language ModelSenior-level Full TimeSan Mateo, CA, United States R16h ago
-
Data Engineer USD 100K-128KData Governance | Data Modeling | Data Security | Databricks | ELT401k match | Adoption Assistance | Community volunteer opportunities | Continuing education support | Dental insuranceMid-level Full TimeSouth Sioux City, NE, United States R17h ago
-
Data Engineer USD 148K-263KAPI | Apache Kafka | Apache Spark | Cassandra | Distributed SystemsDisability insurance | Health insurance | Holiday pay | Learning and development | Life insuranceMid-level Full TimeUSA-Remote Work R19h ago
-
Data Engineer - AWS/Databricks - Mid Level USD 113K-169KAWS | AWS Glue | Amazon IAM | Amazon RDS | Apache SparkMentorship | Personalized development plans | Training reimbursement | Work/life balance focusMid-level Full TimeReston, VA, United States R1d ago
-
Senior-level Full TimeRemote - United States R1d ago
-
Data Reliability Engineer/Software Engineer USD 110K-177KAmazon S3 | BigQuery | CI/CD | Cause analysis | Data Catalog401k match | Flexible spending account | Health savings account | Paid parental leave | Paid time off and holidaysMid-level Full TimeLowell, United States R1d ago
-
Senior Machine Learning Engineering (Remote) USD 108K-270KAWS | Agile | Azure | Bias detection | CI/CDConference and journal article opportunities | Mentorship culture | Remote workSenior-level Full TimeDurham, North Carolina, United States of … R1d ago
-
AI Engineer - Application Development USD 75K-158KAWS | AWS Bedrock | AWS GovCloud | Agno | Amazon BedrockFlexible time off | Learning resourcesMid-level Full Time999 REMOTE, United States R1d ago
-
Senior Data Engineer- Data Science & AI USD 165K-216KAWS Glue | AWS Lambda | AWS S3 | Airflow | Azure DataGreat place to work | Work-life balanceSenior-level Full TimeTelecommuter TN, United States R1d ago
-
Senior Software Engineer USD 186K-220KAWS | Azure | Cloud platform | Docker | Fine TuningContinued Career Development | Employee resource groups | Flexible WFH | Generous PTO | Paid volunteer timeSenior-level Full TimeUS-California-Remote, United States R1d ago
-
AI Solutions Architect USD 126K-225KAir gapped deployment | Air-gapped | Apache Kafka | Apache NiFi | Data PipelinesCareer development | Employee resource groups | Flexible work from home | Generous paid time off | Paid volunteer timeSenior-level Full TimeUS-Washington DC-Remote, United States R1d ago
-
Senior Data Engineer - Anywhere Cloud USD 152K-195KABAC | Access Control | Anomaly Detection | Apache Airflow | Apache KafkaContinued Career Development | Employee resource groups | Flexible WFH policy | Generous PTO policy | Mental and physical wellness programsSenior-level Full TimeUS-California-San Jose, United States R1d ago
-
Actuarial Data Scientist (Hybrid) USD 99K-163KData Analysis | Data Mining | Data Modeling | Data Wrangling | Git401k contribution | Dental insurance | Medical insurance | Non Disclosure Agreement | Paid HolidaysMid-level Full TimeAF-MA Boston Office Congress St, United … R1d ago
-
Distinguished Engineer - Applied AI USD 210K-350KAI Governance | AI Safety | Anthropic API | Database Design | Generative AI401k match | Adoption Assistance | Fertility assistance | Health and wellness benefits | Mental healthcareSenior-level Full TimeCA Palo Alto - JPS, United … R1d ago
-
AWS | Agile | Azure | CI/CD | Data Modeling401k match | Adoption Assistance | Fertility assistance | GEICO Flex program | Hybrid workSenior-level Full TimeMD Bethesda (Office) - JPS, United … R1d ago