Data Engineer - Guidewire
Tasks
- Analyze integration and system requirements
- Build dashboards and reports
- Build data models and schemas
- Conduct root cause analysis
- Create embeddings and vector search datasets
- Create unit scenario and end to end tests
- Design ELT pipelines
- Enrich data with metadata and context
- Implement data governance, security, and compliance
- Monitor generative AI data workflows
- Optimize data storage and retrieval for low latency responses
- Optimize pipeline and storage performance
- Prepare and clean large datasets for model training
- Process structured and unstructured data for generative AI
- Support model training and inference
- Troubleshoot data issues
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Agile Development | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure Event | Azure Event Grid | Azure Key Vault | Azure SQL | Azure SQL Database | CI/CD | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Modeling | Data Quality | Data Security | Data Warehousing | Data integration | Delta Lake | ELT | ETL | Embeddings | Encryption | Event Grid | Key Vault | Metadata Management | MySQL | Oracle | Power BI | SQL | SQL Database | SQL Server | Vector Databases
Education
Associate Degree | Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
SOFTWARE ENGINEER - Apache Spark/Big Data - 7+ yrs of Experience - TS/SCI w/Poly clearance is required - ID A USD 234K-241KAnomaly Detection | Apache Airflow | Apache Spark | Bash | Big Data401k retirement plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeFort George G Meade, United States1h ago
-
Data Management Engineer III USD 93K-163KCI/CD | Data Deduplication | Data Pipelines | Data Standardization | Data TransformationPublic Trust Clearance | Travel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States2h ago
-
Senior Software Engineer, Database Internals AlloyDB USD 174K-252KC plus plus | C# | Code optimization | Compute Technologies | Concurrency ControlEntry-level Full TimeSunnyvale, CA, USA4h ago
-
Staff Software Engineer, AI-Powered GRC Automation USD 207K-300KCloud Platforms | Cloud platform | Continuous controls monitoring | Controls monitoring | Data PipelinesSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA4h ago
-
Senior Staff Software Engineer, BigQuery Core Analytics USD 262K-365KBigQuery | C plus plus | C++ | CI/CD | Distributed SystemsSenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA4h ago
-
Staff Machine Learning Engineer, Embeddings USD 253K-354KA/B | A/B Testing | B testing | C++ | Cloud ComputingCaregiving support | Comprehensive healthcare benefits | Employer 401k match | Family planning support | Flexible vacationSenior-level Full TimeRemote - United States R10h ago
-
GenAI Enablement Specialist USD 170K-231KAgentic Frameworks | Embeddings | LLM | LLM Evaluation | Prompt engineeringBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersExecutive-level Full TimeNew York, NY, United States13h ago
-
Senior-level Full TimePalo Alto, California13h ago
-
Staff AI Engineer USD 215K-285KA/B | A/B Testing | B testing | Behavioral signals | Distributed SystemsSenior-level Full TimePalo Alto, California; San Francisco, California13h ago
-
AI Software Engineer (Vehicle Engineering) USD 125K-175K3D Reconstruction | Agent systems | Agentic AI | Anomaly Detection | CI/CD401k retirement plan | Dental insurance | Employee stock purchase plan | Life insurance | Life insurance coverageSenior-level Full TimeHawthorne, CA14h ago
-
Lead AI Engineer (ML Ops) USD 116K-170KAPIs | AWS | Agile Scrum | Azure | CI/CD401k match | Employee assistance program | Employee charity match | Employee stock purchase plan | Health and wellness allowanceSenior-level Full TimeIrving - 6011 Connection, United States15h ago
-
Senior Machine Learning Engineer, Computer Vision USD 150K-200KAWS | Agile | Airflow | Azure | CI/CD401-k plan | Healthcare benefits | Life insurance | Long-term disability | On-site collaborationSenior-level Full TimeSeattle, Washington, United States16h ago
-
Performance Engineer, Inference Systems USD 350K-850KAutoscaling | Cause analysis | Data Analysis | Data Modeling | Distributed SystemsSenior-level Full TimeSan Francisco, CA | New York …16h ago
-
Full Stack Engineer, Experimentation USD 145K-235KAWS | Agile | Azure | CI/CD | Design PatternsDental insurance | Health insurance | Mental health benefits | Restricted stock units | Vision insuranceSenior-level Full TimeRemote - US R16h ago
-
Senior Data Engineer, Knowledge & Information USD 153K-238KAWS | Alerting | Apache Airflow | Apache Spark | CI/CD401k company match | Dental insurance | Disability insurance | Flexible time off | Health insuranceSenior-level Full TimeUnited States R16h ago
-
Senior Software Engineer, Data USD 144K-216KAWS | Airflow | Azure | DBT | Data LineageDental insurance | Disability insurance | Family-forming benefits | Life insurance | Medical insuranceSenior-level Full TimeSan Jose, California17h ago
-
Compliance, Data Engineering, Dallas, Associate USD 148K-200KAlgorithms | Apache Spark | Batch Processing | C++ | Cloud ComputingSenior-level Full TimeDallas, TX, United States17h ago
-
AI machine learning | Agile | Application Resiliency | CI/CD | Decision supportBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States17h ago
-
Staff AI Engineer USD 170K-220KAPI Development | API Integration | Anthropic API | Artificial Intelligence | Backend Development401k match | Commuter benefits | Employee assistance program | Flexible spending accounts | Gym Fitness Discount ProgramSenior-level Full TimeRemote- US R17h ago
-
Business Analytics Lead - Predictive Modeling USD 130K-140KA/B | A/B Testing | B testing | Business Intelligence | Dashboard Development401k retirement savings plan | Dental coverage | Equity participation | Flexible spending account | Health savings accountSenior-level Full TimeNew York, NY18h ago
-
Quantitative Engineer USD 140K-155KAI Assistant | API Design | AWS | CI/CD | Credit facility401k | Dental insurance | Fitness fund | Health insurance | Learning and development fundSenior-level Full TimeRemote - USA R18h ago
-
AI Prompt Engineer USD 90K-175KAI Agents | Amazon Kiro | Appian AI | Data integration | Google GeminiDisability insurance | Employee assistance program | Employee stock purchase program | Health coverage | Life insuranceMid-level Full TimeMcLean, Virginia18h ago
-
Senior, ML Engineer - Auto Tagger USD 177K-212KAWS | Apache Arrow | Apache Beam | Apache Spark | Cloud platform401k match | Company holiday office closures | Company-paid medical, dental & vision | Disability insurance | Flexible scheduleSenior-level Full TimeAnn Arbor, MI, Remote - US R18h ago
-
Data Science Senior Associate - Digiops Analytics USD 170K-201KA/B | A/B Testing | Adobe Analytics | Advanced Analytics | AlteryxBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersMid-level Full TimeWilmington, DE, United States18h ago
-
Principal Data Engineer USD 133K-165KAccess Control | Azure Data | Azure Data Factory | Azure Databricks | CI/CDSenior-level Full TimeDes Moines, IA, United States19h ago