Insurance Data Engineering - Data Engineer
Tasks
- Build data pipelines
- Build lakehouse tables
- Catalog data assets
- Conduct code reviews
- Contribute to platform architecture
- Design data pipelines
- Develop AWS Glue jobs
- Ensure data quality and integrity
- Implement data governance and lineage
- Integrate data from APIs and databases
- Maintain data pipelines
- Manage scheduling and retries
- Manage schema evolution
- Mentor junior engineers
- Model and load data in Snowflake
- Monitor pipeline performance and cost
- Operate ETL and ELT processes
- Optimize data in Snowflake
- Optimize table maintenance
- Orchestrate workflows
- Process streaming feeds
- Run Spark processing jobs
- Troubleshoot pipeline failures
- Tune pipeline reliability and performance
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Cloud | AWS Cloud Development Kit | AWS CloudFormation | AWS Glue | AWS Lambda | AWS cloud development | Amazon Athena | Amazon EKS | Amazon EMR | Amazon IAM | Amazon Kinesis | Amazon Redshift | Amazon S3 | Apache Airflow | Apache Iceberg | Apache Kafka | Apache Spark | CI/CD | Cloud Development Kit | DBT | Dagster | Data Governance | Data Lineage | Data Observability | Data Quality | Docker | Great Expectations | Infrastructure as Code | Kubernetes | Python | REST API | SQL | Snowflake | Spark Structured Streaming | Structured Streaming | Terraform | “as-code”
Education
N/A
Roles
Related jobs
-
AI Engineer INR 6000KAPI Integration | Backend Development | Databases | Debugging | DeploymentInternational Internship Certificate | Letter of recommendation | Long-term employment opportunity | MentorshipEntry-level Full Time InternshipMumbai, MH, India6h ago
-
Sr. ML Engineer INR 3000K-4000KAPI Development | CI/CD | Cloud Native | Context engineering | Document ingestionAnnual leave | Employee assistance program | Employee wellbeing platform Unmind | Flexible working | Learning and development opportunitiesSenior-level Full TimeChennai, IN6h ago
-
Manager - Data Analyst INR 700K-1200KAgile | Dataiku | ETL | Forecasting | Language ProcessingAnnual leave | Continuous learning culture | Digital wellbeing platform | Employee assistance program | Flexible workingMid-level Full TimeChennai, IN6h ago
-
Data Engineer - Python and Azure Data Bricks INR 1500K-2500KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeCareer growth | Diversity and inclusion | Inclusive cultureMid-level Full TimeIndia - Bengaluru; India - Chennai …6h ago
-
Specialist, Gen AI Solutions INR 3000K-4000KAML | Anomaly Detection | CI/CD | Classification | ClusteringSenior-level Full TimeBangalore, IN6h ago
-
Python,Pyspark Developer INR 2800K-4000KAgile | Data Engineering | Data Processing | Jira | PMI-ACPAnnual leave | Employee assistance program | Flexible working | Life insurance | Medical insuranceSenior-level Full TimeChennai, IN6h ago
-
Specialist / Sr. Specialist, Data Platform Engineer INR 3000K-4000KData Governance | Data Visualization | Data Warehousing | Documentation | DomoRelocation assistance within countrySenior-level Full TimeMumbai, MH, IN6h ago
-
Solutions Architect INR 2000K-3000KAWS | Airflow | Apache Kafka | Apache Spark | AzureBroadband reimbursement | Continuous learning opportunities | Medical insurance | Paid certifications | Personal accident insuranceSenior-level Full TimeBangalore8h ago
-
Deputy Manager - Data Sciences INR 1200K-2000KAgile | Application Integration | Application design | Azure | Azure DataMid-level Full TimeGurgaon, HR, India8h ago
-
Senior Software Engineer , Data Eng INR 3600K-5000KAWS | Apache Airflow | Apache Spark | Azure | CI/CDCommuter benefits | Dental insurance | Disability insurance | Financial wellness support | Health insuranceSenior-level Full TimeBengaluru, India10h ago
-
Senior Data Engineer - Advertising Engineering INR 3470K-4580KAWS | Apache Airflow | Apache Spark | Azure | CI/CDFinancial wellness resources | Flexible remote work Fridays | Mental health support | Time offSenior-level Full TimeBengaluru, India10h ago
-
Staff ML - GenAI Engineer, Voice & Speech INR 3584K-4680KAPI Development | AWS | Audio Alignment | Audio Processing | Cloud ComputingFlexible work hours | Overlap with India and US business hours | Remote work | Work from IndiaSenior-level Full TimeIndia R12h ago
-
AWS | Alerting | Application Security | Authentication | AuthorizationAdditional Holiday Benefits | Continuous learning | Flexible remote first async working | Fully remote | High ownership roleSenior-level Full TimeIndia R13h ago
-
AWS | Agent Frameworks | Alerting | Authentication | AuthorizationCollaborative global teams | Fully remote | High ownership culture | Paid annual leave | Professional developmentSenior-level Full TimeIndia R13h ago
-
.NET | Agent Frameworks | Backend Development | C# | Distributed SystemsCareer growth opportunities | Continuous learning culture | Diversity equity and equal opportunity | Flexible working arrangements | Inclusive workplaceSenior-level Full TimeIndia R13h ago
-
Airflow | BigQuery | CI/CD | Cloud Composer | Cloud FunctionsCareer growth | Collaborative work environment | Inclusive workplace | Professional development and continuous learningSenior-level Full TimeIndia13h ago
-
Sr. Software Engineer (3-6 years of experience in Agentic AI / GenAI / LLM / RAG, Python) INR 3242K-5000KAPI Development | Agentic Workflows | Automation | CI/CD | GenAISenior-level Full TimeBengaluru, India14h ago
-
EY - GDS Consulting - AI And DATA -MS Fabric -Senior INR 1500K-4500KActive Directory | Agent skills | Azure AI | Azure Active Directory | Azure DevOpsSenior-level Full TimeTrivandrum, KL, IN, 69558114h ago
-
EY - GDS Consulting - AI And DATA - Snowflake-Senior INR 2500K-4200KAgile | Alerting | Automated testing | Batch Processing | CI/CDSenior-level Full TimeTrivandrum, KL, IN, 69558114h ago
-
EY - GDS Consulting - AI And DATA -Azure Databricks-Senior INR 2500K-4200KAPI Integration | AWS | Agent workflow | Alerting | Auto LoaderSenior-level Full TimeTrivandrum, KL, IN, 69558114h ago
-
Senior Data Engineer - (Real-Time Streaming) INR 1500K-2500KAPI Integration | Agile | Apache Flink | Apache Kafka | Apache SparkSenior-level Full TimeChennai, Tamil Nadu, India14h ago
-
Sr. Platform Engineer - (Cloudera CDP & DevOps) INR 2000K-4000KAlerting | Ansible | Apache Hive | Apache Spark | ArgoCDSenior-level Full TimeBengaluru, Karnataka, India14h ago
-
Senior AI Engineer INR 2500K-4800K.NET | AI Foundry | AI Search | AWS Bedrock | AWS Bedrock AgentCoreOperational readiness support | Rotational on-callSenior-level Full TimeHyderabad - Phoenix Aquila, India14h ago
-
Senior Lead AI Engineer INR 3242K-5000KAPI Development | Azure DevOps | Azure OpenAI | CI/CD | Data GovernanceSenior-level Full TimeMumbai Central Avenue, India14h ago
-
Data Engineer (SQL, PySpark, Snowflake, DBT, Redshift) - AVP/VP INR 4000K-4000KAWS CDK | AWS CloudFormation | AWS Glue | AWS IAM | AWS LambdaExecutive-level Full TimeGurugram20h ago