Data Engineer II
Tasks
- Build ETL and ELT pipelines
- Create monitoring and observability for pipelines
- Deploy data pipelines to cloud
- Design and build data pipelines
- Develop batch and real-time processing systems
- Develop stream-processing solutions
- Ensure data quality validation and reliability
- Implement infrastructure-as-code
- Mentor junior engineers
- Optimize data processing performance and scalability
- Perform code reviews and design discussions
- Process structured and unstructured data
- Support production deployments and troubleshooting
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS CloudFormation | Airflow | Amazon EMR | Amazon S3 | Apache Flink | Apache Spark | Avro | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | BigQuery | CI/CD | Cloud Storage | Cloud platform | Data Lake Storage | Databricks | Distributed Systems | Docker | ELT | ETL | Gatling | GitHub Actions | Google Cloud | Google Cloud Platform | Google Cloud Storage | Grafana | Hibernate | Infrastructure as Code | JMeter | JUnit | Java | Jenkins | Kafka | Kubernetes | Messaging Systems | Multithreading | ORC | Parquet | Prometheus | Python | REST APIs | Redshift | SQL | Snowflake | Snowflake Schema | Spark Structured Streaming | Splunk | Spring Boot | Spring Security | Star Schema | Structured Streaming | Terraform | Unit Testing | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Related jobs
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAzure | Data masking | Python | RBAC | Row Level SecurityFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAccess Control | Airflow | Amazon Web Services | Cloud platform | DBTEqual employment opportunity | Flexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Cloud platform | DBTFlexible work arrangements | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Apache Airflow | Azure | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Support for wellbeingSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Airflow | Azure | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing support | Work-life balance supportSenior-level Full TimeBengaluru Millenia, India7h ago
-
Software Engineer (Data Engineering & Building APIs) INR 700K-1500KAWS | Algorithms | CI/CD | Data Structures | DevOpsHybrid work flexibilityMid-level Full TimeGurgaon - Cyber Park, India9h ago
-
AWS Infra and DevOps Consultant INR 2200K-3500KAPI Gateway | AWS Audit Manager | AWS Backup | AWS CDK | AWS CodePipelineSenior-level Full Timebangalore, India11h ago
-
Senior-level Full TimeBangalore16h ago
-
Senior-level ContractHyderabad, India16h ago
-
Mid-level Full Timekerala, India17h ago
-
Principal Engineer, Data Engineering (HPC cluster software such as Slurm, NC, LSF or Grid Engine) expereince with 9 to 13 years INR 1600K-3000KAnsible | Backup Management | Docker | Grid Engine | InfinibandSenior-level Full TimeBengaluru, KA, India17h ago
-
Senior-level Full TimeNoida,UP,India R17h ago
-
Lead Software Engineer - Java, Spring boot, AWS Cloud & AI/ML (REST APIs, System Design, SDLC) INR 3000K-5000KAPI Gateway | AWS Glue | AWS Lambda | AWS Step Functions | AgileSenior-level Full TimeBengaluru, Karnataka, India17h ago
-
Senior-level Full TimeIN-MH-Pune17h ago
-
Data Engineer INR 1500K-2500KAWS | AWS Glue | AWS Lake Formation | AWS Step Functions | Amazon AthenaSenior-level Full TimeIN-MH-Pune17h ago
-
Snowflake Developer INR 1500K-2500KApache Airflow | DBT | Data Sharing | Data Warehousing | Dynamic TablesSenior-level Full TimeIN-TN-Chennai17h ago
-
Associate Engineer - AI/ML INR 2500K-4000KDistributed Systems | Edge AI | GPU | LLM Agents | LangchainAccess to cutting edge technologies and tools | Collaborative work environment | Professional growth and development | Training and development opportunitiesMid-level Full TimeBangalore - Karnataka, India - Kalyani …17h ago
-
Technologist, Data Analytics Engineering INR 3000K-4500KAI orchestration | API Management | Artificial Intelligence | Automation | CI/CDSenior-level Full TimeBengaluru, KA, India18h ago
-
GenAI Specialist INR 2000K-5000KAI Studio | Agent systems | Agentic AI | Azure AI | Azure AI StudioLeadership development | Training and development programsSenior-level Full TimeBengaluru, India18h ago
-
Senior Expert D&T MLOps Engineer INR 3000K-4000KDBT | Data Modeling | Data Vault | Databricks | ETLSenior-level Full TimeHyderabad, India19h ago
-
AI Customer Engineer, Conglomerates, Google Cloud INR 3000K-5000KAI Agent Development Kit | Agent Development Kit | Agentic AI | As-a-Service | Audit LoggingSenior-level Full TimeGurugram, Haryana, India; Mumbai, Maharashtra, India19h ago