Big Data Engineer
Tasks
- Automate data pipeline orchestration
- Build Data Lakes and Data Warehouses
- Build ETL/ELT pipelines
- Build low latency scalable data pipelines
- Develop analytics platforms
- Develop real-time data processing pipelines
- Ensure security compliance regulatory requirements
- Explore aligned tools platforms and technologies
- Implement data governance metadata management data quality
- Manage complex workflow dependencies
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Flink | Apache Hadoop | Apache Hive | Apache Kafka | Apache Spark | Data Flow | Data Pipeline Development | Data flow management | Data pipeline | Database Management Systems | ELT | ETL | Flow Management | Git | HBase | Linux | Management systems | MapReduce | Oozie | Pipeline Development | Process Automation | PySpark | Python | Relational Database | Relational Database Management | Relational database management systems | SQL | Shell Scripting | Spark Streaming
Education
N/A
Roles
Related jobs
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAzure | Data masking | Python | RBAC | Row Level SecurityFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAccess Control | Airflow | Amazon Web Services | Cloud platform | DBTEqual employment opportunity | Flexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Cloud platform | DBTFlexible work arrangements | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Apache Airflow | Azure | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Support for wellbeingSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Access Control | Apache Airflow | Azure | Cloud platformFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
IN_Senior Associate_Snowflake_Data Analytics_Advisory_Bangalore INR 2000K-3000KAWS | Airflow | Azure | Cloud platform | DBTFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing support | Work-life balance supportSenior-level Full TimeBengaluru Millenia, India9h ago
-
Software Engineer (Data Engineering & Building APIs) INR 700K-1500KAWS | Algorithms | CI/CD | Data Structures | DevOpsHybrid work flexibilityMid-level Full TimeGurgaon - Cyber Park, India11h ago
-
Senior AI & Automation Manager INR 3000K-5000KAI Agent | AI Agent Development | API | Agent Development | AutomationSenior-level Full TimeBangalore11h ago
-
AWS Infra and DevOps Consultant INR 2200K-3500KAPI Gateway | AWS Audit Manager | AWS Backup | AWS CDK | AWS CodePipelineSenior-level Full Timebangalore, India12h ago
-
Senior-level Full TimeBangalore18h ago
-
Senior-level ContractHyderabad, India18h ago
-
Mid-level Full Timekerala, India18h ago
-
Principal Engineer, Data Engineering (HPC cluster software such as Slurm, NC, LSF or Grid Engine) expereince with 9 to 13 years INR 1600K-3000KAnsible | Backup Management | Docker | Grid Engine | InfinibandSenior-level Full TimeBengaluru, KA, India18h ago
-
Senior-level Full TimeNoida,UP,India R19h ago
-
AWS | Apache Spark | Asynchronous programming | Azure | ChromaDBAccidental death and dismemberment coverage | Dental insurance | Flexible paid time off | Health insurance | Life insuranceMid-level Full TimeIndia R19h ago
-
Lead Software Engineer - Java, Spring boot, AWS Cloud & AI/ML (REST APIs, System Design, SDLC) INR 3000K-5000KAPI Gateway | AWS Glue | AWS Lambda | AWS Step Functions | AgileSenior-level Full TimeBengaluru, Karnataka, India19h ago
-
Senior-level Full TimeIN-MH-Pune19h ago
-
Data Engineer INR 1500K-2500KAWS | AWS Glue | AWS Lake Formation | AWS Step Functions | Amazon AthenaSenior-level Full TimeIN-MH-Pune19h ago
-
Snowflake Developer INR 1500K-2500KApache Airflow | DBT | Data Sharing | Data Warehousing | Dynamic TablesSenior-level Full TimeIN-TN-Chennai19h ago
-
Associate Engineer - AI/ML INR 2500K-4000KDistributed Systems | Edge AI | GPU | LLM Agents | LangchainAccess to cutting edge technologies and tools | Collaborative work environment | Professional growth and development | Training and development opportunitiesMid-level Full TimeBangalore - Karnataka, India - Kalyani …19h ago
-
GenAI Specialist INR 2000K-5000KAI Studio | Agent systems | Agentic AI | Azure AI | Azure AI StudioLeadership development | Training and development programsSenior-level Full TimeBengaluru, India20h ago
-
AI Agents | APIs | Data Privacy | Databases | EvaluationCareer growth opportunities | Global team collaboration | High ownership and autonomy | Remote work opportunityMid-level Full TimeIndia20h ago