Big Data Engineer
Tasks
- Automate data pipelines with orchestration
- Build Data Lakes and Data Warehouses
- Build real-time data processing pipelines
- Build scalable low-latency data pipelines
- Develop ETL and ELT pipelines
- Ensure data quality standards
- Ensure security, compliance, and regulatory requirements
- Explore and evaluate data engineering tools and technologies
- Implement data governance and metadata management
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Flink | Apache Hadoop | Apache Hive | Apache Kafka | Apache Oozie | Apache Spark | Data Flow | Data Governance | Data Pipeline Development | Data Quality | Data flow management | Data pipeline | ELT | ETL | Flow Management | Git | HBase | Linux | MapReduce | Metadata Management | Pipeline Development | PySpark | Python | SQL | Shell Scripting | Spark Streaming
Education
N/A
Roles
Related jobs
-
Principal consultant GenAI LLM Ops Engineer INR 3125K-5000KAWS | Alerting | Azure | Azure OpenAI | BashSenior-level Full TimeIndia-Bangalore5h ago
-
Senior Principal Consultant, Gen_AI Technical Lead INR 2500K-4500KA/B | A/B Testing | AWS | Azure | Azure OpenAISenior-level Full TimeIndia-Bangalore5h ago
-
Software Developer – QA Automation & AI Integration (Backend) INR 1170K-2500KAPI Integration | Automated testing | BDD | CI/CD | CucumberAI certifications | Mentorship | TrainingMid-level Full TimeIndia-Hyderabad5h ago
-
Senior-level Full TimeIN-MH-Pune5h ago
-
Python - AI ML INR 2500K-4225KActiveMQ | Advanced search | Advanced search algorithms | Agile | ChromaDBSenior-level Full TimeIN-AP-Hyderabad5h ago
-
Data Engineer INR 1400K-2000KData Engineering | Data Lake | Data Lineage | Data orchestration | ETLMid-level Full TimeIN-TN-Chennai5h ago
-
AI Engineer (Fresher) – Mobile Radio Tester INR 180K-180KData Preprocessing | Deep learning | Machine Learning | Model Evaluation | PyTorchEntry-level Full TimeNew Delhi6h ago
-
Senior Software Engineer INR 2500K-2829KAWS Glue | AWS Lambda | AWS Step Functions | Amazon EC2 | Amazon EKSFamily well-being benefits | Health benefits | Hybrid work | Paid time offSenior-level Full TimeHyderabad, India6h ago
-
Data Engineer, Play Data Science and Analytics INR 1500K-2000KAI | Automation | C++ | Data Analysis | Data ModelingMid-level Full TimeBengaluru, Karnataka, India7h ago
-
Mid-level Full TimeGurugram, Haryana, India8h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India8h ago
-
Senior Data Engineer-Digital Banking Kotak 811-Regional Sales INR 2000K-2300KAgile | Apache Hadoop | Apache Hive | Apache Spark | Batch ProcessingSenior-level Full TimeBangalore, Karnataka, India8h ago
-
Senior Data Engineer INR 1000K-2000KAmazon DynamoDB | Apache Airflow | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeTelangana, Hyderabad, India8h ago
-
ML Engineer INR 1400K-2000K.NET | API Development | Apache Spark | Azure Cognitive | Azure Cognitive ServicesMid-level Full TimeHyderabad, TG, India8h ago
-
Software Engineer II - Java Fullstack + React + Python(AI/ML) INR 1000K-2200KAWS | AWS CodeDeploy | Agile | CI/CD | CucumberMid-level Full TimeHyderabad, Telangana, India9h ago
-
Principal Engineer, Data Analytics Engineering (Data Engineering , GenAI , Python , SQL) INR 2500K-5000KAWS | AWS Glue | AWS Lambda | Airflow | Amazon BedrockSenior-level Full TimeBengaluru, KA, India9h ago
-
Analytics & BI Consultant – Metabase INR 800K-1200KAgile | Business Intelligence | Common table expressions | Dashboards | Data ReconciliationSenior-level Full TimeGurugram, Haryana, India9h ago
-
Senior-level Full Timebangalore, India10h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India10h ago
-
Vice President - Data Scientist INR 1500K-6000KAWS | Agile | Azure | Data Pipelines | Data VisualizationExecutive-level Full TimeTelangana, India10h ago
-
Data Quality Engineer-Azure Data Platform Testing INR 1200K-1500KAgile | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimePune, Maharashtra, India11h ago
-
Mid-level Full TimeHyderabad, IN11h ago
-
Data Engineer INR 1500K-2000KADLS | AWS | Airflow | Azure | DBTBonuses | Broadband reimbursement | Medical insurance | Paid certifications | Professional development allowanceMid-level Full TimeIndia - Remote R12h ago
-
Agentic AI - Lead Engineer INR 2500K-3500KAPIs | Agentic AI | CI/CD | Data Engineering | Data VisualizationSenior-level Full TimeChennai, Tamil Nadu, India12h ago
-
AI Engineer INR 950K-1050KAWS | Airflow | Algorithms | Apache Spark | Data StructuresGlobal team collaboration | Hybrid workMid-level Full TimeBengaluru, INDIA, India12h ago