Big Data Engineer
Tasks
- Automate data pipelines and workflows using orchestration frameworks like Airflow
- Build data solutions including data lakes warehouses ETL pipelines APIs and analytics platforms
- Develop real-time data processing pipelines
- Develop scalable low-latency data pipelines using Kafka Flink Spark Streaming
- Ensure data security compliance and regulatory requirements
- Implement data governance metadata management and data quality standards
Perks/Benefits
Skills/Tech-stack
Airflow | Data Pipeline Development | Data pipeline | Database Management | Flink | HBase | Hadoop | Hive | Kafka | Linux | MapReduce | Oozie | Pipeline Development | PySpark | Python | Relational Database | Relational Database Management | SQL | Shell Scripting | Spark
Education
Roles
Related jobs
-
Senior-level Full TimeBangalore, Karnataka, India3h ago
-
Lead Data Scientist INR 2500K-3880KAWS | Algorithms | Cloud Computing | Data Analysis | Data VisualizationFlexible work model | Health insurance | Life insurance | Paid time off | Retirement benefitsSenior-level Full TimeHyderabad, India7h ago
-
IT Data Analytics Specialist (ETL Developer) INR 1200K-2000KAirflow | Azure | Azure Data | Azure Data Factory | CI/CDSenior-level Full TimePune, India7h ago
-
Associate Data Engineer INR 1500K-2500KDBT | Data Modeling | Data integration | Datawarehouse | ETL401k plan | Flexible paid time off | Learning and development | Medical benefits | Paid parental leaveMid-level Full TimePune, MH, India8h ago
-
Lead Data Scientist INR 2500K-3880KAI frameworks | AWS | Data Visualization | Generative AI | LangchainFlexible hybrid work model | Health insurance | Life insurance | Paid time off | Retirement benefitsSenior-level Full TimeHyderabad, India8h ago
-
Senior Data Engineer INR 2156K-2600KAWS | Apache Airflow | Apache Spark | Azure Synapse | BigQueryFlexible work hours | Health benefitsSenior-level Full TimeBengaluru, INDIA, India8h ago
-
AI Engineer INR 2500K-3500KAPIs | Data Pipelines | Deep learning | LLMs | Language ProcessingInclusive environment | Remote workMid-level Full TimeIndia8h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Platform Engineering Professional - Data Engineer INR 2520K-3380KAb Initio | Apache Spark | BigQuery | CI/CD | ConfluenceMid-level Full TimeOuter Ring Road, Bellandur, Bengaluru, India14h ago
-
Systems Integration Senior Analyst INR 2500K-4500KAI Services | API Design | Autogen | Azure AI | Azure AI ServicesSenior-level Full TimeNoida, UP, IN14h ago
-
Systems Integration Senior Analyst INR 2500K-4500KAI frameworks | API Design | Autogen | Azure AI | Azure OpenAISenior-level Full TimeNoida, UP, IN14h ago
-
Systems Integration Senior Analyst INR 2500K-4500KAPI Design | Autogen | Azure AI | Azure Search | ClaudeSenior-level Full TimeNoida, UP, IN14h ago
-
Data Engineer - Azure Databricks INR 2520K-3380KAzure Cloud | Azure Data | Azure Data Factory | Data Factory | DatabricksSenior-level Full TimeBangalore, KA, IN14h ago
-
Data Engineer - Azure Databricks INR 2520K-3380KAgile | Azure Cloud | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeBangalore, KA, IN14h ago
-
Data Engineer - Azure Databricks INR 2520K-3380KAzure Cloud | Azure Data | Azure Data Factory | Data Factory | DatabricksSenior-level Full TimeBangalore, KA, IN14h ago
-
Gen AI Engineer / Applied AI Engineer INR 2650K-3500KAI Agents | API Integration | Cloud Run | Data Pipelines | EmbeddingsChallenging data and algorithmic problems | Hands-on experience with Vertex AI and Gemini | Opportunities to build production systems | Work on AI automation impacting workflowsMid-level Full TimeGurugram, Haryana, India14h ago
-
Senior-level Full TimeMadurai, Tamil Nadu, India14h ago
-
Senior Data Engineer – Azure Databricks INR 1200K-2000KAzure Databricks | Azure Synapse | Data Modelling | Data Pipelines | Data WarehousesSenior-level ContractRemote , India R14h ago
-
Data Engineer INR 1500K-1500KBatch scripting | Data Modeling | Database Automation | Database Design | ETLMid-level Full TimePLOT NO-1, S.NO. 77, India14h ago
-
APIs | AWS | Azure | Data Modeling | Data WarehousingCoaching | Development opportunities | Flexible workingSenior-level Full TimeNoida, UP, IN, 20130114h ago
-
Senior-level Full TimeGera Commerzone SEZ, Pune, India14h ago
-
GCP DevOps Architect INR 2500K-4500KAPI Security | Apigee | Artifact Registry | Cloud Functions | Cloud RunCareer growth opportunities | Flexible work arrangementsSenior-level Full TimeBengaluru, India14h ago
-
Software Engineer - Full Stack INR 1340K-1680KAWS | Angular | Azure | Bootstrap | CSSCareer growth | Healthcare | Hybrid work | Online learning | Paid time offMid-level Full TimeIND-Pune-Equifax Analytics-PTEC, India14h ago
-
Software Engineer - Career INR 2829K-3200KAWS | Airflow | Apache Beam | Azure | BigQueryCareer development | Healthcare | Hybrid work | Online learning | Paid time offSenior-level Full TimeIND-Pune-Equifax Analytics-PTEC, India14h ago
-
LMTS | Lead Data Engineer INR 2876K-4500KAgile | Airflow | CI/CD | Cloud Platforms | Data EngineeringCareer growth opportunities | Flexible workSenior-level Full TimeIndia - Hyderabad14h ago