Senior Big Data Engineer
Tasks
- Architect scalable, low-latency data pipelines
- Build ETL/ELT pipelines
- Conduct technical design reviews
- Design data pipeline automation
- Design end-to-end data solutions
- Ensure security and compliance requirements
- Evaluate and recommend improvements to data architecture and processes
- Implement data anomaly detection and alerting
- Maintain data governance and data quality standards
- Orchestrate data workflows
- Translate business requirements into data architecture
Perks/Benefits
Skills/Tech-stack
API | Agile | Alation | Apache Airflow | Apache Flink | Apache Kafka | Apache Spark | Azure | Collibra | DBT | Data Governance | Data Quality | Databricks | DevOps | ELT | ETL | Git | Google Cloud | HBase | Hadoop | Hive | Java | MapReduce | Oozie | OpenMetadata | Python | RDBMS | Scala | Snowflake | Spark Streaming
Education
Roles
Related jobs
-
Sr. Specialist, Data Engineering INR 2771K-4000KAgile | Airflow | BigQuery | CI/CD | Cloud FunctionsRelocation assistance within countrySenior-level Full TimeMumbai, MH, IN2h ago
-
Senior Software Engineer - Ingestion INR 2040K-2800KBackup and Recovery | C++ | Data Capture | Database replication | Distributed SystemsSenior-level Full TimeBengaluru, India7h ago
-
.NET | Agile | Azure | Azure Databricks | C#Senior-level Full TimeKolkata DN 57, India10h ago
-
Mid-level Full TimeBangalore, Karnataka, IN India, 56008710h ago
-
Data Engineer INR 1500K-2000KAgile | Apache Airflow | Apache Kafka | Ascend.io | CI/CDFlexible work modelMid-level Full TimeHyderabad, India16h ago
-
Data Engineer II INR 1000K-1500KAWS | Apache Airflow | Data Governance | Data Quality | Data StructuresMid-level Full TimeBengaluru, India17h ago
-
AI Engineer INR 2500K-5000KAPI-driven | API-driven Development | Agent Development | Agent Development Framework | BigQuerySenior-level Full TimePune, Maharashtra, India17h ago
-
Senior Applied AI Engineer INR 3264K-4800KAPI Integration | GraphQL | HubSpot | JavaScript | LangchainFlexible work hours | Hybrid work model | Mental health counseling | Merit based refresh grants | Paid parental leaveSenior-level Full TimeBangalore, India17h ago
-
Mid-level Full TimeHyderabad, IND-TG, India18h ago
-
Associate Data Engineer INR 1500K-2500KAgile | Apache NiFi | Azure DevOps | C plus plus | Database systemsMid-level Full TimePune, India20h ago
-
Senior-level Full TimeBANGALORE, Karnataka, Inde20h ago
-
Advanced Engineer-AI/ML INR 3264K-4800KAPI Integration | ASR | AUTOSAR | AWS | Audio ProcessingEmployee discounts | Recognition rewards program | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …20h ago
-
Senior Software Engineer - Data License Analytics, Pune INR 3000K-4720KAWS S3 | Agent to Agent | Agent to Agent A2A | Apache Airflow | Apache IcebergSenior-level Full TimePune21h ago
-
Spark, Scala, Hadoop & Quantexa/Senior Consultant Specialist INR 2000K-2800KApache Airflow | Apache Spark | CI/CD | DevOps | ElasticsearchFlexible working hours | On-call support | Shift workSenior-level Full TimeBangalore, Karnataka, India R21h ago
-
Mid-level Full TimeIN-KA-Bangalore21h ago
-
AI Lead and Automation Tester - Document Intelligence INR 2500K-5000KAI Workflow Orchestration | AI workflow | Automation Testing | Clinical Trial Compliance | Clinical trialSenior-level Full TimeIN-TN-Chennai21h ago
-
Senior Consultant INR 2000K-4500KAPI Testing | Adversarial Testing | Amazon SageMaker | Azure Machine Learning | Bias detectionSenior-level Full TimeIN-AP-Hyderabad21h ago
-
Assistant Director - Insights And Analytics INR 3000K-5199KCode reuse | Data Governance | Data Quality | Data Wrangling | Machine LearningExecutive-level Full TimeBangalore, Karnataka, India R21h ago
-
Senior-level Full TimeBangalore - Karnataka - India22h ago
-
Senior-level Full TimeMumbai, India22h ago
-
Java Full Stack Engineer - Professional INR 1200K-3000KAngular | Apache Airflow | Apache Spark | Azure Kubernetes | Azure Kubernetes ServiceSenior-level Full TimeIndia22h ago
-
Senior-level Full Timebengaluru, India23h ago
-
Senior-level Full Timebengaluru, India23h ago
-
Lead Data Platform Engineer INR 2040K-3900KAPI Integration | AWS | AWS CloudWatch | AWS DataZone | AWS GlueEnhanced medical benefits | Family well-being benefits | Hybrid work | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India23h ago
-
Anomaly Detection | Autoencoders | Bayesian optimization | Classification | ClusteringOnsite workEntry-level Full TimeHyderabad, India23h ago