Big Data Engineer
Kochi, Kerala, India
JPY 8600K-10000K (estimate) Senior-level Full Time
Tasks
- Automate data pipeline operations
- Build data models and KPI frameworks
- Create monitoring and observability tools
- Design data ingestion pipelines
- Develop data migration solutions
- Develop data transformation pipelines
- Document data lineage and architecture
- Implement real-time and batch processing solutions
- Optimize Spark workloads
- Support data platform APIs and integrations
Perks/Benefits
- N/A
Skills/Tech-stack
.NET | AWS | AWS Lambda | Amazon EC2 | Apache Airflow | Apache Spark | C# | CI/CD | DataOps | Databricks | Datadog | Delta Lake | Docker | GitHub Actions | Go | Grafana | JavaScript | Jenkins | Kafka | Kubernetes | Linux | MLOps | MLflow | NATS | PostgreSQL | PySpark | Python | SQL | Spotfire | Structured Streaming | Tableau | TeamCity | Terraform | TimescaleDB | Windows
Education
Roles
Related jobs
-
AWS Glue | AWS Lambda | AWS Step Functions | Amazon EC2 | Amazon EMRSenior-level Full TimeNoida, India2h ago
-
Associate Engineer - Data engineer INR 1000K-1700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksCollaborative work environment | Professional development | Training opportunitiesMid-level Full TimeBangalore - Karnataka, India - Kalyani …2h ago
-
Senior-level Full TimeGurugram, India4h ago
-
Intern Masters Embedded Eng INR 300K-300KC# | C++ | Development methodologies | Embedded Systems | Firmware DevelopmentEntry-level Full Time InternshipBengaluru, Karnataka, India4h ago
-
Senior Software Engineer, AI/ML INR 2000K-5000K3D mathematics | Agent systems | Autonomous Agents | C plus plus | Computational GeometrySenior-level Full TimeBengaluru, Karnataka, India4h ago
-
APIs | Agent systems | Cloud platform | Cost Optimization | CrewAIMid-level Full TimeTokyo, Japan4h ago
-
Applied AI/ML Senior Associate INR 2817K-4680KApache Hive | Apache Spark | Keras | Language Processing | Machine LearningSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Software Engineer, Data Science INR 1000K-2000KD3 | Data Governance | Data Modeling | Data Pipelines | ETLHealth and wellness programs | Time awayMid-level Full TimeBengaluru, KA, India R6h ago
-
Entry-level Full TimePune, Maharashtra, India7h ago
-
Data Engineer 3 INR 1500K-2500KApache Hive | Apache Kafka | Apache Spark | Continuous integration | Data CatalogMid-level Full TimePune, Maharashtra, India7h ago
-
Senior Consultant - Data Engineering-Data Analysis INR 2400K-3500KAWS Redshift | Confluence | ETL | Git | NumPySenior-level Full TimeNoida, Uttar Pradesh, India7h ago
-
Lead Software Engineer - Java AWS Kafka INR 2500K-3500KAWS | Agile | Apache Kafka | CI/CD | CachingSenior-level Full TimeBengaluru, Karnataka, India7h ago
-
D&T Lead - Data Engineer INR 1500K-2500KBatch Processing | CI/CD | Cloud Data | Cloud data warehousing | ConfluenceSenior-level Full TimePune, IN7h ago
-
Software Engineer INR 1500K-2200KAgile | Cloud services | Code Reviews | Data Modeling | Data StorageMid-level Full TimePune, India8h ago
-
Mid-level Full TimePune, India8h ago
-
Associate Software Engineer INR 1300K-2000KAgile | Design Patterns | JavaScript | Python | REST APIMid-level Full TimePune, India8h ago
-
Tech Lead Data Engineer INR 1500K-2600KAWS Data | AWS Glue | AWS data warehousing | Amazon Athena | Amazon EKSGreat Place to Work recognitionSenior-level Full TimePune, India8h ago
-
Senior-level Full TimePune, India8h ago
-
Data Engineer INR 1500K-2500KAWS Glue | Agile methodology | Amazon Athena | Amazon EMR | Amazon KinesisMid-level Full TimePune, India8h ago
-
AI Engineer - FDE (Forward Deployed Engineer) INR 1860K-3900KAWS | Agent systems | Apache Spark | Apache Spark SQL | AzureSenior-level Full TimeRemote - India R8h ago
-
Senior Staff Engineer, Supply chain optimization INR 3000K-4500KAMPL | AWS | Agent-Based Simulation | Agent-based | AnyLogicSenior-level Full TimeRemote, India R8h ago
-
Embedded C++ & PLC Automation Engineer-EPS1 INR 1716K-2700KC++ | C++17 | CAN | Embedded Systems | EtherCATSenior-level Full Timebangalore, India9h ago
-
Embedded C++ & PLC Automation Engineer-EPS1 INR 1716K-2700KBuild systems | C++ | C++17 | CAN | Case DevelopmentSenior-level Full Timebangalore, India9h ago
-
Senior AI-ML Platform Engineer / Tech Lead INR 2500K-5000KAI Foundry | API Gateway | API Keys | AWS Bedrock | AWS CloudFormationInclusion and Belonging CultureSenior-level Full TimeBangalore, Karnataka, IN, 5601009h ago
-
AI Agents | Accuracy Metrics | Benchmarking | Data Delivery | Data PipelinesMid-level Full TimeMumbai, Maharashtra, India9h ago