Senior Data Engineer - Big Data & Automation
Tasks
- Apply automated patching
- Automate data analytics jobs using CI CD
- Automate security and regulatory checks
- Build automated pipelines and workflows
- Collaborate with platform and product teams on platform adoption
- Containerize data services using Kubernetes OpenShift
- Design and build data platform
- Implement observability
- Implement security requirements for CDP
- Install configure and maintain Cloudera CDP
- Manage Linux systems and networking
- Onboard customers and applications using self service automation
- Optimize cluster performance
- Troubleshoot Spark Hive and YARN jobs
- Tune Spark jobs and Hive jobs
Perks/Benefits
Skills/Tech-stack
Agile | Ansible | Apache Hive | Apache Spark | ArgoCD | CI/CD | Cloudera CDP | Container Management | Docker | Git | GitHub Actions | Groovy | Hadoop | Jenkins | Kubernetes | Kubernetes (OpenShift | Kubernetes OpenShift container management | Linux | Observability | OpenShift | Podman | Python | Shell | Terraform | YARN
Education
N/A
Roles
Related jobs
-
Expert Data Engineer (Databricks) INR 1500K-2400KAPI Design | API Key | AWS Glue | AWS IAM | AWS LambdaSenior-level Full TimeMaharashtra, Pune, India1h ago
-
Senior Data Engineer (Databricks) INR 2000K-2800KAPI Keys | AWS Glue | AWS IAM | AWS Lambda | AWS S3Senior-level Full TimeMaharashtra, Pune, India1h ago
-
AWS | AWS Glue | AWS Lambda | AWS Step Functions | AgileHybrid work modelSenior-level ContractShaikpet, India3h ago
-
Associate Engineer - Embedded AI & Data Analytics INR 1000K-1141KC plus plus | Classification | Data Modeling | Data Observability | Data PipelinesCollaborative work environment | On-site work schedule | Professional development opportunities | Training opportunitiesMid-level Full TimeBangalore - Karnataka, India - Kalyani …3h ago
-
Advanced Engineer - Embedded C++ Linux INR 1236K-1854KASPICE | CI/CD | Cloud Computing | Communications | Configuration ManagementEmployee discounts | Employee recognition rewards | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …3h ago
-
AI/ML Expert INR 2500K-4500KASR | Agent systems | Conversational AI | Data Analysis | Edge ComputingSenior-level Full Timepune, India6h ago
-
Senior Data Engineer - ETL Framework INR 1500K-2500KApache Doris | Apache Hive | Apache Iceberg | Apache Spark | Apache TezFlexible work policy | Work from homeSenior-level Full TimeIndia6h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India7h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India8h ago
-
AI Engineer - SAP S/4HANA INR 2000K-4500KA2A protocol | AI Core | AI-Hub | Agent systems | Agent to AgentSenior-level Full TimeBangalore, Karnataka, IN8h ago
-
Lead Software Engineer - Java/Python, AWS, Spark INR 1500K-3000KAWS | AWS CloudFormation | AWS Step Functions | Airflow | Amazon S3Senior-level Full TimePune, Maharashtra, India8h ago
-
Consultant INR 1500K-2000KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageMid-level Full TimeBangalore, Karnataka, India8h ago
-
Executive-KDNI - Data & Analytics INR 1500K-5199KAWS | Azure | Dashboarding | Data Governance | Data ModelingExecutive-level Full TimeGurgaon, Haryana, India9h ago
-
Analytics Engineer - SQL & Data Visualization INR 1500K-2500KAgile | Cloud Data | Cloud Data Warehouse | Data Interpretation | Data ModelingMid-level Full TimeBengaluru, KA, India9h ago
-
AES - DE - Generative AI Application Developers INR 2500K-4200KAWS | Agent Loops | Anthropic | Asynchronous programming | AutogenSenior-level Full TimePune, Maharashtra, India9h ago
-
Manager Software Engineer - RLE INR 1800K-2500KAmazon Web Services | Applied AI | Backend architecture | CI/CD | Cloud platformEquity grant | Food Credit | Housing bonus | Medical insurance | Performance bonusMid-level Full TimeIndia10h ago
-
Ai / Data Engineer INR 2500K-3500KAgentic Workflows | Data Ingestion | Deduplication | Elasticsearch | Embedding ModelsMid-level Full TimeIndia10h ago
-
Senior-level Full TimePune10h ago
-
Data Visualization | Financial Modeling | MATLAB | Portfolio construction | PythonMid-level Full TimeBengaluru, Karnataka, India10h ago
-
AWS | Apache Spark | Databricks | GenAI | Hugging FaceFlexible work arrangements | Professional growth opportunitiesSenior-level Full TimeIndia10h ago
-
Anthropic API | Async Programming | Audit Logging | Evaluation | Few-Shot LearningCareer growth | Remote-friendlySenior-level Full TimeIndia10h ago
-
Applied AI ML Senior Associate INR 2500K-4500KAPIs | AWS Bedrock | Azure OpenAI | CI/CD | DatabricksSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India11h ago
-
TC-CS-IAM-AI and Agentic AI Engineer- Senior INR 2000K-4800KAWS | Agile | Autogen | Azure | ClassificationSenior-level Full TimeBengaluru, KA, IN, 56004815h ago
-
Data Architect INR 2600K-3380KAgile | Architecture governance | Architecture standards | Behavior-Driven Development | CI/CDDiversity and inclusion | Remote WorkingSenior-level Full Timehyderabad, India15h ago