Senior Data Engineer - ETL Framework
India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Build and maintain canonical data models
- Build centralized standardized data capabilities
- Capture and document data lineage principles
- Create data exchange standards
- Define data archival and purging standards
- Define data exception handling processes
- Define data quality rules and governance standards
- Define pipeline scheduling and monitoring standards
- Develop and optimize data pipelines
- Develop secured data access matrix
- Drive data management and analytics strategy
- Drive performance engineering CI CD testing and operations
- Implement data reusability and reduce duplicity
- Manage data engineering architecture and design
- Perform performance tuning for Spark jobs
- Recommend scalable high performance data pipeline solutions
Perks/Benefits
Skills/Tech-stack
Apache Doris | Apache Hive | Apache Iceberg | Apache Spark | Apache Tez | CI/CD | Cloud Native | Cloudera Data Platform | Continuous Delivery | Continuous integration | Data Archival | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Security | Data Skew | Data Skew Handling | Data Vault | Data platform | Data profiling | Data purging | Distributed Computing | Executor Tuning | Hadoop | Join optimization | Kubernetes | Memory tuning | Metadata Management | NoSQL | OpenShift | Partitioning | Performance Tuning | Presto | PySpark | Python | SQL | Serialization Optimization | Shuffle Optimization | Skew Handling | Spark SQL | Testing | Workload Migration
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Language: en
Views: 3
Clicks: 1
Saves: 0
Related jobs
-
AI Engineer with RAG / MCP INR 1500K-3000KDocument processing | LLM | Microservices | Model Context Protocol | Prompt engineeringMid-level Full TimeIN-AP-Hyderabad6h ago
-
AWS Data Engineer- Gen AI INR 1500K-2500KAWS CDK | AWS Glue | AWS Step Functions | Amazon RDS | Amazon RedshiftSenior-level Full TimeIN-TN-Chennai6h ago
-
Junior GenAI Developer INR 400K-480KAgentic Workflows | Embeddings | Git | Hybrid search | LLM APIHealth insurance | Language courses | Mentorship | Professional development opportunities | Relocation programEntry-level Full TimeAll Cities, India R7h ago
-
Lead Software Engineer - Data Engineering INR 3500K-5000KAWS | Airflow | Authentication | Authorization | CI/CDSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Engineer - Digital & AI Solutions INR 302K-346KAWS | Agile | Azure | Big Data | Cloud platformCharity Programmes | Discounted gym memberships | Environmental initiatives | Flexible working | Hybrid workingEntry-level Full TimeHyderabad, Telangana, India12h ago
-
AI/ML Engineer_MS INR 2500K-5000KAI Foundry | AI Search | AWS Bedrock | Amazon QuickSight | App ServiceSenior-level Full Timetelengana, India13h ago
-
CLOUD AMAZE - DW Architect (Practice) INR 1500K-2500KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksSenior-level Full TimeIndia13h ago
-
Senior INR 3000K-5000KAI Foundry | Azure AI | Azure AI Foundry | Azure OpenAI | Azure OpenAI ServiceSenior-level Full TimeGurgaon, Haryana, India13h ago
-
AI Engineer II INR 3799K-6000KAgentic AI | Agile methodology | Algorithms | Automated testing | Distributed SystemsSenior-level Full TimeBengaluru, KA, India14h ago
-
Senior-level Full TimeBengaluru, KA, India14h ago
-
Data Analyst Technical Lead (Alteryx) INR 3000K-4000KAPI Integration | Alteryx Designer | Alteryx Server | Data Cleansing | Data ReconciliationUK shiftSenior-level Full TimeNoida, UP, India14h ago
-
Mid-level Full TimeBangalore, Karnataka, India14h ago
-
Software Engineer III - AI/ML, Java/Python, REACT INR 3500K-5000KAPI Contract | API contract management | AWS | Asynchronous processing | CI/CDSenior-level Full TimeMumbai, Maharashtra, India14h ago
-
GCP Data Engineer INR 1500K-2000KAgile | Apache Airflow | Ascend.io | CI/CD | Cloud ComposerFlexible work model | Wellbeing cultureMid-level Full TimeHyderabad, India14h ago
-
Mid-level Full TimeBangalore, Karnataka, India14h ago
-
Senior AI & Automation Engineer (6+ years Generative AI, Agentic AI, LLMs, Python, APIs, MCP) INR 3700K-5500KAgent systems | Agentic AI | Agile | CI/CD | DockerSenior-level Full TimeBengaluru, KA, India14h ago
-
AWS | Adversarial Machine Learning | Amazon SageMaker | Anonymization | AzureCutting-edge AI security work | Flexible working hours | Fully remote | Global cross-functional collaboration | Opportunity to shape AI security best practicesSenior-level Full TimeIndia R15h ago
-
A/B | A/B Testing | AWS | Azure | B testingFlexible working hours | Flextime policy | Hybrid work model | International collaboration | Technical growth opportunitiesSenior-level Full TimeIndia15h ago
-
Senior Software Engineer, Machine Learning INR 2500K-3500KA/B | A/B Testing | Amazon S3 | Apache Airflow | Apache SparkAccident insurance | Commuter benefits | Dental insurance | Disability insurance | Financial wellness supportSenior-level Full TimeBengaluru, India15h ago
-
Manager Software Engineer- ML + NLP+ ppl management INR 3200K-4800KAPIs | BERT | Data Mining | Data Science | Deep learningFlexible work arrangements | Hybrid work environmentSenior-level Full TimeGurgaon - Cyber Park, India19h ago
-
Principal Engineer, Storage Data Science and Analytics INR 3200K-4590KAIOps | Anomaly Detection | Apache Spark | C# | C++Career Development Programs | Flexible work arrangements | Health and wellbeing benefitsSenior-level Full TimeBengaluru, Karnātaka, India19h ago
-
Principal Engineer, Storage Data Science and Analytics INR 3000K-3900KAIOps | Agentic Workflows | Anomaly Detection | C# | C++Senior-level Full TimeBengaluru, Karnātaka, India19h ago
-
Senior-level Full TimeBengaluru Office, India19h ago
-
Senior-level Full TimeBangalore, India R19h ago
-
Databricks -Project Lead-Data Engg INR 1500K-2500KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Data FactoryClient communication support | Flexible to work US shiftsSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …19h ago