Senior Data Engineer
111443-IND-HYDERABAD-INTL HYD WF CENTRE BLK B8 Twr-4, India
INR 2000K-2250K (estimate) Senior-level Full Time
Tasks
- Build scalable migration pipelines with Python and SQL
- Design data migration strategies to GCP
- Document migration architectures processes and runbooks
- Execute data reconciliation and reconciliation reporting
- Handle large volume data transfers with CDC incremental loads and parallel loading
- Implement batch and streaming migrations with Airflow Kafka and Pub Sub
- Manage metadata lineage and governance with Dataplex and Data Catalog
- Migrate data from on premise RDBMS and Hadoop ecosystems
- Migrate structured and semi structured data to BigQuery
- Optimize migration workloads for performance scalability and cost on GCP
- Perform schema mapping data transformation and validation
- Process large scale data migrations with Apache Spark on Dataproc
- Troubleshoot migration issues and support post migration stabilization
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Apigee | BigQuery | Bigtable | CDC | CI/CD | Cloud Composer | Cloud platform | Data Catalog | Dataplex | Dataproc | Google Cloud | Google Cloud Platform | Google Pub/Sub | Hive | Kafka | MongoDB | MySQL | Pub/Sub | Python | REST APIs | SQL
Education
N/A
Roles
Related jobs
-
Systems Integration Senior Specialist INR 2800K-4000KAirflow | Batch Processing | Change Data Capture | Clustering | DBTFlexible shift schedule | Hybrid work | Onsite work as neededSenior-level Full TimeBangalore, KA, IN4h ago
-
Staff Engineer, .Net Core (Optimizely CMS 12, and React) INR 1500K-2400K.NET | .Net Framework | ASP.Net Core | Agile | AuthenticationSenior-level Full TimeRemote, India R5h ago
-
AI/ML Engineer INR 2500K-4500KAI Foundry | AI Search | AWS Bedrock | Agent Engine | Amazon QuickSightSenior-level Full Timetelengana, India5h ago
-
Azure Data & Fabric Developer / ETL pipeline & SQL ( 5 + yrs Assistant Manager - RNA INR 1500K-2200KAzure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeMumbai, MH, India6h ago
-
Data Engineer (Python & SQL) INR 2000K-2400KApache Airflow | Apache Beam | Apache Spark | Batch Processing | BigQueryCareer growth | Healthcare package | Hybrid work | Online learning platform | Paid time offMid-level Full TimeIND-Pune-Equifax Analytics-PEC, India11h ago
-
Mid-level Full TimeIT Centre Bengaluru GDTC, India11h ago
-
AI Software Engineer INR 1340K-1680KAgile | Automation Testing | C# | C++ | Distributed ComputingHealth and wellbeing benefits | Professional development programsSenior-level Full TimeBengaluru, Karnātaka, India11h ago
-
Advanced AI Solution Architect INR 2500K-4500KAI Governance | API Design | AWS | Agentic AI | AzureHybrid work | Relocation supportSenior-level Full TimeIT Centre Bengaluru GDTC, India11h ago
-
Senior Software Engineer/ Gen AI & Python INR 2250K-2829KAI content safety | AI gateway | API Management | Azure AI | Azure AI Content SafetySenior-level Full Time111443-IND-HYDERABAD-INTL HYD WF CENTRE BLK B8 …11h ago
-
Head of World Models (Universal Robots, India) INR 3000K-6000KAI orchestration | Actor-critic | Agent Frameworks | Autogen | DPOExecutive-level Full TimeBangalore, IN11h ago
-
Head of Simulation (Universal Robots, India) INR 3000K-6000KAI orchestration | Actor-Critic methods | Actor-critic | Agent Frameworks | AutogenExecutive-level Full TimeBangalore, IN11h ago
-
AWS | Anomaly Detection | Ansible | AppDynamics | Azure16x5 schedule coverage | On-call rotationSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India11h ago
-
Data Visualization Senior Consultant INR 1431K-2000KAWS Glue | Amazon S3 | Amazon Web Services | Data Ingestion | Data LakesSenior-level Full Timepune, MH, IN11h ago
-
Data Visualization Senior Consultant INR 1431K-2000KAWS Glue | Amazon S3 | Amazon Web Services | Data Lakes | Data ModelingSenior-level Full TimePune, MH, IN11h ago
-
Software Engineer INR 1068K-1910KAgile | Angular | Azure | CI/CD | DatabricksCareer development opportunities | Employee Assistance Program (EAP) | Financial wellness tools | Flexible working environment | LinkedIn LearningMid-level Full TimePune, MH, India15h ago
-
Data/AI Engineer INR 3000K-4000KAKS | Aiohttp | Asyncio | Azure Databricks | Azure DevOpsHybrid work environment | Medical coverageSenior-level Full TimePune, Maharashtra, India15h ago
-
Software Engineer II, Data Engineer INR 2520K-3380KAWS | Algorithms | Apache Airflow | Apache Spark | AzureSenior-level Full TimeBangalore, India17h ago
-
Software Engineer II, Data Engineer INR 2520K-3380KAWS | Apache Airflow | Apache Spark | Azure | Cloud platformSenior-level Full TimeBangalore, India17h ago
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R20h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R20h ago
-
AI Solutions Delivery Engineer INR 1340K-1680KAgent Orchestration | Agent SDK | Agent systems | Anthropic Claude | JSONFlexible work schedule | Hybrid work | Remote workMid-level Full TimeIndia - Pune R20h ago
-
Data Engineer INR 1500K-2000KAWS Lambda | Amazon Redshift | Amazon S3 | Azure Data | Azure Data FactoryMid-level Full TimeAhmedabad, 380015, India21h ago
-
ML / LLM Engineer (Remote) INR 2500K-3000KAmazon Web Services | Azure | Classification | Feature Engineering | Language ModelsRemote workMid-level Full TimeMaharashtra, Pune, India R21h ago
-
Senior-level Full Timehosur road bangalore, India21h ago
-
QA Engineer INR 240K-360KDBT | Data Quality | Data Warehousing | Dimensional Modeling | ETLOff hour support on rotation basisEntry-level Full TimePune City, India21h ago