Data Engineer - Iceberg Migration
Tasks
- Collaborate with stakeholders to validate results
- Convert and validate ETL jobs
- Execute end to end data migration
- Migrate Hive tables to Apache Iceberg
- Review AI generated migration outputs
Perks/Benefits
- N/A
Skills/Tech-stack
ACID upserts | AWS Glue | Amazon S3 | Apache Iceberg | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | CI/CD | Cloudera | Data Lake Storage | Dataframe API | Flink | Git | HDFS | Hadoop | Hive | HiveQL | Iceberg schema evolution | Iceberg time travel | Jenkins | Job optimization | Minio | Nessie | Oozie | PySpark | Scala | Schema evolution | Spark SQL | Spark job optimization | Sqoop | Structured Streaming | Time Travel | Trino | YARN
Education
N/A
Roles
Related jobs
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | AI Assisted Development | CI/CD | Docker | GitHub ActionsBudget for education | Budget for wellness | Local connection meetups | Remote workSenior-level Full TimeChennai, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | KubernetesLocal team connection | Remote workSenior-level Full TimePune City, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAutomated testing | CI/CD | Docker | GitHub Actions | JavaEducation budget | Local team meetups | Personal growth budget | Remote work | Secure local complianceSenior-level Full TimeAhmedabad, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | CI/CD | Cloud Native | Docker | GitHubEducation budget | Local team connection | Periodic meetups | Personal growth budget | Remote workSenior-level Full TimeJalalabad, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JiraEducation budget | Local meetups | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeKolkata, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JavaScriptEducation budget | Local compliance | Local connection | Periodic meet ups | Remote workSenior-level Full TimeNoida, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI Agents | CI/CD | Docker | GitHub Actions | JavaEducation budget | Local office connectivity | Periodic team meetups | Personal growth budget | Remote workSenior-level Full TimeJaipur, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | JiraEducation budget | Legal compliance support for India | Personal growth budget | Remote work | Team meetupsSenior-level Full TimeTrivandrum, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KAI-assisted coding | Assisted coding | CI/CD | Docker | GitHubEducation budget | Local meet-ups | Professional development budget | Remote work | Wellness budgetSenior-level Full TimeHyderabad, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub Actions | Java | KubernetesEducation budget | Local connection | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeKochi, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Cloud Native | Docker | GitHub Actions | JavaEducation budget | Local connection | Personal growth budget | Remote work | Wellness budgetSenior-level Full TimeIndore, India19h ago
-
Software Engineer (AI/DevOps) ID53179 INR 2500K-3200KCI/CD | Docker | GitHub | Java | JiraLocal connection meetups | Remote workSenior-level Full TimeCoimbatore South, India19h ago
-
Data Science AI Engineer, Senior Professional INR 2500K-4500KAPI Development | AWS | Agentic AI | Apache Spark | AzureFully remoteSenior-level Full TimeBangalore, KA, IN, 560100 R1d ago
-
Manufacturing Data Analytics Technical Expert INR 1628K-2146KApache Hive | Apache Spark | Data Historian | Data analytics | Data integrationSenior-level Full TimeBangalore, Karnataka, IN, 5601001d ago
-
Principal Engineer - Generative AI Infra Capabilities INR 2000K-4600KApigee | Arize | Avi | CI/CD | CUDASenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India1d ago
-
Mid-level Full TimeIndia1d ago
-
Lead Assistant Manager INR 500K-1500KAWS | Apache Airflow | Cloud platform | Data Governance | Data LakehouseSenior-level Full TimeIndia1d ago
-
DE&A - Core - Big Data Engineering - ETL INR 1500K-2000KApache Hive | Apache Spark | Batch Processing | BigQuery | Cloud StorageMid-level Full TimeBangalore, Karnataka, India1d ago
-
DE&A - Core - Big Data Engineering - ETL INR 1500K-2000KBatch Processing | BigQuery | Dataflow | Dataproc | ETLMid-level Full TimeBangalore, Karnataka, India1d ago
-
DE&A - Core - Big Data Engineering - ETL INR 1500K-2000KApache Hadoop | Apache Hive | Apache Spark | Batch Processing | Cloud DataflowMid-level Full TimeBangalore, Karnataka, India1d ago
-
DE&A - Core - Big Data Engineering - ETL INR 1500K-2000KApache Hive | Apache Spark | Batch Processing | BigQuery | Cloud StorageMid-level Full TimeBangalore, Karnataka, India1d ago
-
GCP and Databricks Assistant Manager Hiring INR 1500K-2000KApache Spark | BigQuery | Bitbucket | Cloud Composer | Cloud FunctionsMid-level Full TimeBangalore, Karnataka, India1d ago
-
DE&A - Core - Big Data Engineering - ETL INR 2040K-2400KApache Hive | Apache Spark | Batch Processing | BigQuery | Cloud ArchitectureSenior-level Full TimeBangalore, Karnataka, India2d ago
-
Senior-level Full TimeIndia2d ago
-
AI Engineer INR 2000K-2600KAPI-driven | API-driven Development | Agent Development | Agent Development Framework | Agent Development KitMid-level Full TimePune, Maharashtra, India2d ago