EY - GDS Consulting - AI and DATA - AWS Data Engineer - Senior
Kochi, KL, IN, 682313
JPY 8600K-9700K (estimate) Senior-level Full Time
Tasks
- Automate DevOps and CI CD deployments
- Collaborate with data analysts BI developers and data scientists
- Design and implement ETL/ELT workflows
- Design optimize and monitor Spark jobs and SQL models
- Develop data lakehouse pipelines in AWS and Databricks
- Implement and manage Unity Catalog
- Migrate on premises ETL workloads to AWS and Databricks
- Participate in data architecture reviews
- Translate stakeholder data goals into governed solutions
Perks/Benefits
Skills/Tech-stack
AWS | AWS Glue | AWS Lambda | AWS Step Functions | Airflow | Amazon CloudWatch | Amazon Redshift | Amazon S3 | Apache Spark | Azure DevOps | Bronze Silver Gold | CI/CD | CSV | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | GitHub | JSON | Jenkins | Medallion Architecture | Parquet | PySpark | Python | SQL | Step Functions | Unity Catalog | XML
Education
Roles
Related jobs
-
DevOps Engineer, DPaaS Group - Data Platform Department (DPD) JPY 6500K-9000KAWS | Ansible | ArgoCD | Azure | BashSenior-level Full TimeRakuten Crimson House, Japan16h ago
-
Solution Engineer - Tableau (通信 or Retail) JPY 8000K-15000KAgentforce | Business Intelligence | Data 360 | Data Lake | Data VisualizationSenior-level Full TimeJapan - Tokyo16h ago
-
Bash | Cloud infrastructure | Cloud infrastructure as code | Docker | GCPAsynchronous culture | Flexible management | Friendly work environment | Life changing product mission | Opportunity for high impactMid-level Full TimeTokyo, Japan3d ago
-
Lead Machine Learning Engineer JPY 7000K-15000KC++ | Control Theory | Data Processing | Data Processing Pipelines | Deep learningSenior-level Contract Full TimeTokyo4d ago
-
EXD Product Analytics Engineer JPY 3010K-4200KArtificial Intelligence | BI tools | Big Data | Data Science | Data VisualizationMid-level Full TimeAkashi, Hyogo, Japan4d ago
-
AWS | Alicloud | Ansible | Bash | CloudFormationMid-level Full TimeFutakotamagawa Rise Office, Japan4d ago
-
Data Engineer / Data Analyst - Ad & Media Solutions Section, Data Solutions Department, Marketing Solutions Business (A&M) JPY 5000K-8000KAI | BI Development | Big Data | Big data processing | Data AggregationMid-level Full TimeFutakotamagawa Rise Office, Japan4d ago
-
Associate Applied AI Engineer (Madrid/Spain) - Orbit Program JPY 9500K-11000KAI orchestration | Artificial Intelligence | Business value | Business value framing | Change ManagementCareer path | Counseling | Generous PTO | Gym subsidies | Hybrid working optionsMid-level Full TimeTokyo, Japan5d ago
-
Machine Learning Engineer, Recommendation Algorithm JPY 6950K-8550KA/B | A/B Testing | B testing | Data Processing | Feature EngineeringHealth insurance | Relocation supportMid-level Full TimeShibuya, Tokyo, Japan5d ago
-
シニアデータエンジニア ‐ データブリックス JPY 4800K-8400KAWS | Airflow | Apache Spark | Azure | CICDCompetitive salary | Hybrid work system | International collaboration | Knowledge sharing | Professional development resourcesMid-level Full TimeTokyo Prefecture, Japan6d ago
-
V.I.E. - Data Engineer (F/M/D) JPY 4700K-6500KAI | Chemoinformatics | Cloud Platforms | Data Analysis | Data ModelingCareer development opportunities | Diversity and inclusion | International work environmentMid-level Full TimeICT - Innovation Campus Tokyo - …6d ago
-
AI | Containerization | FastAPI | ML deployment | MicroservicesMid-level Full TimeRakuten Crimson House, Japan6d ago
-
AI Evaluation | Chain-of-Thought | Data Analysis | Few-Shot Learning | Few-shotSenior-level Freelance Part TimeJapan R7d ago
-
Airflow | Apache Spark | ArgoCD | BigQuery | Data GovernanceSenior-level Full TimeRakuten Crimson House, Japan7d ago
-
Cassandra | Java | JavaScript | Linux | MariaDBMid-level Full TimeRakuten Crimson House, Japan7d ago
-
Code review | Deployment | Full Stack | Full-Stack Development | Generative ModelsFlexible work schedule | Relocation assistanceMid-level Full TimeTokyo, Japan8d ago
-
Robotics Software Engineer - Navigation JPY 8600K-9700KBehavior Tree | C++ | CI/CD | Docker | LinuxCareer growth | Collaborative environment | Competitive salary | Visa supportSenior-level Full TimeKoto City, Tokyo, Japan10d ago
-
Software Engineer, Advertisement Optimization Group - Merchandising and Advertisement Department (MAD) JPY 8600K-11000KAlgorithms | Asynchronous processing | CI/CD | Cloud Architecture | Data StructuresSenior-level Full TimeRakuten Crimson House, Japan10d ago
-
[CS2501]コネクティドカー（オフボード）_ SDVデータプラットフォーム開発エンジニア/ SDV Data platform Development Engineer（一般層 総括職/担当職） JPY 4100K-7800KAWS | Azure | Cloud platform | Data Governance | Data ManagementEntry-level Full TimeNissan Technical Center - Atsugi (NTC), …10d ago
-
Senior Data Engineer - Databricks (m/f/*) JPY 8000K-9700KApache Spark | CI/CD | Cloud Platforms | Databricks | Delta LakeHybrid work options | International environment | Professional development | Team eventsSenior-level Full TimeTokyo Prefecture, Japan11d ago
-
シニアコンピュータービジョンエンジニア / Senior Computer Vision Engineer JPY 3600K-5268KAI | Agile | C# | C++ | Embedded LinuxSenior-level Full TimeTokyo, Sumida-ku, Japan (Hybrid) R11d ago
-
Bash | Cloud infrastructure | Data Processing | GCP | LinuxCompetitive salary | Flexible work environment | Impactful work | Inclusive cultureMid-level Full TimeYokohama, Japan13d ago
-
Bash | Data Processing | Docker | GCP | PythonCollaborative culture | Fast-growing environment | Impactful work | Opportunity to support people with learning differences | Work in AI and Audio sectorMid-level Full TimeFukuoka, Japan13d ago
-
Data Engineer, City Data Platform Development (Employment for People with Disabilities) JPY 3600K-6000KAWS | C# | C++ | Data Governance | Data SecurityMid-level Full TimeTokyo14d ago
-
EY - GDS Consulting - AI and DATA - Azure Data Engineer - Senior JPY 6500K-9700KAzure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeCoaching | Flexible work arrangements | SupportSenior-level Full TimeKochi, KL, IN, 68231314d ago