DE&A - Core - Advanced Data Engineering - Ab Initio
Tasks
- Build transformation logic
- Create ETL graphs
- Debug ETL graphs
- Define data structures
- Execute real time data processing
- Implement checkpointing and recovery
- Integrate messaging queues
- Manage data lineage
- Manage technical metadata
- Manage version control
- Process data in parallel
- Test ETL graphs
- Use rollup joins scan and reformat
Perks/Benefits
- N/A
Skills/Tech-stack
Ab Initio | Checkpointing | Data Lineage | Data Manipulation | Data Manipulation Language | Data Processing | Development Environment | ETL | Graphical Development Environment | Graphical development | JOIN | Messaging Queues | Metadata Management | Parallel processing | Real Time | Real-time Data | Real-time Data Processing | Recovery | Reformat | Rollup | SCAN | Version control
Education
N/A
Roles
Related jobs
-
Sr.DataOPS Developer INR 2000K-3000KAWS Greengrass | Airflow | Apache Flink | Apache Kafka | Apache Kafka StreamsSenior-level Full TimeBangalore, India2h ago
-
Net Core Azure Cosmos DB Gen AI INR 1800K-3300K.NET | API Management | ASP.Net Core | App Service | Application InsightsMid-level Full TimeIN-KA-Bangalore4h ago
-
Assistant Manager- Python Gen AI with GCP- Bangalore INR 1500K-2500KADK | AI Agents | API Development | Agile | Cloud platformWork from client office 5 days per weekMid-level Full TimeBangalore, Karnataka, India5h ago
-
Database Engineering Associate - Snowflake Developer INR 1500K-2000KAPI Integration | AWS | Access Management | Agile | Batch ProcessingFlexible hybrid work model | Health insurance | Life insurance | Paid time off | Pension/retirement benefitsMid-level Full TimeHyderabad, India R6h ago
-
Business Intelligence Engineer - Snowflake Developer INR 2500K-4132KAgile | Airflow | Bitbucket | CI/CD | Common table expressionsFlexible remote days | Health insurance | Hybrid work model | Life insurance | Paid time offSenior-level Full TimeHyderabad, India R7h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India7h ago
-
AM - Cloud Data Engineer INR 1500K-2500KAWS Lake Formation | Apache Airflow | Apache Kafka | Apache Spark | AzureSenior-level Full TimePune, Maharashtra, India9h ago
-
QE for AI Engineer INR 3000K-4000KAdversarial Testing | Benchmarking | CI/CD | Data Lineage | EvaluationSenior-level Full TimePune, Maharashtra, India9h ago
-
Data & Analytics Engineer - Consultant INR 1500K-3000KData Director | Data Governance | Data Management | Data Quality | ETLSenior-level Full TimeBangalore, Karnataka, India9h ago
-
API Gateway | AWS CloudWatch | AWS Cost Optimization | AWS Glue | AWS LambdaCollaborative environment | Continuing education | Growth opportunities | Professional training opportunities | Remote work flexibilitySenior-level Full TimeIndia12h ago
-
Custom Software Engineer INR 1300K-2000KAgile methodologies | Data Warehousing | Data integration | Databricks | Development LifecycleMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Custom Software Engineer INR 1300K-2000KAgile | Data integration | Databricks | ETL | Microsoft AzureMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Custom Software Engineer INR 1300K-2000KAgile methodologies | Analytics services | Azure Analytics | Azure Analytics Services | Azure DataMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Custom Software Engineer INR 1300K-2000KAgile | Analytics services | Azure Analytics | Azure Analytics Services | Azure DataMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Custom Software Engineer INR 2700K-3500KAnalytics | Automation | Cause analysis | Cloud adoption | Data EngineeringSenior-level Full TimeNavi Mumbai, MDC5C, India17h ago
-
Application Support Engineer INR 2500K-4000KData Processing | Data integration | Distributed Computing | ETL | PySparkSenior-level Full TimeBhubaneswar, BBDC1A, India17h ago
-
Custom Software Engineer INR 1300K-2000KAgile | Analytics services | Azure Analytics | Azure Analytics Services | Azure DataMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Mid-level Full TimeBengaluru, BDC7A, India17h ago
-
Custom Software Engineer INR 1300K-2000KAgile | Data Warehousing | Data integration | Databricks | ETLMid-level Full TimeBengaluru, BDC14A, India17h ago
-
Data Engineer INR 1000K-2000KCloud Data | Cloud data warehousing | Data Modeling | Data Warehousing | ETLMid-level Full TimeIndore, INDC1A, India17h ago
-
Data Engineer INR 1500K-2000KAWS | Airflow | Azure | BigQuery | CI/CDCollaboration with founder | Direct ownershipMid-level Full TimeNagpur, Maharashtra, India17h ago
-
Data Engineer III INR 3000K-4000KAWS Athena | AWS CDK | AWS Glue | AWS Kinesis | AWS Lake FormationAccident insurance | Employee assistance program | Flexible working arrangement | Health insurance | Life insuranceSenior-level Full TimeIndia-Chennai (Ascendas Tech park)17h ago
-
Data Platform Engineer INR 1400K-2000KBig Data | Big data processing | Data Modeling | Data Processing | Data integrationMid-level Full TimeIndore, INDC1A, India17h ago
-
Custom Software Engineer INR 1300K-2000KBuild tool | Configuration Management | Data Build | Data Build Tool | Data ModelingMid-level Full TimeBengaluru, BDC7A, India17h ago
-
Custom Software Engineer INR 3000K-4000KData Transformation | Data integration | Data orchestration | Data pipeline | DatabricksSenior-level Full TimeBengaluru, BDC7A, India17h ago