Data Engineer
Tasks
- Apply medallion architecture patterns
- Build and manage orchestration workflows
- Build ingestion pipelines
- Catalog data assets
- Document pipelines and data models
- Implement CDC
- Implement RBAC
- Implement SCD Type 1
- Implement SCD Type 2
- Implement batch pipelines
- Implement data governance and compliance
- Implement encryption
- Implement schema evolution
- Implement streaming pipelines
- Manage Unity Catalog metadata
- Manage partitions
- Optimize Delta Lake tables
- Perform performance tuning
- Recover failed pipelines
- Support CI/CD pipelines
- Track data lineage
- Troubleshoot pipeline failures
- Validate data quality
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon Web Services | Apache Spark | Azure | Azure Data | Azure Data Factory | CDC | CI/CD | Cloud platform | Data Factory | Data Modeling | Data Quality | Data pipeline | Data pipeline orchestration | Data quality monitoring | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Encryption | Git | Google Cloud | Google Cloud Platform | Lakehouse Architecture | Medallion Architecture | Microsoft Purview | Pipeline Orchestration | PySpark | Quality monitoring | RBAC | SCD Type 1 | SCD Type-2 | SQL | Schema evolution | Type 1 | Type 2) | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Agile | Application performance | Azure | Azure AI | C++Career growth | Continuous learning | Fully remote | Inclusive global work cultureSenior-level Full TimeIndia R6h ago
-
Sr. Computer Scientist - I (Java, Big Data) INR 3150K-4500KApache Ignite | CI/CD | Capacity Planning | Datadog | Distributed SystemsSenior-level Full TimeBangalore, India R13h ago
-
Senior-level Full TimeNoida, India R13h ago
-
Senior-level Full TimeBangalore, India R1d ago
-
C# | MATLAB | NumPy | Pandas | PythonPart-time projects | Project based workSenior-level Full TimeIndia - Remote R2d ago
-
Data Engineering Lead INR 3000K-4000KAmazon Web Services | Artificial Intelligence | Data Governance | Data Pipelines | Data platformAnnual bonus | Health insurance | Knowledge sharing | Learning and development | PensionSenior-level Full TimeRemote, Bangalore R2d ago
-
Senior Data Engineer (Databricks) (Remote) INR 1500K-2040KAI | API Keys | AWS Glue | AWS IAM | AWS LambdaCollaboration | Ownership culture | Remote workSenior-level Full TimeMaharashtra, Pune, India R2d ago
-
AI / ML Engineer (Remote) INR 2500K-4500KAWS | Amazon SageMaker | CI/CD | Computer Vision | Data PreprocessingRemote workSenior-level Full TimeMaharashtra, Pune, India R3d ago
-
Internal Audit Data Analytics, Associate INR 420K-700KAgile | Azure DevOps | Data Warehousing | ETL | N-Gram AnalysisComprehensive healthcare | Flexible time off | Retirement plan | Support for working parents | Tuition reimbursementEntry-level Full TimeMU8-South (A) Wing, 7-10 Floor, Nesco … R3d ago
-
Analyst – Data Engineer INR 800K-1000KAWS | AWS Lambda | Amazon Redshift | Amazon S3 | Apache AirflowCareer development opportunities | Flexible work schedule | In-person collaborationMid-level Full TimeIndia - Bangalore R3d ago
-
Azure Data | Azure Data Factory | Azure Storage | C# | Data FactoryAccess to online learning platforms | Flexible work hours | Health insurance | Leave benefits | Professional development budgetMid-level Full TimeIndia - Remote R3d ago
-
Analyst – Data Engineer INR 800K-1000KAWS | AWS Lambda | Amazon Redshift | Amazon S3 | Apache AirflowFlexible work schedule | In office minimum three days per weekMid-level Full TimeIndia - Bangalore R3d ago
-
Senior-level Full TimeNoida, India R3d ago
-
Fullstack Data Engineer (Remote) INR 800K-2000KAPI Keys | AWS IAM | AWS S3 | AWS VPC | Apache SparkRemote work | UK shift scheduleSenior-level Full TimeMaharashtra, Pune, India R3d ago
-
Apache Airflow | BigQuery | Cloud Functions | Cloud Storage | Cloud platformCommunity forums | Laptop provided | Medical insurance | Mentorship | No weekend workSenior-level Full TimeAhmedabad R4d ago
-
Senior-level Full TimeRemote - India R4d ago
-
Senior-level Full TimeIndia - Remote R4d ago
-
AI Engineer (Full Stack Development) INR 1800K-3000KAPI Integration | Amazon Web Services | Authentication | Cloud Functions | Cloud platformMid-level Full TimeMumbai, Maharashtra, India - Remote R4d ago
-
Azure Bicep | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryCareer development | Inclusive culture | Peer recognition | Technical trainingSenior-level Full TimeIND-Hyderabad-Hybrid, India R4d ago
-
Assct Spclst, Data Analytics INR 2520K-3500KAnomaly Detection | Azure SQL | Azure Synapse | Azure Synapse Analytics | CTEsSenior-level Full TimeBangalore, India R4d ago
-
Sr Data Engineer I Analytics INR 2400K-3380KAgile | Data Modeling | Dimensional Modeling | ETL | Load ScriptSenior-level Full TimeBangalore, Karnataka (Hybrid) R5d ago
-
Software Engineer I (.Net) INR 1500K-2500K.NET | API Automation | API Gateway | AWS | Agentic AICareer development | Community building | Learning opportunities | Work from Anywhere cultureEntry-level Full TimeIndia R5d ago
-
Senior Devops Engineer INR 2500K-4000KALB | AWS Config | AWS GuardDuty | AWS Security | AWS Security HubSenior-level Full TimeIndia - Remote R5d ago
-
Senior Engineer, Customer Data Operations INR 2520K-3880KAWS | AWS Lambda | Airflow | Amazon EC2 | Amazon S3Access to Headspace | Hybrid work model | Mental health days | Retirement savings | Tuition reimbursementSenior-level Full TimeIndia, Bengaluru, Karnataka R5d ago
-
Senior Software Engineer (Data Platform) INR 3000K-3880KAmazon Kinesis | Auto Scaling | Cursor | DBU optimization | Data ContractsDental insurance | Flexible time off | Medical insurance | Paid parental leave | Sabbatical after 5 yearsSenior-level Full TimeIndia (Remote-First) R6d ago