Data Engineer
Tasks
- Apply medallion architecture transformations
- Apply security controls with RBAC encryption and data classification
- Build and manage data workflows and orchestration
- Build ingestion pipelines across AWS Azure GCP
- Document pipelines data models and operational procedures
- Implement CDC and SCD type 1 and 2
- Implement batch and streaming data pipelines
- Implement data governance quality validation and monitoring
- Implement metadata lineage and cataloging
- Manage Delta Lake tables partitions and performance
- Support CI CD pipelines troubleshooting and recovery
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Airflow | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Pipelines | CDC | CI/CD | Cloud Storage | Cloud platform | Data Factory | Data Lineage | Data Modeling | Data Quality | Data Validation | Data profiling | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Encryption | Git | Google Cloud | Google Cloud Platform | Lakehouse | Metadata Management | Microsoft Purview | Power BI | PySpark | RBAC | SCD Type 1 | SCD Type-2 | SQL | Schema evolution | Type 1 | Type 2) | Unity Catalog
Education
Roles
Related jobs
-
AWS | Airflow | DBT | Fine Tuning | Language ModelsBonuses | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, India R8h ago
-
Senior Database Engineer - #3190 (Remote) INR 1500K-2700KAWS IAM | AWS KMS | AWS RDS | Amazon Aurora | BashOn-call rotation | Remote workSenior-level Full TimeChennai, India R12h ago
-
Software Engineer SRE (Site Reliability Engineer) INR 1600K-3000KAWS | Ansible | Apache Airflow | Apache Kafka | Apache SparkSenior-level Full TimeBangalore, India Office (BANGALORE) R12h ago
-
GCP BigQuery,DevOps,Credit Card Domain/Associate Director, Data and Analytics Specialist INR 650K-900KApache Airflow | Apache Kafka | BigQuery | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, Telangana, India R14h ago
-
Senior Staff Engineer, Data Migration Lead INR 3000K-4000KAccess Control | Audit Trail | Cutover planning | Data Governance | Data ManagementSenior-level Full TimeRemote, India R16h ago
-
Python, GenAI, LLM/Consultant Specialist INR 3700K-5000KAccess Management | Active Directory | Active Directory Domain Services | Agent Frameworks | Amazon Web ServicesFlexible working | Professional developmentMid-level Full TimePune, Maharashtra, India R20h ago
-
Associate Engineer-Data Analyst(Hybrid) INR 700K-900KAerospace Engineering | Data Analysis | Data Visualization | Mechanical Engineering | Power BICar lease programme | Contingency leave | Employee scholar programme | Group health insurance | Group personal accident insuranceMid-level Full TimeIN-KA-BENGALURU-NORTHGATE ~ Sy No 2/2 Venkatala … R1d ago
-
Sr Data Engineer - ETL, Power BI & Java INR 2775K-4401KData Governance | Data Modeling | Data Quality | Dimensional Modeling | ELTSenior-level Full TimeIndia Virtual R1d ago
-
Python Spark Developer INR 1200K-2000KAgile methodology | Apache Spark | CI/CD | Data pipeline | ETLHybrid work modelMid-level Full TimeBangalore, India R1d ago
-
AWS | Alerting | Autogen | Data Ingestion | Data PreprocessingSenior-level Full TimeBangalore - Carina, India R1d ago
-
Mid-level Full TimeIN Virtual, India R1d ago
-
Senior Data Platform Support Engineer -GCP Certified INR 3000K-4000KArtifact Registry | Bash | BigQuery | Cloud Composer | Cloud MonitoringHybrid work | On-call rotaSenior-level Full TimePune, India R1d ago
-
Senior ML Engineer INR 4000K-5876KA/B | A/B Testing | AWS | Amazon SageMaker | AzureHybrid work model | Mentorship | Remote work option | Travel as neededSenior-level Full TimeAPAC - India - Bengaluru - … R1d ago
-
Senior-level Full TimeBangalore, India R2d ago
-
AI Engineer II (Remote) USD 39K-72KAWS | CI/CD | Docker | Embeddings | Entity recognitionRemote workMid-level Full TimeIndia, Remote R3d ago
-
Mid-level Full TimeIndia - Remote R3d ago
-
Principal Engineer, Python fullstack (React+Genai) INR 3000K-5000KAPI Integration | AWS | Agent Orchestration | Azure | ComplianceSenior-level Full TimeRemote, India R3d ago
-
Principal Engineer, Machine Learning INR 4000K-5000KArtificial Intelligence | Computer Vision | Deep learning | Docker | Foundation ModelsSenior-level Full TimeRemote, India R3d ago
-
Associate Principal Engineer, Machine Learning INR 1800K-3300KAdversarial Networks | Agent Orchestration | Agent systems | Computer Vision | Deep learningMid-level Full TimeRemote, India R3d ago
-
AI Application Engineer INR 3000K-4408KAWS Lambda | AWS S3 | Amazon Bedrock | CI/CD | Cloud SecurityKnowledge sharing culture | Learning environmentMid-level Full TimeHybrid (Bengaluru, Karnataka, IN); Hybrid (Pune, … R3d ago
-
GenAI LLM Python/Sr. Associate Director, Software Engineering INR 1500K-3000KAWS | Agent Orchestration | Agent systems | Azure | Cloud platformFlexible working | Inclusive and diverse environment | Professional development opportunitiesMid-level Full TimePune, Maharashtra, India R3d ago
-
Software Engineering Senior Analyst - HIH - Evernorth INR 2475K-3380KAWS | Access Control | Auditability | CI | CI/CDSenior-level Full TimeHIH - Hyderabad, India R4d ago
-
SDE III - Backend INR 3500K-5000KAPI Integration | AWS Lambda | Amazon EC2 | Amazon RDS | Amazon SQSCollaborative open minded culture | Learning and development programs | Multicultural experience | Professional growthSenior-level Full TimeIndia R4d ago
-
SDE III - Backend INR 3500K-5000KAWS | Amazon SQS | Docker | EC2 | EmbeddingsLearning and development programs | Professional growthSenior-level Full TimeIndia R4d ago
-
Senior Applied AI Engineer INR 3000K-4000KAPIs | AWS Bedrock | Agent systems | AgentOps | Agentic ArchitecturesOwnership of projects | Professional training and playbooks | Remote-first work environmentSenior-level Full TimeIndia-Remote R4d ago