Data Engineer with Databricks, AWS
Tasks
- Analyze and validate data ingestion pipelines
- Assess data quality
- Collaborate on data mappings and business rules
- Develop SQL queries for reconciliation and testing
- Develop and maintain transformation pipelines
- Implement logging error handling monitoring
- Implement silver and gold data structures
- Participate in testing and production readiness
- Perform data profiling for quality issues
- Support data engineering initiatives using lakehouse architecture
- Troubleshoot pipeline performance and defects
- Validate and reconcile source to bronze and bronze to silver data
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | Amazon S3 | Apache Spark | CDC | CI/CD | Data Transformation | Data Validation | Data Warehouse | Data Warehouse Concepts | Data profiling | Databricks | Delta Lake | ELT | ETL | Git | PySpark | Python | SQL | Terraform | Warehouse concepts
Education
N/A
Roles
Related jobs
-
Design Verification Engineer, PhD, Early Career INR 2500K-6000KAI | AI accelerators | Accelerator architectures | Arithmetic Unit | Bus architectureNone Full TimeBengaluru, Karnataka, India3h ago
-
Software Engineer III - Tableau SRE & Automation INR 2500K-4000KChange Management | DNS | Dynatrace | Firewall | GrafanaSenior-level Full TimeHyderabad, Telangana, India5h ago
-
Senior-level Full TimeHyderabad, IN6h ago
-
Mid-level Full TimeHyderabad, TS, India6h ago
-
Lead / Manager Agentic AI Engineer - Claude Code & Codex INR 2000K-4500KAWS | Asynchronous programming | Autogen | Azure | CI/CDSenior-level Full TimeHyderabad, TS, India7h ago
-
Data Engineer INR 1500K-2200KAnalysis Services | Azure Analysis | Azure Analysis Services | Azure Blob | Azure Blob StorageMid-level Full TimeHyderabad, IN7h ago
-
Senior Data Engineer INR 1500K-2600KAnalysis Services | Azure Analysis | Azure Analysis Services | Azure Blob | Azure Blob StorageSenior-level Full TimeHyderabad, IN7h ago
-
Senior-level Full TimeGurgaon, HR, India8h ago
-
(Senior) AI Engineer - Data & AI Organisation (all genders) 1 INR 2500K-5000KAPI Development | Amazon Web Services | Apache Spark | CI/CD | DBTSenior-level Full TimeBangalore, Karnataka, IN, 5601008h ago
-
(Senior) Software Engineer (all genders) 1 INR 3600K-4720KAI Safety | AWS | Amazon Web Services | Apache Spark | Automated testingSenior-level Full TimeBangalore, Karnataka, IN, 5601008h ago
-
AI Engineer (India, China) INR 2500K-4064KAPIs | AWS | Airflow | Apache Spark | AutogenAnnual leave | Flexible working | Life insurance | Medical insurance | Parental leaveMid-level Full TimeBangalore, IN9h ago
-
Mid-level Full TimeIndia - Bengaluru10h ago
-
Adversarial Training | Arize | Audio Processing | Automatic Speech Recognition | Dataset SynthesisChai or coffee | ESOP | Flexible work environment | Free lunch | Workspace perksEntry-level Full TimeNew Delhi, India10h ago
-
AWS S3 | AWS storage | AWS storage services | Active Directory | Ansible100 percent remote work | Flexible time off | Holiday allowance | Medical insurance | Shift allowanceMid-level Full TimeIndia R11h ago
-
Senior-level Full TimeINDIA - NOIDA- BIRLASOFT OFFICE, IN12h ago
-
Databricks Platform Engineer (AWS) INR 1500K-3500KAWS | AWS Cloud | AWS Cloud Database | Amazon Virtual Private Cloud | Asset bundlesFlexible work arrangement | Hybrid working model | Wellness supportSenior-level Full TimeHyderabad, India R14h ago
-
API Testing | AWS | Azure | BLEU | Bias TestingOnsite workSenior-level Full TimeBengaluru, India14h ago
-
Senior-level Full TimeChennai, Chennai-MRV-AD, IN14h ago
-
Senior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …14h ago
-
Sr Technical Lead-BI INR 1500K-2700KDAX | Data Modeling | Data Warehousing | Dimensional Modeling | ETLSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …14h ago
-
Sr Technical Lead-BI INR 1500K-2700KBI Data modeling | DAX | Data Modeling | Data Warehousing | Dimensional ModelingSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …14h ago
-
AI / ML Engineer INR 2000K-5000KAI Services | Chatbots | Cloud AI | Cloud AI services | Deep learningSenior-level Full TimeChennai, CDC2A, India14h ago
-
Senior-level Full TimeBengaluru, BDC7C, India14h ago
-
Data Engineer INR 1500K-2500KCloud Computing | Data Modeling | Data Pipelines | Data Quality | Data WarehouseSenior-level Full TimeGurugram, DDC5E, India14h ago
-
Data Engineer INR 1500K-2500KAWS DynamoDB | AWS S3 | Amazon Redshift | Data Governance | Data Pipeline ArchitectureSenior-level Full TimeGurugram, DDC5E, India14h ago