Senior Data Engineer - R01563934
INR 1000K-2000K (estimate) Senior-level Full Time
Tasks
- Build Databricks workflows for orchestration
- Build curated datasets for analytics
- Build logging monitoring and alerting
- Design Medallion Architecture
- Design scalable ETL/ELT pipelines
- Develop batch and streaming data pipelines
- Enable ACID transactions
- Enforce schema evolution and CDC
- Implement CI/CD for data pipelines
- Implement Delta Lake performance optimization
- Implement Delta Live Tables
- Implement data quality validation
- Ingest data from databases APIs and files
- Troubleshoot pipeline failures and optimize performance
- Use Unity Catalog for governance and lineage
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | Airflow | Apache Spark | Avro | Azure DevOps | Broadcast joins | CI/CD | Caching | Change Data Capture | Data Capture | Data Governance | Data Modeling | Databricks | Delta | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | Event Hubs | Git | GitHub Actions | JSON | Kafka | MLOps | MLflow | Parquet | Partitioning | PySpark | Python | SQL | Schema evolution | Snowflake Schema | Spark SQL | Spark Streaming | Star Schema | Unity Catalog | Z-Order
Education
Roles
Related jobs
-
Mid-level Full TimeHyderabad, India8h ago
-
Data Scientist (GenAI/Core DS) INR 2000K-3000KAWS | Data Analysis | Data Modeling | Data Visualization | LangchainSenior-level Full TimeGurugram, Haryana, India9h ago
-
Senior-level Full TimeIN-AP-Hyderabad9h ago
-
Senior-level Full TimeIN-AP-Hyderabad9h ago
-
Tech Lead INR 1500K-2400KAmazon Web Services | Data Governance | Data Modeling | Data Quality | DataOpsSenior-level Full TimeIN-TN-Chennai9h ago
-
Senior-level Full TimeIN-AP-Hyderabad9h ago
-
APIs | AWS Kinesis | ClickHouse | DBT | Data GovernanceCareer growth | Flexible collaboration | Inclusive culture | Innovation focus | Remote-friendly work setupSenior-level Full TimeIndia9h ago
-
Senior-level Full Timebangalore, India9h ago
-
Full Stack Engineer – MERN Stack INR 1300K-2500KAPIs | AWS | Azure | Bootstrap | CI/CDCollaborative culture | Growth opportunities | MentorshipMid-level Full TimeMaharashtra, Pune, India9h ago
-
Python AI Engineer INR 1800K-3500KAPI Development | AWS | AWS Lambda | Amazon Bedrock | Amazon SageMakerHybrid workSenior-level Full TimeIndia10h ago
-
Gen AI Engineer INR 2500K-4500KAWS Extract | Document processing | Langchain | Language Models | Large Language ModelsSenior-level Full TimeHyderabad, Telangana, India10h ago
-
Apache Kafka | Codebeamer | Data Validation | Event Driven | Event-driven architectureSenior-level Full Timebangalore, India10h ago
-
Mid-level Full Timebangalore, India10h ago
-
Senior Data Engineer (Python, PySpark, Kafka) INR 1500K-2040KApache Kafka | Data Warehousing | Dimensional Modeling | Flink | GitSenior-level Full Timebangalore, India10h ago
-
Software Engineer (SDE 2) - Full Stack (Frontend), Lakeflow INR 1800K-3000KAPI Integration | AWS | Accessibility | Azure | CSSMid-level Full TimeBengaluru, India11h ago
-
.NET | API Integration | Agents | Automation Anywhere | Azure FunctionsEntry-level Full TimeNashik, MH, India11h ago
-
API performance | API reliability | API scalability | AWS Glue | AWS LambdaAI training | Community forums | Laptop provided | Medical insurance | MentorshipEntry-level Full TimeIndia R11h ago
-
Software Engineer III - Python, AI ML INR 2000K-3500KAgentic Frameworks | Amazon Web Services | Apache Airflow | Azure | Cloud NativeSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
TRQ27-0135-01-OR Consultant - Data & AI Practice INR 400K-600KCBC | CPLEX | Cloud Computing | Constraint Programming | Error HandlingEntry-level Full TimeIndia12h ago
-
Sr. AI Engineer, Backend (Altium) INR 2000K-4500KAPI Development | Agentic Systems | Algorithms | CI/CD | Data StructuresEmployee resource groups | Remote work optionSenior-level Full TimeBangalore, India12h ago
-
Software Engineering INR 1400K-1820KAzure Cosmos | Azure Cosmos DB | Azure Data | Azure Data Factory | Azure EventMid-level Full TimeHyderabad, TS, IN13h ago
-
AWS | AWS CloudWatch | Alerting | Backups | BashSenior-level Full TimeBengaluru, Karnataka, India14h ago
-
Applied Computer Vision Engineer - Data Driven Development INR 2000K-3630KAWS | Apache Spark | Azure | C++ | Convolutional Neural NetworksSenior-level Full Timebengaluru, India14h ago
-
Senior-level Full TimePune, Maharashtra, India14h ago
-
AWS | Amazon Bedrock | Anthropic | Azure | Azure AIFlexible work arrangements | Health insurance | Learning and development | Paid time off | Retirement benefitsSenior-level Full TimeIndia14h ago