Senior Data Engineer - Full Stack
Tasks
- Build event driven near real time data solutions
- Conduct technical design and code reviews mentor engineers
- Create APIs and integrations for governed data
- Create scalable data models and curated datasets
- Design and build data products on Databricks
- Develop batch incremental and streaming data pipelines
- Develop lightweight applications dashboards and interfaces
- Diagnose and resolve issues across data and application components
- Implement data quality security lineage and governance with Databricks and Unity Catalog
- Optimize pipelines streaming queries services and applications for performance and cost
- Partner with stakeholders to define requirements and delivery plans
- Set up automated testing CI CD monitoring alerting and documentation
Perks/Benefits
Skills/Tech-stack
AWS | Alerting | Apache Spark | Azure | CI/CD | Data Modeling | Data Warehousing | Databricks | Delta Lake | Distributed Processing | Event Driven | Event Hubs | Event-driven architecture | FastAPI | Flask | GCP | Git | Kafka | Kinesis | Monitoring | PySpark | Python | REST APIs | SQL | Schema evolution | Streaming Pipelines | Unity Catalog
Education
N/A
Related jobs
-
Senior-level Full TimeBengaluru, Karnataka, India R4h ago
-
Software Development Specialist INR 940K-940KAmazon CloudWatch | Amazon DynamoDB | Amazon S3 | Amazon Web Services | Azure DevOpsMid-level Full TimeBangalore, KA, IN R8h ago
-
Senior Manager INR 1500K-2800KAWS Glue | AWS Secrets | AWS Secrets Manager | Adaptive Query | Adaptive query executionHybrid work modeSenior-level Full TimePune, Maharashtra, India R9h ago
-
Java API & Microservices Engineer / Architect INR 2500K-3500KAPI Testing | Azure | CI/CD | Cassandra | DatadogSenior-level Full TimeBangalore, KA, IN R16h ago
-
Senior-level Full TimeBangalore, KA, IN R16h ago
-
Lead Data Engineer INR 3000K-4000KAPI | Agile | Amazon Web Services | Azure | Azure SynapseHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India R16h ago
-
Senior Data Platform Engineer INR 1500K-3000KARM Templates | AWS | Azure DevOps | Azure DevOps Services | Azure EntraHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India R16h ago
-
Senior DevOps Engineer INR 3000K-4000KAlerting | Apache Spark | BigQuery | CI/CD | DataflowFlexible working | Hybrid workingSenior-level Full TimeHyderabad Knowledge City (LTC), India R16h ago
-
Junior AI Specialist INR 300K-420KAWS | Apache Airflow | DBT | Data Modeling | Deep learningCertification programs | Health insurance | Language courses | Mentorship | Professional developmentEntry-level Full TimePune, India R1d ago
-
Senior-level Full TimeBengaluru East, Karnataka, India R1d ago
-
Staff Data Engineer INR 3242K-4200KCloud Data | Cloud data platform | Continuous integration | Data Engineering | Data QualityFlexible work hours | Mentorship | Remote work optionsSenior-level Full TimeHyderabad, Telangana , India R1d ago
-
Senior Data Engineer- Remote, India INR 1500K-2600KApache Airflow | Apache Kafka | Asset bundles | Azure | Azure DataRemote work in IndiaSenior-level Full TimeIndia R1d ago
-
Testing Engineering Specialist Advisor INR 2475K-3380K.NET | AI Search | AWS | Application Insights | AzureSenior-level Full TimeNoida, UP, IN R1d ago
-
Systems Integration Specialist Advisor_Storage Cloud INR 800K-1099KAWS | Access Control | Automation | Block Storage | CI/CDMid-level Full TimeBangalore, KA, IN R1d ago
-
AVP - Cloud Database Automation Engineer INR 2337K-3056KAWS RDS | Ansible | Auto Scaling | Cassandra | Cloud ConfigFlexible work hours | Work from homeExecutive-level Full TimeHyderabad IN, India R1d ago
-
AI Engineer INR 1500K-4000KAI Agents | API Design | Amazon Web Services | Async task queues | AzureRemote workMid-level Full TimeRemote - Pune, Maharashtra, India R1d ago
-
AI Engineer INR 1800K-3500KAI Foundry | AI Search | AWS Bedrock | Autogen | Azure AIHybrid work model | Onsite work at designated location | Remote work flexibilityNone Full TimeHyderabad, Telangana, India R1d ago
-
Senior Software Engineer - AI/ML INR 3000K-4800KAWS | Agentic design | Agentic design patterns | Agile | AzureSenior-level Full TimeHyderabad, India R1d ago
-
Software Engineer II, Analytics Operations INR 885K-1500KAWS Glue | Airflow | Amazon Athena | Amazon Redshift | Apache SparkMid-level Full TimeHyderabad, India R1d ago
-
Gen AI Site Reliability Engineer (SRE)- Senior Associate-AI Managed Services - Consult INR 2500K-4500KAWS | Alerting | Amazon Bedrock | Automation | Cause analysisSenior-level Full TimeBangalore (SDC) - Bagmane Tech Park, … R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KAttention | Automated retraining | CI/CD | Contrastive Learning | DDPMid-level Full TimeNoida, India R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KApache Spark | Attention Mechanism | Automated retraining | CI/CD | CheckpointingMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer INR 1860K-3000KApache Spark | Attention Mechanisms | Automated retraining | CI/CD | Contrastive LearningMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer 4 INR 3242K-5000KApache Spark | Attention | Automated retraining | CI/CD | CheckpointingSenior-level Full TimeNoida, India R1d ago
-
Mid-level Full TimeRemote Pune-Baroda, India R1d ago