Senior Data Engineer - Full Stack
Tasks
- Build event driven near real time data solutions
- Conduct technical design and code reviews mentor engineers
- Create APIs and integrations for governed data
- Create scalable data models and curated datasets
- Design and build data products on Databricks
- Develop batch incremental and streaming data pipelines
- Develop lightweight applications dashboards and interfaces
- Diagnose and resolve issues across data and application components
- Implement data quality security lineage and governance with Databricks and Unity Catalog
- Optimize pipelines streaming queries services and applications for performance and cost
- Partner with stakeholders to define requirements and delivery plans
- Set up automated testing CI CD monitoring alerting and documentation
Perks/Benefits
Skills/Tech-stack
AWS | Alerting | Apache Spark | Azure | CI/CD | Data Modeling | Data Warehousing | Databricks | Delta Lake | Distributed Processing | Event Driven | Event Hubs | Event-driven architecture | FastAPI | Flask | GCP | Git | Kafka | Kinesis | Monitoring | PySpark | Python | REST APIs | SQL | Schema evolution | Streaming Pipelines | Unity Catalog
Education
N/A
Related jobs
-
Software Development Specialist INR 940K-940KAmazon CloudWatch | Amazon DynamoDB | Amazon S3 | Amazon Web Services | Azure DevOpsMid-level Full TimeBangalore, KA, IN R8h ago
-
Java API & Microservices Engineer / Architect INR 2500K-3500KAPI Testing | Azure | CI/CD | Cassandra | DatadogSenior-level Full TimeBangalore, KA, IN R15h ago
-
Senior-level Full TimeBangalore, KA, IN R15h ago
-
Junior AI Specialist INR 300K-420KAWS | Apache Airflow | DBT | Data Modeling | Deep learningCertification programs | Health insurance | Language courses | Mentorship | Professional developmentEntry-level Full TimePune, India R1d ago
-
Senior-level Full TimeBengaluru East, Karnataka, India R1d ago
-
Staff Data Engineer INR 3242K-4200KCloud Data | Cloud data platform | Continuous integration | Data Engineering | Data QualityFlexible work hours | Mentorship | Remote work optionsSenior-level Full TimeHyderabad, Telangana , India R1d ago
-
Senior Data Engineer- Remote, India INR 1500K-2600KApache Airflow | Apache Kafka | Asset bundles | Azure | Azure DataRemote work in IndiaSenior-level Full TimeIndia R1d ago
-
Testing Engineering Specialist Advisor INR 2475K-3380K.NET | AI Search | AWS | Application Insights | AzureSenior-level Full TimeNoida, UP, IN R1d ago
-
Systems Integration Specialist Advisor_Storage Cloud INR 800K-1099KAWS | Access Control | Automation | Block Storage | CI/CDMid-level Full TimeBangalore, KA, IN R1d ago
-
AVP - Cloud Database Automation Engineer INR 2337K-3056KAWS RDS | Ansible | Auto Scaling | Cassandra | Cloud ConfigFlexible work hours | Work from homeExecutive-level Full TimeHyderabad IN, India R1d ago
-
AI Engineer INR 1500K-4000KAI Agents | API Design | Amazon Web Services | Async task queues | AzureRemote workMid-level Full TimeRemote - Pune, Maharashtra, India R1d ago
-
AI Engineer INR 1800K-3500KAI Foundry | AI Search | AWS Bedrock | Autogen | Azure AIHybrid work model | Onsite work at designated location | Remote work flexibilityNone Full TimeHyderabad, Telangana, India R1d ago
-
Senior Software Engineer - AI/ML INR 3000K-4800KAWS | Agentic design | Agentic design patterns | Agile | AzureSenior-level Full TimeHyderabad, India R1d ago
-
Software Engineer II, Analytics Operations INR 885K-1500KAWS Glue | Airflow | Amazon Athena | Amazon Redshift | Apache SparkMid-level Full TimeHyderabad, India R1d ago
-
Gen AI Site Reliability Engineer (SRE)- Senior Associate-AI Managed Services - Consult INR 2500K-4500KAWS | Alerting | Amazon Bedrock | Automation | Cause analysisSenior-level Full TimeBangalore (SDC) - Bagmane Tech Park, … R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KAttention | Automated retraining | CI/CD | Contrastive Learning | DDPMid-level Full TimeNoida, India R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KApache Spark | Attention Mechanism | Automated retraining | CI/CD | CheckpointingMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer INR 1860K-3000KApache Spark | Attention Mechanisms | Automated retraining | CI/CD | Contrastive LearningMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer 4 INR 3242K-5000KApache Spark | Attention | Automated retraining | CI/CD | CheckpointingSenior-level Full TimeNoida, India R1d ago
-
Mid-level Full TimeRemote Pune-Baroda, India R1d ago
-
Senior Data Engineer INR 3242K-4580KAgile | DBT | Data Lakes | Data Marts | Data ModelingCasual dress code | Financial health resources | Flexible work arrangements | Food benefits | Hybrid work scheduleSenior-level Full TimePune, IN, India R1d ago
-
Senior Data Engineer INR 1500K-2500KApache Cucumber | Apache Kafka | Apache Kafka Connect | Bash | BigQueryFlexible working | Hybrid workingSenior-level Full TimeHyderabad Knowledge Park Tower 2, India R1d ago
-
Mid-level Full TimeNoida, UP, IN R1d ago
-
Senior ML Research Engineer/Scientist INR 3000K-4580KAgent collaboration | Agentic AI | Benchmark design | Context engineering | Data MixtureSenior-level Full TimeHyderabad, India R1d ago
-
Agile | BigQuery | CI/CD | Cloud Functions | Cloud RunContinuous professional development | Flexible workingMid-level Full TimeHyderabad, Telangana, India R1d ago