Data Engineer & BI
Hyderabad, Telangana, India
INR 3000K-4000K (estimate) Mid-level Full Time
Tasks
- Build batch and real time processing workflows
- Build master data synchronization and transactional workflow pipelines
- Build semantic data models for OEE downtime and machine performance
- Define and maintain canonical data models and data contracts
- Design and implement API based and event driven enterprise integrations
- Design scalable data pipelines for IoT data ingestion transformation storage
- Design time series telemetry and asset hierarchy data models
- Develop dashboards and reports with Power BI and Grafana
- Ensure data governance and SOC2 compliance
- Ensure data quality validation and consistency across pipelines
- Implement ETL ETL pipelines for structured and unstructured data
- Implement RBAC secure data access and audit logging
- Implement lakehouse architecture with Bronze Silver Gold layers
- Implement row level security and tenant aware data access
- Optimize Synapse and Fabric queries and workloads
- Optimize data storage formats and partitioning strategies
- Prepare data for predictive maintenance anomaly detection and trend analysis
- Reduce cloud costs using modeling storage lifecycle and query optimization
Perks/Benefits
- N/A
Skills/Tech-stack
API Management | Azure API | Azure API Management | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Event | Azure Event Hubs | Azure IoT | Azure IoT Hub | Azure Service | Azure Service Bus | Azure Synapse | Azure Synapse Analytics | DAX | Data Contracts | Data Factory | Data Lake Storage Gen2 | Data Management | Data Modeling | Data Partitioning | Delta Lake | ELT | ETL | Event Driven | Event Hub | Event Hubs | Event-driven architecture | Grafana | IoT Hub | JWT | Lakehouse | Master Data | Master Data Management | Microsoft Fabric | OAuth 2 | OAuth 2 0 | Parquet | Power BI | PySpark | Python | RBAC | REST API | Row Level Security | SOC 2 | SQL | Series data | Service Bus | Star Schema | Synapse Analytics | Time Series | Time Series Data
Education
N/A
Related jobs
-
API Development | CI/CD | ETL | Git | OpenpyxlDistributed team | Work from homeMid-level Full TimeMaharashtra, Pune, India8h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KAI Search | Apache Airflow | Apache Spark | CI/CD | Change Data CaptureEducation budget | Local connection | Remote work | Wellness budgetSenior-level Full TimePune City, India8h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | CI/CD | Change Data CaptureBudget for education | Budget for wellness | Legal compliance in India | Local connection meetups | Remote workSenior-level Full TimeBangalore North, India8h ago
-
Data Engineer (GCP) ID56375 INR 2400K-3380KApache Airflow | Apache Spark | BigQuery | Bitemporal Modeling | CI/CDCompetitive compensation | Education budget | Local connection | Remote work | Wellness budgetSenior-level Full TimeMumbai, India8h ago
-
Data Engineer (MS) (Indian Citizen only) INR 3000K-4000KCI/CD | Data Quality | Data Validation | Databricks | Date normalizationCollaborative culture | Long-term project stability | Mentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India8h ago
-
Copilot Studio / Agentic AI Engineer INR 2000K-4000KAI Services | API Integration | Agent Orchestration | Autonomous workflows | AzureSenior-level Full TimeMaharashtra, Pune, India9h ago
-
Gen Ai INR 2500K-5000KAlgorithms | Computer Vision | Data Analysis | Data Preprocessing | Data StructuresSenior-level Full TimeIN-TN-Chennai9h ago
-
Mid-level Full TimeIN-AP-Hyderabad9h ago
-
GEN AI Consultant INR 2000K-3000KAI Studio | Azure AI | Azure AI Studio | Azure Cognitive | Azure Cognitive SearchMid-level Full TimeIN-AP-Hyderabad9h ago
-
Gen AI INR 2500K-4500KAWS | Agent systems | Airflow | Asynchronous programming | Attention MechanismsSenior-level Full TimeIN-KA-Bangalore9h ago
-
Mid-level Full TimeIN-KA-Bangalore9h ago
-
Data & AI Specialist INR 1500K-2500KAI Search | Azure AI | Azure AI Search | Azure Data | Azure Data LakeMid-level Full TimeIN-HR-Gurgaon9h ago
-
Senior-level Full TimeIN-AP-Hyderabad9h ago
-
Software Engineer - Data Engineer INR 2520K-3380KAWS | Azure | Data Governance | Data Lakes | Data ModelingSenior-level Full TimeBangalore - Karnataka, India - Kalyani …10h ago
-
Data Scientist - Gen AI ML - Chennai/Pune INR 2000K-3000KAPI Security | Agent systems | Asynchronous programming | CI/CD | Docker401k retirement plan | Dental insurance | Medical insurance | Paid Holidays | Paid time offMid-level Full TimeIndia10h ago
-
Advanced Specialist, Data Governance INR 1800K-2500KApache Airflow | CI/CD | Cloud Composer | Cloud Storage | CollibraMid-level Full TimeChennai, Tamil Nadu, India10h ago
-
Advanced Specialist, Technology Data Analyst INR 1800K-3000KAccess Control | Bash | BigQuery | CI/CD | Capacity Planning24/7 on-call rotationSenior-level Full TimeChennai, Tamil Nadu, India10h ago
-
Sr. Data Engineer - AI ML INR 2475K-3800KAWS ECS | AWS Glue | AWS Lambda | AWS Neptune | AWS Step FunctionsComprehensive benefits package | Flexible work options | Hybrid work model | Life leave days | Professional development daysSenior-level Full TimePune, MH, India10h ago
-
Senior Staff Machine Learning Engineer INR 3000K-3500KAlgorithms | Artificial Intelligence | Code optimization | Design Patterns | JavaFlexible work arrangements | Health benefits | Paid time off | Remote work optionsSenior-level Full TimeBangalore, Karnataka, India10h ago
-
Forward Deployed Engineer, GenAI, Google Cloud INR 2000K-3000KAPIs | Agent systems | Cloud platform | Cost Per Request | CrewAIMid-level Full TimeMumbai, Maharashtra, India; Bengaluru, Karnataka, India11h ago
-
Forward Deployed Engineer, GenAI, Google Cloud INR 2500K-5000KAPIs | Agent systems | Cloud platform | Cost Optimization | Data PipelinesSenior-level Full TimeGurugram, Haryana, India; Bengaluru, Karnataka, India11h ago
-
Software Engineer III, GenAI, Google Cloud INR 2500K-4800KAlgorithms | C++ | Data Structures | Distributed Computing | Google CloudSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
AI Engineering Lead INR 2475K-4500KAWS | AWS Bedrock | Agentic Workflows | Azure | Azure Prompt FlowCompany-provided equipment | English lessons | Flexible working options | Learning opportunities | Mentorship programsSenior-level Full TimeHyderabad, TS, India11h ago
-
Staff Engineer INR 2000K-4000KAI | Agentic Workflows | Cloud Cost Optimization | Cloud Dataflow | Cloud DataprocSenior-level Full TimeChennai, Tamil Nadu, India11h ago
-
Applied AI/ML Associate Senior - Causal ML INR 1400K-2000KA/B | A/B Testing | Apache Hive | Apache Spark | B testingMid-level Full TimeBengaluru, Karnataka, India12h ago