Senior Data Engineer
Tasks
- Automate and monitor data workflows
- Collaborate with architects to establish best practices
- Collaborate with cross-functional teams to deliver datasets
- Create modular dbt data models
- Design data ingestion pipelines
- Develop Python and SQL ingestion and transformation code
- Implement BigQuery partitioning and clustering
- Implement CI/CD workflows
- Implement data quality checks and testing
- Lead data integration pipeline development
- Maintain data workflow orchestration
- Maintain technical documentation
- Mentor junior engineers and conduct code reviews
- Optimize BigQuery queries
- Transform raw data into curated datasets
Perks/Benefits
Skills/Tech-stack
Agile | Apache Airflow | Apache Kafka | CI/CD | Cloud Composer | Cloud Dataflow | Clustering | DBT | Data Governance | Data Quality | Data Validation | Data Warehousing | Databricks | ELT | ETL | Fivetran | Git | Google BigQuery | Kanban | Partitioning | Pub/Sub | Python | Query Optimization | SQL | Scrum | Workflow Orchestration
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Developer | ETL Developer | Engineer | Senior Data Engineer | Senior ETL Developer
Related jobs
-
Software Engineer II INR 1000K-2000KApache Spark | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryMid-level Full TimeIndia, Telangana, Hyderabad3h ago
-
Sr. Specialist, Data Engineering INR 2771K-4000KAgile | Airflow | BigQuery | CI/CD | Cloud FunctionsRelocation assistance within countrySenior-level Full TimeMumbai, MH, IN5h ago
-
Senior-level Full TimeTelangana, India5h ago
-
API Integration | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryMid-level Full TimePune, Maharashtra, India7h ago
-
Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure SQLMid-level Full TimePune, Maharashtra, India7h ago
-
Azure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimePune, Maharashtra, India7h ago
-
IT engineer data lakehouse INR 1500K-2500K3NF | APIs | Audit trails | Azure | Azure DatabricksMobile and flexible working models | Sabbaticals | Training opportunitiesSenior-level Full TimeBengaluru, KA, India7h ago
-
Mid-level Full TimeBangalore, Karnataka, India8h ago
-
Lead Software Engineer INR 2500K-5000KAWS | Asynchronous programming | Authentication | Authorization | AutogenSenior-level Full TimeRemote, India R9h ago
-
API Integration | Angular | Data Visualization | Flask | JavaScriptFlexible remote work | Long term remote contractSenior-level Contract Full TimeIndia9h ago
-
Senior Software Engineer - Ingestion INR 2040K-2800KBackup and Recovery | C++ | Data Capture | Database replication | Distributed SystemsSenior-level Full TimeBengaluru, India10h ago
-
Business Intelligence Analyst - I INR 1500K-2500KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Data FactoryMid-level Full TimeHyderabad, IN, 500 00813h ago
-
.NET | Agile | Azure | Azure Databricks | C#Senior-level Full TimeKolkata DN 57, India13h ago
-
Mid-level Full TimeBangalore, Karnataka, IN India, 56008713h ago
-
Forward Deployment Engineer (AI Solutions)-BK INR 2500K-4500KAI Agents | AWS Bedrock | Amazon Web Services | Angular | Azure OpenAIMid-level Full TimeBengaluru, Karnataka, India13h ago
-
Forward Deployment Engineer (AI Solutions) INR 2500K-5000KAI Agents | AWS Bedrock | Amazon Web Services | Angular | Azure OpenAIMid-level Full TimeBengaluru, Karnataka, India13h ago
-
DE-RCE-Fraud Analytics-Senior-GDSN02 INR 3264K-5000KAWS Glue | AWS Lambda | Amazon EventBridge | Amazon S3 | CI/CDSenior-level Full TimeBengaluru, KA, IN, 56001613h ago
-
Senior-level Full TimePune, MH, IN, 41100513h ago
-
Senior Data Engineer INR 2000K-2500KAutomated testing | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeIN KA Bengaluru, India13h ago
-
TechOps-CloudOps-DataOps-Senior INR 3000K-4000KAutomation | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeBengaluru, KA, IN, 56001613h ago
-
TechOps-CloudOps-DataOps-Staff INR 3000K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Azure DevOpsSenior-level Full TimeBengaluru, KA, IN, 56001613h ago
-
Data Platforms SRE INR 1500K-2500KAWS IAM | Access Management | Active Directory | Azure Active Directory | Azure DevOpsEntry-level Full TimeBangalore, India13h ago
-
Senior-level Full TimeBengaluru, Karnataka, India13h ago
-
Senior Software Engineer - Java Fullstack, Gen AI INR 3500K-4000KAgile | Angular | Artifactory | Azure | Azure GCPSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India13h ago
-
Mid-level Full TimeIN, BANGALORE, India13h ago