Senior Staff - GCP Data Engineer
Bangalore Office BLS2, India
INR 1500K-2400K (estimate) Senior-level Full Time
Tasks
- Architect data lake and data warehouse solutions
- Build real-time streaming pipelines
- Create and optimize BigQuery datasets
- Design and develop ETL/ELT pipelines
- Design data models partitioning and clustering
- Ensure data security encryption and access control
- Implement Change Data Capture solutions
- Implement data lifecycle policies retention and archival
- Implement data marts and aggregation layers
- Implement data quality checks and monitoring
- Implement monitoring and alerting
- Integrate data from databases APIs SaaS and file systems
- Integrate third party data providers and external data feeds
- Maintain data lineage documentation and impact analysis
- Maintain dimensional models and fact tables
- Optimize pipeline performance and cost
- Support self service analytics documentation and data cataloging
- Troubleshoot pipeline issues and implement error handling
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Analytics hub | Apache Airflow | Apache Beam | BigQuery | Bigtable | CDC | CI/CD | Change Data Capture | Cloud Composer | Cloud Dataflow | Cloud Functions | Cloud Monitoring | Cloud PubSub | Cloud SQL | Cloud Storage | Cloud logging | Cloud platform | DBT | Data Capture | Data Lifecycle Management | Data Lineage | Data Modeling | Data Quality | Data lifecycle | Data quality monitoring | Dataform | Dataplex | Dataproc | Dimensional Modeling | Docker | Encryption | Firestore | GKE | Git | Google Cloud | Google Cloud Platform | Java | Kafka | Kubernetes | Lifecycle Management | MySQL | PostgreSQL | Python | Quality monitoring | Resource allocation | SQL | Scheduling | Spark | Terraform
Education
Related jobs
-
AWS | Amazon Kinesis | Amazon Redshift | Apache Airflow | Apache BeamFree games | Hybrid work | Learning and growth opportunities | Medical insurance | Mental health supportEntry-level Full Time地点：Hyderabad, Telangana, India7h ago
-
Mid-level Full TimeLocations: Hyderabad, Telangana, India7h ago
-
AWS CloudFormation | Amazon Kinesis | Amazon Redshift | Amazon Web Services | Apache AirflowCareer growth opportunities | Complimentary games | Family leave | Healthcare coverage | Mental wellbeing supportEntry-level Full TimeLocations: Hyderabad, Telangana, India7h ago
-
GBM - Public Dept-Bengaluru-Associate-Quantitative Engineering INR 1200K-2200KAutomation | C++ | Derivatives pricing | Equity Derivatives | Equity swapsMid-level Full TimeBengaluru, Karnataka, India8h ago
-
Associate Distinguished Engineer (Data Governance Framework, Data Catalog Implementation) INR 1300K-2000KBusiness glossary | Compliance | Data Architecture | Data Cataloging | Data ClassificationAwareness sessions | Continuous learning opportunities | Training programsMid-level Full TimeRemote, India R8h ago
-
Staff Software Engineer, AI/ML and Moma Agentic INR 2500K-5000KArtificial Intelligence | C# | C++ | Database technologies | Distributed SystemsMentorshipSenior-level Full TimeHyderabad, Telangana, India8h ago
-
Senior Staff Software Engineer, Google Cloud Storage INR 3200K-4000KArtificial Intelligence | Benchmarking | C++ | Cloud Storage | Distributed SystemsSenior-level Full TimeHyderabad, Telangana, India8h ago
-
Senior Manager of Software Engineering Analytics Platform INR 3000K-5000KAgile | Artificial Intelligence | CI/CD | Cloud Native | DatabricksSenior-level Full TimeHyderabad, Telangana, India9h ago
-
Senior-level Full TimeHyderabad, TS, India10h ago
-
Software Engineer II - UI + Python Developer + AI / ML INR 2500K-4000KAWS | Agile | CI/CD | CSS | HTMLMid-level Full TimeMumbai, Maharashtra, India12h ago
-
AI Engineer INR 2475K-4200KAgentic Systems | Azure OpenAI | Azure OpenAI APIs | Azure OpenAI SDK | Cloud IntegrationEntry-level Full TimeBengaluru, KA, IN12h ago
-
Data Engineer Advisor INR 3200K-4200KAlerting | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeHyderabad, TG, IN12h ago
-
Machine Learning Engineer - LLM & GenAI INR 2000K-5000KAPI Integration | Caching | Chroma | CrateDB | Embedding OptimizationRemote workMid-level Full TimePune, MH, India13h ago
-
AI/ML Expert INR 2500K-5000KAgent systems | Edge Computing | Embedded Systems | Entity Extraction | Fine TuningSenior-level Full Timepune, India13h ago
-
Associate Distinguished Engineer (Agentic AI Architect) INR 1500K-3000KAI Governance | AI architecture | AI orchestration | APIs | AWSMid-level Full TimeIndia, India13h ago
-
AWS SageMaker | Airflow | Amazon EC2 | Amazon EKS | Computer VisionContinuous learning and development | Flexible schedule | Fully remote | Mentorship opportunitiesMid-level Full TimeIndia R14h ago
-
Data Scientist INR 3000K-4000K3D Perception | Active Learning | Computer Vision | Container Orchestration | Data AnnotationSenior-level Full TimeBengaluru, KA, India14h ago
-
Data and AI/ML Design engineer at NE-VM INR 2000K-3300KAgile | Apache Kafka | C# | CSS | Computer VisionSenior-level Full Timebangalore, India14h ago
-
Senior-level Full Timecoimbatore, India14h ago
-
Senior Data Engineer (Python, PySpark, Kafka) INR 1500K-2700KApache Kafka | Data Warehousing | Dimensional Modeling | Git | GitHubSenior-level Full TimeBangalore, India14h ago
-
Data Engineer_Sr_Python Specialist INR 1500K-2500KAPIs | Alerting | Data Augmentation | Data Lineage | Data ModelingSenior-level Full Timebangalore, India14h ago
-
Senior-level Full Timebangalore, India14h ago
-
AI/ML Developer INR 1500K-2500KAzure | Cloud platform | Data Pipelines | Data Preprocessing | Deep learningMid-level Full Timebangalore, India14h ago
-
Senior-level Full TimeMumbai, Maharashtra, India14h ago
-
Applied Computer Vision Engineer - Data Driven Development INR 4000K-5500KAWS | Apache Spark | Azure | Convolutional Neural Networks | Deep learningSenior-level Full Timebengaluru, India14h ago