Sr Lead Data Engineer With GCP
INDIA - BENGALURU - BIRLASOFT OFFICE, IN
INR 2000K-3500K (estimate) Senior-level Full Time
Tasks
- Build batch data pipelines with Dataproc and PySpark
- Create and manage Dataproc jobs
- Create solution design documents and architecture discussions
- Deploy and manage containerized applications on Google Kubernetes Engine
- Design scalable solutions on Google Cloud
- Develop and optimize datasets and queries in BigQuery
- Develop backend and event driven components with Cloud Functions
- Implement monitoring logging and alerting with Cloud Monitoring
- Participate in client interactions and requirement gathering
- Support AI and ML and generative AI solutions with Vertex AI
- Troubleshoot production issues and debug data pipelines
Perks/Benefits
Skills/Tech-stack
API Integration | Apache Spark | Batch Processing | BigQuery | CI/CD | Cloud Functions | Cloud Monitoring | Cloud platform | Data Engineering | ELT | ETL | GCP Dataproc | Google Cloud | Google Cloud Platform | Google Kubernetes | Google Kubernetes Engine | IAM | Infrastructure as Code | Kubernetes Engine | Networking | PySpark | Python | SQL | Streaming | Terraform | Vertex AI | “as-code”
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeHyderabad, India4h ago
-
Sr. Data Engineer INR 2800K-4000KCloud warehousing | DAX | Dashboard Design | Data Quality | Data StorytellingCommunity outreach programs | Employee Resource Group | Employee stock purchase program | Health and wellness incentivesSenior-level Full TimeBangalore, KA, IN, 5600375h ago
-
Software Engineer INR 1000K-1496KBitbucket | Data Architecture | Data Modeling | Data Pipelines | LinuxEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India5h ago
-
GIS Data Engineer II INR 2000K-2400KAWS | Amazon Athena | Amazon EC2 | Amazon RDS | Amazon S3MentorshipSenior-level Full TimeMumbai, MH, India7h ago
-
C# | C++ | Go | Infrastructure automation | JavaSenior-level Full TimeBengaluru, Karnataka, IND11h ago
-
Amazon S3 | C# | C++ | Data Migration | Data replicationSenior-level Full TimeBengaluru, Karnataka, IND11h ago
-
Software Engineer - Embedded & Functional Safety Software INR 1753K-3440KAgile | Automated testing | C# | C++ | CAN busSenior-level Full TimeBangalore, IN11h ago
-
Software Engineer - Robot Navigation, Localization & SLAM INR 1753K-2372K3D camera | Agile | Automated testing | C++ | CI/CDMid-level Full TimeBangalore, IN11h ago
-
Software Engineer - Robot Connectivity & Wi Fi Systems INR 1000K-2000KAgile | Automated testing | CI/CD | GRPC | GitCustomer-facing workSenior-level Full TimeBangalore, IN11h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agent/tool use | Apache Spark | Azure | ContainersContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India11h ago
-
Senior-level Full TimeMumbai, Maharashtra, India11h ago
-
Apache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeNoida, India11h ago
-
IN_Manager_SSIS_Data & Analytics_Advisory_Noida INR 1500K-2000KAWS | Azure | CI/CD | Cloud Data | Cloud Data PlatformsFlexibility programmes | Inclusive benefits | MentorshipMid-level Full TimeNoida, India11h ago
-
EY - GDS Consulting - AIA - Gen AI - Staff INR 3334K-5000KAWS | AWS Bedrock | Azure OpenAI | CD | CISenior-level Full TimePune, MH, IN, 41101411h ago
-
EY - GDS Consulting - AI And DATA -Snowflake Architect -Manager INR 3000K-4000KAgile | Batch Processing | Cloud Data | Cloud Data Warehouse | DBTCareer development | Coaching and feedback | Flexible work arrangementSenior-level Full TimeHyderabad, TG, IN, 50008111h ago
-
Senior Executive - Information Technology INR 2000K-4225KAdvanced Analytics | Bagging | Boosting | Decision Trees | Machine LearningSenior-level Full TimeAiroli, MH, IN11h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agentic AI | Azure | CI/CD | Code reviewContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India11h ago
-
Software Engineer INR 1400K-2000KAgile | Cloud services | Code review | Data Modeling | Database DesignMid-level Full TimePune, India19h ago
-
Amazon Web Services | Apache Airflow | Apache Kafka | Apache Spark | Cloud platformContract employment | Hybrid work | VIRTUAL interviewSenior-level ContractBangalore North, India21h ago
-
Data Engineer INR 1500K-2040KAgentic AI | Cost Optimization | Data Architecture | Data Modeling | Data platformSenior-level Full TimeIN-MH-Pune22h ago
-
Data Engineer Architect. INR 1500K-2400KAPI Development | AWS | Agentic Engineering | Database optimization | FastAPISenior-level Full TimeIN-MH-Pune22h ago
-
AIML - Tech Lead INR 2500K-4500KApache Airflow | ETL | Entity recognition | Language Processing | Machine LearningSenior-level Full TimeIN-MH-Pune22h ago
-
Senior GenAI Developer INR 2500K-4800KAPI Gateway | AWS Bedrock | Agent systems | Agentic RAG | Amazon LambdaCertification programs | Global client projects | Health insurance | Internship opportunities | Language coursesSenior-level Full TimeAll Cities, India R23h ago
-
AI Solutions Architect INR 3000K-5000KAPI Integration | Agentic AI | Enterprise Integration | LLM orchestration | Language ModelsSenior-level Full TimeNoida, Uttar Pradesh23h ago
-
Senior-level Full TimePune, Maharashtra, India1d ago