Staff Engineer, Big Data
Tasks
- Build serverless applications with GCP Cloud Functions and Cloud Run
- Collaborate with cross-functional teams
- Configure IAM policies and VPC networking
- Containerize applications with Docker and deploy to Kubernetes
- Design scalable big data solutions using GCP
- Develop and deploy Python APIs with FastAPI
- Develop and optimize BigQuery and SQL queries
- Implement CI CD pipelines for build test and deployment
- Implement and maintain data pipelines with Apache Airflow
- Provision GCP resources using Terraform Infrastructure as Code
- Set up monitoring logging alerting and observability
- Troubleshoot production issues and perform root cause analysis
- Write unit tests and apply quality practices
Perks/Benefits
- N/A
Skills/Tech-stack
API Security | Alerting | Apache Airflow | Apache Spark | CI/CD | Cloud Functions | Cloud Run | Cloud platform | Django | Docker | FastAPI | Flask | Git | GitLab CI | GitLab CI/CD | Google BigQuery | Google Cloud | Google Cloud Platform | IAM | Infrastructure as Code | Kubernetes | Load Balancing | Logging | Monitoring | Octopus Deploy | PySpark | Python | SQL | Terraform | VPC | “as-code”
Education
Roles
Big Data Engineer | Cloud Engineer | Cloud) | Data Engineer | Engineer | Software Engineer
Related jobs
-
Data Engineer II INR 1000K-1500KBigQuery | Browser Developer Tools | CSS | Consent Mode | Developer toolsMid-level Full TimeBangalore, Karnataka, IN4h ago
-
Specialist, Business Analytics INR 650K-900KDBT | Domo | FinOps | Git | GitHubRelocation assistance within countryMid-level Full TimeMumbai, MH, IN5h ago
-
Google Cloud Engineer INR 1500K-2500KBigQuery | Cloud Data | Cloud Data Fusion | Cloud SQL | Cloud StorageAccidental life insurance | Company-sponsored certifications | Global client projects | Group health insurance | Group life insuranceSenior-level Full TimeHyderabad, Telangana, India6h ago
-
Senior Data Engineer INR 3000K-4000KAmazon Web Services | Apache Spark | Cloud platform | Continuous Delivery | Data ArchitectureSenior-level Full TimeBangalore, India; Chennai, India; Hyderabad, India; …6h ago
-
Staff Software Engineer INR 3000K-4000KAPI Development | Alerting | Apache Spark | Backend Services | CI/CDSenior-level Full TimeBengaluru, KA, India6h ago
-
Lead Developer (Python, React, AWS, GenAI) INR 3242K-4800KAWS | Agile | Continuous Delivery | Continuous integration | Design PatternsSenior-level Full TimePune, India7h ago
-
Senior Member Technical Staff (MTS 3) - Machine learning INR 2500K-4500KAWS | Anomaly Detection | Azure | Cassandra | Cosmos DBSenior-level Full TimeBengaluru, India7h ago
-
Lead Data Engineer (OIC+Python) INR 3380K-4800KAPI Management | Agile | Apache Airflow | CI/CD | Cloud ComposerSenior-level Full TimeHyderabad, India7h ago
-
Senior Data Engineer, BizTech INR 3080K-4400KAmazon Redshift | Apache Flink | Apache Kafka | Apache Spark | Batch ProcessingSenior-level Full TimeBangalore, India8h ago
-
Senior-level Full TimeIndia10h ago
-
Agile | Alerting | Automation | CI/CD | ClusteringAgile environment | Professional growthSenior-level Full TimeIndia11h ago
-
AI Governance | AWS | Agentic Workflows | CI/CD | Cloud NativeSenior-level Full TimeAPAC - India - Bengaluru - …13h ago
-
API Development | Agent Orchestration | Agent alignment | Anthropic | Async ProgrammingSenior-level Full TimeBengaluru Millenia, India13h ago
-
AWS | Authentication | CI/CD | Docker | Exception HandlingEqual employment opportunity | Flexible work programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimePune, India13h ago
-
AWS | Authentication | CI/CD | Data Modeling | DockerFlexible work arrangements | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimePune, India13h ago
-
AWS | Apache Kafka | Azure | Bootstrap | CSSFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeChennai - Menon Eternity, India13h ago
-
Agile | Apache Spark | Azure Blob | Azure Blob Storage | Azure DataFlexibility programmes | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India13h ago
-
AWS Data Engineer INR 3200K-4580KAWS IAM | AWS KMS | AWS Lambda | AWS Secrets | AWS Secrets ManagerSenior-level Full TimeBangalore, KA, IN R13h ago
-
AWS Data Engineer INR 3200K-4580KAWS IAM | AWS KMS | AWS Lambda | AWS Secrets | AWS Secrets ManagerSenior-level Full TimeBangalore, KA, IN R13h ago
-
Automation Excellence Senior Specialist INR 2500K-3584KAI Governance | AI Services | API | Access Management | Azure AISenior-level Full TimeINDJZ03 - Pune - Weikfield IT …13h ago
-
Automation Excellence Lead INR 2500K-5000K.NET | AI Agent | AI Agent Design Patterns | AI Builder | AI FoundrySenior-level Full TimeINDJZ03 - Pune - Weikfield IT …13h ago
-
Databricks Engineer INR 1000K-6000KAmazon Web Services | Apache Spark | CI/CD | Cloud platform | Data EngineeringDual project acceptable | EST time zone | Long-term contract | Night shift | Remote workSenior-level Full TimeBengaluru, Karnataka, India - Remote R13h ago
-
Databricks and Splunk Engineer - Senior Associate INR 3264K-4500KAmazon Web Services | Analytics | Azure | Cloud Computing | Cyber ThreatEmployee networks | Flexible work/life support | Hybrid work schedule | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeBangalore, India R14h ago
-
Senior Software Engineer INR 2771K-4064KAngular | Apache Hadoop | Apache Hive | Apache Spark | DockerSenior-level Full TimeBangalore, India15h ago
-
HPC Systems Engineer INR 1800K-3000KAnsible | C# | Cause analysis | Configuration Management | DebuggingEvening and weekend maintenance participation | Rotating on-call | Vendor travelMid-level Full TimeMumbai16h ago