Digital Cloud & Data Platform Sr. Engineer
Tasks
- Automate unit testing for data transformations and schemas
- Build and operate real time and batch data processing
- Collaborate with cross functional teams to translate telemetry into insights
- Create streaming and batch ETL reference implementations
- Design data platform architecture and roadmap
- Design resilience with error handling reprocessing and backfill
- Develop reusable ingestion transformation and serving pipelines
- Ensure data quality validation lineage and observability
- Implement data governance, security, and compliance controls
- Implement medallion architecture for structured data
- Monitor platform throughput latency and unit economics
- Optimize storage compute and query performance for cost and scale
- Provide self service access to trusted datasets
- Support System Integration Testing and User Acceptance Testing
Perks/Benefits
- Disability insurance
- Health insurance
- Life insurance
- Paid time off
- Relocation eligible
- Travel opportunities
Skills/Tech-stack
AWS IAM | AWS KMS | AWS Lake Formation | AWS Lambda | AWS Step Functions | Access Control | Amazon S3 | Apache Airflow | Apache Athena | Apache Flink | Apache Iceberg | Apache Spark | Data Governance | Data Lineage | Data Quality | ETL | Event Driven | Event-driven architecture | Lake Formation | Medallion Architecture | Observability | Policy-Based Access Control | Policy-based | Policy-based access | SQL | Schema evolution | Step Functions | Streaming Data | Trino | Unit Testing | Workflow Orchestration
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Data Engineer INR 1000K-1000KApache Airflow | Apache Spark | BigQuery | CI/CD | Cloud BuildEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India9h ago
-
Data Engineer INR 1500K-3000KAPI Development | AWS | Amazon Kinesis | Amazon Redshift | AzureCommunity volunteering opportunities | Discounted gym memberships | Flexible working | Life assurance | Medical insuranceMid-level Full TimeHyderabad, India13h ago
-
Forward Deployed Engineer INR 4500K-7000KAI | APIs | Agentic AI | Developer tools | Enterprise SoftwareHigh-ownership environment | Onsite customer engagement | R and D Collaboration | Structured Product FeedbackSenior-level Full TimeGurugram, Haryana, IN14h ago
-
GCP Data Engineer / Consultant Specialist INR 1500K-2000KAirflow | Alerting | Apache Beam | Automation | BigQueryFlexible working | Inclusive workplace | Opportunities for growth | Professional developmentMid-level Full TimePune, Maharashtra, India R15h ago
-
Senior AI Testing Engineer (Generative AI) INR 2000K-3300KAI Agents | API Testing | AWS | Azure | CI/CDSenior-level Full TimeBangalore North, India15h ago
-
Senior-level Full TimeIN-KA-Bangalore16h ago
-
Mid-level Full TimeBangalore, Karnataka, India17h ago
-
Mid-level Full TimeBangalore, Karnataka, India17h ago
-
Consultant - Python Developer with Gen AI INR 1500K-2200KAI Search | Agile | Azure | Azure OpenAI | CSSMid-level Full TimeBangalore, Karnataka, India17h ago
-
Practice Customer Engineer, Gen AI, Telecom or FSI INR 1800K-3300KAI Agent Development Kit | API Gateway | Agent Development Kit | Agentic AI | Audit LoggingSenior-level Full TimeMumbai, Maharashtra, India17h ago
-
Data Engineer, Play Data Science and Analytics INR 1500K-2500KAI | Automation | C plus plus | Data Infrastructure | Data ModelingMid-level Full TimeBengaluru, Karnataka, India17h ago
-
Data Engineer, YouTube INR 800K-1400KApache Flume | Apache Spark | Cloud Dataflow | Data Analysis | Data ArchitectureMid-level Full TimeBengaluru, Karnataka, India17h ago
-
Practice Customer Engineer, GenAI, Public Sector, Google Cloud INR 2000K-3300KAPIs | Agentic AI | Audit Logging | Cloud Networking | Cloud infrastructureSenior-level Full TimeGurgaon, Haryana, India; Bengaluru, Karnataka, India17h ago
-
Forward Deployed Engineer, Generative AI, Google Cloud INR 1800K-3000KAPI Integration | Agent systems | Cloud platform | CrewAI | Data PipelinesMid-level Full TimeMumbai, Maharashtra, India; Bengaluru, Karnataka, India17h ago
-
Sr Devops Engineer INR 2000K-3200KAWS | AWS CloudFormation | Access Control | Airflow | Amazon ECSDocumentation and standards | On-call rotationSenior-level Full TimePune, Maharashtra, India17h ago
-
Mid-level Full TimePune, MH, India18h ago
-
Senior Engineer - Embedded QA INR 1500K-2094KAPI Testing | Agile | Azure DevOps | CI/CD | Continuous DeploymentSenior-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101318h ago
-
Senior Software Engineer INR 2520K-3100KAzure Cosmos | Azure Cosmos DB | Azure Data | Azure Data Explorer | Azure Data FactoryOn-call rotation | Scalable work environmentSenior-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN; …19h ago
-
Consultant - Python Developer with Gen AI and AI Foundry INR 2000K-3300KAWS | Autogen | Azure | Azure OpenAI | CI/CDSenior-level Full TimeBangalore, Karnataka, India19h ago
-
Consultant INR 1500K-2000KDashboarding | Data Dictionary | Data Governance | Data Modeling | Data QualityMid-level Full TimeGurugram, Haryana, India20h ago
-
Senior Consultant INR 1500K-2146KAlert automation | Dashboard automation | Data Dictionary | Data Governance | Data MappingSenior-level Full TimeIndia20h ago
-
Senior Consultant INR 1500K-2146KAlert automation | Dashboard Development | Data Governance | Data Mapping | Data ModelingSenior-level Full TimeIndia20h ago
-
Senior-KDNI INR 3000K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Azure SQL | Data FactorySenior-level Full TimeBangalore, Karnataka, India20h ago
-
Senior Data Engineer I - Bigdata, GCP, GenAI INR 2000K-3000KAgile | Apache Kafka | Apache Spark | CI/CD | Cloud ComposerSenior-level Full TimeChennai, TN, India21h ago
-
AI Engineer- Associate Consultant INR 1800K-2500KAWS | Agent Orchestration | Autonomous Agents | Azure | ClassificationMid-level Full TimeBangalore, Karnataka, India21h ago