Apache Airflow & AWS (S3/EMR/Bedrock) Data Platform Administrator / Operations Engineer
Pune, Maharashtra, India
INR 2535K-4200K (estimate) Senior-level Full Time
Tasks
- Analyze S3 usage and costs
- Configure Amazon Bedrock model endpoints
- Enable S3 access logging
- Encrypt S3 data with AWS KMS
- Execute retries and backfills
- Govern Bedrock costs
- Handle SLA miss events
- Implement S3 storage tiering and transitions
- Manage Bedrock access controls
- Manage EMR autoscaling
- Manage pipeline failure recovery
- Monitor Apache Airflow DAGs
- Monitor Bedrock invocations
- Optimize Amazon S3 storage with lifecycle policies
- Perform EMR lifecycle management
- Perform root cause analysis for pipeline failures
- Provide SLA based operational support
- Provision and configure Amazon EMR clusters
- Rerun failed task instances
- Resolve DAG execution delays
- Secure S3 with IAM policies and bucket policies
Perks/Benefits
Skills/Tech-stack
AWS | AWS IAM | AWS KMS | Access Logging | Amazon Bedrock | Amazon EMR | Amazon S3 | Apache Airflow | Autoscaling | Backfilling | Bucket policies | Cause analysis | DAG monitoring | Intelligent Tiering | Lifecycle Policies | Object level permissions | Retry strategies | Root Cause Analysis | Root cause | S3 versioning | SLA management | Storage class transitions | Storage lifecycle policies
Education
N/A
Related jobs
-
AWS Data Engineer - Consultant INR 720K-764KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon EMRMid-level Full TimeBangalore, Karnataka, India10h ago
-
AWS Data Engineer - Consultant INR 720K-764KAWS Glue | AWS Lambda | AWS Step Functions | Amazon Athena | Amazon EMRMid-level Full TimeBangalore, Karnataka, India10h ago
-
Gen AI - Consultant INR 2500K-4500KAWS SageMaker | Apache Airflow | Azure Machine Learning | CI/CD | Cloud ComputingSenior-level Full TimeBangalore, Karnataka, India10h ago
-
Gen AI - Consultant INR 2500K-4500KAPI Design | AWS SageMaker | Apache Airflow | Apache Spark | Azure MLAccess to AI research labs and cloud platforms | Certification access | Mentoring opportunities | Work on global clientsSenior-level Full TimeBangalore, Karnataka, India11h ago
-
Software Engineer III-( Java, AWS & Pyspark ) INR 2000K-2200KAWS | AWS Athena | AWS Glue | AWS Kinesis | AWS Lake FormationSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Senior-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Data Engineer INR 2500K-2829KAWS | Apache Airflow | Apache Iceberg | Apache Spark | AzureEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India13h ago
-
Data Engineering INR 2520K-3356KAWS | AWS CloudFormation | Amazon Redshift | Amazon S3 | Backup and RecoveryCode reviews | Continuous improvement | Mentoring and coaching | Onboarding support | Technical assessments participationSenior-level Full TimeIndia13h ago
-
Lead Software Engineer - Python AWS Developer INR 2200K-3500KAPI Versioning | AWS SDK | AWS SDK for Python | AWS Secrets | AWS Secrets ManagerCareer growth | Collaborative team environment | Community of practice involvement | Mentorship | Technology adoption opportunitiesSenior-level Full TimeMumbai, Maharashtra, India14h ago
-
Senior-level Full TimeHyderabad, Telangana, India14h ago
-
Apprentice Software Engineer INR 302K-540KAWS | Azure | BigQuery | Cloud Computing | Data ComplianceContinuous improvement initiatives | Mentoring | Shift handovers | Structured learning sessionsEntry-level Apprenticeship Full TimeChennai, Tamil Nadu, India14h ago
-
AWS Glue | Amazon Redshift | Amazon S3 | Apache Airflow | Apache BeamMid-level Full TimeBengaluru, INDIA, India15h ago
-
Lead Software Engineer - PySpark, BigData, AWS INR 2000K-2200KAWS | AWS CDK | AWS Lambda | Amazon DynamoDB | Amazon ECSSenior-level Full TimeHyderabad, Telangana, India16h ago
-
Senior-level Full TimeBangalore, KA, IN20h ago
-
Senior Software Engineer-Data Engineering INR 1800K-2200KAPI Gateway | AWS | AWS API | AWS API Gateway | AWS CloudFormationSenior-level Full TimeBangalore, Karnataka, India20h ago
-
Mid-level Full TimeBengaluru, BDC7C, India20h ago
-
Mid-level Full TimeBengaluru, BDC7C, India20h ago
-
Mid-level Full TimeBengaluru, BDC7A, India20h ago
-
Custom Software Engineer INR 885K-1475KApache Airflow | Cloud | Data Governance | Data integration | DatabricksMid-level Full TimeMumbai, MDC2B, India20h ago
-
Mid-level Full TimeHyderabad, India20h ago
-
Senior Machine Learning Engineer INR 2500K-3200KAWS | Artificial Intelligence | Azure | Cloud Computing | Cloud platformHealthcare | Paid volunteering days | Retirement planning | Wellbeing initiativesSenior-level Full TimeIND-Bangalore-TowerE,RMZ Infin, India20h ago
-
Senior-level Full TimePune, India20h ago
-
Gen AI Tech Engineering Lead - Senior Vice President INR 2500K-4500KAWS | Agentic Frameworks | Agile | Artificial Intelligence | AzureFull-time roleSenior-level Full TimePLOT NO-1, S.NO. 77, India20h ago
-
Gen AI Tech Engineering Lead - Senior Vice President INR 2500K-4500KAWS | Agentic Frameworks | Agile | Artificial Intelligence | AzureSenior-level Full TimePLOT NO-1, S.NO. 77, India20h ago
-
Senior Product Software Engineer - UI/Gen AI Product Engineering INR 2000K-5000KAWS | Authentication | Authorization | Best practices | BitbucketSenior-level Full TimeIND - Pune, Kalyani Nagar, India20h ago