Senior Data Engineer
Bangalore, Karnataka, IN India, 560087
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Apply quality gates auditability and compliance
- Build Git based CI CD pipelines
- Build end-to-end data pipelines
- Collaborate with product analytics and platform teams on data contracts and SLAs
- Design Iceberg tables and incremental processing
- Design and build scalable data products
- Diagnose data system issues and drive remediation
- Implement AWS access control and tagging with IAM
- Implement Parquet and Iceberg with schema evolution
- Implement dimensional and normalized data models
- Implement query optimization across Hive Impala Snowflake and RDBMS
- Implement transformations with SQL and dbt and PySpark
- Integrate event driven ingestion with Kafka
- Lead AWS data infrastructure across dev test prod
- Manage infrastructure as code with Terraform
- Mentor engineers and drive code quality through reviews and documentation
- Model RAW CURATED and SERVING layers
- Optimize data architectures for performance and cost
- Orchestrate workflows with Airflow and manage SLAs and SLOs
- Own batch and streaming pipeline delivery
- Use GenAI assisted development responsibly
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | AWS IAM | AWS Lambda | Amazon S3 | Amazon SageMaker | Apache Airflow | Apache Iceberg | Apache Kafka | CI/CD | DBT | Data Modeling | Data pipeline | Data pipeline orchestration | Dimensional Modeling | Hive | Impala | Infrastructure as Code | Normalized modeling | Parquet | Performance optimization | Pipeline Orchestration | PostgreSQL | PySpark | Python | Query Optimization | SQL | Schema evolution | Snowflake | Terraform | “as-code”
Education
N/A
Roles
Related jobs
-
Manager Software Engineer- ML + NLP+ ppl management INR 3200K-4800KAPIs | BERT | Data Mining | Data Science | Deep learningFlexible work arrangements | Hybrid work environmentSenior-level Full TimeGurgaon - Cyber Park, India5h ago
-
Principal Engineer, Storage Data Science and Analytics INR 3200K-4590KAIOps | Anomaly Detection | Apache Spark | C# | C++Career Development Programs | Flexible work arrangements | Health and wellbeing benefitsSenior-level Full TimeBengaluru, Karnātaka, India5h ago
-
AWS | Airflow | DBT | Fine Tuning | Language ModelsBonuses | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, India R11h ago
-
AI/ML engineer - Associate Research Scientist INR 1500K-2500KAIOps | API Development | AWS | Anomaly Detection | AutomationContinuous learning and professional development | Employee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India11h ago
-
Mid-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101312h ago
-
Efs Fcrm 169474 INR 1500K-2400KApache Spark | Azure DevOps | CI/CD | Data Partitioning | Data WarehousingMid-level Full TimeHyderabad, IND-TG, India12h ago
-
Planview Administration 169576 INR 1500K-2500KAlerting | Data Governance | Data Modeling | Data Validation | Data cleaningSenior-level Full TimeHyderabad, IND-TG, India12h ago
-
Senior Database Engineer - #3190 (Remote) INR 1500K-2700KAWS IAM | AWS KMS | AWS RDS | Amazon Aurora | BashOn-call rotation | Remote workSenior-level Full TimeChennai, India R15h ago
-
Software Engineer SRE (Site Reliability Engineer) INR 1600K-3000KAWS | Ansible | Apache Airflow | Apache Kafka | Apache SparkSenior-level Full TimeBangalore, India Office (BANGALORE) R15h ago
-
Mid-level Full TimeIN-TN-Chennai16h ago
-
Senior-level Full TimeIN-MH-Pune16h ago
-
Snr AI INR 2500K-5000KAI Search | Amazon SageMaker | Azure AI | Azure AI Search | Azure Machine LearningSenior-level Full TimeIN-HR-Gurgaon16h ago
-
Senior-level Full TimeBengaluru, Karnataka17h ago
-
GCP BigQuery,DevOps,Credit Card Domain/Associate Director, Data and Analytics Specialist INR 650K-900KApache Airflow | Apache Kafka | BigQuery | CI/CD | Cloud ComposerMid-level Full TimeHyderabad, Telangana, India R17h ago
-
Azure Data Engineer INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Functions | Azure SynapseSenior-level Full TimePune, Maharashtra, India18h ago
-
Lead Software Engineer - Python Full Stack, AI/ML INR 4000K-5000KAWS | Agile | Angular | Application Resiliency | Automated testingSenior-level Full TimeMumbai, Maharashtra, India18h ago
-
Senior-level Full Timecoimbatore, India18h ago
-
Digital Marketing Analyst INR 2500K-3380KAWS | Azure | Cloud platform | Customer Analytics | Data GovernanceSenior-level Full TimeChennai, Tamil Nadu, India19h ago
-
Senior Staff Machine Learning Engineer (Java + LLM) INR 3000K-4401KAlgorithm Design | Code Tuning | Data Structures | Data Structures and Algorithms | Design PatternsFlexible work schedule | Health benefits | Paid time off | Remote work optionSenior-level Full TimeBangalore, Karnataka, India19h ago
-
Senior Staff Engineer, Data Migration Lead INR 3000K-4000KAccess Control | Audit Trail | Cutover planning | Data Governance | Data ManagementSenior-level Full TimeRemote, India R19h ago
-
Senior-level Full TimeMumbai, Maharashtra, India20h ago
-
Data Analyst Customer Success Engineer INR 2500K-4064KA/B | A/B Testing | AI workflows | API Integration | AmplitudeClient Customer Interactions | Health insurance | Learning aids | Meal vouchersSenior-level Full TimeHYDERABAD, Telangana, India20h ago
-
AI Platform Engineer INR 3700K-5500KAPI Design | Agent Orchestration | Agent systems | Artificial Intelligence | CI/CDSenior-level Full TimeMumbai, India21h ago
-
Senior-level Full TimeMumbai, India21h ago
-
Deputy Vice President - Data Scientist INR 3300K-4000KAmazon Web Services | Apache Spark | Data Visualization | Gradient boosting | HadoopAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive cultureExecutive-level Full TimeGurugram, India21h ago