Principal Engineer (Big Data Engineer)
Bengaluru, KA, India
R
INR 2800K-4500K (estimate) Senior-level Full Time
Tasks
- Build data pipelines
- Compare technologies for best fit
- Create architecture and design documents
- Define non-functional requirements
- Define process flows
- Design complex data mappings
- Design message broker architecture
- Design system architecture
- Develop ETL to consumption flows
- Develop solution for functional and non functional requirements
- Gather and analyze requirements
- Implement event sourcing
- Lead technical team
- Perform performance tuning
- Resolve issues with root cause analysis
- Review architecture for scalability security and extensibility
- Troubleshoot complex bugs
- Write and review code
Perks/Benefits
Skills/Tech-stack
Amazon Web Services | Apache HBase | Apache Spark | Cassandra | Cloud platform | Data Mapping | Data pipeline | Design Patterns | ETL | Event Sourcing | Extensibility | Functional Requirements | Google Cloud | Google Cloud Platform | Java | MS SQL | Message Broker | Microsoft Azure | Non-functional Requirements | Non-functional) | Performance Tuning | PySpark | Scala | Scalability | Security | System design | Troubleshooting | Web Services
Education
N/A
Related jobs
-
GCP Data Engineer / Consultant Specialist INR 1500K-2000KAirflow | Alerting | Apache Beam | Automation | BigQueryFlexible working | Inclusive workplace | Opportunities for growth | Professional developmentMid-level Full TimePune, Maharashtra, India R11h ago
-
Senior-level Full TimeIndia - Remote R1d ago
-
API Integration | Backend Development | Frontend Frameworks | Generative AI | Google CloudFast paced product culture | Fully remote work | High ownership | Shift-based scheduleSenior-level Full TimeIndia R1d ago
-
Partner AI Deployment Engineer - AWS INR 2000K-3300KAWS Lambda | Amazon EC2 | Amazon Web Services | Enterprise Architecture | Generative AISenior-level Full TimeIndia - Remote R1d ago
-
Manager, Data And Analytics INR 700K-1200KAWS Redshift | Anomaly Detection | Azure | BigQuery | Business IntelligenceMid-level Full TimeBangalore, Karnataka, India R1d ago
-
Machine Learning Engineer INR 1000K-1800KAPI Development | AWS | Amazon Bedrock | Amazon EKS | Amazon SageMakerMid-level Full TimeBangalore,Karnataka,India R1d ago
-
Financial analytics | Java | Project Management | Python | RFlexible time off | Healthcare | Retirement plan | Support for working parents | Tuition reimbursementSenior-level Full TimeM11 - Commers III, 43, 44 … R2d ago
-
Senior QA Engineer INR 1500K-2500KAPI Testing | Apache Avro | Apache Kafka | CI/CD | Contract testingRemote workSenior-level Full TimeIndia - Remote R2d ago
-
Software Engineering Senior Analyst - HIH - Evernorth INR 2500K-3380KAWS | Access Control | Apache Spark | Audit Logging | CI/CDSenior-level Full TimeHIH - Hyderabad, India R2d ago
-
AI Data Engineer INR 3000K-3880KAPIs | AWS | Agile | Customer Data | Customer Data PlatformEmployee incentive programs | Flexible work policies | Headspace app access | Hybrid work model | Mental health daysMid-level Full TimeIndia, Hyderabad, Telangana R2d ago
-
Software development engineer 2 INR 1000K-2200KAgile methodology | Apache Spark | CI/CD | Data pipeline | Data pipeline designHybrid work modelMid-level Full TimeBangalore, India R2d ago
-
Sr. Engineer - Data & ML Platform (Hybrid, IND) INR 2800K-4000KAirflow | Apache Flink | Apache Spark | Blob Storage | CI/CDEmployee networks | Paid adoption leave | Paid parental leave | Professional development opportunities | Vacation and holidaysSenior-level Full TimeBangalore, India R2d ago
-
Sr. Engineer - Data & ML Platform (Hybrid, IND) INR 2800K-4000KAirflow | Apache Flink | Apache Iceberg | Apache Spark | Blob StorageCompetitive vacation and holidays | Employee volunteer opportunities | Hybrid work | Paid adoption leave | Paid parental leaveSenior-level Full TimeBangalore, India R2d ago
-
.NET | API Management | Agile | Angular | AuthenticationCollaborative cross-functional team | Fully remote within India | Hands on GenAI projects | Innovation-focused environmentSenior-level Full TimeIndia R4d ago
-
Agile | Airflow | Azure | Azure Data | Azure Data FactoryCareer development | Fully remote | Inclusive collaborative environment | Long-term contract | MentorshipSenior-level Full TimeIndia R4d ago
-
Airflow | Apache Superset | Business Intelligence | Cloud platform | Complex joinsAnnual vacation | Conference support | English learning discounts | Fully remote | Health reimbursementSenior-level Full TimeIndia R4d ago
-
AWS | Azure | CI/CD | CircleCI | CloudFormationCareer growth | Collaborative innovation driven engineering culture | Continuous learning | Fully remote friendly work environment | MentorshipSenior-level Full TimeIndia R4d ago
-
AWS | Azure | CD pipelines | CI/CD | CI/CD pipelinesCareer growth | Flexible engagement | Fully remoteSenior-level Full TimeIndia R4d ago
-
Data Governance | Data Modeling | Data Quality | Data Security | Data WarehousingCareer growth | Collaborative environment | Continuous learning | Flexible day shift | Fully remote workSenior-level Full TimeIndia R4d ago
-
Sr. Data Engineer INR 2800K-4000KAgile | Apache Airflow | Apache Hive | Apache Spark | Batch ProcessingAnnual leave | Contributory pension scheme | Diversity forums | Hybrid work | Mindfulness app accessSenior-level Full TimePune, India R5d ago
-
Data Engineering Operations Lead INR 3000K-4000KAgile | Apache Airflow | Apache Hive | Apache Spark | BigQueryAnnual leave | Away from keyboard time | Contributory pension scheme | Diversity forums and networking groups | Fitness Wellbeing ProgrammesSenior-level Full TimePune, India R5d ago
-
GenAI Engineering INR 2200K-3500KAWS | Azure | Bias detection | Continuous Deployment | Data PrivacyHybrid work schedule | Learning and development programs | MentoringSenior-level Full TimeBengaluru - GTP, India R5d ago
-
Senior Data Engineer INR 2520K-4000KAirflow | Batch Processing | Bitbucket | CICD | Data LakeHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R5d ago
-
Senior Data Engineer INR 2520K-4000KApache Airflow | Batch Processing | Bitbucket | CI/CD | Data LakePaid volunteer days | Work from home flexibilitySenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R5d ago
-
Senior Data Engineer INR 2520K-4000KAmazon DynamoDB | Amazon Redshift | Apache Airflow | Apache Spark | Batch ProcessingHealth and wellbeing resources | Paid volunteer daysSenior-level Full TimeBengaluru - India - Bengaluru, 560071 … R5d ago