Senior Data Engineer
Tasks
- Analyze pipelines and improve reliability scalability
- Build real-time data pipelines with Kafka
- Build scalable data ingestion pipelines
- Collaborate on data quality dashboards and reporting
- Design lakehouse architectures with Apache Iceberg
- Develop Apache Spark batch and streaming jobs
- Establish observability with monitoring logging and alerting
- Implement data quality checks and error monitoring
- Implement versioning time travel incremental processing and ACID compliance
- Optimize Iceberg table design
- Orchestrate workflows with DBT and Airflow
- Perform code reviews and apply engineering best practices
- Use AI tools to accelerate ETL and ELT development
- Use AWS services for cost and performance optimization
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS Lambda | AWS Step Functions | Alerting | Amazon EMR | Amazon S3 | Amazon Web Services | Apache Airflow | Apache Iceberg | Apache Kafka | Apache Spark | CI/CD | DBT | Data Quality | Data Validation | Kafka Streams | Logging | Monitoring | Python | SQL | Snowflake | Step Functions | Web Services
Education
Roles
Related jobs
-
Senior-level Full TimeHyderabad, India5h ago
-
Sr. Data Engineer INR 2800K-4000KCloud warehousing | DAX | Dashboard Design | Data Quality | Data StorytellingCommunity outreach programs | Employee Resource Group | Employee stock purchase program | Health and wellness incentivesSenior-level Full TimeBangalore, KA, IN, 5600376h ago
-
Software Engineer INR 1000K-1496KBitbucket | Data Architecture | Data Modeling | Data Pipelines | LinuxEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India6h ago
-
Data Scientist- GenAI Engineer INR 2000K-3000KA/B | A/B Testing | B testing | Causal Inference | ClusteringMid-level Full TimeNoida, Uttar Pradesh, India6h ago
-
GIS Data Engineer II INR 2000K-2400KAWS | Amazon Athena | Amazon EC2 | Amazon RDS | Amazon S3MentorshipSenior-level Full TimeMumbai, MH, India8h ago
-
C# | C++ | Go | Infrastructure automation | JavaSenior-level Full TimeBengaluru, Karnataka, IND12h ago
-
Amazon S3 | C# | C++ | Data Migration | Data replicationSenior-level Full TimeBengaluru, Karnataka, IND12h ago
-
Software Engineer - Embedded & Functional Safety Software INR 1753K-3440KAgile | Automated testing | C# | C++ | CAN busSenior-level Full TimeBangalore, IN12h ago
-
Software Engineer - Robot Navigation, Localization & SLAM INR 1753K-2372K3D camera | Agile | Automated testing | C++ | CI/CDMid-level Full TimeBangalore, IN12h ago
-
Software Engineer - Robot Connectivity & Wi Fi Systems INR 1000K-2000KAgile | Automated testing | CI/CD | GRPC | GitCustomer-facing workSenior-level Full TimeBangalore, IN12h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agent/tool use | Apache Spark | Azure | ContainersContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India12h ago
-
Apache Spark | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeNoida, India12h ago
-
IN_Manager_SSIS_Data & Analytics_Advisory_Noida INR 1500K-2000KAWS | Azure | CI/CD | Cloud Data | Cloud Data PlatformsFlexibility programmes | Inclusive benefits | MentorshipMid-level Full TimeNoida, India12h ago
-
EY - GDS Consulting - AIA - Gen AI - Staff INR 3334K-5000KAWS | AWS Bedrock | Azure OpenAI | CD | CISenior-level Full TimePune, MH, IN, 41101412h ago
-
EY - GDS Consulting - AI And DATA -Snowflake Architect -Manager INR 3000K-4000KAgile | Batch Processing | Cloud Data | Cloud Data Warehouse | DBTCareer development | Coaching and feedback | Flexible work arrangementSenior-level Full TimeHyderabad, TG, IN, 50008112h ago
-
Senior Executive - Information Technology INR 2000K-4225KAdvanced Analytics | Bagging | Boosting | Decision Trees | Machine LearningSenior-level Full TimeAiroli, MH, IN12h ago
-
AI Software Engineer INR 1753K-2600KAWS | Agentic AI | Azure | CI/CD | Code reviewContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offEntry-level Full TimeIN - HYDERABAD ORION, India12h ago
-
EY - GDS Consulting - AIA - Gen AI - Senior INR 3000K-5000KAWS | Amazon Bedrock | Azure OpenAI | DynamoDB | EmbeddingsSenior-level Full TimePune, MH, IN, 41101412h ago
-
Software Engineer INR 1400K-2000KAgile | Cloud services | Code review | Data Modeling | Database DesignMid-level Full TimePune, India20h ago
-
Amazon Web Services | Apache Airflow | Apache Kafka | Apache Spark | Cloud platformContract employment | Hybrid work | VIRTUAL interviewSenior-level ContractBangalore North, India22h ago
-
Data Engineer Architect. INR 1500K-2400KAPI Development | AWS | Agentic Engineering | Database optimization | FastAPISenior-level Full TimeIN-MH-Pune23h ago
-
AIML - Tech Lead INR 2500K-4500KApache Airflow | ETL | Entity recognition | Language Processing | Machine LearningSenior-level Full TimeIN-MH-Pune23h ago
-
Senior GenAI Developer INR 2500K-4800KAPI Gateway | AWS Bedrock | Agent systems | Agentic RAG | Amazon LambdaCertification programs | Global client projects | Health insurance | Internship opportunities | Language coursesSenior-level Full TimeAll Cities, India R1d ago
-
Senior-level Full TimePune, Maharashtra, India1d ago
-
Software Engineer II - Gen AI/ Agentic AI INR 1500K-2500KAWS | Amazon Elastic Kubernetes Service | Amazon SageMaker | Data Processing | Data integrationMid-level Full TimeHyderabad, Telangana, India1d ago