Lead Software Engineer, Data Engineering
Tasks
- Architect data ingestion pipelines
- Automate data workflow operations
- Build ML ready datasets
- Build data lakehouse architectures
- Collaborate with cross-functional teams
- Deploy ML models in production
- Deploy data APIs
- Design data transformation pipelines
- Develop ELT pipelines
- Develop ETL pipelines
- Develop data orchestration pipelines
- Implement data governance frameworks
- Implement data lineage frameworks
- Implement data quality frameworks
- Integrate streaming data pipelines
- Integrate structured data sources
- Integrate unstructured data sources
- Mentor junior engineers
- Monitor data workflows
- Operationalize data pipelines
- Serve datasets for model training
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Apache Superset | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure DevOps | Azure Event | Azure Event Hubs | Azure Machine Learning | Azure Service | Azure Service Bus | Azure Synapse | Azure Synapse Analytics | C# | CI/CD | Cosmos DB | DBT | Data Factory | Data Lake | Data Modeling | Databricks | Delta Lake | Docker | ELT | ETL | Event Hubs | Git | GitHub Actions | Java | Kafka | Kubernetes | MLOps | Machine Learning | MongoDB | MySQL | PostgreSQL | Power BI | PyTorch | Python | SQL | SQL Server | Scala | Service Bus | Synapse Analytics | Tableau | TensorFlow
Education
N/A
Related jobs
-
Software Engineering Professional INR 1500K-2500KAI orchestration | AIOps | APIs | AWS | Agent FrameworksMid-level Full TimeBengaluru, IN, 5601032h ago
-
Software Engineering Professional INR 1500K-2500KAI orchestration | AIOps | API Integration | AWS | Agent FrameworksMid-level Full TimeBengaluru, IN, 5601032h ago
-
Software Engineer INR 2156K-2894KAWS ECS | AWS Lambda | Amazon ECR | Amazon S3 | Amazon SNSDevelopment opportunities | Reward and recognition | Volunteering | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India4h ago
-
Software Engineer - Associate INR 1500K-3000KAWS | AWS Lambda | Amazon ECR | Amazon ECS | Amazon S3Development opportunities | Reward & recognition | Volunteering opportunities | Wellness programs | Work-life balanceMid-level Full TimeHyderabad, India4h ago
-
Staff Engineer (Python/AWS) INR 2500K-3465KAPI Gateway | AWS | AWS CDK | Aurora MySQL | Aurora PostgresDEI focus | Development opportunities | Reward and recognition | Volunteering | WellnessSenior-level Full TimeHyderabad, India4h ago
-
Senior-level Full TimeBengaluru, KA, India6h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Apache Spark | CI/CD | Data Modeling | Data QualityBike to work | Educational assistance | Employee assistance program | Employee discount | Fitness classesSenior-level Full TimePune, MH, India6h ago
-
Senior Software Engineer - SAST INR 2475K-3465KCompiler programming | Integration Testing | Java | Kotlin | Parser generatorSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Staff Software Engineer (Python) INR 2000K-4000KAPIs | AWS | Authentication | Batch Processing | Cause analysisCollaborative team environment | Leadership development | Training programsSenior-level Full TimeMumbai, MH, India6h ago
-
Software Engineer II INR 2000K-2525KAWS | CI/CD | Data Structures | Docker | EC2Professional development | Training coursesMid-level Full TimeMumbai, India7h ago
-
Generative AI Developer INR 1400K-1700KCloud Functions | Cloud Storage | Cloud platform | Compute Engine | GeminiDigital learning platform access | Elder care | Flexible work | Health checks | Insurance with top-upsMid-level Full TimeBangalore, IN9h ago
-
AI Engineer INR 2000K-4000KAWS Bedrock | AWS Bedrock Agents | AWS Bedrock Guardrails | AWS VPC | Agile methodologiesSenior-level Full TimeIndia Office11h ago
-
Tech Lead AI Engineer INR 1800K-3000KAI Evaluation | AWS Bedrock | AWS Lambda | AWS SageMaker | Agile methodologySenior-level Full TimeIndia Office11h ago
-
Sr Software Engineer – AI Ops INR 2000K-3440KAPI Gateway | AWS Lambda | Amazon API | Amazon API Gateway | Amazon CloudWatchExtended hours flexibilitySenior-level Full TimeIndia Office11h ago
-
Data Engineer INR 2000K-3000KApache Kafka | Apache Spark | Cloud Computing | Data Modeling | Data WarehousingFlexible working | Hybrid workingSenior-level Full TimeHyderabad Knowledge Park Tower 2, India11h ago
-
ANSI SQL | AWS | Access Control | Apache Airflow | Apache OozieSenior-level Full TimePune - Hinjewadi (Ascendas), India11h ago
-
Cloud data engineer INR 2800K-4000KAPI Integration | AWS Glue | AWS Lambda | AWS Step Functions | Amazon RedshiftSenior-level Full TimePune/Mumbai, MH, IN11h ago
-
Principal, Disaster Recovery Engineer INR 3000K-4200KAWS | Air-gapped | Air-gapped environments | Azure | BashSenior-level Full TimePune, India11h ago
-
Staff Data Engineer INR 2400K-3380KAWS Cloud | AWS Glue | Agile | Amazon Athena | Amazon CloudWatchSenior-level Full TimeBangalore - Manyata Tech Park Road, …11h ago
-
ETL Pentaho Developer, AS INR 2200K-4000KData Marts | Data Modeling | Data Quality | Data Warehousing | Data integrationAccident insurance | Childcare reimbursement | Employee assistance program | Flexible leave policy | Gender neutral parental leaveEntry-level Full TimeBangalore, Velankani Tech Park, India11h ago
-
Software Engineering Manager, Data Platform INR 933K-1141KBackend Development | Data Engineering | Data Ingestion | Data Pipelines | Data ProcessingHybrid workMid-level Full TimeBangalore14h ago
-
App Engineer (AI & Automation) INR 2000K-3465KAI Assisted Development | AI-Assisted Development Tools | AI-powered automation | API Integration | Claude CodeMid-level Full TimeIndia16h ago
-
Senior-level Full TimeBengaluru, India18h ago
-
HR Tech Data Engineer INR 3000K-4000KAPI Integration | AWS | Access Control | Active Directory | Anthropic APICollaboration culture | Flexible learning opportunities | Inclusive work environment | Training and supportSenior-level Full TimeIndia (Hyderabad)19h ago
-
Data Engineer INR 1500K-2525KAPI Integration | Apache Airflow | Azure Data | Azure Data Factory | Azure SynapseMid-level Full TimeIndia - Hybrid R20h ago