Lead Software Engineer, Data Engineering
Tasks
- Architect data ingestion pipelines
- Automate data workflow operations
- Build ML ready datasets
- Build data lakehouse architectures
- Collaborate with cross-functional teams
- Deploy ML models in production
- Deploy data APIs
- Design data transformation pipelines
- Develop ELT pipelines
- Develop ETL pipelines
- Develop data orchestration pipelines
- Implement data governance frameworks
- Implement data lineage frameworks
- Implement data quality frameworks
- Integrate streaming data pipelines
- Integrate structured data sources
- Integrate unstructured data sources
- Mentor junior engineers
- Monitor data workflows
- Operationalize data pipelines
- Serve datasets for model training
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Spark | Apache Superset | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure DevOps | Azure Event | Azure Event Hubs | Azure Machine Learning | Azure Service | Azure Service Bus | Azure Synapse | Azure Synapse Analytics | C# | CI/CD | Cosmos DB | DBT | Data Factory | Data Lake | Data Modeling | Databricks | Delta Lake | Docker | ELT | ETL | Event Hubs | Git | GitHub Actions | Java | Kafka | Kubernetes | MLOps | Machine Learning | MongoDB | MySQL | PostgreSQL | Power BI | PyTorch | Python | SQL | SQL Server | Scala | Service Bus | Synapse Analytics | Tableau | TensorFlow
Education
N/A
Related jobs
-
Staff Machine Learning Engineer L5 INR 2520K-3500KAWS | Algorithms | Apache Airflow | Azure | Data StructuresSenior-level Full TimeGurugram, India7h ago
-
Principal AI Engineer INR 2000K-4500KAPI Integration | AWS Bedrock | Agent Orchestration | CI/CD | Deep learningSenior-level Full TimeBangalore, KA, IN8h ago
-
Staff LiDAR Algorithms Engineer INR 2500K-4500K3D Geometry | ASPICE | C++ | Calibration | Cloud processingSenior-level Full TimeBangalore15h ago
-
Mid-level Full TimeBengaluru, KA, India17h ago
-
AI Engineer / AI Developer INR 1800K-3300KAI Agents | Alerting | Amazon SQS | Async Programming | CI/CDSenior-level Full TimeBangalore, India Office (BANGALORE) R17h ago
-
Mid-level Full TimeBangalore, India Office (BANGALORE) R18h ago
-
AI Engineer – Python & Snowflake INR 1800K-2700KAPI Development | AWS | Airflow | Azure | Data ModelingMid-level Full TimeHyderabad, Hyderabad, IN18h ago
-
Senior AI Engineer (Full-Stack) (Remote) INR 2500K-5000KAWS | Active Directory | Anthropic | ArcGIS REST | ArcGIS REST APIClient facing ownership | Remote workSenior-level Full TimeMaharashtra, Pune, India R18h ago
-
Staff Artificial Intelligence & Machine Learning Engineer INR 2500K-4500KAPIs | Agentic Workflows | Chunking | Compliance | Cost OptimizationSenior-level Full TimeBengaluru, KA, India18h ago
-
Senior Database Engineer - #3180 (Remote) INR 1500K-2700KAWS CDK | AWS IAM | AWS KMS | Amazon Aurora | Amazon CloudFormationSenior-level Full TimeChennai, India R18h ago
-
Lead AI Software Engineer INR 2000K-4500KAI Agents | AWS | Azure | CI/CD | Cloud platformCareer growth | Career growth in technology domain | Continuous learningSenior-level Full TimePune City, India19h ago
-
GCP Data Engineer INR 1500K-2500KAirflow | BigQuery | Cloud Composer | Cloud Functions | Cloud platformSenior-level Full TimeIndia - Pune19h ago
-
Entry-level Full TimeCoimbatore, India19h ago
-
Senior-level Full TimeHyderabad, India19h ago
-
Entry-level Full TimeCoimbatore, India19h ago
-
Senior-level Full TimeIN-HR-Gurgaon19h ago
-
Senior-level Full TimeIN-KA-Bangalore19h ago
-
Senior Engineer - Embedded QA INR 1500K-2094KAgile | CI/CD | Continuous Deployment | Continuous integration | Data Driven TestingSenior-level Full TimeHadapsar, Pune, Maharashtra, IND, 411013 R20h ago
-
Senior-level Full TimeBengaluru, Karnataka, India21h ago
-
AI Engineer - AI, Data & Platforms INR 3000K-4200KAgentic Workflows | Apache Airflow | Apache Spark | DBT | Dagster4 day in office 1 day flexible work arrangement | Flexible work arrangementSenior-level Full TimeGurugram21h ago
-
Staff Software Engineer, Play Games Discovery INR 2500K-4500KData Pipelines | Data Processing | Data Structures | Data Structures and Algorithms | Distributed ComputingSenior-level Full TimeBengaluru, Karnataka, India21h ago
-
Senior Software Engineer (AI,Java/Python) INR 2475K-3500KAPI Development | AWS | Agent Frameworks | Agent systems | Amazon SQSFamily wellbeing benefits | Health benefits | Paid time off | Wellbeing benefits | Wellness supportSenior-level Full TimeHyderabad, India22h ago
-
Access Control | Autogen | CI/CD | Chroma | Continuous MonitoringEntry-level Full TimeHyderabad, Telangana, India22h ago
-
Senior-level Full TimeIndia22h ago
-
DevOps Track Sr.Engineer INR 2229K-4000KAPI Gateway | APM | AWS CloudWatch | AWS Systems Manager | AWS systemsSenior-level Full TimeIndia22h ago