Lead Software Engineer - Data Engineering
Chennai, Tamil Nadu, India
INR 3417K-4580K (estimate) Senior-level Full Time
Tasks
- Apply software engineering principles and testing
- Architect AWS data platforms
- Automate workflows and continuous improvement
- Build data ingestion pipelines for batch and near real time
- Build end to end pipelines with AWS services
- Collaborate with product managers and stakeholders
- Define CI/CD standards
- Design data ingestion frameworks
- Drive agile execution
- Ensure system scalability and resiliency
- Establish engineering best practices
- Implement CI/CD pipelines with Jenkins or Azure DevOps
- Implement data modeling governance and metadata management
- Ingest structured and unstructured data sources
- Lead and mentor data engineering teams
- Operationalize ML models with analytics workflows
- Optimize Snowflake data platforms
- Translate business needs into data solutions
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS CloudFormation | Agile | Azure DevOps | CI/CD | Data Governance | Data Ingestion | Data Modeling | Distributed Systems | ELT | EMR | ETL | Git | GitHub | Glue | Jenkins | Kafka | Lambda | Metadata Management | Python | Redshift | S3 | SQL | Snowflake | Spark Streaming
Education
Related jobs
-
Data Engineer II INR 2040K-3380KAWS CloudFormation | Amazon Kinesis | Amazon Redshift | Amazon Web Services | Apache AirflowComplimentary games | Family leave | Healthcare coverage | Mental well-being support | Paid time offSenior-level Full TimeLocations: Hyderabad, Telangana, India3h ago
-
Data Engineer II INR 2040K-3380KAWS | Amazon Kinesis | Amazon Redshift | Apache Airflow | Apache BeamComplimentary games | Family leave | Healthcare coverage | Hybrid work model | Mental well-being supportSenior-level Full TimeLocations: Hyderabad, Telangana, India3h ago
-
Data Engineer II INR 2200K-3465KAmazon Kinesis | Amazon Redshift | Amazon Web Services | Apache Airflow | Apache BeamFree games | Healthcare benefits | Hybrid work schedule | Mental health support | Paid time offMid-level Full Time地点:Hyderabad, Telangana, India3h ago
-
Data Engineer II INR 2200K-3465KAWS | Amazon Kinesis | Amazon Redshift | Apache Airflow | Apache BeamFamily leave | Free games | Health insurance | Hybrid work | Mental health supportMid-level Full Time地点:Hyderabad, Telangana, India3h ago
-
Senior Data Engineer INR 1500K-2500KAWS Glue | AWS Lambda | Amazon EMR | Amazon S3 | Apache AirflowSenior-level Full TimeSaidapet3h ago
-
Senior-level Full TimeHyderabad, India3h ago
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
ETL Data with QA Tester INR 1500K-2500KDBT | Data Aggregation | Data Quality | Data Transformation | Data ValidationRemote workSenior-level Full TimeIN-AP-Hyderabad4h ago
-
Agentic AI | Alembic | Autogen | Azure AI | Azure OpenAIEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India4h ago
-
Agile | Airflow | Azure | Azure Data | Azure Data FactoryEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time off | Wellness plansMid-level Full TimeChennai, TN, India5h ago
-
Software Engineer III - Python Developer + LLM / Gen AI INR 2500K-5000KAWS | Agile | Application Resiliency | CI/CD | Cloud ComputingSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Sr. Expert INR 2516K-3356KAlgorithms | Cloud Computing | Data Aggregation | Data Engineering | Data IngestionEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeVadodara, GJ, India6h ago
-
API Development | Amazon Redshift | Amazon Web Services | Apache Spark | AzureSenior-level Full TimeBangalore, Karnataka, India7h ago
-
Manager - AI Application Development INR 2500K-5000KAI Foundry | Agent systems | Arize Phoenix | Autogen | Azure AISenior-level Full TimeGurgaon, Haryana, India7h ago
-
Lead ML Engineer INR 2500K-5000KAgile | Autogen | Automl | Azure | CI/CDEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India8h ago
-
Data & Analytics Engineer - AM INR 1500K-2500KApache Spark | Data Modeling | Data Warehousing | Databricks | Delta LakeSenior-level Full TimeBangalore, Karnataka, India8h ago
-
Support Engineer – Developer Portal INR 600K-700KAI Automation | API Integration | Agentic AI | Automation Anywhere | ExcelAccommodation support | Equal opportunity employment | Inclusive cultureEntry-level Full TimeHyderabad, India9h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Amazon Aurora | Apache Airflow | Apache Kafka | Apache SparkAccreditations | Bike to work | Career development | Educational assistance | Employee assistance programSenior-level Full TimeMumbai, MH, India9h ago
-
Data Engineer INR 3000K-4000KAWS | Cloud Computing | Data Architecture | Data Governance | Data ManagementAccreditation support | Bike to work | Career development program | Educational assistance | Employee assistance programMid-level Full TimeBengaluru, KA, India10h ago
-
Mid-level Full TimeNoida, Uttar Pradesh, India10h ago
-
Sr. AI Engineer - Data Pipelines & Context Systems INR 4000K-5449KABAC | API Design | Asynchronous programming | Auditability | ChunkingFlexible work arrangementsSenior-level Full TimeBangalore10h ago
-
Algorithms | Apache Airflow | Argo Workflows | Azure | Azure ComputeCollaborative team environment | Flexible work environment | Opportunity to innovate | Opportunity to work on impactful projectsSenior-level Full TimeIndia11h ago
-
Manager, GenAI Full Stack Developer INR 1800K-3000KAPI Gateway | AWS | Azure | CI/CD | Context ManagementMid-level Full TimeGurugram, Haryana, India11h ago
-
Manager, RAG/LLM Specialist INR 2000K-3000KEmbedding Models | Hybrid search | Langchain | Llamaindex | LoRAMid-level Full TimeGurugram, Haryana, India11h ago
-
Deputy Manager INR 1500K-2700KAirflow | Apache Beam | Apache Kafka | App Engine | Artifact RegistryMid-level Full TimeChennai, Tamil Nadu, India11h ago