Software Development Engineer II - Data
Tasks
- Architect and implement high scale data pipelines
- Build data lakehouse solutions
- Deploy scalable secure data infrastructure on AWS
- Design data models for structured and semi structured data
- Develop streaming and batch processing pipelines
- Implement data quality checks and observability
- Optimize pipeline performance and reliability
- Orchestrate ELT workflows with Airflow
Perks/Benefits
Skills/Tech-stack
AWS ECS | AWS Lambda | Apache Airflow | Apache Flink | Apache Iceberg | Apache Pinot | Apache Spark | Automated testing | CDC | CI/CD | Data Governance | Data Lineage | Data Modeling | Data Observability | Docker | Druid | Go | Infrastructure as Code | Kafka | Kinesis | Kubernetes | Node.js | Parquet | Partitioning | Pulsar | Python | Query Optimization | Real Time | Real-time analytics | Rockset | S3 | SQL | Scala | Schema Design | Snowflake | Streaming Data | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Senior-level Full TimeIN-HR-Gurgaon8h ago
-
Cloud Data Engineer INR 1500K-2500KAnsible | Apache Hadoop | Apache Spark | CI/CD | Cloud platformTravel opportunities up to 30 percentMid-level Full TimeBengaluru, Karnataka, India; Hyderabad, Telangana, India10h ago
-
Azure DevOps | Bash | CI/CD | Centralized Logging | Distributed tracingLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India14h ago
-
Azure DevOps | Bash | CI/CD | Distributed tracing | DockerLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India14h ago
-
Azure DevOps | Bash | CI/CD | Distributed tracing | DockerLearning culture | MentorshipMid-level Full TimeBangalore, Karnataka, India14h ago
-
Senior Associate Developer INR 1500K-2000KAWS EMR | AWS Glue | AWS Lambda | AWS S3 | Amazon RedshiftMid-level Full TimeMumbai, Maharashtra, India17h ago
-
Software Engineering LMTS - Hyderabad (Java + Spark + AI/ML) INR 2800K-4500KAWS | Agile | Apache Iceberg | Apache Spark | AzureCross-team collaboration | Hybrid work environment | Mentorship | Technical roadmapping leadershipSenior-level Full TimeIndia - Hyderabad21h ago
-
Data & AI Scientist INR 3000K-5000KABAC | Airflow | Audit Logging | Azure DevOps | Azure MLFlexible working options | Hybrid workingSenior-level Full TimeHyderabad Knowledge Park Tower 2, India21h ago
-
Lead Azure Data Engineer INR 500K-2000KAutomation | Azure Data | Azure Data Factory | Azure Synapse | Azure Synapse AnalyticsMid-level Full TimeBengaluru, Karnataka, India21h ago
-
Senior Software Engineer INR 3000K-6000KAPI | Asynchronous processing | CI/CD | Distributed Systems | DockerMid-level Full TimeBengaluru, Karnataka, India21h ago
-
Data Engineer (Python, Spark) INR 1500K-2000KAWS Glue | Airflow | Amazon Web Services | Apache Flink | Apache HiveMid-level Full TimeAPAC - India - Bengaluru - …21h ago
-
Senior Engineer Digital & AI Solutions INR 3000K-5000KAPI Development | Agent systems | CI/CD | Cloud Computing | ContainerizationFood allowance | Gratuity | Group term life insurance | Internet allowance | Medical insuranceSenior-level Full TimeHyderabad, Telengana, India1d ago
-
Staff Machine Learning Engineer INR 3000K-4500KAPIs | Agentic Workflows | Airflow | Anomaly Detection | Bias detectionAgile work environment | Collaborative culture | In-office three days weeklySenior-level Full TimeBangalore R1d ago
-
AI Observability | AWS | Azure | CI/CD | DockerCareer advancement | Flexible work environment | Fully remote | Professional development | Work-life balanceSenior-level Full TimeIndia R1d ago
-
Senior-level Full TimeIndia1d ago
-
API Testing | Apache Airflow | Azure | Bitbucket | CI/CDEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Senior-level Full TimeIN-TN-Chennai1d ago
-
Data Integration Engineer INR 1500K-2500KData Pipeline Testing | Data Validation | Data pipeline | ELT | ETLSenior-level Full TimeIndia1d ago
-
Senior-level Full TimeIndia - Bengaluru; India - Chennai …1d ago
-
AI Software Engineer - Senior INR 2500K-4800KAWS | Agent Orchestration | Agile | Autogen | Automated testingSenior-level Full TimePune, Maharashtra, India1d ago
-
AI Software Engineer - Senior INR 2500K-5000KAI Governance | AI orchestration | APIs | Agent systems | AgileSenior-level Full TimePune, Maharashtra, India1d ago
-
Senior Data Engineer INR 1500K-2500KAlerting | Apache Pinot | BigQuery | Cloud platform | Cost OptimizationSenior-level Full TimeBangalore, India1d ago
-
Senior Manager - C2 INR 3000K-5000KAI Observability | Docker | Embedding Models | Evaluation | Generative AISenior-level Full TimeGurugram, Haryana, India1d ago
-
Java Full Stack Engineer - Professional INR 1300K-2200KAmazon Web Services | Apache Kafka | AppDynamics | Automation | Cause analysisClient office work requirement | Hybrid work modelMid-level Full TimeIndia R1d ago
-
Executive-level Full TimeMumbai, Maharashtra, India1d ago