Data Engineer
Chennai, Tamil Nadu, India
INR 1500K-2600K (estimate) Mid-level Full Time
Tasks
- Architect AI ready data platform components for ML consumption
- Architect scalable cost effective data platforms
- Build and maintain feature stores on Vertex AI for low latency serving
- Build pipelines for embedding generation and vector store population and RAG retrieval
- Define SLAs implement alerting and lead incident resolution
- Define reusable engineering patterns pipeline templates transformation standards and naming conventions
- Design CI CD pipelines with automated testing and deployment gates
- Design data models using dimensional and normalization and data vault
- Drive DataOps practices with automated testing data contracts and observability
- Ensure data security encryption and governance compliance
- Execute testing strategies for regulated workloads with parallel validation
- Implement SCD strategies and historical data management
- Lead code reviews and improve quality testability and maintainability
- Lead end to end design and delivery of data engineering solutions on GCP
- Mentor junior engineers through pairing and feedback
- Migrate legacy data structures to GCP with validation
- Monitor and optimize GCP costs and implement efficiency improvements
- Provision and govern cloud infrastructure with Terraform
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | BigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud Composer Airflow | Cloud Storage | Cloud platform | Composer (Airflow) | DBT | Data Contracts | Data Governance | Data Modeling | Data Vault | DataOps | Dataflow | Dataform | Dataplex | Dimensional Modeling | Embeddings | Encryption | Feature Store | GitHub Copilot | Google Cloud | Google Cloud Platform | Google Gemini | Kimball | LLM-Ops | Observability | PubSub | Python | RAG | SCD | SQL | Terraform | Third Normal Form | Vector Databases | Vertex AI
Education
N/A
Roles
Related jobs
-
Senior-level Full TimeBANGALORE, Karnataka, Inde3h ago
-
Advanced Engineer-AI/ML INR 3264K-4800KAPI Integration | ASR | AUTOSAR | AWS | Audio ProcessingEmployee discounts | Recognition rewards program | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …4h ago
-
Senior Software Engineer - Data License Analytics, Pune INR 3000K-4720KAWS S3 | Agent to Agent | Agent to Agent A2A | Apache Airflow | Apache IcebergSenior-level Full TimePune4h ago
-
AI Lead and Automation Tester - Document Intelligence INR 2500K-5000KAI Workflow Orchestration | AI workflow | Automation Testing | Clinical Trial Compliance | Clinical trialSenior-level Full TimeIN-TN-Chennai4h ago
-
Senior-level Full TimeBangalore - Karnataka - India5h ago
-
Lead Data Platform Engineer INR 2040K-3900KAPI Integration | AWS | AWS CloudWatch | AWS DataZone | AWS GlueEnhanced medical benefits | Family well-being benefits | Hybrid work | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India6h ago
-
Anomaly Detection | Autoencoders | Bayesian optimization | Classification | ClusteringOnsite workEntry-level Full TimeHyderabad, India6h ago
-
AI Agent Development Kit | API Gateway | Agent Development Kit | Agentic AI | Audit LoggingSenior-level Full TimeMumbai, Maharashtra, India6h ago
-
AI/ML Infrastructure Software Engineer INR 3000K-5000KC++ | JAX | Keras | Machine Learning | Machine Learning FairnessSenior-level Full TimeBengaluru, Karnataka, India6h ago
-
Data Engineer INR 1300K-2000KAPI Integration | Data Modeling | Data Transformation | Data pipeline | Power BIMid-level Full TimeHyderabad, Telangana, India6h ago
-
Senior QA Engineer (Data Platform Testing) INR 2500K-3000KAPI Gateway | API Testing | AWS Lambda | CI/CD | CloudWatchSenior-level Full TimeHyderabad, India7h ago
-
Consultant - Cloud Data Engineer INR 1500K-2500KApache Airflow | Apache Kafka | Apache Spark | Azure | Azure DataMid-level Full TimeGurgaon, Haryana, India7h ago
-
Generative AI Lead INR 3264K-5876KAI Governance | APIs | Cloud Architecture | Cloud infrastructure | Data LakeHybrid workSenior-level Full TimeHyderabad, India7h ago
-
Senior-level Full TimeHyderabad, India7h ago
-
Software Engineering Professional 2, AI Engineering INR 3000K-4800KAI Governance | AI Ops | API Integration | Amazon Q | Automated testingEntry-level Full TimeBengaluru, IN, 5601038h ago
-
Sr. Principal Business Systems Analyst 1 INR 3700K-5000KAgentic Frameworks | Azure DevOps | Big Data | C# | C++Senior-level Full TimeBengaluru, KA, IN8h ago
-
AWS | Ansible | Azure | Bash | CI/CDMid-level Full TimeHyderabad, Telangana, India10h ago
-
Mid-level Full TimeBangalore, KA, IN, 56007611h ago
-
Engineering Technical Leader – Embedded Software INR 3000K-3500KBash | Board Support | Board Support Package | Bootloader | C plus plusSenior-level Full TimeBengaluru, India11h ago
-
Data Engineer INR 1000K-2000KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | CI/CDMid-level Full TimeHyderabad, Telangana, India12h ago
-
AWS | Airbyte | Airflow | Azure | CI/CDCollaboration across teams | Inclusive work environment | Learning opportunities | Professional growth | Remote workSenior-level Full TimeIndia12h ago
-
Auditability | Cloud infrastructure | Databases | Deployment Automation | IntegrationSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 56006612h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 56006612h ago
-
Applied AI ML Lead INR 4000K-5572KAI Agents | Data Governance | Data Management | Deep learning | Experiment trackingSenior-level Full TimeBengaluru, Karnataka, India12h ago