Data Engineer
Chennai, Tamil Nadu, India
INR 1500K-2600K (estimate) Mid-level Full Time
Tasks
- Architect AI ready data platform components for ML consumption
- Architect scalable cost effective data platforms
- Build and maintain feature stores on Vertex AI for low latency serving
- Build pipelines for embedding generation and vector store population and RAG retrieval
- Define SLAs implement alerting and lead incident resolution
- Define reusable engineering patterns pipeline templates transformation standards and naming conventions
- Design CI CD pipelines with automated testing and deployment gates
- Design data models using dimensional and normalization and data vault
- Drive DataOps practices with automated testing data contracts and observability
- Ensure data security encryption and governance compliance
- Execute testing strategies for regulated workloads with parallel validation
- Implement SCD strategies and historical data management
- Lead code reviews and improve quality testability and maintainability
- Lead end to end design and delivery of data engineering solutions on GCP
- Mentor junior engineers through pairing and feedback
- Migrate legacy data structures to GCP with validation
- Monitor and optimize GCP costs and implement efficiency improvements
- Provision and govern cloud infrastructure with Terraform
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | BigQuery | CI/CD | Cloud Build | Cloud Composer | Cloud Composer Airflow | Cloud Storage | Cloud platform | Composer (Airflow) | DBT | Data Contracts | Data Governance | Data Modeling | Data Vault | DataOps | Dataflow | Dataform | Dataplex | Dimensional Modeling | Embeddings | Encryption | Feature Store | GitHub Copilot | Google Cloud | Google Cloud Platform | Google Gemini | Kimball | LLM-Ops | Observability | PubSub | Python | RAG | SCD | SQL | Terraform | Third Normal Form | Vector Databases | Vertex AI
Education
N/A
Roles
Related jobs
-
Advanced Engineer-AI/ML INR 3264K-4800KAPI Integration | ASR | AUTOSAR | AWS | Audio ProcessingEmployee discounts | Recognition rewards program | Training opportunities | Tuition reimbursement | Wellness benefitsSenior-level Full TimeBangalore - Karnataka, India - Kalyani …3h ago
-
Senior Software Engineer - Data License Analytics, Pune INR 3000K-4720KAWS S3 | Agent to Agent | Agent to Agent A2A | Apache Airflow | Apache IcebergSenior-level Full TimePune3h ago
-
AI Lead and Automation Tester - Document Intelligence INR 2500K-5000KAI Workflow Orchestration | AI workflow | Automation Testing | Clinical Trial Compliance | Clinical trialSenior-level Full TimeIN-TN-Chennai4h ago
-
Senior-level Full TimeBangalore - Karnataka - India4h ago
-
Lead Data Platform Engineer INR 2040K-3900KAPI Integration | AWS | AWS CloudWatch | AWS DataZone | AWS GlueEnhanced medical benefits | Family well-being benefits | Hybrid work | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India5h ago
-
Anomaly Detection | Autoencoders | Bayesian optimization | Classification | ClusteringOnsite workEntry-level Full TimeHyderabad, India5h ago
-
AI Agent Development Kit | API Gateway | Agent Development Kit | Agentic AI | Audit LoggingSenior-level Full TimeMumbai, Maharashtra, India5h ago
-
AI/ML Infrastructure Software Engineer INR 3000K-5000KC++ | JAX | Keras | Machine Learning | Machine Learning FairnessSenior-level Full TimeBengaluru, Karnataka, India5h ago
-
Data Engineer INR 1300K-2000KAPI Integration | Data Modeling | Data Transformation | Data pipeline | Power BIMid-level Full TimeHyderabad, Telangana, India5h ago
-
Consultant - Cloud Data Engineer INR 1500K-2500KApache Airflow | Apache Kafka | Apache Spark | Azure | Azure DataMid-level Full TimeGurgaon, Haryana, India6h ago
-
Generative AI Lead INR 3264K-5876KAI Governance | APIs | Cloud Architecture | Cloud infrastructure | Data LakeHybrid workSenior-level Full TimeHyderabad, India6h ago
-
Senior-level Full TimeHyderabad, India6h ago
-
Software Engineering Professional 2, AI Engineering INR 3000K-4800KAI Governance | AI Ops | API Integration | Amazon Q | Automated testingEntry-level Full TimeBengaluru, IN, 5601037h ago
-
Sr. Principal Business Systems Analyst 1 INR 3700K-5000KAgentic Frameworks | Azure DevOps | Big Data | C# | C++Senior-level Full TimeBengaluru, KA, IN7h ago
-
Mid-level Full TimeBangalore, KA, IN, 56007610h ago
-
Engineering Technical Leader – Embedded Software INR 3000K-3500KBash | Board Support | Board Support Package | Bootloader | C plus plusSenior-level Full TimeBengaluru, India10h ago
-
Data Engineer INR 1000K-2000KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | CI/CDMid-level Full TimeHyderabad, Telangana, India11h ago
-
AWS | Airbyte | Airflow | Azure | CI/CDCollaboration across teams | Inclusive work environment | Learning opportunities | Professional growth | Remote workSenior-level Full TimeIndia11h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 56006611h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 56006611h ago
-
Applied AI ML Lead INR 4000K-5572KAI Agents | Data Governance | Data Management | Deep learning | Experiment trackingSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Senior Analytics Engineer – Data Applications INR 2771K-4064KAirflow | Anomaly Detection | Apache Spark | CI/CD | DBTSenior-level Full TimeBengaluru, INDIA, India12h ago
-
.NET | APIs | Automation | C Sharp | CrewAIAccess to AI tools | Accidental insurance | Casual leave | Earned Leave | Flexible autonomySenior-level Full TimeIndia12h ago
-
Senior Data Engineer INR 3000K-4000KA/B | A/B Testing | AWS | Apache Airflow | Apache HiveAccident insurance | Disability insurance | Financial wellness support | Flexible work schedule | Healthcare benefitsSenior-level Full TimeBengaluru, India12h ago
-
Lead - AI/ML Engineer (Generative AI Agent Systems) INR 3700K-5000KAI Agent Development Kit | Agent Development Kit | Agent Engine | Autonomous Agents | Cloud platformSenior-level Contract Full TimePune, MH, India13h ago