Data Engineer II, Data & Analytics
Tasks
- Build RAG pipelines using vector databases
- Build multi step AI agents and automated workflows with orchestration frameworks
- Build reusable API frameworks
- Collaborate with enterprise data and analytics platform teams
- Define data product strategy with stakeholders
- Design and develop RESTful APIs for data products
- Develop ETL ELT pipelines for lakehouse architectures
- Develop data engineering solutions using data lakes data warehouses and data marts
- Enhance data products and analytics ready datasets
- Implement data governance policies procedures and standards
- Implement incremental processing upserts and merges with Delta Lake
- Integrate LLM APIs into enterprise workflows
- Maintain data models with data architects
- Optimize data storage and retrieval performance and scalability
- Transform validate and develop data pipelines using SQL and Python
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | AWS | Data Governance | Data Lake | Data Mart | Data Privacy | Data Security | Data Warehouse | Databricks | Delta Lake | Django | ELT | ETL | FastAPI | Flask | Incremental processing | JWT | LLM APIs | Lakehouse | Langchain | Llamaindex | OAuth 2.0 | OWASP | OpenSearch | PGVector | Pinecone | Prompt engineering | PySpark | Python | RAG | RBAC | REST API | Retrieval-Augmented Generation | SQL | Upserts Merge | Vector Databases
Education
N/A
Roles
Related jobs
-
Entry-level Full TimeBellandur, Karnataka, India3h ago
-
Entry-level Full TimePune City, India9h ago
-
Lead Software Engineer - Pyspark, AWS, Databricks INR 3000K-5000KAWS | Agile | Apache Kafka | Application Resiliency | Automated testingSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Sr Lead Software Engineer - Agentic AI INR 3500K-5000KA2A SDK | APIs | AWS Lambda | Amazon EKS | Amazon S3Senior-level Full TimeBengaluru, Karnataka, India10h ago
-
Data & Analytics Engineer - AM INR 1500K-2500KApache Spark | Data Modeling | Data Warehousing | Databricks | Delta LakeSenior-level Full TimeBangalore, Karnataka, India10h ago
-
Senior-level Full TimeIN-AP-Hyderabad10h ago
-
AWS DE- Gen AI INR 2000K-3300KAWS Glue | AWS Identity | AWS Identity and Access Management | AWS Identity and Access Management IAM | AWS LambdaSenior-level Full TimeIN-TN-Chennai10h ago
-
Data Engineer INR 1600K-2500KAmazon Web Services | Apache Spark | Cloud platform | Docker | Google CloudSenior-level Full TimeIN-AP-Hyderabad10h ago
-
Apache Airflow | Apache Spark | BigQuery | CI/CD | Cloud ComposerMid-level Full TimePune, Maharashtra, India R11h ago
-
Lead Software Engineer - Lead Data Architect INR 3000K-5000KAWS | CI/CD | Cassandra | Data Governance | Data LineageSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Lead Data Engineer - Strategic Data Provisioning Data Engineer INR 3845K-5000KAI for Data | AI/ML | AWS | Agile | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Senior Product Development Engineer (Embedded System Firmware and Python, C programming) with 5 to 8 years of experience INR 660K-3000KARM | ASIC | Arc | Bitbucket | C#Entry-level Full TimeBengaluru, KA, India12h ago
-
Data Engineer INR 1500K-2200KAPI Integration | Data Modeling | Data Pipelines | Data Transformation | Data VisualizationMid-level Full TimeHyderabad, Telangana, India12h ago
-
Mid-level Full TimeHadapsar, Pune, Maharashtra, IND, 41101312h ago
-
Senior AI/ML Engineer – Generative AI INR 3700K-5000KA/B | A/B Testing | AI orchestration | API Integration | Agent systemsSenior-level Full TimeIndia13h ago
-
Mid-level Full TimeChennai, TN, India14h ago
-
AWS Lambda | Amazon EMR | Amazon S3 | Amazon Web Services | Apache AirflowEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India14h ago
-
Software Engineer II INR 1000K-2000KApache Spark | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactoryMid-level Full TimeIndia, Telangana, Hyderabad14h ago
-
AI Assisted Development | APIs | AWS | AWS Lambda | AgileSenior-level Full TimeBengaluru, Karnataka, India14h ago
-
Assistant Manager- AI Engineer INR 1200K-2500KAmazon Web Services | Autogen | Azure | Cloud Computing | ContainerizationMid-level Full TimeBangalore, Karnataka, India14h ago
-
Senior AI Engineer I INR 3264K-5876KAPIs | Apache Airflow | BigQuery | Cloud platform | Data PipelinesSenior-level Full TimeBengaluru, KA, India15h ago
-
Sr. Specialist, Data Engineering INR 2771K-4000KAgile | Airflow | BigQuery | CI/CD | Cloud FunctionsRelocation assistance within countrySenior-level Full TimeMumbai, MH, IN15h ago
-
Data and Digital Initiatives INR 3000K-4064KAPI | Cloud Computing | Data Analysis | Data Preprocessing | Data WranglingContinuous professional development | Flexible workingSenior-level Full TimeMumbai, Maharashtra, India16h ago
-
Data Warehouse, ETL, Cloud / Consultant Specialist INR 2800K-3900KAWS | Agile | Ansible | Apache Spark | CDCFlexible working | Inclusive work environment | Opportunities for growth | Professional developmentSenior-level Full TimePune, Maharashtra, India R16h ago
-
Lead Assistant Manager INR 2000K-3900KAWS IAM | Access Tokens | Amazon Web Services | Audit Logging | CI/CDSenior-level Full TimeIndia16h ago