Backend & AI Engineer (Search + AI Agent Orchestration)

Description

We are looking for an experienced and ambitious Backend & AI Engineer with deep expertise in search systems, data pipelines, and AI agent orchestration. You will design, implement, and maintain backend infrastructures that power advanced AI-driven products, focusing on multi-agent architectures, retrieval-augmented generation (RAG), and cutting-edge search technologies.

Responsibilities

• Architect & Implement Search & AI Systems
• Research, design, implement, and maintain search systems and products utilizing AI at their core (e.g., RAG pipelines, advanced NLP, Web3 Agents Marketplace and protocol and more).
• Engineer and optimize RAG-based AI search and chat systems into our products (e.g., web3index.com).
• AI Agent Development & Orchestration
• Develop and integrate AI agent frameworks capable of orchestrating multiple agents for tasks like data retrieval, data enrichment, and automated decision-making.
• Collaborate with data scientists and ML engineers to create agent-based protocols that streamline AI-driven processes (e.g., chaining or coordinating multiple AI systems).
• Evaluate and implement best practices for multi-agent communication, coordination, and lifecycle management.
• Backend & Data Pipeline Engineering
• Build and maintain backend systems handling large-scale scraping, search indexing, and querying pipelines.
• Implement scalable APIs (REST, GraphQL) that serve search and agent-based results with high availability and low latency.
• Optimize backend systems for performance, scalability, and cost efficiency, ensuring robust monitoring and alerts.
• Develop verifiable execution deployment pipelines for web3 AI agents that execute yield bearing strategies on chain.
• Leverage traditional keyword search (e.g., Lucene/Elasticsearch), as well as semantic vector-based search (FAISS, Milvus, Pinecone, etc.).
• Integrate embeddings (OpenAI, Hugging Face) and advanced NLP models into production-grade systems.
• Perform team wide collaboration & product alignment
• Work closely with frontend and product teams to align backend capabilities with end-user requirements.
• Collaborate with cross-functional teams to integrate AI and search functionalities into various data products.
• Stay up-to-date with advancements in search technologies, NLP models, AI agent architectures, and frameworks.

Requirements

Required Skills & Qualifications

• Backend Engineering (4+ years)
• Proven experience in designing, building, and scaling highly data-intensive products and web scraping pipelines.
• Strong proficiency in Python, with a track record of delivering production-grade products and APIs.
• Search & Vector Databases
• Expertise in keyword-based search systems like Lucene/Elasticsearch.
• Experience with vector databases (e.g., Pinecone, Vespa, Weaviate, Milvus, FAISS, mongodb) for semantic search.
• Hands-on experience implementing RAG frameworks with AI search.
• AI & NLP
• Strong understanding of NLP models, embeddings, and large language models (LLMs).
• Familiarity with Langchain, Llamaindex or similar frameworks for building advanced AI and agent-based solutions.
• Experience integrating AI/ML models into production systems (e.g., via endpoints, containers, or cloud services).
• AI Agent Orchestration using langraph, crewai, autogen etc..
• Experience designing and managing multi-agent architectures or agent-based systems is a significant advantage.
• Knowledge of agent communication protocols, state management, and orchestration patterns.
• Data Pipelines & Databases
• Proficiency in building scalable data pipelines for ingesting, transforming, and indexing structured/unstructured data.
• Strong knowledge of SQL and NoSQL databases (e.g., PostgreSQL, MongoDB).
• Cloud & Distributed Systems
• Familiarity with AWS, GCP, or Azure for deploying large-scale backend infrastructure.
• Solid understanding of distributed systems, caching, and high-performance architectures.
• Monitoring & Analytics
• Experience with production monitoring, logging, and analytics frameworks (Prometheus, Grafana, Datadog, etc.).

Preferred Soft Skills

• Self-Driven & Independent
• Able to take ownership of complex tasks while aligning with broader team objectives.
• Collaboration & Communication
• Strong communication skills to clearly articulate technical concepts to diverse stakeholders.
• A team player who thrives in a collaborative, innovative environment.
• Organization & Adaptability
• Excellent time-management and organizational skills, able to handle multiple priorities.
• Eager to learn and stay updated with cutting-edge AI, agent orchestration, and search technologies.

What You’ll Gain

• The opportunity to pioneer next-generation AI agent orchestration and RAG-based systems.
• A dynamic environment where you can push boundaries in NLP, search, and distributed systems.
• Collaboration with a talented team across data science, product, and platform engineering.
• Competitive compensation, benefits, and the chance to shape high-impact AI-driven products.

If you’re excited about merging world-class backend engineering with cutting-edge AI agent architectures, we’d love to hear from you! Join us on our mission to revolutionize how data is searched, transformed, and leveraged at scale.