ORIN.AI

Overview

ORIN.AI answers questions with visualisations. You ask something like "compare cloud market share", and instead of paragraphs the app returns a typed dashboard object (pie, bar, line, timeline, table, comparison and more) that the frontend renders with Recharts or Mermaid. Authenticated users get a memory layer in Pinecone, so repeat questions can be answered from earlier conversations.

What I built

• A /api/generate route that validates the request with Zod, resolves the Appwrite user, and runs either the memory-aware pipeline or an anonymous one.
• A LangChain multi-agent setup with four tools (classify visualisation, retrieve context, generate dashboard, validate UI schema) driven by an AgentExecutor, with a direct tool-chain fallback that calls the four tools in order if the agent fails.
• A strict dashboardSchema in Zod (type enum, 5-120 char title, 1-100 data rows, optional axis and colour config) that every response must pass before it reaches the client.
• A per-user memory manager on Pinecone with Hugging Face multilingual-e5-large embeddings (OpenAI or a local fallback if unavailable). Matches above 0.9 similarity short-circuit to a "From Memory" answer.
• A model router that maps intents (classification, summarisation, generation) to Gemini 2.5 Flash over the REST API with JSON response mode. It began as an Azure OpenAI router, which is why the function is still named invokeAzureChat.
• Chart renderers for pie, bar, line, area, radar, scatter, funnel, gauge, heatmap, treemap and Mermaid diagrams, plus Appwrite email and Google OAuth login and query logging.

Architecture

The API route hands queries to a pipeline that checks memory first, then runs the agent tools, then validates the result.

flowchart LR
  subgraph client["Client"]
    chat["Chat UI"]
    charts["Chart and Mermaid renderers"]
  end
  subgraph server["Next.js server"]
    api["api generate route"]
    pipe["Memory-aware pipeline"]
    agent["LangChain agent with 4 tools"]
    val["Zod dashboard schema"]
  end
  subgraph data["Data"]
    pine[("Pinecone user memory")]
    aw[("Appwrite auth and query logs")]
  end
  subgraph ext["External"]
    gem["Gemini 2.5 Flash"]
    hf["Hugging Face embeddings"]
  end
  chat -->|"POST query"| api
  api --> aw
  api --> pipe
  pipe --> pine
  pine --- hf
  pipe --> agent
  agent --> gem
  agent --> val
  val -->|"dashboard JSON"| charts
  pipe -->|"log query"| aw

System design

For a signed-in user, the pipeline tries memory before spending model calls, and falls back from the LangChain agent to a direct Gemini agent if needed.

sequenceDiagram
  participant U as User
  participant R as API route
  participant M as Memory manager
  participant A as LangChain agent
  participant G as Gemini
  participant W as Appwrite
  U->>R: POST query
  R->>W: Get session user
  R->>M: Search similar memories
  alt Similarity above 0.9
    M-->>R: Stored context
    R-->>U: From Memory text answer
  else No close match
    R->>A: Query plus user context
    A->>G: Classify visualisation type
    A->>M: Retrieve context if RAG needed
    A->>G: Generate dashboard JSON
    A->>A: Validate against schema
    A-->>R: Dashboard
    R->>W: Log query and type
    R->>M: Store conversation
    R-->>U: Dashboard JSON to render
  end

Decisions & trade-offs

• Schema-first output: the model must produce JSON that passes Zod, so the renderer never has to guess. It means some answers get forced into a chart shape when plain text would do.
• Every LLM step has a fallback (agent to direct tool chain to Gemini agent, keyword classifier when the model call fails). The app stays usable, but failures are quieter and harder to spot.
• Hugging Face embeddings first for memory to avoid tying memory to one paid provider, at the cost of slower cold-start inference calls.

What I'd do next

• Replace the placeholder "external knowledge" retrieval with real web search so current-events questions are grounded.
• Rename the router and remove the leftover Azure configuration now that it only calls Gemini.