💾Memory

Memory allows you to chat with AI as if AI has the memory of previous conversations.

Human: hi I am bob

AI: Hello Bob! It's nice to meet you. How can I assist you today?

Human: what's my name?

AI: Your name is Bob, as you mentioned earlier.

Under the hood, these conversations are stored in arrays or databases, and provided as context

to LLM. For example:

You are an assistant to a human, powered by a large language model trained by OpenAI.

Whether the human needs help with a specific question or just wants to have a conversation about a particular topic, you are here to assist.

Current conversation:

{history}

Separate conversations for multiple users

UI & Embedded Chat

By default, UI and Embedded Chat will automatically separate different users conversations. This is done by generating a unique chatId for each new interaction. That logic is handled under the hood by THub.

Prediction API

You can separate the conversations for multiple users by specifying a unique sessionId

1. For every memory node, you should be able to see a input parameter Session ID

In the /api/v1/prediction/{your-chatflowid} POST body request, specify the sessionId in overrideConfig

{

"question": "hello!",

"overrideConfig": {

"sessionId": "user1"

}

}

Message API

· GET /api/v1/chatmessage/{your-chatflowid}

· DELETE /api/v1/chatmessage/{your-chatflowid}

All conversations can be visualized and managed from UI as well:

For OpenAI Assistant, Threads will be used to store conversations.

1)Buffer Memory

The Buffer Memory node stores the entire conversation history in memory and makes it available for the model during interactions.

Key Features:

• Complete Conversation Storage: Stores all messages from the conversation without trimming. • Context Preservation: Helps the model maintain full context across the interaction. • Simple Integration: Easy to use with chat models and agents. • Real-Time Updates: Continuously updates memory as the conversation progresses.

Setup Requirements:

1. Add the Buffer Memory node to the canvas.

2. No mandatory configuration is required.

3. Optionally configure Additional Parameters if needed.

4. Connect the memory node to a chat model or agent.

Use Cases:

• Applications requiring full conversation history. • Chatbots that depend on complete context understanding.

2)Buffer Window Memory

The Buffer Window Memory node stores only a fixed number of recent messages instead of the entire conversation.

Key Features:

• Limited Context Storage: Stores only the latest messages based on defined window size. • Memory Control: Prevents excessive memory usage. • Improved Performance: Reduces token usage by limiting context. • Configurable Window Size: Allows control over how many messages are retained.

Setup Requirements:

1. Add the Buffer Window Memory node to the canvas.

2. Set the Size parameter (number of messages to retain).

3. Configure Additional Parameters if required.

4. Connect the node to a chat model or agent.

Use Cases:

• Applications needing only recent context. • Systems where token optimization is important.

3)Conversation Summary Buffer Memory

The Conversation Summary Buffer Memory node summarizes past conversations and maintains a compressed version of the interaction.

Key Features:

• Summarized Memory Storage: Stores condensed summaries instead of full conversation. • Token Optimization: Reduces token usage significantly. • Context Retention: Maintains important information through summaries. • Dynamic Updates: Continuously updates summaries as conversation evolves.

Setup Requirements:

1. Add the Conversation Summary Buffer Memory node to the canvas.

2. Connect a Chat Model for generating summaries.

3. Set the Max Token Limit for summary size.

4. Configure Additional Parameters if required.

5. Connect the node to a chat model or agent.

Use Cases:

• Long conversations requiring memory efficiency. • Applications needing balance between context and performance.

4)Conversation Summary Memory

The Conversation Summary Memory node maintains a summarized version of the conversation to efficiently manage long interactions.

Key Features:

• Summarized Context: Stores compressed summaries instead of full conversations. • Token Efficiency: Reduces token usage while retaining key information. • Continuous Updating: Updates summaries dynamically as the conversation grows. • Improved Performance: Enables handling of long conversations efficiently.

Setup Requirements:

1. Add the Conversation Summary Memory node to the canvas.

2. Connect a Chat Model for generating summaries.

3. Configure Additional Parameters if required.

4. Connect the node to a chat model or agent.

Use Cases:

• Long-running conversations with limited token usage. • Applications requiring summarized context instead of full history.

5)DynamoDB Chat Memory

The DynamoDB Chat Memory node stores conversation history in AWS DynamoDB for persistent and scalable memory.

Key Features:

• Persistent Storage: Stores chat history in DynamoDB across sessions. • Scalable Architecture: Supports large-scale applications. • Cloud Integration: Seamlessly integrates with AWS services. • Reliable Data Storage: Ensures durability and availability of conversation data.

Setup Requirements:

1. Create a DynamoDB table in AWS.

2. Note the Table Name, Partition Key, and Region.

3. Add the DynamoDB Chat Memory node to the canvas.

4. Connect Credential > provide AWS credentials.

5. Enter Table Name and Partition Key.

6. Select Region.

7. Configure Additional Parameters if required.

Use Cases:

• Applications requiring persistent chat history. • Cloud-based scalable chatbot systems.

6)Mem0 Memory Node

The Mem0 node integrates THub with Mem0, a memory system designed to provide persistent memory capabilities for AI chatflows.

Key Features:

• Persistent Memory Storage: Mem0 offers persistent memory storage for THub chatflows, ensuring that conversation history is retained across sessions.

• Seamless Integration: It integrates seamlessly with existing THub templates and is compatible with various LLM nodes.

• Custom Configurations: Supports custom memory configurations, allowing developers to tailor memory settings to specific needs

• Performance Advantages: According to Mem0, their system achieves 26% higher accuracy than OpenAI Memory, 91% lower latency, and 90% token savings.

Setup Requirements:

1. Flowise Installation: Ensure Flowise is installed (NodeJS >= 18.15.0 required).

2. Access Flowise UI: Navigate to the THub UI at http://localhost:3000.

3. Mem0 API Key: Obtain your Mem0 API key from the Mem0 dashboard.

4. Integration: Replace the default Buffer Memory with Mem0 Memory in your Flowise chatflow.

Use Cases:

• Applications requiring long-term memory retention across user sessions.

• Scenarios where efficient memory usage and low latency are critical.

7)MongoDB Atlas Chat Memory

The MongoDB Atlas Chat Memory node stores conversation data in MongoDB Atlas for persistent and flexible memory management.

Key Features:

• Persistent Storage: Stores chat data in MongoDB Atlas. • Flexible Schema: Supports dynamic data structures. • Cloud Database Integration: Works with MongoDB Atlas clusters. • Scalable and Reliable: Handles large volumes of conversation data.

Setup Requirements:

1. Create a MongoDB Atlas cluster.

2. Create a Database and Collection.

3. Obtain connection credentials.

4. Add the MongoDB Atlas Chat Memory node to the canvas.

5. Connect Credential > provide MongoDB connection details.

6. Enter Database Name and Collection Name.

7. Configure Additional Parameters if required.

Use Cases:

• Applications needing persistent and flexible storage. • Chatbots requiring scalable database-backed memory.

8)Redis-Backed Chat Memory

The Redis-Backed Chat Memory node stores conversation history in Redis for fast and efficient memory access.

Key Features:

• In-Memory Storage: Uses Redis for fast data access. • Persistent Option: Can persist data depending on Redis configuration. • Low Latency: Provides quick read and write operations. • Scalable Usage: Suitable for high-performance applications.

Setup Requirements:

1. Set up a Redis instance.

2. Add the Redis-Backed Chat Memory node to the canvas.

3. Connect Credential > provide Redis connection details.

4. Configure Additional Parameters if required.

Use Cases:

• Real-time chat applications. • Systems requiring fast memory access.

9)Upstash Redis-Backed Chat Memory

The Upstash Redis-Backed Chat Memory node uses Upstash Redis (serverless) to store and manage chat memory.

Key Features:

• Serverless Redis: Uses Upstash for managed Redis services. • High Availability: Reliable cloud-based memory storage. • Fast Retrieval: Optimized for quick data access. • Easy Integration: Simple setup with REST URL.

Setup Requirements:

1. Create an Upstash Redis database. Link: https://console.upstash.com/

2. Copy the Upstash Redis REST URL.

3. Add the Upstash Redis-Backed Chat Memory node to the canvas.

4. Connect Credential if required.

5. Paste the Upstash Redis REST URL.

6. Configure Additional Parameters if needed.

Use Cases:

• Serverless chat applications. • Cloud-based memory storage solutions.

10)Zep Memory

Zep is long-term memory store for LLM applications. It stores, summarizes, embeds, indexes, and enriches LLM app / chatbot histories, and exposes them via simple, low-latency APIs.

Guide to Deploy Zep to Render

You can easily deploy Zep to cloud services like Render, Flyio. If you prefer to test it locally, you can also spin up a docker container by following their quick guide.

In this example, we are going to deploy to Render.

1. Head over to Zep Repo and click Deploy to Render

2. This will bring you to Render's Blueprint page and simply click Create New Resources

3.When the deployment is done, you should see 3 applications created on your dashboard.

4.Simply click the first one called zep and copy the deployed URL

Guide to Deploy Zep to Digital Ocean (via Docker)

Use in THub UI

1. Back to THub application, simply create a new canvas or use one of the templates from marketplace. In this example, we are going to use Simple Conversational Chain

2. Replace Buffer Memory with Zep Memory. Then replace the Base URL with the Zep URL you have copied above

3. Save the chatflow and test it out to see if conversations are remembered.

4. Now try clearing the chat history, you should see that it is now unable to remember the previous conversations.

Zep Authentication

Zep allows you to secure your instance using JWT authentication. We'll be using the zepcli command line utility here.

1. Generate a secret and the JWT token

After downloaded the ZepCLI:

You will first get your SECRET Token:

Then you will get JWT Token:

2. Configure Auth environment variables

Set the following environment variables in your Zep server environment:

1. Configure Credential on THub

Add a new credential for Zep, and put in the JWT Token in the API Key field:

3. Use the created credential on Zep node

In the Zep node Connect Credential, select the credential you have just created. And that's it!

Threads

Threds is only used when an OpenAI Assistant is being used. It is a conversation session between an Assistant and a user.Threads store messages and automatically handle truncation to fit content into a model’s context.

Separate conversations for multiple users

UI & Embedded Chat

By default, UI and Embedded Chat will automatically separate threads for multiple users conversations. This is done by generating a unique chatId for each new interaction.That logic is handled under the hood by THub.

Prediction API

POST /api/v1/prediction/{your-chatflowid}, specify the chatId . Same thread will be used for

the same chatId

Message API

· GET /api/v1/chatmessage/{your-chatflowid}

· DELETE /api/v1/chatmessage/{your-chatflowid}

You can also filter via chatId - /api/v1/chatmessage/{your-chatflowid}?chatId={your-chatid}

All conversations can be visualized and managed from UI as well:

10) Zep Memory - Cloud Node

The Zep Memory (Cloud) node uses Zep’s managed cloud service to store and manage chat memory.

Key Features:

• Managed Service: No need to maintain infrastructure. • Persistent Memory: Stores conversations across sessions. • Scalable Storage: Handles large-scale applications. • Easy Integration: Quick setup using credentials.

Setup Requirements:

1. Sign up for Zep Cloud. Link: https://www.getzep.com/

2. Obtain API credentials.

3. Add the Zep Memory - Cloud node to the canvas.

4. Connect Credential > provide API key/details.

5. Configure Additional Parameters if required.

Use Cases:

• Production-grade applications. • Scalable AI chat systems.