# 🧠Large Language Models(LLM)

#### **Features**

•   Natural Language Processing (NLP): LLMs can understand, interpret, and generate human language.

•   Contextual Understanding: They can grasp the context of conversations or text, providing relevant responses.

•   Multilingual Capabilities: Many LLMs support multiple languages, broadening their applicability.

•   Scalability: These models can handle a wide range of applications, from chatbots to complex data analysis.

#### **Applications**

•  Chatbots and Virtual Assistants: Enhance customer service by providing accurate and timely responses.

•   Content Creation: Assist in generating articles, reports, and other written content.

•   Translation Services: Improve the accuracy and efficiency of translating text between languages.

•    Data Analysis: Aid in interpreting and summarizing large datasets.

#### 1)AWS Bedrock

The **AWS Bedrock** LLM node integrates Amazon's fully managed foundation model service into THub, allowing users to leverage a wide range of AI models — including Amazon Titan, Anthropic Claude, and Meta Llama — within their workflows.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2FEfQhErDqaypJ4L4ggeMp%2Fimage.png?alt=media&#x26;token=fc5c2c4a-788b-4d8a-9a1e-ad8d54cae4cf" alt=""><figcaption></figcaption></figure>

#### **Setup Requirements:**

To configure this node, you'll need the following:

* **AWS Credential**: An IAM user or role with `bedrock:InvokeModel` permission.
* **Region**: The AWS region where Bedrock is enabled on your account.
* **Model Name**: The foundation model you want to invoke (e.g. `amazon.titan-tg1-large`).
* **Custom Model Name** *(optional)*: The ARN of a fine-tuned or custom model if you've provisioned one in Bedrock.

#### **Key Features:**

* Access multiple foundation model providers (Amazon, Anthropic, Meta, AI21) from a single node.
* Native AWS IAM authentication — no separate API key management required.
* Supports custom and fine-tuned models via model ARN.
* Cache toggle to reduce repeated API calls for identical prompts.

#### **Use Cases:**

* Summarising documents and reports
* Answering questions over enterprise data
* Calling child flows with AI-generated output
* SQL Q\&A and data analysis
* Web scraping Q\&A

#### 2)Azure OpenAI

The **Azure OpenAI** LLM node integrates Microsoft Azure's hosted OpenAI service into THub, allowing users to leverage GPT-4, GPT-3.5, and other OpenAI models within their workflows — with enterprise-grade compliance and data residency.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2FhYiWOYeHfBFHYHDijYEb%2Fimage.png?alt=media&#x26;token=542fe795-976d-402d-8b43-42bf114e9a51" alt=""><figcaption></figcaption></figure>

#### **Setup Requirements:**

To configure this node, you'll need the following:

* **Connect Credential**: An Azure OpenAI credential containing your Azure endpoint URL and API key.
* **Model Name**: Your Azure **deployment name** — the name you gave the model when deploying it in the Azure portal (e.g. `text-davinci-003`).
* **Temperature** *(optional)*: Controls response randomness. Range `0.0` to `2.0`. Default is `0.9`.

#### **Key Features:**

* Uses OpenAI's GPT models hosted entirely within your Azure subscription.
* Data stays within your chosen Azure region — meets enterprise compliance and data residency requirements.
* Billed through your existing Azure account.
* Cache toggle to reuse responses for repeated prompts.

#### **Use Cases:**

* Interacting with APIs
* Multiple document Q\&A
* Calling child flows
* Data summarisation and transformation
* SQL Q\&A

#### 3)Cohere

The **Cohere** LLM node integrates Cohere's enterprise language models into THub, allowing users to leverage Cohere's suite of instruction-following and NLP models within their workflows.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2F6Bmj0mG0lh6yJiMQAaBZ%2Fimage.png?alt=media&#x26;token=87c2caf4-3259-4013-9eb7-bfb9f6f4e657" alt=""><figcaption></figcaption></figure>

#### **Setup Requirements:**

To configure this node, you'll need the following:

* **Connect Credential**: A Cohere API key from your Cohere dashboard.
* **Model Name**: The Cohere model to use (e.g. `command`, `command-r`, `command-r-plus`).
* **Temperature** *(optional)*: Controls response randomness. Range `0.0` to `1.0`. Default is `0.7`.
* **Max Tokens** *(optional)*: Maximum length of the generated response. Leave blank to use the model's default.

#### **Key Features:**

* Purpose-built models for enterprise NLP tasks — summarisation, classification, and generation.
* `command-r-plus` supports complex multi-step reasoning and long documents.
* Lightweight `command-light` model for fast, cost-efficient tasks.
* Cache toggle to reuse responses for repeated prompts.

#### **Use Cases:**

* Document summarisation
* Text classification and tagging
* Multiple document Q\&A
* Data upsertion
* Web scraping Q\&A

#### 4)GoogleVertex AI

The **Google Vertex AI** LLM node integrates Google Cloud's managed AI platform into THub, allowing users to leverage Google's PaLM 2 and Gemini foundation models within their workflows.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2FDEZUuHYfw4Suu9iGERla%2Fimage.png?alt=media&#x26;token=7e4a97b4-e060-4e75-92c8-1d6229f5da4a" alt=""><figcaption></figcaption></figure>

#### Setup Requirements:

To configure this node, you'll need the following:

* **Connect Credential**: A Google Cloud service account JSON key with the **Vertex AI User** role (`roles/aiplatform.user`).
* **Model Name**: The Vertex AI model to invoke (e.g. `text-bison`, `gemini-pro`).
* **Temperature** *(optional)*: Controls response randomness. Range `0.0` to `1.0`. Default is `0.7`.

#### **Key Features:**

* Access Google's PaLM 2 and Gemini model families from a single node.
* Runs within your GCP project — billed through Google Cloud.
* `gemini-pro` supports extended context and stronger reasoning tasks.
* Cache toggle to reuse responses for repeated prompts.

#### **Use Cases:**

* Calling child flows
* Interacting with APIs
* SQL Q\&A
* Multiple document Q\&A
* Data upsertion

#### 5)Ollama

The **Ollama** LLM node integrates a locally hosted Ollama instance into THub, allowing users to run open-source foundation models entirely within their own infrastructure — with no data sent to external APIs.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2FXDX6su8e4y3blWFNHxUo%2Fimage.png?alt=media&#x26;token=70ae4a5b-bfaf-44f1-b83f-6e4b48ac5c81" alt=""><figcaption></figcaption></figure>

#### **Setup Requirements:**

To configure this node, you'll need the following:

* **Base URL**: The address where your Ollama server is running (default: `http://localhost:11434`).
* **Model Name**: The name of the model you have pulled locally (e.g. `llama3`, `mistral`, `phi3`). Run `ollama list` in your terminal to see available models.
* **Temperature** *(optional)*: Controls response randomness. Range `0.0` to `1.0`. Default is `0.9`.

#### **Key Features:**

* Fully self-hosted — no data leaves your infrastructure.
* No per-token API costs; runs on your own hardware.
* Supports a wide range of open-source models: Llama 3, Mistral, Phi-3, Gemma, Code Llama, and more.
* Cache toggle to reduce local inference overhead for repeated prompts.

#### **Use Cases:**

* Privacy-sensitive document Q\&A
* Offline and air-gapped workflow automation
* Web scraping Q\&A
* SQL Q\&A
* Data upsertion

#### 6)OpenAI

The **OpenAI** LLM node integrates OpenAI's API directly into THub, allowing users to leverage GPT-4, GPT-4o, GPT-3.5, and other flagship models within their workflows.

<figure><img src="https://1720595571-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxWXmt1Z68dgle5JORrEw%2Fuploads%2F9MNmljjZDnaFO4ujOs1R%2Fimage.png?alt=media&#x26;token=f4d296c1-c44f-4cd7-a76d-84ea701026ef" alt=""><figcaption></figcaption></figure>

#### **Setup Requirements:**

To configure this node, you'll need the following:

* **Connect Credential**: An OpenAI API key from your OpenAI account.
* **Model Name**: The OpenAI model to use (e.g. `gpt-4o`, `gpt-3.5-turbo`, `gpt-3.5-turbo-instruct`).
* **Temperature** *(optional)*: Controls response randomness. Range `0.0` to `2.0`. Default is `0.7`.

#### **Key Features:**

* Direct access to OpenAI's full model lineup — GPT-4o, GPT-4 Turbo, GPT-3.5, and legacy completion models.
* Simplest setup among all LLM nodes — just an API key and model name.
* Supports both chat models (`gpt-4o`, `gpt-3.5-turbo`) and legacy completion models (`gpt-3.5-turbo-instruct`).
* Cache toggle to reuse responses for repeated prompts.

#### **Use Cases:**

* Calling child flows
* Interacting with APIs
* Multiple document Q\&A
* SQL Q\&A
* Data upsertion
* Web scraping Q\&A


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.thub.tech/langchain/large-language-models-llm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.