🧬Embeddings

Embeddings can be used to create a numerical representation of textual data. This numerical representation is useful because it can be used to find similar documents.

An embedding is a vector (list) of floating-point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness.

They are commonly used for:

· Search (where results are ranked by relevance to a query string)

· Clustering (where text strings are grouped by similarity)

· Recommendations (where items with related text strings are recommended)

· Anomaly detection (where outliers with little relatedness are identified)

· Diversity measurement (where similarity distributions are analyzed)

· Classification (where text strings are classified by their most similar label)

1)AWS Bedrock Embeddings

AWSBedrock embedding models to generate embeddings for a given text.

Setup

AWS Console

1. Log in to AWS Console

2. In the search bar, search for Bedrock and open Amazon Bedrock

1. Click Model access from the left panel

2. Request access to embedding models such as

· amazon.titan-embed-text-v1 · amazon.titan-embed-text-v2 · cohere.embed-english-v3

1. Go to AWS IAM to generate credentials

Link: https://console.aws.amazon.com/iam/

1. Create AWS Access Credentials

IAM → Users → Create User → Create Access Key

Copy the following details

· Access Key · Secret Access Key

1. Open THub Canvas

2. Go to Embeddings > drag AWS Bedrock Embeddings node

3. Click Connect Credential → Create New

Enter the credentials

· AWS Access Key · AWS Secret Access Key

1. Select Region

Example

· us-east-1 · ap-south-1 · eu-west-1

1. Select Model Name

Example

· amazon.titan-embed-text-v1 · amazon.titan-embed-text-v2 · cohere.embed-english-v3

1. Select Cohere Input Type (only if Cohere model is used)

Options

· search_document · search_query · classification · clustering

1. Voila 🎉, you have successfully created AWS Bedrock Embeddings node in THub

2)Azure OpenAI Embeddings

Prerequisite

1. Log in or sign up to Azure

2. Create your Azure OpenAI and wait for approval approximately 10 business days

3. Your API key will be available at Azure OpenAI > click name_azure_openai > click Click here to manage keys

Setup

Azure OpenAI Embeddings

1. Click Go to Azure OpenaAI Studio

2. Click Deployments

3. Click Create new deployment

4. Select as shown below and click Create

5. Successfully created Azure OpenAI Embeddings

· Deployment name: text-embedding-ada-002

· Instance name: top right conner

THub

1. Embeddings > drag Azure OpenAI Embeddings node

2. Connect Credential > click Create New

3. Copy & Paste each details (API Key, Instance & Deployment name, API Version) into Azure OpenAI Embeddings credential

4. Voila 🎉, you have created Azure OpenAI Embeddings node in THub

3)Google GenerativeAI Embeddings

Google Generative API to generate embeddings for a given text.

Google Generative AI Embeddings

Prerequisite

1. Go to Google AI Studio

2. Sign in with your Google account

3. Click Create API Key and copy the generated API Key

Setup

THub

1. Embeddings > drag Google Generative AI Embeddings node

2. Connect Credential > click Create New

3. Paste the Google Generative AI API Key in the credential

4. Select Model Name

5. Select Task Type

Example: · RETRIEVAL_DOCUMENT · RETRIEVAL_QUERY

1. Voila 🎉, you have created Google Generative AI Embeddings node in THub

4)Google VertexAI Embeddings

Google vertexAI API to generate embeddings for a given text.

Prerequisite

1. Go to Google Cloud Console

2. Create or select a Project

3. Enable Vertex AI API

4. Create Service Account credentials

Go to IAM & Admin → Service Accounts

1. Create a Service Account and generate a JSON Key

Setup

THub

1. Embeddings > drag Google Vertex AI Embeddings node

2. Connect Credential > click Create New

3. Upload or paste the Service Account JSON credentials

4. Select Model Name

5. Select Region

6. Voila 🎉, you have created Google Vertex AI Embeddings node in THub

5)Ollama Embeddings

Generate embeddings for a given text using opensource model on Ollama.

Prerequisite

1. Install Ollama

2. Start Ollama on your system

3. Pull an embedding supported model

Example command

ollama pull llama2

Setup

THub

1. Embeddings > drag Ollama Embeddings node

2. Enter Base URL

3. Enter Model Name

4. Voila 🎉, you have created Ollama Embeddings node in THub

6)OpenAI Embeddings

OpenAI API to generate embeddings for a given text.

OpenAI Embeddings

Prerequisite

1. Go to OpenAI Platform

2. Sign in or create an account

3. Generate an API Key

Setup

THub

1. Embeddings > drag OpenAI Embeddings node

2. Connect Credential > click Create New

3. Paste the OpenAI API Key in the credential

4. Select Model Name Example: text-embedding-ada-002

5. Voila 🎉, you have created OpenAI Embeddings node in THub

11)OpenAI Embeddings Custom

OpenAI API to generate embeddings for a given text.

Prerequisite

1. Go to OpenAI Platform Link: https://platform.openai.com/

2. Sign in or create an account

3. Generate an API Key Link: https://platform.openai.com/api-keys

Setup

THub

1. Embeddings > drag OpenAI Embeddings Custom node

2. Connect Credential > click Create New

3. Paste the OpenAI API Key in the credential

4. Enter Model Name Example: custom embedding model name

5. Voila 🎉, you have created OpenAI Embeddings Custom node in THub