> For the complete documentation index, see [llms.txt](https://docs.thub.tech/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.thub.tech/langchain/text-splitters.md).
# ✂️Text Splitters
As simple as this sounds, there is a lot of potential complexity here. Ideally, you want to keep the semantically related pieces of text together. What "semantically related" means could depend on the type of text. This notebook showcases several ways to do that.
**At a high level, text splitters work as following:**
1\. Split the text up into small, semantically meaningful chunks (often sentences).
2\. Start combining these small chunks into a larger chunk until you reach a certain size (as measured by some function).
3\. Once you reach that size, make that chunk its own piece of text and then start creating a new chunk of text with some overlap (to keep context between chunks).
**That means there are two different axes along which you can customize your text splitter:**
1\. How the text is split
2\. How the chunk size is measured
#### 1)Character Text Splitter
Splits only on one type of character (defaults to "\n\n").
#### 2)Code Text Splitter
Split documents based on language-specific syntax.
#### 3)Html-To-Markdown Text Splitter
Converts Html to Markdown and then split your content into documents based on the Markdown headers.
#### 4)Markdown Text Splitter
Split your content into documents based on the Markdown headers.
#### 5)Recursive Character Text Splitter
Split documents recursively by different characters - starting with "\n\n", then "\n", then " ".
#### 6)Token Text Splitter
Splits a raw text string by first converting the text into BPE tokens, then split these tokens into chunks and convert the tokens within a single chunk back into text.
#### **7)Recursive JSON Text Splitter**
The Recursive JSON Text Splitter operates by traversing the JSON data recursively, ensuring that each chunk adheres to specified size constraints. It maintains the hierarchical structure of the JSON, which is crucial for preserving the context and relationships within the data.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.thub.tech/langchain/text-splitters.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.