Documents

zenpyre.documents

Contain utilities for documents.

zenpyre.documents.DocumentConsistencyError

Bases: ValueError

Raised when two documents share an id but have different page_content or metadata.

zenpyre.documents.DocumentHasher

Bases: BaseHasher[Document]

Hasher for LangChain Document objects.

This hasher delegates to hash_document, which computes a hash from the document's page_content and metadata, so two documents with equal content and metadata produce the same hash regardless of object identity.

Example

>>> from langchain_core.documents import Document
>>> from coola.hashing import HasherRegistry
>>> from zenpyre.documents import DocumentHasher
>>> registry = HasherRegistry()
>>> hasher = DocumentHasher()
>>> hasher
DocumentHasher()
>>> doc = Document(page_content="hello", metadata={"source": "test"})
>>> len(hasher.hash(doc, registry=registry))
64

zenpyre.documents.assign_ids

assign_ids(
    docs: list[Document], *, force: bool = False
) -> list[Document]

Assign a stable UUID to each document that does not already have one.

Iterates over docs and sets :attr:~langchain_core.documents.Document.id on any document whose id is None, using :func:~zenpyre.documents.hash_document_uuid to derive a deterministic UUID from the document's content and metadata. Documents that already have an ID are left unchanged unless force=True.

.. note:: This function mutates the documents in place and also returns the same list, allowing it to be used inline.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to assign IDs to.	required
force	bool	If True, recomputes and overwrites the ID for every document, even those that already have one. Defaults to False.	False

Returns:

Type	Description
list[Document]	The same list of :class:~langchain_core.documents.Document
list[Document]	instances, with id set on any document that previously had
list[Document]	id=None, or on all documents if force=True.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import assign_ids
>>> docs = [
...     Document(page_content="Hello"),
...     Document(page_content="World", id="existing-id"),
... ]
>>> docs = assign_ids(docs)
>>> docs[0].id is not None
True
>>> docs[1].id
'existing-id'
>>> docs = assign_ids(docs, force=True)
>>> docs[1].id != "existing-id"
True

zenpyre.documents.check_document_consistency

check_document_consistency(
    docs: list[Document], *, raise_error: bool = False
) -> bool

Check that documents sharing the same id have the same page_content and metadata.

Documents with id=None are ignored, since None does not identify a single logical document. metadata is compared via a canonical JSON serialization (:func:json.dumps with sort_keys=True), so metadata key order does not affect equality. This means metadata values must be JSON-serializable.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to check.	required
raise_error	bool	If True, raises :class:DocumentConsistencyError on the first inconsistency found. If False, logs a warning for each inconsistent document and continues checking the rest.	False

Returns:

Type	Description
bool	True if all documents are consistent, False if at least
bool	one inconsistency was found. Always returns True when
bool	raise_error=True, because an inconsistency raises instead of
bool	returning False.

Raises:

Type	Description
DocumentConsistencyError	if raise_error=True and two documents with the same id have different page_content or metadata.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import check_document_consistency
>>> docs = [
...     Document(id="1", page_content="A", metadata={"source": "a.txt"}),
...     Document(id="1", page_content="A", metadata={"source": "a.txt"}),
... ]
>>> check_document_consistency(docs)
True

zenpyre.documents.copy_ids_to_metadata

copy_ids_to_metadata(
    documents: list[Document],
    metadata_key: str = "source_id",
) -> list[Document]

Copy each document's id into its metadata under metadata_key.

Text splitters generally copy a parent document's metadata onto every chunk they produce, but they do not preserve the parent's id (each chunk gets its own, usually None unless assigned later). Storing the parent id in metadata before splitting means every resulting chunk retains a reference back to the document it came from, under chunk.metadata[metadata_key].

Documents are mutated in place and the same list is returned. Documents whose id is None are left untouched, so no key is added for them.

Parameters:

Name	Type	Description	Default
documents	list[Document]	The documents to tag. Mutated in place.	required
metadata_key	str	The metadata key to store the id under. Defaults to "source_id".	'source_id'

Returns:

Type	Description
list[Document]	The same list of documents that was passed in.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import copy_ids_to_metadata
>>> docs = [Document(page_content="Hello", id="doc-1")]
>>> copy_ids_to_metadata(docs)
>>> docs[0].metadata["source_id"]
'doc-1'

zenpyre.documents.deduplicate_documents

deduplicate_documents(
    docs: list[Document], log: bool = False
) -> list[Document]

Remove duplicate documents from a list.

Two documents are considered duplicates only if their id, page_content, and metadata are all equal. metadata is compared via a canonical JSON serialization (:func:json.dumps with sort_keys=True), so metadata key order does not affect equality. This means metadata values must be JSON-serializable.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to deduplicate.	required
log	bool	If True, log the initial and final number of documents, along with the number of duplicates removed.	False

Returns:

Type	Description
list[Document]	A new list containing the first occurrence of each unique
list[Document]	(id, page_content, metadata) combination, in the original
list[Document]	relative order. The input list is not modified.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import deduplicate_documents
>>> docs = [
...     Document(page_content="A", metadata={"source": "a.txt"}),
...     Document(page_content="B", metadata={"source": "b.txt"}),
...     Document(page_content="A", metadata={"source": "a.txt"}),
... ]
>>> result = deduplicate_documents(docs)
>>> [doc.page_content for doc in result]
['A', 'B']

zenpyre.documents.filter_by_metadata

filter_by_metadata(
    docs: list[Document], metadata_key: str, value: Any
) -> list[Document]

Filter a list of documents by the value of a metadata key.

Returns a new list containing only documents whose metadata contains metadata_key with a value equal to value. Documents missing metadata_key are excluded.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to filter.	required
metadata_key	str	The metadata key to filter by.	required
value	Any	The value to match against. Documents whose metadata_key equals this value are kept.	required

Returns:

Type	Description
list[Document]	A new list of :class:~langchain_core.documents.Document
list[Document]	instances whose metadata matches the filter. The original
list[Document]	list is not modified.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import filter_by_metadata
>>> docs = [
...     Document(page_content="A", metadata={"category": "Science"}),
...     Document(page_content="B", metadata={"category": "Cooking"}),
...     Document(page_content="C", metadata={"category": "Science"}),
... ]
>>> result = filter_by_metadata(docs, "category", "Science")
>>> [doc.page_content for doc in result]
['A', 'C']

zenpyre.documents.filter_by_metadata_range

filter_by_metadata_range(
    docs: list[Document],
    metadata_key: str,
    lower: Any = None,
    upper: Any = None,
) -> list[Document]

Filter a list of documents by a range of values for a metadata key.

Returns a new list containing only documents whose metadata contains metadata_key with a value within the specified range [lower, upper] (inclusive on both ends). Either bound can be set to None to indicate no constraint on that side. If both bounds are None, all documents that contain metadata_key are returned. Documents missing metadata_key are always excluded.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to filter.	required
metadata_key	str	The metadata key to filter by.	required
lower	Any	The inclusive lower bound. Pass None (the default) for no lower bound.	None
upper	Any	The inclusive upper bound. Pass None (the default) for no upper bound.	None

Returns:

Type	Description
list[Document]	A new list of :class:~langchain_core.documents.Document
list[Document]	instances whose metadata_key value falls within
list[Document]	[lower, upper]. The original list is not modified.

Raises:

Type	Description
TypeError	If the metadata values are not comparable with the provided bounds (e.g. comparing str with int).

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import filter_by_metadata_range
>>> docs = [
...     Document(page_content="A", metadata={"page": 1}),
...     Document(page_content="B", metadata={"page": 5}),
...     Document(page_content="C", metadata={"page": 10}),
... ]
>>> result = filter_by_metadata_range(docs, "page", lower=2, upper=8)
>>> [doc.page_content for doc in result]
['B']
>>> result = filter_by_metadata_range(docs, "page", lower=5)
>>> [doc.page_content for doc in result]
['B', 'C']
>>> result = filter_by_metadata_range(docs, "page", upper=5)
>>> [doc.page_content for doc in result]
['A', 'B']

zenpyre.documents.filter_by_metadata_values

filter_by_metadata_values(
    docs: list[Document],
    metadata_key: str,
    values: set[Any],
) -> list[Document]

Filter a list of documents by checking if a metadata value is in a set.

Returns a new list containing only documents whose metadata contains metadata_key with a value that is a member of values. Documents missing metadata_key are excluded.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to filter.	required
metadata_key	str	The metadata key to filter by.	required
values	set[Any]	The set of accepted values. Documents whose metadata_key is in this set are kept.	required

Returns:

Type	Description
list[Document]	A new list of :class:~langchain_core.documents.Document
list[Document]	instances whose metadata_key value is in values. The
list[Document]	original list is not modified.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import filter_by_metadata_values
>>> docs = [
...     Document(page_content="A", metadata={"category": "Science"}),
...     Document(page_content="B", metadata={"category": "Cooking"}),
...     Document(page_content="C", metadata={"category": "Technology"}),
...     Document(page_content="D", metadata={"category": "Science"}),
... ]
>>> result = filter_by_metadata_values(docs, "category", {"Science", "Technology"})
>>> sorted(doc.page_content for doc in result)
['A', 'C', 'D']

zenpyre.documents.format_documents

format_documents(
    documents: list[Document],
    include_metadata: bool = False,
    output_format: str = "xml",
) -> str

Concatenate a list of LangChain documents into a single LLM- friendly string, in either XML or Markdown format.

This is a convenience dispatcher over :func:format_documents_as_xml and :func:format_documents_as_markdown. See those functions for details on how each format renders documents and metadata.

Parameters:

Name	Type	Description	Default
documents	list[Document]	The documents to concatenate.	required
include_metadata	bool	If True, include each document's metadata above its content, sorted alphabetically by key. Defaults to False.	False
output_format	str	Either "xml" or "markdown". Defaults to "xml".	'xml'

Returns:

Type	Description
str	A single string with one document block per document, in the same
str	order as documents. Returns an empty string if documents
str	is empty.

Raises:

Type	Description
ValueError	If output_format is not "xml" or "markdown".

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import format_documents
>>> docs = [
...     Document(page_content="The cat sat on the mat."),
... ]
>>> print(format_documents(docs, output_format="xml"))
<document id="1">
The cat sat on the mat.
</document>
>>> print(format_documents(docs, output_format="markdown"))
## Document 1
<BLANKLINE>
The cat sat on the mat.

zenpyre.documents.format_documents_as_json

format_documents_as_json(
    documents: list[Document],
    include_metadata: bool = False,
) -> str

Concatenate a list of LangChain documents into a single LLM- friendly JSON string.

Each document is rendered as an object with an ``id`` field and a
``content`` field. When ``include_metadata`` is ``True``, a
``metadata`` field (a JSON object, keys sorted alphabetically) is
also included.

Parameters:

Name	Type	Description	Default
documents	list[Document]	The documents to concatenate.	required
include_metadata	bool	If True, include each document's metadata as a nested object. Defaults to False.	False

Returns:

Type	Description
str	A JSON array (as a string) with one object per document, in the
str	same order as documents. Returns "[]" if documents is
str	empty.

Example:

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import format_documents_as_json
>>> docs = [
...     Document(
...         page_content="The cat sat on the mat.",
...         metadata={"source": "story.txt", "author": "Alice"},
...     ),
... ]
>>> print(format_documents_as_json(docs))
[
  {
    "id": 1,
    "content": "The cat sat on the mat."
  }
]
>>> print(format_documents_as_json(docs, include_metadata=True))
[
  {
    "id": 1,
    "metadata": {
      "author": "Alice",
      "source": "story.txt"
    },
    "content": "The cat sat on the mat."
  }
]
>>> format_documents_as_json([])
'[]'

zenpyre.documents.format_documents_as_markdown

format_documents_as_markdown(
    documents: list[Document],
    include_metadata: bool = False,
) -> str

Concatenate a list of LangChain documents into a single LLM- friendly Markdown string.

Each document is rendered under its own level-2 heading (## Document N) so the LLM can distinguish document boundaries. When include_metadata is True, each document's metadata is rendered as a Markdown bullet list above its content, sorted alphabetically by key.

Parameters:

Name	Type	Description	Default
documents	list[Document]	The documents to concatenate.	required
include_metadata	bool	If True, include each document's metadata (as a bullet list, sorted by key) above its content. Defaults to False.	False

Returns:

Type	Description
str	A single string with one ## Document N section per document,
str	in the same order as documents. Returns an empty string if
str	documents is empty.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import format_documents_as_markdown
>>> docs = [
...     Document(
...         page_content="The cat sat on the mat.",
...         metadata={"source": "story.txt", "author": "Alice"},
...     ),
...     Document(
...         page_content="The dog chased the ball.",
...         metadata={"source": "story.txt", "author": "Bob"},
...     ),
... ]
>>> print(format_documents_as_markdown(docs))
## Document 1
<BLANKLINE>
The cat sat on the mat.
<BLANKLINE>
## Document 2
<BLANKLINE>
The dog chased the ball.
>>> print(format_documents_as_markdown(docs, include_metadata=True))
## Document 1
<BLANKLINE>
- author: Alice
- source: story.txt
<BLANKLINE>
The cat sat on the mat.
<BLANKLINE>
## Document 2
<BLANKLINE>
- author: Bob
- source: story.txt
<BLANKLINE>
The dog chased the ball.
>>> format_documents_as_markdown([])
''

zenpyre.documents.format_documents_as_xml

format_documents_as_xml(
    documents: list[Document],
    include_metadata: bool = False,
) -> str

Concatenate a list of LangChain documents into a single LLM- friendly XML-tagged string.

Each document is rendered as a clearly delimited <document> block so the LLM can distinguish document boundaries. When include_metadata is True, each document's metadata is rendered above its content as key: value lines, sorted alphabetically by key.

Parameters:

Name	Type	Description	Default
documents	list[Document]	The documents to concatenate.	required
include_metadata	bool	If True, include each document's metadata (as key: value lines, sorted by key) above its content. Defaults to False.	False

Returns:

Type	Description
str	A single string with one <document> block per document, in the
str	same order as documents. Returns an empty string if
str	documents is empty.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import format_documents_as_xml
>>> docs = [
...     Document(
...         page_content="The cat sat on the mat.",
...         metadata={"source": "story.txt", "author": "Alice"},
...     ),
...     Document(
...         page_content="The dog chased the ball.",
...         metadata={"source": "story.txt", "author": "Bob"},
...     ),
... ]
>>> print(format_documents_as_xml(docs))
<document id="1">
The cat sat on the mat.
</document>
<BLANKLINE>
<document id="2">
The dog chased the ball.
</document>
>>> print(format_documents_as_xml(docs, include_metadata=True))
<document id="1">
author: Alice
source: story.txt
<BLANKLINE>
The cat sat on the mat.
</document>
<BLANKLINE>
<document id="2">
author: Bob
source: story.txt
<BLANKLINE>
The dog chased the ball.
</document>
>>> format_documents_as_xml([])
''

zenpyre.documents.get_document_id_lengths

get_document_id_lengths(
    documents: Iterable[Document], *, sort: bool = False
) -> list[tuple[Any, int]]

Compute the number of characters in each document's page_content.

Parameters:

Name	Type	Description	Default
documents	Iterable[Document]	A list, generator, or other iterable of langchain_core.documents.Document objects. Consumed exactly once; if a generator/iterator is passed in, it will be exhausted by this call.	required
sort	bool	If True, sort the output by character count, from shortest to longest. If False, the output preserves the order of documents.	False

Returns:

Type	Description
list[tuple[Any, int]]	A list of (document_id, char_count) tuples, one per input
list[tuple[Any, int]]	document. A document whose page_content is not a string
list[tuple[Any, int]]	(e.g. None) is treated as having a length of 0.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import get_document_id_lengths
>>> docs = [
...     Document(id="a", page_content="hello"),
...     Document(id="b", page_content="hello world"),
... ]
>>> get_document_id_lengths(docs)
[('a', 5), ('b', 11)]
>>> get_document_id_lengths(docs, sort=True)
[('a', 5), ('b', 11)]

zenpyre.documents.get_document_length

get_document_length(document: Document) -> int

Compute the number of characters in a document's page_content.

Parameters:

Name	Type	Description	Default
document	Document	The langchain_core.documents.Document to measure.	required

Returns:

Type	Description
int	The length, in characters, of page_content, or 0 if
int	page_content is not a string (e.g. None).

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import get_document_length
>>> get_document_length(Document(page_content="hello"))
5

zenpyre.documents.get_document_lengths

get_document_lengths(
    documents: Iterable[Document],
) -> list[int]

Compute the number of characters in each document's page_content.

Parameters:

Name	Type	Description	Default
documents	Iterable[Document]	A list, generator, or other iterable of langchain_core.documents.Document objects. Consumed exactly once; if a generator/iterator is passed in, it will be exhausted by this call.	required

Returns:

Type	Description
list[int]	A list of character counts, one per input document, in the
list[int]	same order as documents. A document whose page_content
list[int]	is not a string (e.g. None) is treated as having a length
list[int]	of 0.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import get_document_lengths
>>> docs = [
...     Document(id="a", page_content="hello"),
...     Document(id="b", page_content="hello world"),
... ]
>>> get_document_lengths(docs)
[5, 11]

zenpyre.documents.get_longest_document

get_longest_document(
    documents: Iterable[Document],
    *,
    ignore_empty: bool = False,
    treat_whitespace_as_empty: bool = False
) -> Document | None

Find the document with the longest page_content.

Streams through documents one at a time and keeps only the current longest document, so memory usage is O(1) regardless of how many documents are processed (aside from whatever the input iterable itself holds in memory).

Parameters:

Name	Type	Description	Default
documents	Iterable[Document]	A list, generator, or other iterable of langchain_core.documents.Document objects. Consumed exactly once; if a generator/iterator is passed in, it will be exhausted by this call.	required
ignore_empty	bool	If True, documents whose page_content is empty are skipped, so the longest non-empty document is returned instead.	False
treat_whitespace_as_empty	bool	If True, a page_content that contains only whitespace is also considered empty for the purpose of ignore_empty. Has no effect if ignore_empty is False.	False

Returns:

Type	Description
Document | None	The first document with the largest page_content length
Document | None	(ties broken by the earliest occurrence in documents), or
Document | None	None if documents is empty or, when ignore_empty is
Document | None	True, if every document is empty (or whitespace-only, when
Document | None	treat_whitespace_as_empty is also True).

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import get_longest_document
>>> docs = [
...     Document(id="a", page_content="hello world"),
...     Document(id="b", page_content=""),
...     Document(id="c", page_content="hi"),
... ]
>>> get_longest_document(docs).id
'a'

zenpyre.documents.get_shortest_document

get_shortest_document(
    documents: Iterable[Document],
    *,
    ignore_empty: bool = False,
    treat_whitespace_as_empty: bool = False
) -> Document | None

Find the document with the shortest page_content.

Streams through documents one at a time and keeps only the current shortest document, so memory usage is O(1) regardless of how many documents are processed (aside from whatever the input iterable itself holds in memory).

Parameters:

Name	Type	Description	Default
documents	Iterable[Document]	A list, generator, or other iterable of langchain_core.documents.Document objects. Consumed exactly once; if a generator/iterator is passed in, it will be exhausted by this call.	required
ignore_empty	bool	If True, documents whose page_content is empty are skipped, so the shortest non-empty document is returned instead.	False
treat_whitespace_as_empty	bool	If True, a page_content that contains only whitespace is also considered empty for the purpose of ignore_empty. Has no effect if ignore_empty is False.	False

Returns:

Type	Description
Document | None	The first document with the smallest page_content length
Document | None	(ties broken by the earliest occurrence in documents), or
Document | None	None if documents is empty or, when ignore_empty is
Document | None	True, if every document is empty (or whitespace-only, when
Document | None	treat_whitespace_as_empty is also True).

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import get_shortest_document
>>> docs = [
...     Document(id="a", page_content="hello world"),
...     Document(id="b", page_content=""),
...     Document(id="c", page_content="hi"),
... ]
>>> get_shortest_document(docs).id
'b'
>>> get_shortest_document(docs, ignore_empty=True).id
'c'

zenpyre.documents.hash_document

hash_document(doc: Document, length: int = 64) -> str

Compute a stable, reproducible hash of a LangChain document.

Combines the document's page_content and metadata into a single canonical string and hashes it. Metadata is serialised via :func:json.dumps with sort_keys=True to guarantee a consistent ordering regardless of the dict insertion order.

Parameters:

Name	Type	Description	Default
doc	Document	The :class:~langchain_core.documents.Document to hash.	required
length	int	The desired length of the returned hex string. Must be an even number between 2 and 128 inclusive. Defaults to 64.	64

Returns:

Type	Description
str	A lowercase hexadecimal string of exactly length characters
str	that uniquely identifies the document's content and metadata.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import hash_document
>>> doc = Document(page_content="Hello", metadata={"source": "cats.txt"})
>>> len(hash_document(doc))
64

zenpyre.documents.hash_document_uuid

hash_document_uuid(doc: Document) -> str

Compute a stable, reproducible UUID for a LangChain document.

Uses :func:uuid.uuid5 (SHA-1 based) with a fixed project-specific namespace to derive a deterministic UUID from the document's page_content and metadata. Metadata is serialised via :func:json.dumps with sort_keys=True to guarantee a consistent ordering regardless of dict insertion order.

The returned UUID can be assigned directly to :attr:~langchain_core.documents.Document.id, which LangChain expects to be a UUID string. This makes re-indexing idempotent — adding the same document twice with the same ID upserts rather than duplicates.

Parameters:

Name	Type	Description	Default
doc	Document	The :class:~langchain_core.documents.Document to hash.	required

Returns:

Type	Description
str	A lowercase UUID string of the form
str	'xxxxxxxx-xxxx-5xxx-xxxx-xxxxxxxxxxxx'.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import hash_document_uuid
>>> doc = Document(page_content="Hello", metadata={"source": "cats.txt"})
>>> hash_document_uuid(doc)

zenpyre.documents.hash_documents

hash_documents(
    docs: list[Document], length: int = 64
) -> str

Compute a stable, reproducible hash of a list of LangChain documents.

Hashes each document individually via :func:hash_document and combines the results into a single canonical string, then hashes that. The order of documents matters — two lists with the same documents in a different order will produce different hashes.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to hash.	required
length	int	The desired length of the returned hex string. Must be an even number between 2 and 128 inclusive. Defaults to 64.	64

Returns:

Type	Description
str	A lowercase hexadecimal string of exactly length characters
str	that uniquely identifies the list's content, metadata, and
str	ordering.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import hash_documents
>>> docs = [
...     Document(page_content="Hello", metadata={"source": "a.txt"}),
...     Document(page_content="World", metadata={"source": "b.txt"}),
... ]
>>> len(hash_documents(docs))
64

zenpyre.documents.is_document_empty

is_document_empty(
    document: Document,
    *,
    treat_whitespace_as_empty: bool = False
) -> bool

Determine if a document's page_content is empty.

Parameters:

Name	Type	Description	Default
document	Document	The langchain_core.documents.Document to check.	required
treat_whitespace_as_empty	bool	If True, a page_content that contains only whitespace is also considered empty.	False

Returns:

Type	Description
bool	True if page_content is the empty string (or is not a
bool	string, e.g. None), or, if treat_whitespace_as_empty is
bool	True, if page_content is non-empty but contains only
bool	whitespace.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import is_document_empty
>>> is_document_empty(Document(page_content=""))
True
>>> is_document_empty(Document(page_content="hello"))
False
>>> is_document_empty(Document(page_content="  "), treat_whitespace_as_empty=True)
True

zenpyre.documents.is_document_whitespace_only

is_document_whitespace_only(document: Document) -> bool

Determine if a document's page_content is non-empty but contains only whitespace.

Parameters:

Name	Type	Description	Default
document	Document	The langchain_core.documents.Document to check.	required

Returns:

Type	Description
bool	True if page_content is a non-empty string that
bool	contains only whitespace characters. False if
bool	page_content is the empty string, is not a string (e.g.
bool	None), or contains any non-whitespace character.

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import is_document_whitespace_only
>>> is_document_whitespace_only(Document(page_content="  \n"))
True
>>> is_document_whitespace_only(Document(page_content=""))
False
>>> is_document_whitespace_only(Document(page_content="hello"))
False

zenpyre.documents.sort_by_metadata

sort_by_metadata(
    docs: list[Document],
    metadata_key: str,
    *,
    keep_missing: bool = True,
    reverse: bool = False
) -> list[Document]

Sort a list of documents by the value of a metadata key.

Documents are sorted in ascending order by the value of metadata_key by default, or descending order if reverse=True. Documents that do not contain metadata_key in their metadata are placed at the end of the result by default, or removed entirely if keep_missing=False.

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to sort.	required
metadata_key	str	The metadata key to sort by.	required
keep_missing	bool	If True (the default), documents without metadata_key in their metadata are kept and placed at the end of the result. If False, they are excluded from the result entirely.	True
reverse	bool	If True, the result is sorted in descending order. Defaults to False, matching the behaviour of :func:sorted.	False

Returns:

Type	Description
list[Document]	A new sorted list of :class:~langchain_core.documents.Document
list[Document]	instances. The original list is not modified.

Raises:

Type	Description
TypeError	If the metadata values for metadata_key are not mutually comparable (e.g. mixing str and int).

Example

>>> from langchain_core.documents import Document
>>> from zenpyre.documents import sort_by_metadata
>>> docs = [
...     Document(page_content="B", metadata={"source": "b.txt"}),
...     Document(page_content="A", metadata={"source": "a.txt"}),
...     Document(page_content="C"),
... ]
>>> sorted_docs = sort_by_metadata(docs, "source")
>>> [doc.metadata.get("source") for doc in sorted_docs]
['a.txt', 'b.txt', None]
>>> sorted_docs = sort_by_metadata(docs, "source", reverse=True)
>>> [doc.metadata.get("source") for doc in sorted_docs]
['b.txt', 'a.txt', None]
>>> sorted_docs = sort_by_metadata(docs, "source", keep_missing=False)
>>> [doc.metadata.get("source") for doc in sorted_docs]
['a.txt', 'b.txt']