Store

docculus.store

Contain store utilities for docculus.

docculus.store.BaseDocumentStore

Bases: ABC

Abstract base class for document stores.

Defines the common interface that all document store implementations must provide. The API mirrors :class:persista.store.BaseStore, with documents (keyed by their id) taking the place of key-value pairs.

To implement a custom document store, subclass :class:BaseDocumentStore and implement all abstract methods.

Implementations are expected to support use as a context manager (with SomeDocumentStore(...) as store: ...), which calls :meth:open on entry and :meth:close on exit.

Constructing a document store does not connect to the underlying backend: implementations must defer that to :meth:open/ :meth:aopen, so every other method (including :meth:close) raises until the store has been opened, either explicitly or via the context manager.

docculus.store.BaseDocumentStore.closed abstractmethod property

closed: bool

Indicate whether the store is closed.

Returns:

Type	Description
bool	True if the store has been closed, False if it is
bool	open and ready to use.

docculus.store.BaseDocumentStore.aclear abstractmethod async

aclear() -> None

Async equivalent of :meth:clear.

docculus.store.BaseDocumentStore.aclose abstractmethod async

aclose() -> None

Async equivalent of :meth:close.

docculus.store.BaseDocumentStore.acontains abstractmethod async

acontains(doc_id: str) -> bool

Async equivalent of :meth:contains.

docculus.store.BaseDocumentStore.acontains_many abstractmethod async

acontains_many(doc_ids: list[str]) -> list[bool]

Async equivalent of :meth:contains_many.

docculus.store.BaseDocumentStore.acount abstractmethod async

acount() -> int

Async equivalent of :meth:count.

docculus.store.BaseDocumentStore.adelete abstractmethod async

adelete(doc_id: str) -> None

Async equivalent of :meth:delete.

docculus.store.BaseDocumentStore.adelete_many abstractmethod async

adelete_many(doc_ids: list[str]) -> None

Async equivalent of :meth:delete_many.

docculus.store.BaseDocumentStore.afilter abstractmethod async

afilter(**metadata_filters: Any) -> list[Document]

Async equivalent of :meth:filter.

docculus.store.BaseDocumentStore.aget abstractmethod async

aget(doc_id: str) -> Document | None

Async equivalent of :meth:get.

docculus.store.BaseDocumentStore.aget_many abstractmethod async

aget_many(doc_ids: list[str]) -> list[Document | None]

Async equivalent of :meth:get_many.

docculus.store.BaseDocumentStore.aiter_batches abstractmethod

aiter_batches(
    batch_size: int = 32,
) -> AsyncIterator[list[Document]]

Async equivalent of :meth:iter_batches.

docculus.store.BaseDocumentStore.akeys abstractmethod

akeys() -> AsyncIterator[str]

Async equivalent of :meth:keys.

docculus.store.BaseDocumentStore.aopen abstractmethod async

aopen() -> None

Async equivalent of :meth:open.

docculus.store.BaseDocumentStore.aset_many abstractmethod async

aset_many(docs: list[Document]) -> None

Async equivalent of :meth:set_many.

docculus.store.BaseDocumentStore.avalues async

avalues(batch_size: int = 32) -> AsyncIterator[Document]

Async equivalent of :meth:values.

docculus.store.BaseDocumentStore.clear abstractmethod

clear() -> None

Remove every document from the store.

This is equivalent to resetting the store to empty, without closing it.

docculus.store.BaseDocumentStore.close abstractmethod

close() -> None

Close the store and release any underlying resources (e.g. database connections, file handles).

Implementations should make repeated calls to close() safe (i.e. idempotent), since :meth:__exit__ calls it unconditionally and callers may also close a store manually before using it as a context manager.

docculus.store.BaseDocumentStore.contains abstractmethod

contains(doc_id: str) -> bool

Check if a document ID exists in the store.

Parameters:

Name	Type	Description	Default
doc_id	str	The document ID to check.	required

Returns:

Type	Description
bool	True if the document exists in the store, False
bool	otherwise.

docculus.store.BaseDocumentStore.contains_many abstractmethod

contains_many(doc_ids: list[str]) -> list[bool]

Check which document IDs exist in the store.

Parameters:

Name	Type	Description	Default
doc_ids	list[str]	The document IDs to check.	required

Returns:

Type	Description
list[bool]	A list of booleans, in the same order as doc_ids,
list[bool]	where each entry is True if the corresponding document
list[bool]	ID exists in the store and False otherwise.

docculus.store.BaseDocumentStore.count abstractmethod

count() -> int

Return the total number of documents in the store.

Returns:

Type	Description
int	The number of documents currently stored.

docculus.store.BaseDocumentStore.delete abstractmethod

delete(doc_id: str) -> None

Delete a document by its ID.

IDs that do not exist should be silently ignored.

Parameters:

Name	Type	Description	Default
doc_id	str	The ID of the document to delete.	required

docculus.store.BaseDocumentStore.delete_many abstractmethod

delete_many(doc_ids: list[str]) -> None

Delete multiple documents by their IDs.

IDs that do not exist should be silently ignored.

Parameters:

Name	Type	Description	Default
doc_ids	list[str]	The IDs of the documents to delete.	required

docculus.store.BaseDocumentStore.filter abstractmethod

filter(**metadata_filters: Any) -> list[Document]

Retrieve documents matching all provided metadata filters.

All filters should be combined with AND. Each keyword argument matches the corresponding metadata key exactly.

Parameters:

Name	Type	Description	Default
**metadata_filters	Any	Key-value pairs where each key is a metadata field name and the value is the exact value to match. Calling with no arguments should return all documents.	{}

Returns:

Type	Description
list[Document]	A list of matching
list[Document]	class:~langchain_core.documents.Document instances.

docculus.store.BaseDocumentStore.get abstractmethod

get(doc_id: str) -> Document | None

Retrieve a single document by its ID.

Parameters:

Name	Type	Description	Default
doc_id	str	The document ID to look up.	required

Returns:

Name	Type	Description
The	Document | None	class:~langchain_core.documents.Document, or
	Document | None	None if not found.

docculus.store.BaseDocumentStore.get_many abstractmethod

get_many(doc_ids: list[str]) -> list[Document | None]

Retrieve multiple documents by their IDs.

Parameters:

Name	Type	Description	Default
doc_ids	list[str]	The document IDs to look up.	required

Returns:

Name	Type	Description
	list[Document | None]	A list the same length as doc_ids, with the
corresponding	list[Document | None]	class:~langchain_core.documents.Document
	list[Document | None]	for each ID that exists, or None for IDs not found.

docculus.store.BaseDocumentStore.iter_batches abstractmethod

iter_batches(
    batch_size: int = 32,
) -> Generator[list[Document], None, None]

Yield documents in batches, avoiding loading the whole store into memory at once.

This is the scalable equivalent of :meth:values: instead of materializing every document as a single list, it streams them from the database in chunks of batch_size.

Parameters:

Name	Type	Description	Default
batch_size	int	The maximum number of documents to yield per batch. Must be a positive integer.	32

Yields:

Type	Description
list[Document]	Lists of documents, each with at most batch_size
list[Document]	elements, in the same order as :meth:values. The last
list[Document]	batch may contain fewer than batch_size documents.

Example

>>> from docculus.store import BaseDocumentStore
>>> from langchain_core.documents import Document
>>> store: BaseDocumentStore = ...  # doctest: +SKIP
>>> store.set_many(
...     [Document(id=str(i), page_content=str(i)) for i in range(5)]
... )  # doctest: +SKIP
>>> for batch in store.iter_batches(batch_size=2):  # doctest: +SKIP
...     print(len(batch))
...
2
2
1

docculus.store.BaseDocumentStore.keys abstractmethod

keys() -> Iterator[str]

Iterate over the IDs of all documents in the store.

Yields:

Type	Description
str	Every document ID currently in the store.

docculus.store.BaseDocumentStore.open abstractmethod

open() -> None

Connect to the underlying backend and prepare the store for use (e.g. open a database connection, create a directory).

The constructor must not do this itself: implementations connect lazily, only once open() (or :meth:__enter__) is called. Implementations should make repeated calls to open() safe (i.e. idempotent), since a store may be reopened after :meth:close.

docculus.store.BaseDocumentStore.set_many abstractmethod

set_many(docs: list[Document]) -> None

Add or replace documents in the store.

Documents whose id already exists should be replaced (upsert semantics).

Parameters:

Name	Type	Description	Default
docs	list[Document]	The list of :class:~langchain_core.documents.Document instances to add. Each document must have an id.	required

Raises:

Type	Description
ValueError	If any document has no id.

docculus.store.BaseDocumentStore.values

values(batch_size: int = 32) -> Iterator[Document]

Lazily iterate over all documents without loading them all into memory at once.

Parameters:

Name	Type	Description	Default
batch_size	int	The batch size used internally when pulling documents from the underlying store. Does not affect the granularity of what is yielded — documents are always yielded one at a time.	32

Yields:

Name	Type	Description
One	Document	class:~langchain_core.documents.Document at a time,
	Document	in the same order as :meth:iter_batches.

docculus.store.DocumentStore

Bases: BaseDocumentStore, MultilineDisplayMixin

Implement a document store backed by a :class:persista.store.BaseStore.

Documents are keyed by their id in the underlying key-value store, so the id itself is not duplicated in the stored value. metadata_mode controls how a document's metadata is represented in the stored value:

• "single": the metadata dict is stored as a single nested value under the "metadata" key.
• "flat": each metadata field is stored as its own top-level key in the stored value, alongside "page_content".

Parameters:

Name	Type	Description	Default
store	BaseStore	The underlying key-value store.	required
metadata_mode	MetadataMode	How document metadata is represented in the values stored in store.	'flat'

docculus.store.DuckDBDocumentStore

Bases: DocumentStore

Implement a :class:~docculus.store.document.DocumentStore backed by a DuckDB database, with document metadata stored as a single JSON column.

This is a convenience wrapper around :class:persista.store.TypedDuckDBStore that configures it with a fixed schema suitable for storing :class:~langchain_core.documents.Document instances: a page_content text column and a metadata JSON column. It is equivalent to constructing a :class:~docculus.store.document.DocumentStore with metadata_mode="single" around such a store.

Use this store when document metadata schemas vary from one document to another, or when the metadata fields do not need to be queried directly with SQL. Use :class:TypedDuckDBDocumentStore instead when metadata fields should be stored as their own SQL columns.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the DuckDB database file, or ":memory:" for an in-memory database.	':memory:'
**kwargs	Any	Additional keyword arguments passed to :class:persista.store.TypedDuckDBStore.	{}

Example

>>> from docculus.store import DuckDBDocumentStore
>>> from langchain_core.documents import Document
>>> with DuckDBDocumentStore() as store:  # doctest: +SKIP
...     store.set_many(
...         [Document(id="1", page_content="hello", metadata={"author": "Alice"})]
...     )
...     store.get("1")
...
Document(id='1', page_content='hello', metadata={'author': 'Alice'})

docculus.store.InMemoryDocumentStore

Bases: DocumentStore

Implement a :class:~docculus.store.document.DocumentStore backed by a plain in-memory dictionary store.

This is a convenience wrapper around :class:persista.store.InMemoryStore, useful for tests, examples, and other scenarios where documents do not need to be persisted across processes. Since the underlying store is a Python dict rather than a SQL database, its metadata_mode is fixed to the :class:~docculus.store.document.DocumentStore default ("flat").

Example

>>> from docculus.store import InMemoryDocumentStore
>>> from langchain_core.documents import Document
>>> with InMemoryDocumentStore() as store:  # doctest: +SKIP
...     store.set_many(
...         [Document(id="1", page_content="hello", metadata={"author": "Alice"})]
...     )
...     store.get("1")
...
Document(id='1', page_content='hello', metadata={'author': 'Alice'})

docculus.store.SQLiteDocumentStore

Bases: DocumentStore

Implement a :class:~docculus.store.document.DocumentStore backed by a SQLite database, with document metadata stored as a single JSON column.

This is a convenience wrapper around :class:persista.store.TypedSQLiteStore that configures it with a fixed schema suitable for storing :class:~langchain_core.documents.Document instances: a page_content text column and a metadata JSON column. It is equivalent to constructing a :class:~docculus.store.document.DocumentStore with metadata_mode="single" around such a store.

Use this store when document metadata schemas vary from one document to another, or when the metadata fields do not need to be queried directly with SQL. Use :class:TypedSQLiteDocumentStore instead when metadata fields should be stored as their own SQL columns.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the SQLite database file, or ":memory:" for an in-memory database.	':memory:'
**kwargs	Any	Additional keyword arguments passed to :class:persista.store.TypedSQLiteStore.	{}

Example

>>> from docculus.store import SQLiteDocumentStore
>>> from langchain_core.documents import Document
>>> with SQLiteDocumentStore() as store:  # doctest: +SKIP
...     store.set_many(
...         [Document(id="1", page_content="hello", metadata={"author": "Alice"})]
...     )
...     store.get("1")
...
Document(id='1', page_content='hello', metadata={'author': 'Alice'})

docculus.store.TypedDuckDBDocumentStore

Bases: DocumentStore

Implement a :class:~docculus.store.document.DocumentStore backed by a DuckDB database, with each document metadata field stored as its own typed SQL column.

This is a convenience wrapper around :class:persista.store.TypedDuckDBStore that configures it with a page_content text column plus one column per metadata field declared in metadata_schema. It is equivalent to constructing a :class:~docculus.store.document.DocumentStore with metadata_mode="flat" around such a store.

Use this store when document metadata follows a known, fixed schema and individual metadata fields should be queryable as SQL columns. Use :class:DuckDBDocumentStore instead when documents may have arbitrary or varying metadata.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the DuckDB database file, or ":memory:" for an in-memory database.	':memory:'
metadata_schema	dict[str, str] | None	A mapping from metadata field name to its SQL column type declaration (e.g. {"author": "TEXT"}). None is equivalent to an empty mapping, i.e. no metadata columns beyond page_content.	None
**kwargs	Any	Additional keyword arguments passed to :class:persista.store.TypedDuckDBStore.	{}

Example

>>> from docculus.store import TypedDuckDBDocumentStore
>>> from langchain_core.documents import Document
>>> with TypedDuckDBDocumentStore(
...     metadata_schema={"author": "TEXT"}
... ) as store:  # doctest: +SKIP
...     store.set_many(
...         [Document(id="1", page_content="hello", metadata={"author": "Alice"})]
...     )
...     store.get("1")
...
Document(id='1', page_content='hello', metadata={'author': 'Alice'})

docculus.store.TypedSQLiteDocumentStore

Bases: DocumentStore

Implement a :class:~docculus.store.document.DocumentStore backed by a SQLite database, with each document metadata field stored as its own typed SQL column.

This is a convenience wrapper around :class:persista.store.TypedSQLiteStore that configures it with a page_content text column plus one column per metadata field declared in metadata_schema. It is equivalent to constructing a :class:~docculus.store.document.DocumentStore with metadata_mode="flat" around such a store.

Use this store when document metadata follows a known, fixed schema and individual metadata fields should be queryable as SQL columns. Use :class:SQLiteDocumentStore instead when documents may have arbitrary or varying metadata.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the SQLite database file, or ":memory:" for an in-memory database.	':memory:'
metadata_schema	dict[str, str] | None	A mapping from metadata field name to its SQL column type declaration (e.g. {"author": "TEXT"}). None is equivalent to an empty mapping, i.e. no metadata columns beyond page_content.	None
**kwargs	Any	Additional keyword arguments passed to :class:persista.store.TypedSQLiteStore.	{}

Example

>>> from docculus.store import TypedSQLiteDocumentStore
>>> from langchain_core.documents import Document
>>> with TypedSQLiteDocumentStore(
...     metadata_schema={"author": "TEXT"}
... ) as store:  # doctest: +SKIP
...     store.set_many(
...         [Document(id="1", page_content="hello", metadata={"author": "Alice"})]
...     )
...     store.get("1")
...
Document(id='1', page_content='hello', metadata={'author': 'Alice'})

docculus.store.factory

Contain factories for document stores.

docculus.store.factory.BaseDocumentStoreFactory

Bases: ABC

Abstract base class for :class:~docculus.store.BaseDocumentStore factories.

Subclasses implement :meth:make_document_store to instantiate and return a configured :class:~docculus.store.BaseDocumentStore object. This pattern decouples document store creation from the rest of the codebase, making it easy to swap how a document store is built (e.g. a shared instance vs. a fresh one per call) without changing call sites.

Example

>>> from docculus.store import BaseDocumentStore, InMemoryDocumentStore
>>> from docculus.store.factory import BaseDocumentStoreFactory
>>> class MyDocumentStoreFactory(BaseDocumentStoreFactory):
...     def make_document_store(self) -> BaseDocumentStore:
...         return InMemoryDocumentStore()
...
>>> factory = MyDocumentStoreFactory()
>>> store = factory.make_document_store()

docculus.store.factory.BaseDocumentStoreFactory.make_document_store abstractmethod

make_document_store() -> BaseDocumentStore

Create and return a configured BaseDocumentStore instance.

Returns:

Name	Type	Description
A	BaseDocumentStore	class:~docculus.store.BaseDocumentStore
	BaseDocumentStore	instance ready for use.

docculus.store.factory.ConfigurableDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

A concrete document store factory that accepts either a pre-built :class:~docculus.store.BaseDocumentStore instance or a configuration dictionary.

When a dict is provided it is resolved at each :meth:make_document_store call via :func:~coola.factory.resolve_object, which uses objectory to instantiate the configured class. When an instance is provided it is returned as-is.

Parameters:

Name	Type	Description	Default
document_store	BaseDocumentStore | dict[str, Any]	A fully configured :class:~docculus.store.BaseDocumentStore instance, or a :class:dict containing an objectory factory specification (must include a "_target_" key pointing to the fully-qualified class name).	required

Example

>>> from docculus.store import InMemoryDocumentStore
>>> from docculus.store.factory import ConfigurableDocumentStoreFactory
>>> factory = ConfigurableDocumentStoreFactory(InMemoryDocumentStore())
>>> store = factory.make_document_store()

docculus.store.factory.DocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

A concrete document store factory that wraps a pre-built :class:~docculus.store.BaseDocumentStore instance.

Use this when the document store is already instantiated and you simply want to wrap it in the :class:~BaseDocumentStoreFactory interface — for example, when injecting a fixed document store into a component that expects a factory.

Parameters:

Name	Type	Description	Default
document_store	BaseDocumentStore	A fully configured :class:~docculus.store.BaseDocumentStore instance to return from :meth:make_document_store.	required

Example

>>> from docculus.store import InMemoryDocumentStore
>>> from docculus.store.factory import DocumentStoreFactory
>>> factory = DocumentStoreFactory(InMemoryDocumentStore())
>>> store = factory.make_document_store()

docculus.store.factory.DuckDBDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

Implement a document store factory that builds a new :class:~docculus.store.DuckDBDocumentStore on each call.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the DuckDB database file, or ":memory:" for an in-memory database.	':memory:'
**kwargs	Any	Additional keyword arguments passed to :class:~docculus.store.DuckDBDocumentStore.	{}

Example

>>> from docculus.store.factory import DuckDBDocumentStoreFactory
>>> factory = DuckDBDocumentStoreFactory()
>>> store = factory.make_document_store()

docculus.store.factory.InMemoryDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

Implement a document store factory that builds a new :class:~docculus.store.InMemoryDocumentStore on each call.

Example

>>> from docculus.store.factory import InMemoryDocumentStoreFactory
>>> factory = InMemoryDocumentStoreFactory()
>>> store = factory.make_document_store()

docculus.store.factory.SQLiteDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

Implement a document store factory that builds a new :class:~docculus.store.SQLiteDocumentStore on each call.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the SQLite database file, or ":memory:" for an in-memory database.	':memory:'
**kwargs	Any	Additional keyword arguments passed to :class:~docculus.store.SQLiteDocumentStore.	{}

Example

>>> from docculus.store.factory import SQLiteDocumentStoreFactory
>>> factory = SQLiteDocumentStoreFactory()
>>> store = factory.make_document_store()

docculus.store.factory.StoreDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

A concrete document store factory that builds its backing store from a :class:~persista.store.factory.BaseStoreFactory.

Use this when the underlying key-value store needs to be freshly created (e.g. a new connection, a new in-memory dict) each time a :class:~docculus.store.DocumentStore is requested, rather than sharing one store instance across every document store.

Parameters:

Name	Type	Description	Default
store_factory	BaseStoreFactory	The factory used to create the backing key-value store passed to each :class:~docculus.store.DocumentStore built by :meth:make_document_store.	required
metadata_mode	MetadataMode	Forwarded to each created :class:~docculus.store.DocumentStore. See :class:~docculus.store.DocumentStore for details.	'flat'

Example

>>> from docculus.store.factory import StoreDocumentStoreFactory
>>> from persista.store.factory import StoreFactory
>>> from persista.store import InMemoryStore
>>> factory = StoreDocumentStoreFactory(StoreFactory(InMemoryStore()))
>>> store = factory.make_document_store()
>>> store.open()

docculus.store.factory.TypedDuckDBDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

Implement a document store factory that builds a new :class:~docculus.store.TypedDuckDBDocumentStore on each call.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the DuckDB database file, or ":memory:" for an in-memory database.	':memory:'
metadata_schema	dict[str, str] | None	A mapping from metadata field name to its SQL column type declaration (e.g. {"author": "TEXT"}). None is equivalent to an empty mapping, i.e. no metadata columns beyond page_content.	None
**kwargs	Any	Additional keyword arguments passed to :class:~docculus.store.TypedDuckDBDocumentStore.	{}

Example

>>> from docculus.store.factory import TypedDuckDBDocumentStoreFactory
>>> factory = TypedDuckDBDocumentStoreFactory(metadata_schema={"author": "TEXT"})
>>> store = factory.make_document_store()

docculus.store.factory.TypedSQLiteDocumentStoreFactory

Bases: BaseDocumentStoreFactory, MultilineDisplayMixin

Implement a document store factory that builds a new :class:~docculus.store.TypedSQLiteDocumentStore on each call.

Parameters:

Name	Type	Description	Default
database	Path | str	The path to the SQLite database file, or ":memory:" for an in-memory database.	':memory:'
metadata_schema	dict[str, str] | None	A mapping from metadata field name to its SQL column type declaration (e.g. {"author": "TEXT"}). None is equivalent to an empty mapping, i.e. no metadata columns beyond page_content.	None
**kwargs	Any	Additional keyword arguments passed to :class:~docculus.store.TypedSQLiteDocumentStore.	{}

Example

>>> from docculus.store.factory import TypedSQLiteDocumentStoreFactory
>>> factory = TypedSQLiteDocumentStoreFactory(metadata_schema={"author": "TEXT"})
>>> store = factory.make_document_store()