dulwich.object_filters

module documentation

Object filtering for Git partial clone and pack generation.

This module implements Git's object filter specifications for partial clone, as documented in: https://git-scm.com/docs/rev-list-options#Documentation/rev-list-options.txt---filterltfilter-specgt

Filter specifications control which objects are included when generating packs, enabling partial clone (downloading only needed objects) and similar operations.

Supported filter specs: - blob:none - Exclude all blobs - blob:limit=<n>[kmg] - Exclude blobs larger than n bytes/KB/MB/GB - tree:<depth> - Exclude trees beyond specified depth - sparse:oid=<oid> - Use sparse specification from object - combine:<filter>+<filter>+... - Combine multiple filters

Class BlobLimitFilter Filter that excludes blobs larger than a specified size.
Class BlobNoneFilter Filter that excludes all blobs.
Class CombineFilter Filter that combines multiple filters with AND logic.
Class FilterSpec Base class for all filter specifications.
Class SparseOidFilter Filter that uses a sparse specification from an object.
Class TreeDepthFilter Filter that excludes trees beyond a specified depth.
Function filter_pack_objects Filter a list of object IDs based on a filter specification.
Function filter_pack_objects_with_paths Filter objects for a pack with full path and depth tracking.
Function parse_filter_spec Parse a filter specification string.
Function _parse_size Parse a size specification like '100', '10k', '5m', '1g'.
def filter_pack_objects(object_store: BaseObjectStore, object_ids: list[ObjectID], filter_spec: FilterSpec) -> list[ObjectID]:

Filter a list of object IDs based on a filter specification.

This function examines each object and excludes those that don't match the filter criteria (e.g., blobs that are too large, trees beyond max depth).

Note

This function currently supports blob size filtering. Tree depth filtering requires additional path/depth tracking which is not yet implemented.

Parameters
object_store:BaseObjectStoreObject store to retrieve objects from
object_ids:list[ObjectID]List of object IDs to filter
filter_spec:FilterSpecFilter specification to apply
Returns
list[ObjectID]Filtered list of object IDs that should be included in the pack
def filter_pack_objects_with_paths(object_store: BaseObjectStore, wants: list[ObjectID], filter_spec: FilterSpec, *, progress: Callable[[bytes], None] | None = None) -> list[ObjectID]:

Filter objects for a pack with full path and depth tracking.

This function performs a complete tree traversal starting from the wanted commits, tracking paths and depths to enable proper filtering for sparse:oid and tree:<depth> filters.

Parameters
object_store:BaseObjectStoreObject store to retrieve objects from
wants:list[ObjectID]List of commit/tree/blob IDs that are wanted
filter_spec:FilterSpecFilter specification to apply
progress:Callable[[bytes], None] | NoneOptional progress callback
Returns
list[ObjectID]Filtered list of object IDs that should be included in the pack
def parse_filter_spec(spec: str | bytes, object_store: BaseObjectStore | None = None) -> FilterSpec:

Parse a filter specification string.

Examples

>>> parse_filter_spec("blob:none")
BlobNoneFilter()
>>> parse_filter_spec("blob:limit=1m")
BlobLimitFilter(limit=1048576)
>>> parse_filter_spec("tree:0")
TreeDepthFilter(max_depth=0)
Parameters
spec:str | bytesFilter specification (e.g., 'blob:none', 'blob:limit=1m')
object_store:BaseObjectStore | NoneOptional object store for loading sparse specifications
Returns
FilterSpecParsed FilterSpec object
Raises
ValueErrorIf spec is not a valid filter specification
def _parse_size(size_str: str) -> int:

Parse a size specification like '100', '10k', '5m', '1g'.

Parameters
size_str:strSize string with optional unit suffix
Returns
intSize in bytes
Raises
ValueErrorIf size_str is not a valid size specification