Classes for dealing with packed git objects.
A pack is a compact representation of a bunch of objects, stored using deltas where possible.
They have two parts, the pack file, which stores the data, and an index that tells you where the data is.
To find an object you look in all of the index files 'til you find a match for the object name. You then use the pointer got from this as a pointer in to the corresponding packfile.
| Class | |
Abstract iterator over pack data based on delta chains. |
| Class | |
Pack index that is based on a file. |
| Class | |
Wrapper for file-like object that computes hash of its data. |
| Class | |
Pack index that is stored entirely in memory. |
| Class | |
Protocol for objects that can contain git objects. |
| Class | |
A Git pack object. |
| Class | |
Generator for pack data chunks. |
| Class | |
The data contained in a packfile. |
| Class | |
Container for objects packed in a pack file. |
| Class | |
An index in to a packfile. |
| Class | |
Version 1 Pack Index file. |
| Class | |
Version 2 Pack Index file. |
| Class | |
Version 3 Pack Index file. |
| Class | |
Delta chain iterator that yields index entries. |
| Class | |
Delta chain iterator that yields ShaFile objects. |
| Class | |
Class to verify a pack stream as it is being read. |
| Class | |
Class to read a pack stream. |
| Class | |
Wrapper for file-like object that remembers the SHA1 of its data. |
| Class | |
Wrapper for file-like object that remembers the SHA1 of its data. |
| Class | |
Class encapsulating an object unpacked from a pack file. |
| Class | |
Delta chain iterator that yield unpacked objects. |
| Class | |
Abstract base class for a stream of unpacked objects. |
| Exception | |
Raised when a pack file unexpectedly disappears. |
| Exception | |
Delta objects could not be resolved. |
| Function | apply |
Based on the similar function in git's patch-delta.c. |
| Function | bisect |
Find a SHA in a data blob with sorted SHAs. |
| Function | chunks |
Get the total length of a sequence of chunks. |
| Function | compute |
Hash a portion of a file into a new SHA. |
| Function | deltas |
Create deltas from sorted objects. |
| Function | deltify |
Generate deltas for pack objects. |
| Function | extend |
Extend a pack file with more objects. |
| Function | find |
Find deltas in a pack that can be reused. |
| Function | full |
Create an UnpackedObject from a ShaFile. |
| Function | generate |
Create pack data from objects. |
| Function | iter |
Return the hexdigest of the SHA1 over a set of names. |
| Function | load |
Load an index file by path. |
| Function | load |
Load an index file from a file-like object. |
| Function | obj |
Compute the SHA for a numeric type and object chunks. |
| Function | pack |
Yield chunks for a pack header. |
| Function | pack |
Generate chunks for a pack object. |
| Function | pack |
Create a pack object header for the given object info. |
| Function | pack |
Create pack data from objects. |
| Function | read |
Read the header of a pack file. |
| Function | read |
Read zlib data from a buffer. |
| Function | sort |
Sort objects for optimal delta compression. |
| Function | take |
Read bytes marked with most significant bit. |
| Function | unpack |
Unpack a Git object. |
| Function | verify |
Read from stream, verify hash, then yield verified chunks. |
| Function | write |
Write a new pack data file. |
| Function | write |
Write a new pack data file. |
| Function | write |
Write a new pack data file. |
| Function | write |
Write a pack header for the given number of objects. |
| Function | write |
Write a pack index file. |
| Function | write |
Write a new pack index file. |
| Function | write |
Write a new pack index file. |
| Function | write |
Write a new pack index file in v3 format. |
| Function | write |
Write pack object to a file. |
| Function | write |
Write a new pack data file. |
| Constant | DEFAULT |
Undocumented |
| Constant | DEFAULT |
Undocumented |
| Constant | DEFAULT |
Undocumented |
| Constant | DELTA |
Undocumented |
| Constant | OFS |
Undocumented |
| Constant | PACK |
Undocumented |
| Constant | REF |
Undocumented |
| Type Variable | T |
Undocumented |
| Type Alias | |
Undocumented |
| Type Alias | |
Undocumented |
| Type Alias | |
Undocumented |
| Type Alias | |
Undocumented |
| Variable | has |
Undocumented |
| Variable | |
Undocumented |
| Function | _compute |
Compute the size of a unresolved object for use with LRUSizeCache. |
| Function | _create |
Use python difflib to work out how to transform base_buf to target_buf. |
| Function | _create |
Wrapper for Rust create_delta to match Python API. |
| Function | _delta |
Undocumented |
| Function | _encode |
Undocumented |
| Function | _load |
Load contents from a file, preferring mmap when possible. |
| Constant | _MAX |
Undocumented |
| Constant | _ZLIB |
Undocumented |
IO[ bytes], hash_func: Callable[ [], HashObject], start_ofs: int = 0, end_ofs: int = 0, buffer_size: int = 1 << 16) -> HashObject:
¶
Hash a portion of a file into a new SHA.
Returns: A new SHA object updated with data read from the file.
| Parameters | |
f:IO[ | A file-like object to read from that supports seek(). |
hashCallable[ | A callable that returns a new HashObject. |
startint | The offset in the file to start reading at. |
endint | The offset in the file to end reading at, relative to the end of the file. |
bufferint | A buffer size for reading. |
| Returns | |
HashObject | Undocumented |
Iterator[ tuple[ ShaFile, bytes | None]], window_size: int | None = None, progress: Callable[ ..., None] | None = None) -> Iterator[ UnpackedObject]:
¶
Create deltas from sorted objects.
| Parameters | |
objects:Iterator[ | Iterator of sorted objects to deltify |
windowint | None | Delta window size; None for default |
progress:Callable[ | Optional progress reporting callback |
| Returns | |
Iterator[ | Iterator of UnpackedObject entries |
Iterator[ ShaFile] | Iterator[ tuple[ ShaFile, bytes | None]], *, window_size: int | None = None, progress: Callable[ ..., None] | None = None) -> Iterator[ UnpackedObject]:
¶
Generate deltas for pack objects.
- Returns: Iterator over type_num, object id, delta_base, content
- delta_base is None for full text entries
| Parameters | |
objects:Iterator[ | An iterable of (object, path) tuples to deltify. |
windowint | None | Window size; None for default |
progress:Callable[ | Optional progress reporting callback |
| Returns | |
Iterator[ | Undocumented |
BinaryIO, object_ids: Set[ RawObjectID], get_raw: Callable[ [ (RawObjectID | ObjectID)], tuple[ int, bytes]], object_format: ObjectFormat, *, compression_level: int = -1, progress: Callable[ [ bytes], None] | None = None) -> tuple[ bytes, list[ tuple[ RawObjectID, int, int]]]:
¶
Extend a pack file with more objects.
The caller should make sure that object_ids does not contain any objects that are already in the pack
PackedObjectContainer, object_ids: Set[ ObjectID], *, other_haves: Set[ ObjectID] | None = None, progress: Callable[ ..., None] | None = None) -> Iterator[ UnpackedObject]:
¶
Find deltas in a pack that can be reused.
| Parameters | |
container:PackedObjectContainer | Pack container to search for deltas |
objectSet[ | Set of object IDs to find deltas for |
otherSet[ | Set of other object IDs we have |
progress:Callable[ | Optional progress reporting callback |
| Returns | |
Iterator[ | Iterator of UnpackedObject entries that can be reused |
Create an UnpackedObject from a ShaFile.
| Parameters | |
o:ShaFile | ShaFile object to convert |
| Returns | |
UnpackedObject | UnpackedObject with full object data |
PackedObjectContainer, object_ids: Sequence[ tuple[ ObjectID, PackHint | None]], delta_window_size: int | None = None, deltify: bool | None = None, reuse_deltas: bool = True, ofs_delta: bool = True, other_haves: set[ ObjectID] | None = None, progress: Callable[ ..., None] | None = None) -> Iterator[ UnpackedObject]:
¶
Create pack data from objects.
Returns: Tuples with (type_num, hexdigest, delta base, object chunks)
Load an index file by path.
Returns: A PackIndex loaded from the given path
| Parameters | |
path:str | os.PathLike[ | Path to the index file |
objectObjectFormat | Hash algorithm used by the repository |
| Returns | |
PackIndex | Undocumented |
str | os.PathLike[ str], f: IO[ bytes] | _GitFile, object_format: ObjectFormat) -> PackIndex:
¶
Load an index file from a file-like object.
Returns: A PackIndex loaded from the given file
| Parameters | |
path:str | os.PathLike[ | Path for the index file |
f:IO[ | File-like object |
objectObjectFormat | Hash algorithm used by the repository |
| Returns | |
PackIndex | Undocumented |
int, object: list[ bytes] | tuple[ bytes | int, list[ bytes]], object_format: ObjectFormat, *, compression_level: int = -1) -> Iterator[ bytes]:
¶
Generate chunks for a pack object.
Returns: Chunks
| Parameters | |
type:int | Numeric type of the object |
object:list[ | Object to write |
objectObjectFormat | Object format (hash algorithm) to use |
compressionint | the zlib compression level |
| Returns | |
Iterator[ | Undocumented |
int, delta_base: bytes | int | None, size: int, object_format: ObjectFormat) -> bytearray:
¶
Create a pack object header for the given object info.
Returns: A header for a packed object.
| Parameters | |
typeint | Numeric type of the object. |
deltabytes | int | None | Delta base offset or ref, or None for whole objects. |
size:int | Uncompressed object size. |
objectObjectFormat | Object format (hash algorithm) to use. |
| Returns | |
bytearray | Undocumented |
Sequence[ ShaFile] | Sequence[ tuple[ ShaFile, bytes | None]] | Sequence[ tuple[ ShaFile, PackHint | None]], *, deltify: bool | None = None, delta_window_size: int | None = None, ofs_delta: bool = True, progress: Callable[ ..., None] | None = None) -> tuple[ int, Iterator[ UnpackedObject]]:
¶
Create pack data from objects.
Returns: Tuples with (type_num, hexdigest, delta base, object chunks)
| Parameters | |
objects:Sequence[ | Pack objects |
deltify:bool | None | Whether to deltify pack objects |
deltaint | None | Delta window size |
ofsbool | Whether to use offset deltas |
progress:Callable[ | Optional progress reporting callback |
| Returns | |
tuple[ | Undocumented |
Callable[ [ int], bytes], unpacked: UnpackedObject, include_comp: bool = False, buffer_size: int = _ZLIB_BUFSIZE) -> bytes:
¶
Read zlib data from a buffer.
This function requires that the buffer have additional data following the compressed data, which is guaranteed to be the case for git pack files.
Returns: Leftover unused data from the decompression.
| Parameters | |
readCallable[ | Read function that returns at least one byte, but may return less than the requested size. |
unpacked:UnpackedObject | An UnpackedObject to write result data to. If its crc32 attr is not None, the CRC32 of the compressed bytes will be computed using this starting CRC32. After this function, will have the following attrs set: * comp_chunks (if include_comp is True) * decomp_chunks * decomp_len * crc32 |
includebool | If True, include compressed data in the result. |
bufferint | Size of the read buffer. |
| Returns | |
bytes | Undocumented |
| Raises | |
zlib.error | if a decompression error occurred. |
Callable[ [ int], bytes], hash_func: Callable[ [], HashObject], read_some: Callable[ [ int], bytes] | None = None, compute_crc32: bool = False, include_comp: bool = False, zlib_bufsize: int = _ZLIB_BUFSIZE) -> tuple[ UnpackedObject, bytes]:
¶
Unpack a Git object.
- Returns: A tuple of (unpacked, unused), where unused is the unused data
leftover from decompression, and unpacked in an UnpackedObject with the following attrs set:
- obj_chunks (for non-delta types)
- pack_type_num
- delta_base (for delta types)
- comp_chunks (if include_comp is True)
- decomp_chunks
- decomp_len
- crc32 (if compute_crc32 is True)
| Parameters | |
readCallable[ | Read function that blocks until the number of requested bytes are read. |
hashCallable[ | Hash function to use for computing object IDs. |
readCallable[ | Read function that returns at least one byte, but may not return the number of bytes requested. |
computebool | If True, compute the CRC32 of the compressed data. If False, the returned CRC32 will be None. |
includebool | If True, include compressed data in the result. |
zlibint | An optional buffer size for zlib operations. |
| Returns | |
tuple[ | Undocumented |
Callable[ [ int], bytes], expected_hash: bytes, hash_algo: str, progress: Callable[ [ bytes], None] | None = None) -> Iterator[ bytes]:
¶
Read from stream, verify hash, then yield verified chunks.
This function downloads data to a temporary file (in-memory for small files, on-disk for large ones) while computing its hash. Only after the hash is verified to match expected_hash will it yield any data. This prevents corrupted or malicious data from reaching the caller.
| Parameters | |
readCallable[ | Function to read bytes (like file.read or HTTP response reader) |
expectedbytes | Expected hash as hex string bytes (e.g., b'a3b2c1...') |
hashstr | Hash algorithm name ('sha1' or 'sha256') |
progress:Callable[ | Optional progress callback |
| Returns | |
Iterator[ | Undocumented |
| Yields | |
| Chunks of verified data (only after hash verification succeeds) |
| Raises | |
ValueError | If hash doesn't match or algorithm unsupported |
str, objects: Sequence[ ShaFile] | Sequence[ tuple[ ShaFile, bytes | None]], object_format: ObjectFormat, *, deltify: bool | None = None, delta_window_size: int | None = None, compression_level: int = -1) -> tuple[ bytes, bytes]:
¶
Write a new pack data file.
Returns: Tuple with checksum of pack file and index file
| Parameters | |
filename:str | Path to the new pack file (without .pack extension) |
objects:Sequence[ | Objects to write to the pack |
objectObjectFormat | Object format |
deltify:bool | None | Whether to deltify pack objects |
deltaint | None | Delta window size |
compressionint | the zlib compression level |
| Returns | |
tuple[ | Undocumented |
Callable[ [ bytes], None] | Callable[ [ (bytes | bytearray | memoryview)], int] | IO[ bytes], records: Iterator[ UnpackedObject], object_format: ObjectFormat, *, num_records: int | None = None, progress: Callable[ ..., None] | None = None, compression_level: int = -1) -> tuple[ dict[ bytes, tuple[ int, int]], bytes]:
¶
Write a new pack data file.
Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum
| Parameters | |
write:Callable[ | Write function to use |
records:Iterator[ | Iterator over type_num, object_id, delta_base, raw |
objectObjectFormat | Object format (hash algorithm) to use |
numint | None | Number of records (defaults to len(records) if None) |
progress:Callable[ | Function to report progress to |
compressionint | the zlib compression level |
| Returns | |
tuple[ | Undocumented |
Callable[ [ bytes], None] | Callable[ [ (bytes | bytearray | memoryview)], int] | IO[ bytes], container: PackedObjectContainer, object_ids: Sequence[ tuple[ ObjectID, PackHint | None]], object_format: ObjectFormat, *, delta_window_size: int | None = None, deltify: bool | None = None, reuse_deltas: bool = True, compression_level: int = -1, other_haves: set[ ObjectID] | None = None) -> tuple[ dict[ bytes, tuple[ int, int]], bytes]:
¶
Write a new pack data file.
Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum
| Parameters | |
write:Callable[ | write function to use |
container:PackedObjectContainer | PackedObjectContainer |
objectSequence[ | Sequence of (object_id, hint) tuples to write |
objectObjectFormat | Object format (hash algorithm) to use |
deltaint | None | Sliding window size for searching for deltas; Set to None for default window size. |
deltify:bool | None | Whether to deltify objects |
reusebool | Whether to reuse existing deltas |
compressionint | the zlib compression level to use |
otherset[ | Set of additional object IDs the receiver has |
| Returns | |
tuple[ | Undocumented |
IO[ bytes], entries: Iterable[ tuple[ bytes, int, int | None]], pack_checksum: bytes, progress: Callable[ ..., None] | None = None, version: int | None = None) -> bytes:
¶
Write a pack index file.
| Parameters | |
f:IO[ | File-like object to write to. |
entries:Iterable[ | List of (checksum, offset, crc32) tuples |
packbytes | Checksum of the pack file. |
progress:Callable[ | Progress function (not currently used) |
version:int | None | Pack index version to use (1, 2, or 3). If None, defaults to DEFAULT_PACK_INDEX_VERSION. |
| Returns | |
bytes | SHA of the written index file |
| Raises | |
ValueError | If an unsupported version is specified |
IO[ bytes], entries: Iterable[ tuple[ bytes, int, int | None]], pack_checksum: bytes) -> bytes:
¶
Write a new pack index file.
Returns: The SHA of the written index file
| Parameters | |
f:IO[ | A file-like object to write to |
entries:Iterable[ | List of tuples with object name (sha), offset_in_pack, and crc32_checksum. |
packbytes | Checksum of the pack file. |
| Returns | |
bytes | Undocumented |
IO[ bytes], entries: Iterable[ tuple[ bytes, int, int | None]], pack_checksum: bytes) -> bytes:
¶
Write a new pack index file.
Returns: The checksum of the index file written
| Parameters | |
f:IO[ | File-like object to write to |
entries:Iterable[ | List of tuples with object name (sha), offset_in_pack, and crc32_checksum. |
packbytes | Checksum of the pack file. |
| Returns | |
bytes | Undocumented |
IO[ bytes], entries: Iterable[ tuple[ bytes, int, int | None]], pack_checksum: bytes, hash_format: int = 1) -> bytes:
¶
Write a new pack index file in v3 format.
Returns: The SHA of the index file written
| Parameters | |
f:IO[ | File-like object to write to |
entries:Iterable[ | List of tuples with object name (sha), offset_in_pack, and crc32_checksum. |
packbytes | Checksum of the pack file. |
hashint | Hash algorithm identifier (1 = SHA-1, 2 = SHA-256) |
| Returns | |
bytes | Undocumented |
Callable[ [ bytes], int], type: int, object: list[ bytes] | tuple[ bytes | int, list[ bytes]], object_format: ObjectFormat, *, sha: HashObject | None = None, compression_level: int = -1) -> int:
¶
Write pack object to a file.
Returns: CRC32 checksum of the written object
| Parameters | |
write:Callable[ | Write function to use |
type:int | Numeric type of the object |
object:list[ | Object to write |
objectObjectFormat | Object format (hash algorithm) to use |
sha:HashObject | None | Optional SHA-1 hasher to update |
compressionint | the zlib compression level |
| Returns | |
int | Undocumented |
Callable[ [ bytes], None] | IO[ bytes], objects: Sequence[ ShaFile] | Sequence[ tuple[ ShaFile, bytes | None]], object_format: ObjectFormat, *, delta_window_size: int | None = None, deltify: bool | None = None, compression_level: int = -1) -> tuple[ dict[ bytes, tuple[ int, int]], bytes]:
¶
Write a new pack data file.
Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum
| Parameters | |
write:Callable[ | write function to use |
objects:Sequence[ | Sequence of (object, path) tuples to write |
objectObjectFormat | Object format (hash algorithm) to use |
deltaint | None | Sliding window size for searching for deltas; Set to None for default window size. |
deltify:bool | None | Whether to deltify objects |
compressionint | the zlib compression level to use |
| Returns | |
tuple[ | Undocumented |