Codecs

BinPickle supports codecs to compress buffer content. These are similar in spirit to numcodecs, but automatically handle some cases such as splitting arrays into blocks and can reduce copying in some situations.

binpickle.codecs.make_codec(codec, *, null_as_none=False, list_is_tuple=False)

Resolve a codec into a BinPickle-compatible codec.

Parameters

codec (obj) –

The codec to resolve into a codec. Can be one of:

  • None (returns Null)

  • A Codec object (returned as-is)

  • A string (look up codec by name and return with default options)

  • A tuple (name, config) (pass to get_config())

  • A list (wrapped in Chain)

  • A numcodecs.abc.Codec (wrapped in NC and returned)

Returns

the codec.

Return type

Codec

Codec API

class binpickle.codecs.Codec

Base class for a codec.

NAME

the name for this codec, used by get_codec() and in index entries.

Type

str

encode(buf)

Encode a buffer.

Parameters

buf (bytes-like) – the buffer to encode.

Returns

the encoded data

Return type

bytes-like

abstract encode_to(buf, out)

Encode a buffer to a binary output stream.

Parameters

  • buf (bytes-like) – the buffer to encode.

  • out (file-like) – the output stream. Must have a write method taking a bytes.

decode(buf)

Decode a buffer.

Parameters

buf (bytes-like) – the buffer to decode.

Returns

the decoded data

Return type

bytes-like

abstract decode_to(buf, out)

Decode a buffer into a bytearray.

Parameters

  • buf (bytes-like) – the buffer to decode.

  • out (bytearray) – the bytearray to receive the output. This method will resize the bytearray as needed to accomodate the output.

abstract config()

Get a JSON-serializable configuration for this codec. It should be able to be passed as **kwargs to the constructor.

Codec Implementations

Null codec

class binpickle.codecs.Null

Null codec (passthrough).

encode(buf)

Encode a buffer.

Parameters

buf (bytes-like) – the buffer to encode.

Returns

the encoded data

Return type

bytes-like

encode_to(buf, out)

Encode a buffer to a binary output stream.

Parameters

  • buf (bytes-like) – the buffer to encode.

  • out (file-like) – the output stream. Must have a write method taking a bytes.

decode(buf, length=None)

Decode a buffer.

Parameters

buf (bytes-like) – the buffer to decode.

Returns

the decoded data

Return type

bytes-like

decode_to(buf, out)

Decode a buffer into a bytearray.

Parameters

  • buf (bytes-like) – the buffer to decode.

  • out (bytearray) – the bytearray to receive the output. This method will resize the bytearray as needed to accomodate the output.

config()

Get a JSON-serializable configuration for this codec. It should be able to be passed as **kwargs to the constructor.

Chain codec

class binpickle.codecs.Chain(codecs=())

Codec that chains together other codecs in sequence. The codecs are applied in the provided order for encoding, and reverse order for decoding.

encode(buf)

Encode a buffer.

Parameters

buf (bytes-like) – the buffer to encode.

Returns

the encoded data

Return type

bytes-like

encode_to(buf, w)

Encode a buffer to a binary output stream.

Parameters

  • buf (bytes-like) – the buffer to encode.

  • out (file-like) – the output stream. Must have a write method taking a bytes.

decode(buf)

Decode a buffer.

Parameters

buf (bytes-like) – the buffer to decode.

Returns

the decoded data

Return type

bytes-like

decode_to(buf, out)

Decode a buffer into a bytearray.

Parameters

  • buf (bytes-like) – the buffer to decode.

  • out (bytearray) – the bytearray to receive the output. This method will resize the bytearray as needed to accomodate the output.

config()

Get a JSON-serializable configuration for this codec. It should be able to be passed as **kwargs to the constructor.

Blosc codec

class binpickle.codecs.Blosc(name='blosclz', level=9, shuffle=1, blocksize=1073741824)

Blosc codec.

encode_to(buf, out)

Encode a buffer to a binary output stream.

Parameters

  • buf (bytes-like) – the buffer to encode.

  • out (file-like) – the output stream. Must have a write method taking a bytes.

decode_to(buf, out)

Decode a buffer into a bytearray.

Parameters

  • buf (bytes-like) – the buffer to decode.

  • out (bytearray) – the bytearray to receive the output. This method will resize the bytearray as needed to accomodate the output.

config()

Get a JSON-serializable configuration for this codec. It should be able to be passed as **kwargs to the constructor.

Gzip codec

class binpickle.codecs.GZ(level=9)

Zlib (gzip-compatible) codec.

encode(buf)

Encode a buffer.

Parameters

buf (bytes-like) – the buffer to encode.

Returns

the encoded data

Return type

bytes-like

encode_to(buf, out)

Encode a buffer to a binary output stream.

Parameters

  • buf (bytes-like) – the buffer to encode.

  • out (file-like) – the output stream. Must have a write method taking a bytes.

decode(buf)

Decode a buffer.

Parameters

buf (bytes-like) – the buffer to decode.

Returns

the decoded data

Return type

bytes-like

decode_to(buf, out)

Decode a buffer into a bytearray.

Parameters

  • buf (bytes-like) – the buffer to decode.

  • out (bytearray) – the bytearray to receive the output. This method will resize the bytearray as needed to accomodate the output.

config()

Get a JSON-serializable configuration for this codec. It should be able to be passed as **kwargs to the constructor.

NumCodecs

BinPickle also supports any codec from numcodecs through the NC wrapper. This is automatically used by the make_codec() function, so you can also pass a NumCodecs codec directly to binpickle.BinPickler.compressed().