PyBuffer (Jython API documentation)
Date: 2024-08-18

org.python.core

## Interface PyBuffer

- All Superinterfaces:
    java.lang.AutoCloseable, BufferProtocol, PyBUF
  All Known Implementing Classes:
    Base1DBuffer, BaseArrayBuffer, BaseBuffer, BaseNIOBuffer, SimpleBuffer, SimpleNIOBuffer, SimpleStringBuffer, SimpleWritableBuffer, Strided1DBuffer, Strided1DNIOBuffer, Strided1DWritableBuffer, ZeroByteBuffer

      public interface PyBuffer
      extends PyBUF, BufferProtocol, java.lang.AutoCloseable

  The Jython buffer API for access to a byte array within an exporting object. This interface is the counterpart of the CPython `Py_buffer` struct. Several concrete types implement this interface in order to provide tailored support for different storage organisations.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Interface and Description
    --- | ---
    `static class` | `PyBuffer.Pointer` Deprecated.

  - ### Field Summary

    - ### Fields inherited from interface org.python.core.PyBUF

      `ANY_CONTIGUOUS, AS_ARRAY, C_CONTIGUOUS, CONTIG, CONTIG_RO, CONTIGUITY, F_CONTIGUOUS, FORMAT, FULL, FULL_RO, INDIRECT, IS_C_CONTIGUOUS, IS_F_CONTIGUOUS, MAX_NDIM, NAVIGATION, ND, RECORDS, RECORDS_RO, SIMPLE, STRIDED, STRIDED_RO, STRIDES, WRITABLE`

  - ### Method Summary

    All Methods Instance Methods Abstract Methods
    Modifier and Type | Method and Description
    --- | ---
    `byte` | `byteAt(int... indices)` Return the byte indexed from an N-dimensional buffer with item size one.
    `byte` | `byteAt(int index)` Return the byte indexed from a one-dimensional buffer with item size one.
    `int` | `byteIndex(int... indices)` Convert a multi-dimensional item index to an absolute byte index in the storage shared by the exporter.
    `int` | `byteIndex(int index)` Convert an item index (for a one-dimensional buffer) to an absolute byte index in the storage shared by the exporter.
    `void` | `close()` An alias for `release()` to satisfy `AutoCloseable`.
    `void` | `copyFrom(byte[] src, int srcPos, int destIndex, int count)` Copy from a slice of a (Java) byte array into the buffer starting at a given destination item-index.
    `void` | `copyFrom(PyBuffer src)` Copy the whole of another `PyBuffer` into this buffer.
    `void` | `copyTo(byte[] dest, int destPos)` Copy the contents of the buffer to the destination byte array.
    `void` | `copyTo(int srcIndex, byte[] dest, int destPos, int count)` Copy a simple slice of the buffer-view to the destination byte array, defined by a starting item-index in the source buffer and the `count` of items to copy.
    `PyBuffer.Pointer` | `getBuf()` Return a structure describing the slice of a byte array that holds the data being exported to the consumer.
    `PyBuffer` | `getBuffer(int flags)` Method by which the consumer requests the buffer from the exporter.
    `PyBuffer` | `getBufferSlice(int flags, int start, int count)` Equivalent to `getBufferSlice(int, int, int, int)` with stride 1.
    `PyBuffer` | `getBufferSlice(int flags, int start, int count, int stride)` Get a `PyBuffer` that represents a slice of the current one described in terms of a start index, number of items to include in the slice, and the stride in the current buffer.
    `java.lang.String` | `getFormat()` A format string in the language of Python structs describing how the bytes of each item should be interpreted.
    `java.nio.ByteBuffer` | `getNIOByteBuffer()` Obtain a `ByteBuffer` giving access to the bytes that hold the data being exported by the original object.
    `BufferProtocol` | `getObj()` Return the underlying exporting object (or `null` if no object implementing the `BufferProtocol` is in that role).
    `PyBuffer.Pointer` | `getPointer(int... indices)` Return a structure describing the position in a byte array of a single item from the data being exported to the consumer, in the case that array may be multi-dimensional.
    `PyBuffer.Pointer` | `getPointer(int index)` Return a structure describing the position in a byte array of a single item from the data being exported to the consumer.
    `boolean` | `hasArray()` Report whether the exporter is able to offer direct access to the exported storage as a Java byte array (through the API that involves class `PyBuffer.Pointer`), or only supports the abstract API.
    `int` | `intAt(int... indices)` Return the unsigned byte value indexed from an N-dimensional buffer with item size one.
    `int` | `intAt(int index)` Return the unsigned byte value indexed from a one-dimensional buffer with item size one.
    `boolean` | `isReleased()` True only if the buffer has been released with (the required number of calls to) `release()` or some equivalent operation.
    `void` | `release()` A buffer is (usually) a view onto to the internal state of an exporting object, and that object may have to restrict its behaviour while the buffer exists.
    `void` | `storeAt(byte value, int... indices)` Store the given byte at the indexed location in of an N-dimensional buffer with item size one.
    `void` | `storeAt(byte value, int index)` Store the given byte at the indexed location in of a one-dimensional buffer with item size one.
    `java.lang.String` | `toString()` The toString() method of a buffer reproduces the byte values in the buffer (treated as unsigned integers) as the character codes of a `String`.

    - ### Methods inherited from interface org.python.core.PyBUF

      `getItemsize, getLen, getNdim, getShape, getStrides, getSuboffsets, isContiguous, isReadonly`

- - ### Method Detail

    - #### getObj

          BufferProtocol getObj()

      Return the underlying exporting object (or `null` if no object implementing the `BufferProtocol` is in that role). This will often be a `PyObject`.
      Returns:
        exporting object (or `null`)

    - #### byteAt

          byte byteAt(int index)
               throws java.lang.IndexOutOfBoundsException

      Return the byte indexed from a one-dimensional buffer with item size one. This is part of the fully-encapsulated API: the buffer implementation exported takes care of navigating the structure of the buffer. Results are undefined where the number of dimensions is not one or if `itemsize`>1.
      Parameters:
        `index` - to retrieve from
      Returns:
        the item at index, which is a byte
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### intAt

          int intAt(int index)
             throws java.lang.IndexOutOfBoundsException

      Return the unsigned byte value indexed from a one-dimensional buffer with item size one. This is part of the fully-encapsulated API: the exporter takes care of navigating the structure of the buffer. Results are undefined where the number of dimensions is not one or if `itemsize`>1.
      Parameters:
        `index` - to retrieve from
      Returns:
        the item at index, treated as an unsigned byte, `=0xff & byteAt(index)`
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### storeAt

          void storeAt(byte value,
                       int index)
                throws java.lang.IndexOutOfBoundsException

      Store the given byte at the indexed location in of a one-dimensional buffer with item size one. This is part of the fully-encapsulated API: the buffer implementation exported takes care of navigating the structure of the buffer. Results are undefined where the number of dimensions is not one or if `itemsize`>1.
      Parameters:
        `value` - to store
        `index` - to location
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### byteAt

          byte byteAt(int... indices)
               throws java.lang.IndexOutOfBoundsException

      Return the byte indexed from an N-dimensional buffer with item size one. This is part of the fully-encapsulated API: the buffer implementation exported takes care of navigating the structure of the buffer. The indices must be correct in number and range for the array shape. Results are undefined where `itemsize`>1.
      Parameters:
        `indices` - specifying location to retrieve from
      Returns:
        the item at location, which is a byte
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### intAt

          int intAt(int... indices)
             throws java.lang.IndexOutOfBoundsException

      Return the unsigned byte value indexed from an N-dimensional buffer with item size one. This is part of the fully-encapsulated API: the buffer implementation exported takes care of navigating the structure of the buffer. The indices must be correct in number and range for the array shape. Results are undefined where `itemsize`>1.
      Parameters:
        `indices` - specifying location to retrieve from
      Returns:
        the item at location, treated as an unsigned byte, `=0xff & byteAt(index)`
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### storeAt

          void storeAt(byte value,
                       int... indices)
                throws java.lang.IndexOutOfBoundsException

      Store the given byte at the indexed location in of an N-dimensional buffer with item size one. This is part of the fully-encapsulated API: the exporter takes care of navigating the structure of the buffer. The indices must be correct in number and range for the array shape. Results are undefined where `itemsize`>1.
      Parameters:
        `value` - to store
        `indices` - specifying location to store at
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### copyTo

          void copyTo(byte[] dest,
                      int destPos)
               throws java.lang.IndexOutOfBoundsException,
                      PyException

      Copy the contents of the buffer to the destination byte array. The number of bytes will be that returned by `PyBUF.getLen()`, and the order is the storage order in the exporter. (Note: Correct ordering for multidimensional arrays, including those with indirection needs further study.)
      Parameters:
        `dest` - destination byte array
        `destPos` - byte-index in the destination array of the byte [0]
      Throws:
        `java.lang.IndexOutOfBoundsException` - if the destination cannot hold it
        `PyException`

    - #### copyTo

          void copyTo(int srcIndex,
                      byte[] dest,
                      int destPos,
                      int count)
               throws java.lang.IndexOutOfBoundsException,
                      PyException

      Copy a simple slice of the buffer-view to the destination byte array, defined by a starting item-index in the source buffer and the `count` of items to copy. This may validly be done only for a one-dimensional buffer, as the meaning of the starting item-index is otherwise not defined. `count*itemsize` bytes will be occupied in the destination.
      Parameters:
        `srcIndex` - starting item-index in the source buffer
        `dest` - destination byte array
        `destPos` - byte-index in the destination array of the source item [0,...]
        `count` - number of items to copy
      Throws:
        `java.lang.IndexOutOfBoundsException` - if access out of bounds in source or destination
        `PyException`

    - #### copyFrom

          void copyFrom(byte[] src,
                        int srcPos,
                        int destIndex,
                        int count)
                 throws java.lang.IndexOutOfBoundsException,
                        PyException

      Copy from a slice of a (Java) byte array into the buffer starting at a given destination item-index. This may validly be done only for a one-dimensional buffer, as the meaning of the destination index is not otherwise defined. `count*itemsize` bytes will be read from the source.
      Parameters:
        `src` - source byte array
        `srcPos` - location in source of first byte to copy
        `destIndex` - starting item-index in the destination (i.e. `this`)
        `count` - number of items to copy in
      Throws:
        `java.lang.IndexOutOfBoundsException` - if access out of bounds in source or destination
        `PyException` - `TypeError` if read-only buffer

    - #### copyFrom

          void copyFrom(PyBuffer src)
                 throws java.lang.IndexOutOfBoundsException,
                        PyException

      Copy the whole of another `PyBuffer` into this buffer. This may validly be done only for buffers that are consistent in their dimensions. When it is necessary to copy partial buffers, this may be achieved using a buffer slice on the source or destination.
      Parameters:
        `src` - source buffer
      Throws:
        `java.lang.IndexOutOfBoundsException` - if access out of bounds in source or destination
        `PyException` - `TypeError` if read-only buffer

    - #### getBuffer

          PyBuffer getBuffer(int flags)
                      throws PyException

      Method by which the consumer requests the buffer from the exporter. The consumer provides information on its ability to understand buffer navigation. Each consumer requesting a buffer in this way, when it has finished using it, should make a corresponding call to `release()` on the buffer it obtained, or `close()` using try-with-resources, since some objects alter their behaviour while buffers are exported.

      When a `PyBuffer` is the target, the same checks are carried out on the consumer flags, and a return will normally be a reference to that buffer. A Jython `PyBuffer` keeps count of these re-exports in order to match them with the number of calls to `release()`. When the last matching `release()` arrives it is considered "final", and release actions may then take place on the exporting object. After the final release of a buffer, a call to `getBuffer` should raise an exception.

      Specified by:
        `getBuffer` in interface `BufferProtocol`
      Parameters:
        `flags` - specifying features demanded and the navigational capabilities of the consumer
      Returns:
        exported buffer
      Throws:
        `PyException` - `BufferError` when expectations do not correspond with the buffer

    - #### release

          void release()

      A buffer is (usually) a view onto to the internal state of an exporting object, and that object may have to restrict its behaviour while the buffer exists. The consumer must therefore say when it has finished with the buffer if the exporting object is to be released from this constraint. Each consumer that obtains a reference to a buffer by means of a call to `BufferProtocol.getBuffer(int)` or `getBuffer(int)` should make a matching call to `release()`. The consumer may be sharing the `PyBuffer` with other consumers and the buffer uses the pairing of `getBuffer` and `release` to manage the lock on behalf of the exporter. It is an error to make more than one call to `release` for a single call to `getBuffer`.

    - #### close

          void close()

      An alias for `release()` to satisfy `AutoCloseable`.
      Specified by:
        `close` in interface `java.lang.AutoCloseable`

    - #### isReleased

          boolean isReleased()

      True only if the buffer has been released with (the required number of calls to) `release()` or some equivalent operation. The consumer may be sharing the reference with other consumers and the buffer only achieves the released state when all consumers who called `getBuffer` have called `release`.

    - #### getBufferSlice

          PyBuffer getBufferSlice(int flags,
                                  int start,
                                  int count)

      Equivalent to `getBufferSlice(int, int, int, int)` with stride 1.
      Parameters:
        `flags` - specifying features demanded and the navigational capabilities of the consumer
        `start` - index in the current buffer
        `count` - number of items in the required slice
      Returns:
        a buffer representing the slice

    - #### getBufferSlice

          PyBuffer getBufferSlice(int flags,
                                  int start,
                                  int count,
                                  int stride)

      Get a `PyBuffer` that represents a slice of the current one described in terms of a start index, number of items to include in the slice, and the stride in the current buffer. A consumer that obtains a `PyBuffer` with `getBufferSlice` must release it with `release()` just as if it had been obtained with `getBuffer(int)`

      Suppose that x(i) denotes the ith element of the current buffer, that is, the byte retrieved by `this.byteAt(i)` or the unit indicated by `this.getPointer(i)`. A request for a slice where `start` = s, `count` = N and `stride` = m, results in a buffer y such that y(k) = x(s+km) where k=0..(N-1). In Python terms, this is the slice x[s : s+(N-1)m+1 : m] (if m>0) or the slice x[s : s+(N-1)m-1 : m] (if m<0). Implementations should check that this range is entirely within the current buffer.

      In a simple buffer backed by a contiguous byte array, the result is a strided PyBuffer on the same storage but where the offset is adjusted by s and the stride is as supplied. If the current buffer is already strided and/or has an item size larger than single bytes, the new `start` index, `count` and `stride` will be translated from the arguments given, through this buffer's stride and item size. The caller always expresses `start` and `strides` in terms of the abstract view of this buffer.

      Parameters:
        `flags` - specifying features demanded and the navigational capabilities of the consumer
        `start` - index in the current buffer
        `count` - number of items in the required slice
        `stride` - index-distance in the current buffer between consecutive items in the slice
      Returns:
        a buffer representing the slice

    - #### byteIndex

          int byteIndex(int index)
                 throws java.lang.IndexOutOfBoundsException

      Convert an item index (for a one-dimensional buffer) to an absolute byte index in the storage shared by the exporter. The storage exported as a `PyBuffer` is a linearly-indexed sequence of bytes, although it may not actually be a heap-allocated Java `byte[]` object. The purpose of this method is to allow the exporter to define the relationship between the item index (as used in `byteAt(int)`) and the byte-index (as used with the `ByteBuffer` returned by `getNIOByteBuffer()`). See `byteIndex(int[])` for discussion of the multi-dimensional case.
      Parameters:
        `index` - item-index from consumer
      Returns:
        corresponding byte-index in actual storage
      Throws:
        `java.lang.IndexOutOfBoundsException`

    - #### byteIndex

          int byteIndex(int... indices)

      Convert a multi-dimensional item index to an absolute byte index in the storage shared by the exporter. The storage exported as a `PyBuffer` is a linearly-indexed sequence of bytes, although it may not actually be a heap-allocated Java `byte[]` object. The purpose of this method is to allow the exporter to define the relationship between the item index (as used in `byteAt(int...)` and the byte-index (as used with the `ByteBuffer` returned by `getNIOByteBuffer()`).
      Parameters:
        `indices` - n-dimensional item-index from consumer
      Returns:
        corresponding byte-index in actual storage

    - #### getNIOByteBuffer

          java.nio.ByteBuffer getNIOByteBuffer()

      Obtain a `ByteBuffer` giving access to the bytes that hold the data being exported by the original object. The position of the buffer is at the first byte of the item with zero index (quite possibly not the lowest valid byte-index), the limit of the buffer is beyond the largest valid byte index, and the mark is undefined.

      For a one-dimensional contiguous buffer, the limit is one byte beyond the last item, so that consecutive reads from the `ByteBuffer` return the data in order. Assuming the following client code where `obj` has type `BufferProtocol`:

           PyBuffer a = obj.getBuffer(PyBUF.SIMPLE);
           int itemsize = a.getItemsize();
           ByteBuffer bb = a.getNIOBuffer();

      the item with index `k` is in `bb` at positions `bb.pos()+k*itemsize` to `bb.pos()+(k+1)*itemsize - 1` inclusive. And if `itemsize==1`, the item is simply the byte at position `bb.pos()+k`.

      If the buffer is multidimensional or non-contiguous (strided), the buffer position is still the (first byte of) the item at index `[0]` or `[0,...,0]`. However, it is necessary to navigate `bb` using the `shape`, `strides` and maybe `suboffsets` provided by the API.

      Returns:
        a `ByteBuffer` onto the exported data contents.

    - #### hasArray

          boolean hasArray()

      Report whether the exporter is able to offer direct access to the exported storage as a Java byte array (through the API that involves class `PyBuffer.Pointer`), or only supports the abstract API. See also `PyBUF.AS_ARRAY`.
      Returns:
        true if array access is supported, false if it is not.

    - #### getBuf

          PyBuffer.Pointer getBuf()

      Return a structure describing the slice of a byte array that holds the data being exported to the consumer. For a one-dimensional contiguous buffer, assuming the following client code where `obj` has type `BufferProtocol`:

           PyBuffer a = obj.getBuffer(PyBUF.SIMPLE);
           int itemsize = a.getItemsize();
           PyBuffer.Pointer b = a.getBuf();

      the item with index `k` is in the array `b.storage` at index `[b.offset + k*itemsize]` to `[b.offset + (k+1)*itemsize - 1]` inclusive. And if `itemsize==1`, the item is simply the byte `b.storage[b.offset + k]`

      If the buffer is multidimensional or non-contiguous, `storage[offset]` is still the (first byte of) the item at index [0] or [0,...,0]. However, it is necessary to navigate `b.storage` using the `shape`, `strides` and maybe `suboffsets` provided by the API.

      Returns:
        structure defining the byte[] slice that is the shared data

    - #### getPointer

          PyBuffer.Pointer getPointer(int index)

      Return a structure describing the position in a byte array of a single item from the data being exported to the consumer. For a one-dimensional contiguous buffer, assuming the following client code where `obj` has type `BufferProtocol`:

           int k = ... ;
           PyBuffer a = obj.getBuffer(PyBUF.FULL);
           int itemsize = a.getItemsize();
           PyBuffer.Pointer b = a.getPointer(k);

      the item with index `k` is in the array `b.storage` at index `[b.offset]` to `[b.offset + itemsize - 1]` inclusive. And if `itemsize==1`, the item is simply the byte `b.storage[b.offset]`

      Essentially this is a method for computing the offset of a particular index. The client is free to navigate the underlying buffer `b.storage` without respecting these boundaries.

      Parameters:
        `index` - in the buffer to position the pointer
      Returns:
        structure defining the byte[] slice that is the shared data

    - #### getPointer

          PyBuffer.Pointer getPointer(int... indices)

      Return a structure describing the position in a byte array of a single item from the data being exported to the consumer, in the case that array may be multi-dimensional. For a 3-dimensional contiguous buffer, assuming the following client code where `obj` has type `BufferProtocol`:

           int i, j, k;
           // ... calculation that assigns i, j, k
           PyBuffer a = obj.getBuffer(PyBUF.FULL);
           int itemsize = a.getItemsize();
           PyBuffer.Pointer b = a.getPointer(i,j,k);

      the item with index `[i,j,k]` is in the array `b.storage` at index `[b.offset]` to `[b.offset + itemsize - 1]` inclusive. And if `itemsize==1`, the item is simply the byte `b.storage[b.offset]`

      Essentially this is a method for computing the offset of a particular index. The client is free to navigate the underlying buffer `b.storage` without respecting these boundaries. If the buffer is non-contiguous, the above description is still valid (since a multi-byte item must itself be contiguously stored), but in any additional navigation of `b.storage[]` to other items, the client must use the shape, strides and sub-offsets provided by the API. Normally one starts `b = a.getBuf()` in order to establish the offset of index [0,...,0].

      Parameters:
        `indices` - multidimensional index at which to position the pointer
      Returns:
        structure defining the byte[] slice that is the shared data

    - #### getFormat

          java.lang.String getFormat()

      A format string in the language of Python structs describing how the bytes of each item should be interpreted. Irrespective of the `PyBUF.FORMAT` bit in the consumer's call to `getBuffer`, a valid `format` string is always returned (difference from CPython).

      Jython only implements "B" so far, and it is debatable whether anything fancier than "<n>B" can be supported in Java.

      Returns:
        the format string

    - #### toString

          java.lang.String toString()

      The toString() method of a buffer reproduces the byte values in the buffer (treated as unsigned integers) as the character codes of a `String`.
      Overrides:
        `toString` in class `java.lang.Object`

Jython homepage