Package dev.vortex.api

Interface ScanOptions

@Immutable public interface ScanOptions
Scan configuration passed to DataSource.scan(ScanOptions).

All fields are optional. A call to of() returns a default that reads every row and column.

  • Method Details

    • projection

      Optional<Expression> projection()
      Projection expression. If empty, all columns are returned.

    • filter

      Optional<Expression> filter()
      Filter expression applied before returning rows.

    • rowRangeBegin

      OptionalLong rowRangeBegin()
      Inclusive start of the row range to read.

    • rowRangeEnd

      OptionalLong rowRangeEnd()
      Exclusive end of the row range to read.

    • selectionIndices

      Optional<long[]> selectionIndices()
      Sorted ascending, unique row indices that should be included in (or excluded from) the scan, depending on selectionMode().

      The native scan validates this contract and rejects indices that are not strictly ascending.

    • selectionRoaringBitmap

      Optional<byte[]> selectionRoaringBitmap()
      Portable serialized Roaring64NavigableMap row selection.

    • selectionMode

      @Default default ScanOptions.SelectionMode selectionMode()
      Meaning of the row selection payload.

    • limit

      OptionalLong limit()
      Maximum row count to return. Absent means "no limit".

    • ordered

      @Default default boolean ordered()
      If true, the scan preserves the original row order across partitions.

    • splitConcurrency

      OptionalInt splitConcurrency()
      The most splits of each partition the native scan decodes at once, in all, and the most of its batches converted to Arrow ahead of the caller. Absent keeps the native default, which scales with the machine's cores.

      Each split being decoded holds the decoded chunks of every column it overlaps, so a scan running alongside others, such as one Spark task of several in an executor, can lower this to reduce memory held by concurrent decoding and Arrow conversion. A scan with both a filter and a limit may also retain one trimmed array per split while preparing output. This does not cap total native memory or I/O read-ahead.

    • validateSplitConcurrency

      @Check default void validateSplitConcurrency()

    • validateSelectionPayload

      @Check default void validateSelectionPayload()

    • of

      static ScanOptions of()

    • builder

      static dev.vortex.api.ImmutableScanOptions.Builder builder()

    • includeRows

      static ScanOptions includeRows(long... rowIndices)
      Scan only the rows at the given sorted ascending, unique row indices.

    • excludeRows

      static ScanOptions excludeRows(long... rowIndices)
      Scan all rows except the given sorted ascending, unique row indices.

    • includeRows

      static ScanOptions includeRows(org.roaringbitmap.longlong.Roaring64NavigableMap rowSelection)
      Scan only the rows in the given Roaring bitmap.

    • excludeRows

      static ScanOptions excludeRows(org.roaringbitmap.longlong.Roaring64NavigableMap rowSelection)
      Scan all rows except the rows in the given Roaring bitmap.