Java API

The Vortex Java API provides bindings for the Vortex library, enabling Java applications to work with Vortex arrays and files.

The API is split into two main components:

  • Vortex JNI: Core JNI bindings for Vortex functionality

  • Vortex Spark: Apache Spark integration for reading and writing Vortex files

API Documentation

Installation

The Java API can be included in your project using Gradle or Maven. See the Java quickstart for the dependency setup and for examples that write, read, filter, and project Vortex files, and the Spark user guide for the Spark connector.

Compatibility

The Java bindings are supported on the following architectures:

  • x86_64 Linux

  • ARM64 Linux

  • Apple Silicon macOS

  • x86_64 Windows

They support any Linux distribution with a GLIBC version >= 2.31. This includes

  • Amazon Linux 2023 or newer

  • Ubuntu 20.04 or newer

Usage Example

Here’s a basic example of using the Vortex Java API to read a Vortex file:

import dev.vortex.api.DataSource;
import dev.vortex.api.Partition;
import dev.vortex.api.Scan;
import dev.vortex.api.ScanOptions;
import dev.vortex.api.Session;
import dev.vortex.arrow.ArrowAllocation;
import org.apache.arrow.memory.BufferAllocator;
import org.apache.arrow.vector.VectorSchemaRoot;
import org.apache.arrow.vector.ipc.ArrowReader;

BufferAllocator allocator = ArrowAllocation.rootAllocator();
Session session = Session.create();
DataSource source = DataSource.open(session, "path/to/file.vortex");

// A scan yields partitions; each partition is read as a stream of Arrow batches.
Scan scan = source.scan(ScanOptions.of());
while (scan.hasNext()) {
    Partition partition = scan.next();
    try (ArrowReader reader = partition.scanArrow(allocator)) {
        while (reader.loadNextBatch()) {
            VectorSchemaRoot batch = reader.getVectorSchemaRoot();
            System.out.println("read " + batch.getRowCount() + " rows");
        }
    }
}

Data crosses the JNI boundary as Arrow record batches, so the buffers stay in native memory and are read from Java through the Arrow C Data Interface.