Harnessing the power of SIMD with the Java Vector API

In the world of high-performance computing, the use of SIMD (single instruction, multiple data) instructions can significantly improve the performance of certain types of calculations. SIMD allows processors to perform the same operation on multiple data points simultaneously, making it ideal for tasks such as numerical calculations, image processing, and multimedia operations. With Java 17, developers now have access to the Vector API, a feature that allows them to harness the power of SIMD directly within their Java applications.

In this article, we’ll explore what the Vector API is, how it works, and provide examples that demonstrate its use.

Understanding SIMD and its importance

Before diving into the Vector API, it’s crucial to understand the concept of SIMD and why it’s important for performance optimization. Traditional processors execute instructions serially, meaning that each instruction operates on one data element at a time. However, many modern processors include SIMD instruction sets, such as SSE (Streaming SIMD Extensions) and AVX (Advanced Vector Extensions), which allow parallel processing of multiple data elements within a single instruction.

This parallelism is especially useful for tasks that involve repetitive operations on large arrays or datasets. By using SIMD instructions, programmers can achieve significant performance improvements by exploiting the inherent parallelism of the underlying hardware.

Introducing the Vector API

The Vector API, introduced in Java 16 as an incubator module (jdk.incubator.vector) and became a standard feature in Java 17, provides a set of classes and methods for performing SIMD operations directly within Java code. The API abstracts the low-level details of SIMD instructions and allows developers to write portable and efficient vectorized code without resorting to platform-specific assembly language or external libraries.

The core components of the Vector API include vector types, operations, and factories. Types of vectors represent SIMD vectors of various sizes and data types, such as integers, floating-point numbers, and Boolean values. Operations include arithmetic, logical, and comparison operations that can be performed on vector elements. Factories are used to create vector instances and perform conversions between vector and scalar types.

Getting started with the Vector API

To use the Vector API from Java 17, your environment must be equipped with JDK version 17. The API is located within java.util.vector package, providing classes and methods for vector operations. A simple example of adding two arrays of integers using the Vector API demonstrates its ease of use and efficiency over traditional loop-based methods.

Example 1: Adding two arrays by elements

To demonstrate the use of the Vector API, let’s consider a simple example of adding two arrays elementwise using SIMD instructions. We’ll start by creating two arrays of floating point numbers and then add them together in parallel using the Vector API.

import java.util.Arrays;
import jdk.incubator.vector.*;
public class VectorExample 
    public static void main(String[] args) 
        int length = 8; // Number of elements in the arrays
        float[] array1 = new float[length];
        float[] array2 = new float[length];
        float[] result = new float[length];

        // Initialize arrays with random values
        Arrays.setAll(array1, i -> (float) Math.random());
        Arrays.setAll(array2, i -> (float) Math.random());

        // Perform addition using Vector API
        try (var vscope = VectorScope.create()) 
            VectorSpecies<Float> species = FloatVector.SPECIES_256;
            int i = 0;
            for (; i < length - species.length(); i += species.length()) 
                FloatVector a = FloatVector.fromArray(species, array1, i);
                FloatVector b = FloatVector.fromArray(species, array2, i);
                FloatVector sum = a.add(b);
                sum.intoArray(result, i);
            
            for (; i < length; i++) 
                result[i] = array1[i] + array2[i];
            
        
        // Print the result
        System.out.println("Result: " + Arrays.toString(result));
    

In this example we create two arrays – array1 and array2 – which contains floating point random numbers. Then we use FloatVector a class for performing SIMD addition of corresponding elements from two arrays. The VectorScope class is used to manage the scope of vectorization and ensure proper resource cleanup.

Example 2: Calculation of the dot product

Another common operation that benefits from SIMD parallelism is calculating the dot product of two vectors. Let’s show how to calculate the dot product of two swing arrays using the Vector API.

import java.util.Arrays;
import jdk.incubator.vector.*;

public class DotProductExample 
    public static void main(String[] args) 
        int length = 8; // Number of elements in the arrays
        float[] array1 = new float[length];
        float[] array2 = new float[length];

        // Initialize arrays with random values
        Arrays.setAll(array1, i -> (float) Math.random());
        Arrays.setAll(array2, i -> (float) Math.random());

        // Perform dot product using Vector API
        try (var vscope = VectorScope.create()) 
            VectorSpecies<Float> species = FloatVector.SPECIES_256;
            int i = 0;
            FloatVector sum = species.create();
            for (; i < length - species.length(); i += species.length()) 
                FloatVector a = FloatVector.fromArray(species, array1, i);
                FloatVector b = FloatVector.fromArray(species, array2, i);
                sum = sum.add(a.mul(b));
            
            float dotProduct = sum.reduceLanes(VectorOperators.ADD);
            for (; i < length; i++) 
                dotProduct += array1[i] * array2[i];
            
            System.out.println("Dot Product: " + dotProduct);
        
    

In this example, we calculate the dot product of two strings array1 and array2 using SIMD parallelism. We use each other FloatVector class to perform SIMD multiplication of corresponding elements and then accumulate the result using vector reduction.

Example 3: Additional operations

Doubled, with zeros where it was originally <= 4: In addition to basic arithmetic, the Vector API supports a wide range of operations, including logical operations, bitwise operations, and conversions. For example, the following example demonstrates vector multiplication and conditional masking, showing the API's versatility for complex data processing tasks.

import jdk.incubator.vector.IntVector;
import jdk.incubator.vector.VectorMask;
import jdk.incubator.vector.VectorSpecies;

public class AdvancedVectorExample 
    public static void example(int[] vals) 
        VectorSpecies<Integer> species = IntVector.SPECIES_256;
        // Initialize vector from integer array
        IntVector vector = IntVector.fromArray(species, vals, 0);
        // Perform multiplication
        IntVector doubled = vector.mul(2);
        // Apply conditional mask
        VectorMask<Integer> mask = vector.compare(VectorMask.Operator.GT, 4);
        // Output the result
        System.out.println(Arrays.toString(doubled.blend(0, mask).toArray()));
    

Here we start by defining a VectorSpecies with a guy IntVector.SPECIES_256, which means we are working with 256-bit integer vectors. This type selection means that, depending on the hardware, the vector can contain multiple integers within those 256 bits, allowing parallel operations on them. Then we initialize our IntVector from an array of integers, vals, using this type. This step converts our scalar array of integers into a vectorized form that can be processed in parallel.

After that, multiply each element in our vector by 2. The mul method performs this operation in parallel on all elements within IntVector, effectively doubling each value. This is a significant advantage over traditional loop-based approaches, where each multiplication would be processed sequentially.

Then we create a VectorMask by comparing each element in the original vector to the value 4 using compare method with GT (greater than) operator. This operation creates a mask where every position in the vector that has a value greater than 4 is set to trueand all other positions are set to false.

Then we use blend the way to apply our mask on doubled vector. This method takes two arguments: the value to concatenate with (0 in this case) and the mask. For each position in the vector where the mask is trueoriginal value from doubled was retained. Where’s the mask falsevalue is replaced by 0. This effectively nullifies every element in doubled the vector that was created from the value of u vals it was 4 or less.

Insights and Considerations

When integrating the Vector API into applications, consider the following:

• Data reconciliation: For optimal performance, ensure that data structures align with vector sizes. A mismatch can lead to performance degradation due to additional processing steps.
• Loop vectorization: Manually vectorizing loops can lead to significant performance improvements, especially in nested loops or complex algorithms. However, it requires careful consideration of loop boundaries and vector sizes.
• Hardware compatibility: Although the Vector API is designed to be hardware independent, performance gains may vary depending on the SIMD capabilities of the underlying hardware. Testing and benchmarking on target hardware is key to understanding potential performance improvements.

By incorporating these advanced examples and considerations, developers can better leverage the Vector API in Java to write more efficient, effective, and scalable applications. Whether for scientific computing, machine learning, or any computationally intensive task, the Vector API provides a powerful set of tools to exploit the full capabilities of modern hardware.

Conclusion

The Vector API in Java provides developers with a powerful tool to take advantage of the performance of SIMD instructions in their Java applications. By abstracting away the complexities of SIMD programming, the Vector API enables developers to write efficient and portable code that takes advantage of the parallelism offered by modern CPU architectures.

While the examples provided in this article demonstrate basic use of the Vector API, developers can explore more advanced features and optimizations to further improve the performance of their applications. Whether for numerical calculations, image processing, or multimedia operations, the Vector API allows Java developers to unlock the full potential of SIMD parallelism without sacrificing portability or ease of development. Experimenting with different data types, vector lengths, and operations can help developers maximize the performance benefits of SIMD in their Java applications.