com.carrotsearch.hppcrt.overview.html Maven / Gradle / Ivy
High Performance Primitive Collections Realtime(HPPC-RT)
High Performance Primitive Collections Realtime (HPPC-RT) library provides
(fork of HPPC of Carrotsearch)
Fundamental data structures (maps, sets, lists, stacks, queues, heaps, sorts) generated for
combinations of object and primitive types to conserve JVM memory and speed
up execution. The Realtime fork intend of extending collections while tweaking and optimizations to remove any dynamic allocations at runtime,
and low variance execution times.
Why HPPC?
The Java Collections package is in many ways excellent, but it cannot
be used for primitive types and autoboxing introduced in Java 1.5
kills the runtime performance (increased memory use, garbage collector overhead).
This library has different design goals than Java Collections (and many other
collection packages). For example, the internals of each class are open
and subject to free hacking and tuning up to one's wishes.
Why not other primitive collection libraries?
There are a few projects implementing collections over primitive types:
fastutil,
PCJ,
GNU Trove,
Apache Mahout (ported COLT collections),
Apache Primitive Collections.
Some of these are LGPL-ed, a license type many commercial companies
tend to avoid at all costs; other projects above are no longer
maintained or complete. There is also a tendency to write tightly encapsulated
code (private internals), implementing the API of standard Java packages and
striving for fast error-recovery. Although these are good programming
practices, they are not always practical (in many computationally intense applications
access to the collection class' internals is a bliss). HPPC has a slightly
different set of goals. If you need Java collections compliance, try
fastutil, it is very, very good.
Assumptions and goals of HPPC
We assume that:
- The programmer knows what he's doing and may wish to access internal
storage of a given collection's class directly (not using predefined
iterators, for example).
- Vast majority of scenarios use single-threaded access. There is little
point in repeated verification of concurrent modifications, for example.
- The programs (algorithms) that use primitive collections are unit-tested
and regression-tested during development, so their behavior in production
systems does not have to be restrictively verified (we can assume the code
using the collection class won't violate the given set of assertions).
From these assumptions stem the following design drivers:
- Data types in the HPPC are as simple as possible and expose
internal storage for any optimisations one may wish to do from within the
end-application code.
- Verification of parameter and state validity is done optionally
using assertions. This means that contracts are only checked if requested at
runtime (using Java 1.4+
-ea switch). When the algorithm is tested and verified,
it can run with no additional overhead from contract checks.
- We tried to avoid complex interface hierarchies, although certain
interfaces are defined for clarity. The programmer should still choose
a proper data structure at design-time and he should know what he's doing.
- HPPC provides utilities for the most common tasks like filtering, iterating
or joining collections, but these utilities are usually more expensive than implementing
a low-level direct access to the data storage (which may be done if needed).
- There is no special support for data serialization.
- The implementation is not thread safe and does not attempt to provide
fast-failing concurrency problems detection.
Design and implementation assumptions
- We want HPPC class templates to be implemented using regular Java (with generics)
so that typical programming tools can be used for development, testing, etc.
- We want HPPC class templates to be usable as generic-collection classes (for boxed
numeric types or any other
Object types), but at the same time we want
specialized classes automatically-generated for primitive-types
(to limit memory consumption and boost performance due to JIT optimisations).
Interfaces and their relation to Java Collections API
HPPC is not strictly modeled after Java Collections API, although we did
try to make the APIs look similar enough for comfortable use.
One particular thing largerly missing in HPPC are view projections
(sublists or views over the collection of keys or values). Certain classes provide such views
(like {@linkplain com.carrotsearch.hppcrt.ObjectObjectOpenHashMap
ObjectObjectOpenHashMap}), but for the most part, specific methods are
provided that accept ranges or closure-like filters. If performance is still unsatisfactory,
the internals of each class are available for direct manipulation.
Rough relationships between Java Collections classes and HPPC classes are
presented in the table below.
Relationships between HPPC and Java Collections.
Java Collections
HPPC (primitives)
HPPC (generics)
bit sets
java.util.BitSetBitSetn/a
array-backed lists
java.util.ArrayList, java.util.Vector[type]ArrayList{@linkplain com.carrotsearch.hppcrt.ObjectArrayList
ObjectArrayList<T>}
stacks
java.util.Stack[type]Stack{@linkplain com.carrotsearch.hppcrt.ObjectStack
ObjectStack<T>}
deques
java.util.ArrayDeque[type]ArrayDeque{@linkplain com.carrotsearch.hppc.ObjectArrayDeque
ObjectArrayDeque<T>}
hash maps (dictionaries)
java.util.HashMap[keyType][valueType]OpenHashMap
{@linkplain com.carrotsearch.hppcrt.ObjectObjectOpenHashMap
ObjectObjectOpenHashMap<K, V>}
[keyType]ObjectOpenHashMap<V>
Object[valueType]OpenHashMap<K>
The method-level API of the corresponding types is also similar, but distinct
differences exist (consult the JavaDoc of each class).
Interfaces and container inheritance
HPPC's interfaces are modeled to encapsulate
the common functionality enforced from all the classes that
implement a given interface. The interface hierarchy is loosely inspired by STL.
An overview of interfaces and their relationship to data structures implemented
in HPPC is depicted graphically in {@link com.carrotsearch.hppcrt.ObjectContainer}.