com.sleepycat.je.SecondaryAssociation Maven / Gradle / Ivy
/*-
* Copyright (C) 2002, 2018, Oracle and/or its affiliates. All rights reserved.
*
* This file was distributed by Oracle as part of a version of Oracle Berkeley
* DB Java Edition made available at:
*
* http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html
*
* Please see the LICENSE file included in the top-level directory of the
* appropriate version of Oracle Berkeley DB Java Edition for a copy of the
* license and additional information.
*/
package com.sleepycat.je;
import java.util.Collection;
/**
* @hidden
* For internal use only.
*
* Provides a way to create an association between primary and secondary
* databases that is not limited to a one-to-many association.
*
* By implementing this interface, a secondary database may be associated with
* one or more primary databases. For example, imagine an application that
* wishes to partition data from a single logical "table" into multiple primary
* databases to take advantage of the performance benefits of {@link
* Environment#removeDatabase} for a single partition, while at the same having
* a single index database for the entire "table". For read operations via a
* secondary database, JE reads the primary key from the secondary database and
* then must read the primary record from the primary database. The mapping
* from a primary key to its primary database is performed by the user's
* implementation of the {@link #getPrimary} method.
*
* In addition, a primary database may be divided into subsets of keys where
* each subset may be efficiently indexed by zero or more secondary databases.
* This effectively allows multiple logical "tables" per primary database.
* When a primary record is written, the secondaries for its logical "table"
* are returned by the user's implementation of {@link #getSecondaries}.
* During a primary record write, because {@link #getSecondaries} is called to
* determine the secondary databases, only the key creators/extractors for
* those secondaries are invoked. For example, if you have a single primary
* database with 10 logical "tables", each of which has 5 distinct secondary
* databases, when a record is written only 5 key creators/extractors will be
* invoked, not 50.
*
* Configuring a SecondaryAssociation
*
* When primary and secondary databases are associated using a {@code
* SecondaryAssociation}, the databases must all be configured with the same
* {@code SecondaryAssociation} instance. A common error is to forget to
* configure the {@code SecondaryAssociation} on the primary database.
*
* A {@code SecondaryAssociation} is configured using {@link
* DatabaseConfig#setSecondaryAssociation} and {@link
* SecondaryConfig#setSecondaryAssociation} for a primary and secondary
* database, respectively. When calling {@link
* Environment#openSecondaryDatabase}, null must be passed for the
* primaryDatabase parameter when a {@code SecondaryAssociation} is configured.
*
* Note that when a {@code SecondaryAssociation} is configured, {@code true}
* may not be passed to the {@link SecondaryConfig#setAllowPopulate} method.
* Population of new secondary databases in an existing {@code
* SecondaryAssociation} is done differently, as described below.
*
* Adding and removing secondary associations
*
* The state information defining the association between primary and secondary
* databases is maintained by the application, and made available to JE by
* implementing the {@code SecondaryAssociation} interface. The application
* may add/remove databases to/from the association without any explicit
* coordination with JE. However, certain rules need to be followed to
* maintain data integrity.
*
* In the simplest case, there is a fixed (never changing) set of primary and
* secondary databases and they are added to the association at the time they
* are created, before writing or reading records. In this case, since reads
* and writes do not occur concurrently with changes to the association, no
* special rules are needed. For example:
*
* - Open the databases.
* - Add databases to the association.
* - Begin writing and reading.
*
*
* In other cases, primary and secondary databases may be added to an already
* established association, along with concurrent reads and writes. The rules
* for doing so are described below.
*
* Adding an empty (newly created) primary database is no more complex than in
* the simple case above, assuming that writes to the primary database do not
* proceed until after it has been added to the association.
*
* - Open the new primary database.
* - Add the primary database to the association such that {@link
* #getSecondaries} returns the appropriate list when passed a key in the
* new primary database, and {@link #getPrimary} returns the new database
* when passed a key it contains.
* - Begin writing to the new primary database.
*
*
* Using the procedure above for adding a primary database, records will be
* indexed in secondary databases as they are added to the primary database,
* and will be immediately available to secondary index queries.
*
* Alternatively, records can be added to the primary database without making
* them immediately available to secondary index queries by using the following
* procedure. Records will be indexed in secondary databases as they are added
* but will not be returned by secondary queries until after the last step.
*
* - Open the new primary database.
* - Modify the association such that {@link #getSecondaries} returns the
* appropriate list when passed a key in the new primary database, but
* {@link #getPrimary} returns null when passed a key it contains.
* - Write records in the new primary database.
* - Modify the association such that {@link #getPrimary} returns the new
* database when passed a key it contains.
*
*
* The following procedure should be used for removing an existing (non-empty)
* primary database from an association.
*
* - Remove the primary database from the association, such that {@link
* #getPrimary} returns null for all keys in the primary.
* - Disable read and write operations on the database and ensure that
* all in-progress operations have completed.
* - At this point the primary database may be closed and removed
* (e.g., with {@link Environment#removeDatabase}), if desired.
* - For each secondary database associated with the removed primary
* database, {@link SecondaryDatabase#deleteObsoletePrimaryKeys} should be
* called to process all secondary records.
* - At this point {@link #getPrimary} may throw an exception (rather
* than return null) when called for a primary key in the removed database,
* if desired.
*
*
* The following procedure should be used for adding a new (empty) secondary
* database to an association, and to populate the secondary incrementally
* from its associated primary database(s).
*
* - Open the secondary database and call {@link
* SecondaryDatabase#startIncrementalPopulation}. The secondary may not be
* used (yet) for read operations.
* - Add the secondary database to the association, such that the
* collection returned by {@link #getSecondaries} includes the new
* database, for all primary keys that should be indexed.
* - For each primary database associated with the new secondary
* database, {@link Database#populateSecondaries} should be called to
* process all primary records.
* - Call {@link SecondaryDatabase#endIncrementalPopulation}. The
* secondary database may now be used for read operations.
*
*
* The following procedure should be used for removing an existing (non-empty)
* secondary database from an association.
*
* - Remove the secondary database from the association, such that it is
* not included in the collection returned by {@link #getSecondaries}.
* - Disable read and write operations on the database and ensure that
* all in-progress operations have completed.
* - The secondary database may now be closed and removed, if
* desired.
*
*
* Other Implementation Requirements
*
* The implementation must use data structures for storing and accessing
* association information that provide happens-before semantics. In
* other words, when the association is changed, other threads calling the
* methods in this interface must see the changes as soon as the change is
* complete.
*
* The implementation should use non-blocking data structures to hold
* association information to avoid blocking on the methods in this interface,
* which may be frequently called from many threads.
*
* The simplest way to meet the above two requirements is to use concurrent
* structures such as those provided in the java.util.concurrent package, e.g.,
* ConcurrentHashMap.
*
* For an example implementation, see:
* test/com/sleepycat/je/test/SecondaryAssociationTest.java
*/
public interface SecondaryAssociation {
/*
* Implementation note on concurrent access and latching.
*
* When adding/removing primary/secondary DBs to an existing association,
* concurrency issues arise. This section explains how they are handled,
* assuming the rules described in the javadoc below are followed when a
* custom association is configured.
*
* There are two cases:
*
* 1) A custom association is NOT configured. An internal association is
* created that manages the one-many association between a primary and its
* secondaries.
*
* 2) A custom association IS configured. It is used internally.
*
* For case 1, no custom association, DB addition and removal take place in
* the Environment.openSecondaryDatabase and SecondaryDatabase.close
* methods, respectively. These methods, and the changes to the internal
* association, are protected by acquiring the secondary latch
* (EnvironmentImpl.getSecondaryAssociationLock) exclusively. All write
* operations that use the association -- puts and deletes -- acquire this
* latch shared. Therefore, write operations and changes to the
* association do not take place concurrently. Read operations via a
* secondary DB/cursor do not acquire this latch, so they can take place
* concurrently with changes to the association; however, read operations
* only call getPrimary and they expect it may return null at any time (the
* operation ignores the record in that case). It is impossible to close
* a primary while it is being read via a secondary, because secondaries
* must be closed before their associated primary. (And the application is
* responsible for finishing reads before closing any DB.)
*
* For case 2, a custom association, the secondary latch does not provide
* protection because modifications to the association take place in the
* application domain, not in the Environment.openSecondaryDatabase and
* SecondaryDatabase.close methods. Instead, protection is provided by the
* rules described in the javadoc here. Namely:
*
* Primary/secondary DBs are added to the association AFTER opening them
* (of course). The DBs will not be be accessed until they are added to
* the association.
*
* Primary/secondary DBs are removed from the association BEFORE closing
* them. The application is responsible for ensuring they are no longer
* accessed before they are closed.
*
* When a primary DB is added, its secondaries will be kept in sync as
* soon as writing begins, since it must be added to the association
* before writes are allowed.
*
* When a secondary DB is added, the incremental population procedure
* ensures that it is fully populated before being accessed.
*/
/**
* Returns true if there are no secondary databases in the association.
*
* This method is used by JE to optimize for the case where a primary
* has no secondary databases. This allows a {@code SecondaryAssociation}
* to be configured on a primary database even when it has no associated
* secondaries, with no added overhead, and to allow for the possibility
* that secondaries will be added later.
*
* For example, when a primary database has no secondaries, no internal
* latching is performed by JE during writes. JE determines that no
* secondaries are present by calling {@code isEmpty}, prior to doing the
* write operation.
*
* @throws RuntimeException if an unexpected problem occurs. The exception
* will be thrown to the application, which should then take appropriate
* action.
*/
boolean isEmpty();
/**
* Returns the primary database for the given primary key. This method
* is called during read operations on secondary databases that are
* configured with this {@code SecondaryAssociation}.
*
* This method should return null when the primary database has been
* removed from the association, or when it should not be included in the
* results for secondary queries. In this case, the current operation will
* treat the secondary record as if it does not exist, i.e., it will be
* skipped over.
*
* @throws RuntimeException if an unexpected problem occurs. The exception
* will be thrown to the application, which should then take appropriate
* action.
*/
Database getPrimary(DatabaseEntry primaryKey);
/**
* Returns the secondary databases associated with the given primary key.
* This method is called during write operations on primary databases that
* are configured with this {@code SecondaryAssociation}.
*
* @throws RuntimeException if an unexpected problem occurs. The exception
* will be thrown to the application, which should then take appropriate
* action.
*/
Collection getSecondaries(DatabaseEntry primaryKey);
}