org.elasticsearch.snapshots.package-info Maven / Gradle / Ivy

Show more of this group Show more artifacts with this name
Show all versions of elasticsearch Show documentation
Elasticsearch subproject :server
There is a newer version: 8.13.2
/*
 * Licensed to Elasticsearch under one or more contributor
 * license agreements. See the NOTICE file distributed with
 * this work for additional information regarding copyright
 * ownership. Elasticsearch licenses this file to you under
 * the Apache License, Version 2.0 (the "License"); you may
 * not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 */

/**
 * This package exposes the Elasticsearch Snapshot functionality.
 *
 * Preliminaries
 *
 * There are two communication channels between all nodes and master in the snapshot functionality:
 * 
 * The master updates the cluster state by adding, removing or altering the contents of its custom entry
 * {@link org.elasticsearch.cluster.SnapshotsInProgress}. All nodes consume the state of the {@code SnapshotsInProgress} and will start or
 * abort relevant shard snapshot tasks accordingly.
 * Nodes that are executing shard snapshot tasks report either success or failure of their snapshot task by submitting a
 * {@link org.elasticsearch.snapshots.SnapshotShardsService.UpdateIndexShardSnapshotStatusRequest} to the master node that will update the
 * snapshot's entry in the cluster state accordingly.
 * 
 *
 * Snapshot Creation
 * Snapshots are created by the following sequence of events:
 * 
 * An invocation of {@link org.elasticsearch.snapshots.SnapshotsService#createSnapshot} enqueues a cluster state update to create
 * a {@link org.elasticsearch.cluster.SnapshotsInProgress.Entry} in the cluster state's {@code SnapshotsInProgress}. This initial snapshot
 * entry has its state set to {@code INIT} and an empty map set for the state of the individual shard's snapshots.
 *
 * After the snapshot's entry with state {@code INIT} is in the cluster state, {@link org.elasticsearch.snapshots.SnapshotsService}
 * determines the primary shards' assignments for all indices that are being snapshotted and updates the existing
 * {@code SnapshotsInProgress.Entry} with state {@code STARTED} and adds the map of {@link org.elasticsearch.index.shard.ShardId} to
 * {@link org.elasticsearch.cluster.SnapshotsInProgress.ShardSnapshotStatus} that tracks the assignment of which node is to snapshot which
 * shard. All shard snapshots are executed on the shard's primary node. Thus all shards for which the primary node was found to have a
 * healthy copy of the shard are marked as being in state {@code INIT} in this map. If the primary for a shard is unassigned, it is marked
 * as {@code MISSING} in this map. In case the primary is initializing at this point, it is marked as in state {@code WAITING}. In case a
 * shard's primary is relocated at any point after its {@code SnapshotsInProgress.Entry} has moved to state {@code STARTED} and thus been
 * assigned to a specific cluster node, that shard's snapshot will fail and move to state {@code FAILED}.
 *
 * The new {@code SnapshotsInProgress.Entry} is then observed by
 * {@link org.elasticsearch.snapshots.SnapshotShardsService#clusterChanged} on all nodes and since the entry is in state {@code STARTED}
 * the {@code SnapshotShardsService} will check if any local primary shards are to be snapshotted (signaled by the shard's snapshot state
 * being {@code INIT}). For those local primary shards found in state {@code INIT}) the snapshot process of writing the shard's data files
 * to the snapshot's {@link org.elasticsearch.repositories.Repository} is executed. Once the snapshot execution finishes for a shard an
 * {@code UpdateIndexShardSnapshotStatusRequest} is sent to the master node signaling either status {@code SUCCESS} or {@code FAILED}.
 * The master node will then update a shard's state in the snapshots {@code SnapshotsInProgress.Entry} whenever it receives such a
 * {@code UpdateIndexShardSnapshotStatusRequest}.
 *
 * If as a result of the received status update requests, all shards in the cluster state are in a completed state, i.e are marked as
 * either {@code SUCCESS}, {@code FAILED} or {@code MISSING}, the {@code SnapshotShardsService} will update the state of the {@code Entry}
 * itself and mark it as {@code SUCCESS}. At the same time {@link org.elasticsearch.snapshots.SnapshotsService#endSnapshot} is executed,
 * writing the metadata necessary to finalize the snapshot in the repository to the repository.
 *
 * After writing the final metadata to the repository, a cluster state update to remove the snapshot from the cluster state is
 * submitted and the removal of the snapshot's {@code SnapshotsInProgress.Entry} from the cluster state completes the snapshot process.
 * 
 * 
 *
 * Deleting a Snapshot
 *
 * Deleting a snapshot can take the form of either simply deleting it from the repository or (if it has not completed yet) aborting it
 * and subsequently deleting it from the repository.
 *
 * Aborting a Snapshot
 *
 * 
 * Aborting a snapshot starts by updating the state of the snapshot's {@code SnapshotsInProgress.Entry} to {@code ABORTED}.
 *
 * The snapshot's state change to {@code ABORTED} in cluster state is then picked up by the {@code SnapshotShardsService} on all nodes.
 * Those nodes that have shard snapshot actions for the snapshot assigned to them, will abort them and notify master about the shards
 * snapshot status accordingly. If the shard snapshot action completed or was in state {@code FINALIZE} when the abort was registered by
 * the {@code SnapshotShardsService}, then the shard's state will be reported to master as {@code SUCCESS}.
 * Otherwise, it will be reported as {@code FAILED}.
 *
 * Once all the shards are reported to master as either {@code SUCCESS} or {@code FAILED} the {@code SnapshotsService} on the master
 * will finish the snapshot process as all shard's states are now completed and hence the snapshot can be completed as explained in point 4
 * of the snapshot creation section above.
 * 
 *
 * Deleting a Snapshot from a Repository
 *
 * 
 * Assuming there are no entries in the cluster state's {@code SnapshotsInProgress}, deleting a snapshot starts by the
 * {@code SnapshotsService} creating an entry for deleting the snapshot in the cluster state's
 * {@link org.elasticsearch.cluster.SnapshotDeletionsInProgress}.
 *
 * Once the cluster state contains the deletion entry in {@code SnapshotDeletionsInProgress} the {@code SnapshotsService} will invoke
 * {@link org.elasticsearch.repositories.Repository#deleteSnapshot} for the given snapshot, which will remove files associated with the
 * snapshot from the repository as well as update its meta-data to reflect the deletion of the snapshot.
 *
 * After the deletion of the snapshot's data from the repository finishes, the {@code SnapshotsService} will submit a cluster state
 * update to remove the deletion's entry in {@code SnapshotDeletionsInProgress} which concludes the process of deleting a snapshot.
 * 
 */
package org.elasticsearch.snapshots;