org.elasticsearch.snapshots.package-info Maven / Gradle / Ivy
The newest version!
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
/**
* This package exposes the Elasticsearch Snapshot functionality.
*
* Preliminaries
*
* There are two communication channels between all nodes and master in the snapshot functionality:
*
* - The master updates the cluster state by adding, removing or altering the contents of its custom entry
* {@link org.elasticsearch.cluster.SnapshotsInProgress}. All nodes consume the state of the {@code SnapshotsInProgress} and will start or
* abort relevant shard snapshot tasks accordingly.
* - Nodes that are executing shard snapshot tasks report either success or failure of their snapshot task by submitting a
* {@link org.elasticsearch.snapshots.SnapshotShardsService.UpdateIndexShardSnapshotStatusRequest} to the master node that will update the
* snapshot's entry in the cluster state accordingly.
*
*
* Snapshot Creation
* Snapshots are created by the following sequence of events:
*
* - An invocation of {@link org.elasticsearch.snapshots.SnapshotsService#createSnapshot} enqueues a cluster state update to create
* a {@link org.elasticsearch.cluster.SnapshotsInProgress.Entry} in the cluster state's {@code SnapshotsInProgress}. This initial snapshot
* entry has its state set to {@code INIT} and an empty map set for the state of the individual shard's snapshots.
*
* - After the snapshot's entry with state {@code INIT} is in the cluster state, {@link org.elasticsearch.snapshots.SnapshotsService}
* determines the primary shards' assignments for all indices that are being snapshotted and updates the existing
* {@code SnapshotsInProgress.Entry} with state {@code STARTED} and adds the map of {@link org.elasticsearch.index.shard.ShardId} to
* {@link org.elasticsearch.cluster.SnapshotsInProgress.ShardSnapshotStatus} that tracks the assignment of which node is to snapshot which
* shard. All shard snapshots are executed on the shard's primary node. Thus all shards for which the primary node was found to have a
* healthy copy of the shard are marked as being in state {@code INIT} in this map. If the primary for a shard is unassigned, it is marked
* as {@code MISSING} in this map. In case the primary is initializing at this point, it is marked as in state {@code WAITING}. In case a
* shard's primary is relocated at any point after its {@code SnapshotsInProgress.Entry} has moved to state {@code STARTED} and thus been
* assigned to a specific cluster node, that shard's snapshot will fail and move to state {@code FAILED}.
*
* - The new {@code SnapshotsInProgress.Entry} is then observed by
* {@link org.elasticsearch.snapshots.SnapshotShardsService#clusterChanged} on all nodes and since the entry is in state {@code STARTED}
* the {@code SnapshotShardsService} will check if any local primary shards are to be snapshotted (signaled by the shard's snapshot state
* being {@code INIT}). For those local primary shards found in state {@code INIT}) the snapshot process of writing the shard's data files
* to the snapshot's {@link org.elasticsearch.repositories.Repository} is executed. Once the snapshot execution finishes for a shard an
* {@code UpdateIndexShardSnapshotStatusRequest} is sent to the master node signaling either status {@code SUCCESS} or {@code FAILED}.
* The master node will then update a shard's state in the snapshots {@code SnapshotsInProgress.Entry} whenever it receives such a
* {@code UpdateIndexShardSnapshotStatusRequest}.
*
* - If as a result of the received status update requests, all shards in the cluster state are in a completed state, i.e are marked as
* either {@code SUCCESS}, {@code FAILED} or {@code MISSING}, the {@code SnapshotShardsService} will update the state of the {@code Entry}
* itself and mark it as {@code SUCCESS}. At the same time {@link org.elasticsearch.snapshots.SnapshotsService#endSnapshot} is executed,
* writing the metadata necessary to finalize the snapshot in the repository to the repository.
*
* - After writing the final metadata to the repository, a cluster state update to remove the snapshot from the cluster state is
* submitted and the removal of the snapshot's {@code SnapshotsInProgress.Entry} from the cluster state completes the snapshot process.
*
*
*
* Deleting a Snapshot
*
* Deleting a snapshot can take the form of either simply deleting it from the repository or (if it has not completed yet) aborting it
* and subsequently deleting it from the repository.
*
* Aborting a Snapshot
*
*
* - Aborting a snapshot starts by updating the state of the snapshot's {@code SnapshotsInProgress.Entry} to {@code ABORTED}.
*
* - The snapshot's state change to {@code ABORTED} in cluster state is then picked up by the {@code SnapshotShardsService} on all nodes.
* Those nodes that have shard snapshot actions for the snapshot assigned to them, will abort them and notify master about the shards
* snapshot status accordingly. If the shard snapshot action completed or was in state {@code FINALIZE} when the abort was registered by
* the {@code SnapshotShardsService}, then the shard's state will be reported to master as {@code SUCCESS}.
* Otherwise, it will be reported as {@code FAILED}.
*
* - Once all the shards are reported to master as either {@code SUCCESS} or {@code FAILED} the {@code SnapshotsService} on the master
* will finish the snapshot process as all shard's states are now completed and hence the snapshot can be completed as explained in point 4
* of the snapshot creation section above.
*
*
* Deleting a Snapshot from a Repository
*
*
* - Assuming there are no entries in the cluster state's {@code SnapshotsInProgress}, deleting a snapshot starts by the
* {@code SnapshotsService} creating an entry for deleting the snapshot in the cluster state's
* {@link org.elasticsearch.cluster.SnapshotDeletionsInProgress}.
*
* - Once the cluster state contains the deletion entry in {@code SnapshotDeletionsInProgress} the {@code SnapshotsService} will invoke
* {@link org.elasticsearch.repositories.Repository#deleteSnapshot} for the given snapshot, which will remove files associated with the
* snapshot from the repository as well as update its meta-data to reflect the deletion of the snapshot.
*
* - After the deletion of the snapshot's data from the repository finishes, the {@code SnapshotsService} will submit a cluster state
* update to remove the deletion's entry in {@code SnapshotDeletionsInProgress} which concludes the process of deleting a snapshot.
*
*/
package org.elasticsearch.snapshots;