com.amazonaws.services.kinesisfirehose.AmazonKinesisFirehose Maven / Gradle / Ivy
Show all versions of aws-java-sdk-osgi Show documentation
/*
* Copyright 2011-2016 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not
* use this file except in compliance with the License. A copy of the License is
* located at
*
* http://aws.amazon.com/apache2.0
*
* or in the "license" file accompanying this file. This file is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
* permissions and limitations under the License.
*/
package com.amazonaws.services.kinesisfirehose;
import com.amazonaws.*;
import com.amazonaws.regions.*;
import com.amazonaws.services.kinesisfirehose.model.*;
/**
* Interface for accessing Firehose.
*
* Amazon Kinesis Firehose API Reference
*
* Amazon Kinesis Firehose is a fully-managed service that delivers real-time
* streaming data to destinations such as Amazon Simple Storage Service (Amazon
* S3), Amazon Elasticsearch Service (Amazon ES), and Amazon Redshift.
*
*/
public interface AmazonKinesisFirehose {
/**
* The region metadata service name for computing region endpoints. You can
* use this value to retrieve metadata (such as supported regions) of the
* service.
*
* @see RegionUtils#getRegionsForService(String)
*/
String ENDPOINT_PREFIX = "firehose";
/**
* Overrides the default endpoint for this client
* ("https://firehose.us-east-1.amazonaws.com"). Callers can use this method
* to control which AWS region they want to work with.
*
* Callers can pass in just the endpoint (ex:
* "firehose.us-east-1.amazonaws.com") or a full URL, including the protocol
* (ex: "https://firehose.us-east-1.amazonaws.com"). If the protocol is not
* specified here, the default protocol from this client's
* {@link ClientConfiguration} will be used, which by default is HTTPS.
*
* For more information on using AWS regions with the AWS SDK for Java, and
* a complete list of all available endpoints for all AWS services, see: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=
* 3912
*
* This method is not threadsafe. An endpoint should be configured when
* the client is created and before any service requests are made. Changing
* it afterwards creates inevitable race conditions for any service requests
* in transit or retrying.
*
* @param endpoint
* The endpoint (ex: "firehose.us-east-1.amazonaws.com") or a full
* URL, including the protocol (ex:
* "https://firehose.us-east-1.amazonaws.com") of the region specific
* AWS endpoint this client will communicate with.
*/
void setEndpoint(String endpoint);
/**
* An alternative to {@link AmazonKinesisFirehose#setEndpoint(String)}, sets
* the regional endpoint for this client's service calls. Callers can use
* this method to control which AWS region they want to work with.
*
* By default, all service endpoints in all regions use the https protocol.
* To use http instead, specify it in the {@link ClientConfiguration}
* supplied at construction.
*
* This method is not threadsafe. A region should be configured when the
* client is created and before any service requests are made. Changing it
* afterwards creates inevitable race conditions for any service requests in
* transit or retrying.
*
* @param region
* The region this client will communicate with. See
* {@link Region#getRegion(com.amazonaws.regions.Regions)} for
* accessing a given region. Must not be null and must be a region
* where the service is available.
*
* @see Region#getRegion(com.amazonaws.regions.Regions)
* @see Region#createClient(Class,
* com.amazonaws.auth.AWSCredentialsProvider, ClientConfiguration)
* @see Region#isServiceSupported(String)
*/
void setRegion(Region region);
/**
*
* Creates a delivery stream.
*
*
* CreateDeliveryStream is an asynchronous operation that immediately
* returns. The initial status of the delivery stream is
* CREATING. After the delivery stream is created, its status
* is ACTIVE and it now accepts data. Attempts to send data to
* a delivery stream that is not in the ACTIVE state cause an
* exception. To check the state of a delivery stream, use
* DescribeDeliveryStream.
*
*
* The name of a delivery stream identifies it. You can't have two delivery
* streams with the same name in the same region. Two delivery streams in
* different AWS accounts or different regions in the same AWS account can
* have the same name.
*
*
* By default, you can create up to 20 delivery streams per region.
*
*
* A delivery stream can only be configured with a single destination,
* Amazon S3, Amazon Elasticsearch Service, or Amazon Redshift. For correct
* CreateDeliveryStream request syntax, specify only one destination
* configuration parameter: either S3DestinationConfiguration,
* ElasticsearchDestinationConfiguration, or
* RedshiftDestinationConfiguration.
*
*
* As part of S3DestinationConfiguration, optional values
* BufferingHints, EncryptionConfiguration, and
* CompressionFormat can be provided. By default, if no
* BufferingHints value is provided, Firehose buffers data up to 5 MB
* or for 5 minutes, whichever condition is satisfied first. Note that
* BufferingHints is a hint, so there are some cases where the
* service cannot adhere to these conditions strictly; for example, record
* boundaries are such that the size is a little over or under the
* configured buffering size. By default, no encryption is performed. We
* strongly recommend that you enable encryption to ensure secure data
* storage in Amazon S3.
*
*
* A few notes about RedshiftDestinationConfiguration:
*
*
* -
*
* An Amazon Redshift destination requires an S3 bucket as intermediate
* location, as Firehose first delivers data to S3 and then uses
* COPY syntax to load data into an Amazon Redshift table. This
* is specified in the
* RedshiftDestinationConfiguration.S3Configuration parameter
* element.
*
*
* -
*
* The compression formats SNAPPY or ZIP cannot be
* specified in RedshiftDestinationConfiguration.S3Configuration
* because the Amazon Redshift COPY operation that reads from
* the S3 bucket doesn't support these compression formats.
*
*
* -
*
* We strongly recommend that the username and password provided is used
* exclusively for Firehose purposes, and that the permissions for the
* account are restricted for Amazon Redshift INSERT
* permissions.
*
*
*
*
* Firehose assumes the IAM role that is configured as part of destinations.
* The IAM role should allow the Firehose principal to assume the role, and
* the role should have permissions that allows the service to deliver the
* data. For more information, see Amazon S3 Bucket Access in the Amazon Kinesis Firehose Developer
* Guide.
*
*
* @param createDeliveryStreamRequest
* Contains the parameters for CreateDeliveryStream.
* @return Result of the CreateDeliveryStream operation returned by the
* service.
* @throws InvalidArgumentException
* The specified input parameter has an value that is not valid.
* @throws LimitExceededException
* You have already reached the limit for a requested resource.
* @throws ResourceInUseException
* The resource is already in use and not available for this
* operation.
* @sample AmazonKinesisFirehose.CreateDeliveryStream
*/
CreateDeliveryStreamResult createDeliveryStream(
CreateDeliveryStreamRequest createDeliveryStreamRequest);
/**
*
* Deletes a delivery stream and its data.
*
*
* You can delete a delivery stream only if it is in ACTIVE or
* DELETING state, and not in the CREATING state.
* While the deletion request is in process, the delivery stream is in the
* DELETING state.
*
*
* To check the state of a delivery stream, use
* DescribeDeliveryStream.
*
*
* While the delivery stream is DELETING state, the service may
* continue to accept the records, but the service doesn't make any
* guarantees with respect to delivering the data. Therefore, as a best
* practice, you should first stop any applications that are sending records
* before deleting a delivery stream.
*
*
* @param deleteDeliveryStreamRequest
* Contains the parameters for DeleteDeliveryStream.
* @return Result of the DeleteDeliveryStream operation returned by the
* service.
* @throws ResourceInUseException
* The resource is already in use and not available for this
* operation.
* @throws ResourceNotFoundException
* The specified resource could not be found.
* @sample AmazonKinesisFirehose.DeleteDeliveryStream
*/
DeleteDeliveryStreamResult deleteDeliveryStream(
DeleteDeliveryStreamRequest deleteDeliveryStreamRequest);
/**
*
* Describes the specified delivery stream and gets the status. For example,
* after your delivery stream is created, call DescribeDeliveryStream
* to see if the delivery stream is ACTIVE and therefore ready
* for data to be sent to it.
*
*
* @param describeDeliveryStreamRequest
* Contains the parameters for DescribeDeliveryStream.
* @return Result of the DescribeDeliveryStream operation returned by the
* service.
* @throws ResourceNotFoundException
* The specified resource could not be found.
* @sample AmazonKinesisFirehose.DescribeDeliveryStream
*/
DescribeDeliveryStreamResult describeDeliveryStream(
DescribeDeliveryStreamRequest describeDeliveryStreamRequest);
/**
*
* Lists your delivery streams.
*
*
* The number of delivery streams might be too large to return using a
* single call to ListDeliveryStreams. You can limit the number of
* delivery streams returned, using the Limit parameter. To determine
* whether there are more delivery streams to list, check the value of
* HasMoreDeliveryStreams in the output. If there are more delivery
* streams to list, you can request them by specifying the name of the last
* delivery stream returned in the call in the
* ExclusiveStartDeliveryStreamName parameter of a subsequent call.
*
*
* @param listDeliveryStreamsRequest
* Contains the parameters for ListDeliveryStreams.
* @return Result of the ListDeliveryStreams operation returned by the
* service.
* @sample AmazonKinesisFirehose.ListDeliveryStreams
*/
ListDeliveryStreamsResult listDeliveryStreams(
ListDeliveryStreamsRequest listDeliveryStreamsRequest);
/**
*
* Writes a single data record into an Amazon Kinesis Firehose delivery
* stream. To write multiple data records into a delivery stream, use
* PutRecordBatch. Applications using these operations are referred
* to as producers.
*
*
* By default, each delivery stream can take in up to 2,000 transactions per
* second, 5,000 records per second, or 5 MB per second. Note that if you
* use PutRecord and PutRecordBatch, the limits are an
* aggregate across these two operations for each delivery stream. For more
* information about limits and how to request an increase, see Amazon
* Kinesis Firehose Limits.
*
*
* You must specify the name of the delivery stream and the data record when
* using PutRecord. The data record consists of a data blob that can
* be up to 1,000 KB in size, and any kind of data, for example, a segment
* from a log file, geographic location data, web site clickstream data,
* etc.
*
*
* Firehose buffers records before delivering them to the destination. To
* disambiguate the data blobs at the destination, a common solution is to
* use delimiters in the data, such as a newline (\n) or some
* other character unique within the data. This allows the consumer
* application(s) to parse individual data items when reading the data from
* the destination.
*
*
* The PutRecord operation returns a RecordId, which is a
* unique string assigned to each record. Producer applications can use this
* ID for purposes such as auditability and investigation.
*
*
* If the PutRecord operation throws a
* ServiceUnavailableException, back off and retry. If the exception
* persists, it is possible that the throughput limits have been exceeded
* for the delivery stream.
*
*
* Data records sent to Firehose are stored for 24 hours from the time they
* are added to a delivery stream as it attempts to send the records to the
* destination. If the destination is unreachable for more than 24 hours,
* the data is no longer available.
*
*
* @param putRecordRequest
* Contains the parameters for PutRecord.
* @return Result of the PutRecord operation returned by the service.
* @throws ResourceNotFoundException
* The specified resource could not be found.
* @throws InvalidArgumentException
* The specified input parameter has an value that is not valid.
* @throws ServiceUnavailableException
* The service is unavailable, back off and retry the operation. If
* you continue to see the exception, throughput limits for the
* delivery stream may have been exceeded. For more information
* about limits and how to request an increase, see Amazon Kinesis Firehose Limits.
* @sample AmazonKinesisFirehose.PutRecord
*/
PutRecordResult putRecord(PutRecordRequest putRecordRequest);
/**
*
* Writes multiple data records into a delivery stream in a single call,
* which can achieve higher throughput per producer than when writing single
* records. To write single data records into a delivery stream, use
* PutRecord. Applications using these operations are referred to as
* producers.
*
*
* Each PutRecordBatch request supports up to 500 records. Each
* record in the request can be as large as 1,000 KB (before 64-bit
* encoding), up to a limit of 4 MB for the entire request. By default, each
* delivery stream can take in up to 2,000 transactions per second, 5,000
* records per second, or 5 MB per second. Note that if you use
* PutRecord and PutRecordBatch, the limits are an aggregate
* across these two operations for each delivery stream. For more
* information about limits and how to request an increase, see Amazon
* Kinesis Firehose Limits.
*
*
* You must specify the name of the delivery stream and the data record when
* using PutRecord. The data record consists of a data blob that can
* be up to 1,000 KB in size, and any kind of data, for example, a segment
* from a log file, geographic location data, web site clickstream data, and
* so on.
*
*
* Firehose buffers records before delivering them to the destination. To
* disambiguate the data blobs at the destination, a common solution is to
* use delimiters in the data, such as a newline (\n) or some
* other character unique within the data. This allows the consumer
* application(s) to parse individual data items when reading the data from
* the destination.
*
*
* The PutRecordBatch response includes a count of any failed
* records, FailedPutCount, and an array of responses,
* RequestResponses. The FailedPutCount value is a count of
* records that failed. Each entry in the RequestResponses array
* gives additional information of the processed record. Each entry in
* RequestResponses directly correlates with a record in the request
* array using the same ordering, from the top to the bottom of the request
* and response. RequestResponses always includes the same number of
* records as the request array. RequestResponses both successfully
* and unsuccessfully processed records. Firehose attempts to process all
* records in each PutRecordBatch request. A single record failure
* does not stop the processing of subsequent records.
*
*
* A successfully processed record includes a RecordId value, which
* is a unique value identified for the record. An unsuccessfully processed
* record includes ErrorCode and ErrorMessage values.
* ErrorCode reflects the type of error and is one of the following
* values: ServiceUnavailable or InternalFailure.
* ErrorMessage provides more detailed information about the
* error.
*
*
* If FailedPutCount is greater than 0 (zero), retry the request. A
* retry of the entire batch of records is possible; however, we strongly
* recommend that you inspect the entire response and resend only those
* records that failed processing. This minimizes duplicate records and also
* reduces the total bytes sent (and corresponding charges).
*
*
* If the PutRecordBatch operation throws a
* ServiceUnavailableException, back off and retry. If the exception
* persists, it is possible that the throughput limits have been exceeded
* for the delivery stream.
*
*
* Data records sent to Firehose are stored for 24 hours from the time they
* are added to a delivery stream as it attempts to send the records to the
* destination. If the destination is unreachable for more than 24 hours,
* the data is no longer available.
*
*
* @param putRecordBatchRequest
* Contains the parameters for PutRecordBatch.
* @return Result of the PutRecordBatch operation returned by the service.
* @throws ResourceNotFoundException
* The specified resource could not be found.
* @throws InvalidArgumentException
* The specified input parameter has an value that is not valid.
* @throws ServiceUnavailableException
* The service is unavailable, back off and retry the operation. If
* you continue to see the exception, throughput limits for the
* delivery stream may have been exceeded. For more information
* about limits and how to request an increase, see Amazon Kinesis Firehose Limits.
* @sample AmazonKinesisFirehose.PutRecordBatch
*/
PutRecordBatchResult putRecordBatch(
PutRecordBatchRequest putRecordBatchRequest);
/**
*
* Updates the specified destination of the specified delivery stream. Note:
* Switching between Elasticsearch and other services is not supported. For
* Elasticsearch destination, you can only update an existing Elasticsearch
* destination with this operation.
*
*
* This operation can be used to change the destination type (for example,
* to replace the Amazon S3 destination with Amazon Redshift) or change the
* parameters associated with a given destination (for example, to change
* the bucket name of the Amazon S3 destination). The update may not occur
* immediately. The target delivery stream remains active while the
* configurations are updated, so data writes to the delivery stream can
* continue during this process. The updated configurations are normally
* effective within a few minutes.
*
*
* If the destination type is the same, Firehose merges the configuration
* parameters specified in the UpdateDestination request with the
* destination configuration that already exists on the delivery stream. If
* any of the parameters are not specified in the update request, then the
* existing configuration parameters are retained. For example, in the
* Amazon S3 destination, if EncryptionConfiguration is not specified
* then the existing EncryptionConfiguration is maintained on the
* destination.
*
*
* If the destination type is not the same, for example, changing the
* destination from Amazon S3 to Amazon Redshift, Firehose does not merge
* any parameters. In this case, all parameters must be specified.
*
*
* Firehose uses the CurrentDeliveryStreamVersionId to avoid race
* conditions and conflicting merges. This is a required field in every
* request and the service only updates the configuration if the existing
* configuration matches the VersionId. After the update is applied
* successfully, the VersionId is updated, which can be retrieved
* with the DescribeDeliveryStream operation. The new
* VersionId should be uses to set
* CurrentDeliveryStreamVersionId in the next
* UpdateDestination operation.
*
*
* @param updateDestinationRequest
* Contains the parameters for UpdateDestination.
* @return Result of the UpdateDestination operation returned by the
* service.
* @throws InvalidArgumentException
* The specified input parameter has an value that is not valid.
* @throws ResourceInUseException
* The resource is already in use and not available for this
* operation.
* @throws ResourceNotFoundException
* The specified resource could not be found.
* @throws ConcurrentModificationException
* Another modification has already happened. Fetch VersionId
* again and use it to update the destination.
* @sample AmazonKinesisFirehose.UpdateDestination
*/
UpdateDestinationResult updateDestination(
UpdateDestinationRequest updateDestinationRequest);
/**
* Shuts down this client object, releasing any resources that might be held
* open. This is an optional method, and callers are not expected to call
* it, but can if they want to explicitly release any open resources. Once a
* client has been shutdown, it should not be used to make any more
* requests.
*/
void shutdown();
/**
* Returns additional metadata for a previously executed successful request,
* typically used for debugging issues where a service isn't acting as
* expected. This data isn't considered part of the result data returned by
* an operation, so it's available through this separate, diagnostic
* interface.
*
* Response metadata is only cached for a limited period of time, so if you
* need to access this extra diagnostic information for an executed request,
* you should use this method to retrieve it as soon as possible after
* executing a request.
*
* @param request
* The originally executed request.
*
* @return The response metadata for the specified request, or null if none
* is available.
*/
ResponseMetadata getCachedResponseMetadata(AmazonWebServiceRequest request);
}