AWS S3 Storage Service

AWS S3 Storage Service

Scheme: aws-s3
Syntax: aws-s3:bucketNameOrArn
Description: The aws-s3 component is used for storing and retrieving objecct from Amazon S3 Storage Service.
Deprecated:false
Async:false
Maven: org.apache.camel/camel-aws/2.18.1.redhat-000025

The aws-s3 component is used for storing and retrieving objecct from Amazon S3 Storage Service.

Name Kind Group Required Default Type Enum Description
bucketNameOrArn path common true java.lang.String Bucket name or ARN
accessKey parameter common java.lang.String Amazon AWS Access Key
amazonS3Client parameter common com.amazonaws.services.s3.AmazonS3 Reference to a `com.amazonaws.services.sqs.AmazonS3` in the link:registry.html[Registry].
amazonS3Endpoint parameter common java.lang.String The region with which the AWS-S3 client wants to work with.
pathStyleAccess parameter common boolean Whether or not the S3 client should use path style access
policy parameter common java.lang.String *Camel 2.8.4*: The policy for this queue to set in the `com.amazonaws.services.s3.AmazonS3#setBucketPolicy()` method.
proxyHost parameter common java.lang.String *Camel 2.16*: To define a proxy host when instantiating the SQS client
proxyPort parameter common java.lang.Integer *Camel 2.16*: Specify a proxy port to be used inside the client definition.
secretKey parameter common java.lang.String Amazon AWS Secret Key
autocloseBody parameter consumer true boolean If this option is true and includeBody is true, then the S3Object.close() method will be called on exchange completion
bridgeErrorHandler parameter consumer boolean Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN/ERROR level and ignored.
deleteAfterRead parameter consumer true boolean Delete objects from S3 after they have been retrieved. The delete is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted.

If this option is false, then the same objects will be retrieve over and over again on the polls. Therefore you need to use the Idempotent Consumer EIP in the route to filter out duplicates. You can filter using the {@link S3Constants#BUCKET_NAME} and {@link S3Constants#KEY} headers, or only the {@link S3Constants#KEY} header.

fileName parameter consumer java.lang.String To get the object from the bucket with the given file name
includeBody parameter consumer true boolean *Camel 2.17*: If it is true, the exchange body will be set to a stream to the contents of the file. If false, the headers will be set with the S3 object metadata, but the body will be null.
maxMessagesPerPoll parameter consumer 10 int Gets the maximum number of messages as a limit to poll at each polling.

Is default unlimited, but use 0 or negative number to disable it as unlimited.

prefix parameter consumer java.lang.String *Camel 2.10.1*: The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.
sendEmptyMessageWhenIdle parameter consumer boolean If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead.
exceptionHandler parameter consumer (advanced) org.apache.camel.spi.ExceptionHandler To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this options is not in use. By default the consumer will deal with exceptions, that will be logged at WARN/ERROR level and ignored.
exchangePattern parameter consumer (advanced) org.apache.camel.ExchangePattern InOnly
RobustInOnly
InOut
InOptionalOut
OutOnly
RobustOutOnly
OutIn
OutOptionalIn
Sets the exchange pattern when the consumer creates an exchange.
pollStrategy parameter consumer (advanced) org.apache.camel.spi.PollingConsumerPollStrategy A pluggable org.apache.camel.PollingConsumerPollingStrategy allowing you to provide your custom implementation to control error handling usually occurred during the poll operation before an Exchange have been created and being routed in Camel.
deleteAfterWrite parameter producer boolean *Camel 2.11.0*: Delete file object after the S3 file has been uploaded
multiPartUpload parameter producer boolean *Camel 2.15.0*: If it is true, camel will upload the file with multi part format, the part size is decided by the option of `partSize`
operation parameter producer org.apache.camel.component.aws.s3.S3Operations copyObject
deleteBucket
listBuckets
*Camel 2.18*: The operation to do in case the user don't want to do only an upload
partSize parameter producer 26214400 long *Camel 2.15.0*: Setup the partSize which is used in multi part upload, the default size is 25M.
region parameter producer java.lang.String The region where the bucket is located. This option is used in the `com.amazonaws.services.s3.model.CreateBucketRequest`.
serverSideEncryption parameter producer java.lang.String *Camel 2.16*: Sets the server-side encryption algorithm when encrypting the object using AWS-managed keys. For example use AES256.
storageClass parameter producer java.lang.String *Camel 2.8.4*: The storage class to set in the `com.amazonaws.services.s3.model.PutObjectRequest` request.
synchronous parameter advanced false boolean Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported).
backoffErrorThreshold parameter scheduler int The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in.
backoffIdleThreshold parameter scheduler int The number of subsequent idle polls that should happen before the backoffMultipler should kick-in.
backoffMultiplier parameter scheduler int To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured.
delay parameter scheduler 500 long Milliseconds before the next poll. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).
greedy parameter scheduler boolean If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages.
initialDelay parameter scheduler 1000 long Milliseconds before the first poll starts. You can also specify time values using units, such as 60s (60 seconds), 5m30s (5 minutes and 30 seconds), and 1h (1 hour).
runLoggingLevel parameter scheduler TRACE org.apache.camel.LoggingLevel TRACE
DEBUG
INFO
WARN
ERROR
OFF
The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that.
scheduledExecutorService parameter scheduler java.util.concurrent.ScheduledExecutorService Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool.
scheduler parameter scheduler none org.apache.camel.spi.ScheduledPollConsumerScheduler none
spring
quartz2
To use a cron scheduler from either camel-spring or camel-quartz2 component
schedulerProperties parameter scheduler java.util.Map To configure additional properties when using a custom scheduler or any of the Quartz2, Spring based scheduler.
startScheduler parameter scheduler true boolean Whether the scheduler should be auto started.
timeUnit parameter scheduler MILLISECONDS java.util.concurrent.TimeUnit NANOSECONDS
MICROSECONDS
MILLISECONDS
SECONDS
MINUTES
HOURS
DAYS
Time unit for initialDelay and delay options.
useFixedDelay parameter scheduler true boolean Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details.

aws-s3 consumer

A Consumer of messages from the Amazon Web Service Simple Storage Service AWS S3