Interface HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder

  • All Superinterfaces:
    org.apache.camel.builder.EndpointConsumerBuilder, org.apache.camel.EndpointConsumerResolver
    All Known Subinterfaces:
    HdfsEndpointBuilderFactory.HdfsEndpointBuilder
    Enclosing interface:
    HdfsEndpointBuilderFactory

    public static interface HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder
    extends org.apache.camel.builder.EndpointConsumerBuilder
    Builder for endpoint consumers for the HDFS component.
    • Method Detail

      • connectOnStartup

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder connectOnStartup​(boolean connectOnStartup)
        Whether to connect to the HDFS file system on starting the producer/consumer. If false then the connection is created on-demand. Notice that HDFS may take up till 15 minutes to establish a connection, as it has hardcoded 45 x 20 sec redelivery. By setting this option to false allows your application to startup, and not block for up till 15 minutes. The option is a: boolean type. Default: true Group: common
      • connectOnStartup

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder connectOnStartup​(String connectOnStartup)
        Whether to connect to the HDFS file system on starting the producer/consumer. If false then the connection is created on-demand. Notice that HDFS may take up till 15 minutes to establish a connection, as it has hardcoded 45 x 20 sec redelivery. By setting this option to false allows your application to startup, and not block for up till 15 minutes. The option will be converted to a boolean type. Default: true Group: common
      • fileSystemType

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder fileSystemType​(String fileSystemType)
        Set to LOCAL to not use HDFS but local java.io.File instead. The option will be converted to a org.apache.camel.component.hdfs.HdfsFileSystemType type. Default: HDFS Group: common
      • fileType

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder fileType​(String fileType)
        The file type to use. For more details see Hadoop HDFS documentation about the various files types. The option will be converted to a org.apache.camel.component.hdfs.HdfsFileType type. Default: NORMAL_FILE Group: common
      • bridgeErrorHandler

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder bridgeErrorHandler​(boolean bridgeErrorHandler)
        Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. The option is a: boolean type. Default: false Group: consumer
      • bridgeErrorHandler

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder bridgeErrorHandler​(String bridgeErrorHandler)
        Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. The option will be converted to a boolean type. Default: false Group: consumer
      • sendEmptyMessageWhenIdle

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder sendEmptyMessageWhenIdle​(boolean sendEmptyMessageWhenIdle)
        If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. The option is a: boolean type. Default: false Group: consumer
      • sendEmptyMessageWhenIdle

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder sendEmptyMessageWhenIdle​(String sendEmptyMessageWhenIdle)
        If the polling consumer did not poll any files, you can enable this option to send an empty message (no body) instead. The option will be converted to a boolean type. Default: false Group: consumer
      • streamDownload

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder streamDownload​(boolean streamDownload)
        Sets the download method to use when not using a local working directory. If set to true, the remote files are streamed to the route as they are read. When set to false, the remote files are loaded into memory before being sent into the route. The option is a: boolean type. Default: false Group: consumer
      • streamDownload

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder streamDownload​(String streamDownload)
        Sets the download method to use when not using a local working directory. If set to true, the remote files are streamed to the route as they are read. When set to false, the remote files are loaded into memory before being sent into the route. The option will be converted to a boolean type. Default: false Group: consumer
      • maxMessagesPerPoll

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder maxMessagesPerPoll​(int maxMessagesPerPoll)
        To define a maximum messages to gather per poll. By default a limit of 100 is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Values can only be greater than 0. Notice: If this option is in use then the limit will be applied on the valid files. For example if you have 100000 files and use maxMessagesPerPoll=500, then only the first 500 files will be picked up. The option is a: int type. Default: 100 Group: filter
      • maxMessagesPerPoll

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder maxMessagesPerPoll​(String maxMessagesPerPoll)
        To define a maximum messages to gather per poll. By default a limit of 100 is set. Can be used to set a limit of e.g. 1000 to avoid when starting up the server that there are thousands of files. Values can only be greater than 0. Notice: If this option is in use then the limit will be applied on the valid files. For example if you have 100000 files and use maxMessagesPerPoll=500, then only the first 500 files will be picked up. The option will be converted to a int type. Default: 100 Group: filter
      • backoffErrorThreshold

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffErrorThreshold​(int backoffErrorThreshold)
        The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. The option is a: int type. Group: scheduler
      • backoffErrorThreshold

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffErrorThreshold​(String backoffErrorThreshold)
        The number of subsequent error polls (failed due some error) that should happen before the backoffMultipler should kick-in. The option will be converted to a int type. Group: scheduler
      • backoffIdleThreshold

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffIdleThreshold​(int backoffIdleThreshold)
        The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. The option is a: int type. Group: scheduler
      • backoffIdleThreshold

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffIdleThreshold​(String backoffIdleThreshold)
        The number of subsequent idle polls that should happen before the backoffMultipler should kick-in. The option will be converted to a int type. Group: scheduler
      • backoffMultiplier

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffMultiplier​(int backoffMultiplier)
        To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. The option is a: int type. Group: scheduler
      • backoffMultiplier

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder backoffMultiplier​(String backoffMultiplier)
        To let the scheduled polling consumer backoff if there has been a number of subsequent idles/errors in a row. The multiplier is then the number of polls that will be skipped before the next actual attempt is happening again. When this option is in use then backoffIdleThreshold and/or backoffErrorThreshold must also be configured. The option will be converted to a int type. Group: scheduler
      • greedy

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder greedy​(boolean greedy)
        If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages. The option is a: boolean type. Default: false Group: scheduler
      • greedy

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder greedy​(String greedy)
        If greedy is enabled, then the ScheduledPollConsumer will run immediately again, if the previous run polled 1 or more messages. The option will be converted to a boolean type. Default: false Group: scheduler
      • repeatCount

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder repeatCount​(long repeatCount)
        Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever. The option is a: long type. Default: 0 Group: scheduler
      • repeatCount

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder repeatCount​(String repeatCount)
        Specifies a maximum limit of number of fires. So if you set it to 1, the scheduler will only fire once. If you set it to 5, it will only fire five times. A value of zero or negative means fire forever. The option will be converted to a long type. Default: 0 Group: scheduler
      • runLoggingLevel

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder runLoggingLevel​(org.apache.camel.LoggingLevel runLoggingLevel)
        The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. The option is a: org.apache.camel.LoggingLevel type. Default: TRACE Group: scheduler
      • runLoggingLevel

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder runLoggingLevel​(String runLoggingLevel)
        The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. The option will be converted to a org.apache.camel.LoggingLevel type. Default: TRACE Group: scheduler
      • scheduledExecutorService

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder scheduledExecutorService​(ScheduledExecutorService scheduledExecutorService)
        Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. The option is a: java.util.concurrent.ScheduledExecutorService type. Group: scheduler
      • scheduledExecutorService

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder scheduledExecutorService​(String scheduledExecutorService)
        Allows for configuring a custom/shared thread pool to use for the consumer. By default each consumer has its own single threaded thread pool. The option will be converted to a java.util.concurrent.ScheduledExecutorService type. Group: scheduler
      • scheduler

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder scheduler​(Object scheduler)
        To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler. The option is a: java.lang.Object type. Default: none Group: scheduler
      • scheduler

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder scheduler​(String scheduler)
        To use a cron scheduler from either camel-spring or camel-quartz component. Use value spring or quartz for built in scheduler. The option will be converted to a java.lang.Object type. Default: none Group: scheduler
      • schedulerProperties

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder schedulerProperties​(String key,
                                                                                           Object value)
        To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler. The option is a: java.util.Map<java.lang.String, java.lang.Object> type. The option is multivalued, and you can use the schedulerProperties(String, Object) method to add a value (call the method multiple times to set more values). Group: scheduler
      • schedulerProperties

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder schedulerProperties​(Map values)
        To configure additional properties when using a custom scheduler or any of the Quartz, Spring based scheduler. The option is a: java.util.Map<java.lang.String, java.lang.Object> type. The option is multivalued, and you can use the schedulerProperties(String, Object) method to add a value (call the method multiple times to set more values). Group: scheduler
      • useFixedDelay

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder useFixedDelay​(boolean useFixedDelay)
        Controls if fixed delay or fixed rate is used. See ScheduledExecutorService in JDK for details. The option is a: boolean type. Default: true Group: scheduler
      • kerberosConfigFileLocation

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder kerberosConfigFileLocation​(String kerberosConfigFileLocation)
        The location of the kerb5.conf file (https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html). The option is a: java.lang.String type. Group: security
      • kerberosKeytabLocation

        default HdfsEndpointBuilderFactory.HdfsEndpointConsumerBuilder kerberosKeytabLocation​(String kerberosKeytabLocation)
        The location of the keytab file used to authenticate with the kerberos nodes (contains pairs of kerberos principals and encrypted keys (which are derived from the Kerberos password)). The option is a: java.lang.String type. Group: security