The spark component can be used to send RDD or DataFrame jobs to Apache Spark cluster.
| Name | Kind | Group | Required | Default | Type | Enum | Description |
|---|---|---|---|---|---|---|---|
| endpointType | path | producer | true | org.apache.camel.component.spark.EndpointType | rdd dataframe hive |
Type of the endpoint (rdd, dataframe, hive). | |
| bridgeErrorHandler | parameter | consumer | boolean | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN/ERROR level and ignored. | |||
| exceptionHandler | parameter | consumer (advanced) | org.apache.camel.spi.ExceptionHandler | To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this options is not in use. By default the consumer will deal with exceptions, that will be logged at WARN/ERROR level and ignored. | |||
| exchangePattern | parameter | consumer (advanced) | org.apache.camel.ExchangePattern | InOnly RobustInOnly InOut InOptionalOut OutOnly RobustOutOnly OutIn OutOptionalIn |
Sets the exchange pattern when the consumer creates an exchange. | ||
| collect | parameter | producer | true | boolean | Indicates if results should be collected or counted. | ||
| dataFrame | parameter | producer | org.apache.spark.sql.DataFrame | DataFrame to compute against. | |||
| dataFrameCallback | parameter | producer | org.apache.camel.component.spark.DataFrameCallback | Function performing action against an DataFrame. | |||
| rdd | parameter | producer | org.apache.spark.api.java.JavaRDDLike | RDD to compute against. | |||
| rddCallback | parameter | producer | org.apache.camel.component.spark.RddCallback | Function performing action against an RDD. | |||
| synchronous | parameter | advanced | false | boolean | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). |