The Enterprise Integration Patterns (EIP) book

The purpose of a "patterns" book is not to advocate new techniques that the authors have invented, but rather to document existing best practices within a particular field. By doing this, the authors of a patterns book hope to spread knowledge of best practices and promote a vocabulary for discussing architectural designs.
One of the most famous patterns books is Design Patterns: Elements of Reusable Object-oriented Software by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides, commonly known as the "Gang of Four" (GoF) book. Since the publication of Design Patterns, many other pattern books, of varying quality, have been written. One famous patterns book is called Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe and Bobby Woolf. It is common for people to refer to this book by its initials EIP. As the subtitle of EIP suggests, the book focuses on design patterns for asynchronous messaging systems. The book discusses 65 patterns. Each pattern is given a textual name and most are also given a graphical symbol, intended to be used in architectural diagrams.

The Camel project

Camel (http://camel.apache.org) is an open-source, Java-based project that helps the user implement many of the design patterns in the EIP book. Because Camel implements many of the design patterns in the EIP book, it would be a good idea for people who work with Camel to have the EIP book as a reference.

Online documentation for Camel

The documentation is all under the Documentation category on the right-side menu of the Camel website (also available in PDF form. Camel-related books are also available, in particular the Camel in Action book, presently serving as the Camel bible--it has a free Chapter One (pdf), which is highly recommended to read to get more familiar with Camel.

A useful tip for navigating the online documentation

The breadcrumbs at the top of the online Camel documentation can help you navigate between parent and child subsections.
For example, If you are on the "Languages" documentation page then the left-hand side of the reddish bar contains the following links.

As you might expect, clicking on "Apache Camel" takes you back to the home page of the Apache Camel project, and clicking on "Documentation" takes you to the main documentation page. You can interpret the "Architecture" and "Languages" buttons as indicating you are in the "Languages" section of the "Architecture" chapter. Adding browser bookmarks to pages that you frequently reference can also save time.

Online Javadoc documentation

The Apache Camel website provides Javadoc documentation. It is important to note that the Javadoc documentation is spread over several independent Javadoc hierarchies rather than being all contained in a single Javadoc hierarchy. In particular, there is one Javadoc hierarchy for the core APIs of Camel, and a separate Javadoc hierarchy for each component technology supported by Camel. For example, if you will be using Camel with ActiveMQ and FTP then you need to look at the Javadoc hierarchies for the core API and Spring API.

Concepts and terminology fundamental to Camel

In this section some of the concepts and terminology that are fundamental to Camel are explained. This section is not meant as a complete Camel tutorial, but as a first step in that direction.

Endpoint

The term endpoint is often used when talking about inter-process communication. For example, in client-server communication, the client is one endpoint and the server is the other endpoint. Depending on the context, an endpoint might refer to an address, such as a host:port pair for TCP-based communication, or it might refer to a software entity that is contactable at that address. For example, if somebody uses "www.example.com:80" as an example of an endpoint, they might be referring to the actual port at that host name (that is, an address), or they might be referring to the web server (that is, software contactable at that address). Often, the distinction between the address and software contactable at that address is not an important one.
Some middleware technologies make it possible for several software entities to be contactable at the same physical address. For example, CORBA is an object-oriented, remote-procedure-call (RPC) middleware standard. If a CORBA server process contains several objects then a client can communicate with any of these objects at the same physical address (host:port), but a client communicates with a particular object via that object's logical address (called an IOR in CORBA terminology), which consists of the physical address (host:port) plus an id that uniquely identifies the object within its server process. (An IOR contains some additional information that is not relevant to this present discussion.) When talking about CORBA, some people may use the term "endpoint" to refer to a CORBA server's physical address, while other people may use the term to refer to the logical address of a single CORBA object, and other people still might use the term to refer to any of the following:

• The physical address (host:port) of the CORBA server process
• The logical address (host:port plus id) of a CORBA object.
• The CORBA server process (a relatively heavyweight software entity)
• A CORBA object (a lightweight software entity)

Because of this, you can see that the term endpoint is ambiguous in at least two ways. First, it is ambiguous because it might refer to an address or to a software entity contactable at that address. Second, it is ambiguous in the granularity of what it refers to: a heavyweight versus lightweight software entity, or physical address versus logical address. It is useful to understand that different people use the term endpoint in slightly different (and hence ambiguous) ways because Camel's usage of this term might be different to whatever meaning you had previously associated with the term.
Camel provides out-of-the-box support for endpoints implemented with many different communication technologies. Here are some examples of the Camel-supported endpoint technologies.

• A JMS queue.
• A web service.
• A file. A file may sound like an unlikely type of endpoint, until you realize that in some systems one application might write information to a file and, later, another application might read that file.
• An FTP server.
• An email address. A client can send a message to an email address, and a server can read an incoming message from a mail server.
• A POJO (plain old Java object).

In a Camel-based application, you create (Camel wrappers around) some endpoints and connect these endpoints with routes, which I will discuss later in Section 4.8 ("Routes, RouteBuilders and Java DSL"). Camel defines a Java interface called Endpoint. Each Camel-supported endpoint has a class that implements this Endpoint interface. As I discussed in Section 3.3 ("Online Javadoc documentation"), Camel provides a separate Javadoc hierarchy for each communications technology supported by Camel. Because of this, you will find documentation on, say, the JmsEndpoint class in the JMS Javadoc hierarchy, while documentation for, say, the FtpEndpoint class is in the FTP Javadoc hierarchy.

CamelContext

A CamelContext object represents the Camel runtime system. You typically have one CamelContext object in an application. A typical application executes the following steps.

1. Create a CamelContext object.
2. Add endpoints – and possibly Components, which are discussed in Section 4.5 ("Components") – to the CamelContext object.
3. Add routes to the CamelContext object to connect the endpoints.
4. Invoke the start() operation on the CamelContext object. This starts Camel-internal threads that are used to process the sending, receiving and processing of messages in the endpoints.
5. Eventually invoke the stop() operation on the CamelContext object. Doing this gracefully stops all the endpoints and Camel-internal threads.

Note that the CamelContext.start() operation does not block indefinitely. Rather, it starts threads internal to each Component and Endpoint and then start() returns. Conversely, CamelContext.stop() waits for all the threads internal to each Endpoint and Component to terminate and then stop() returns.
If you neglect to call CamelContext.start() in your application then messages will not be processed because internal threads will not have been created.
If you neglect to call CamelContext.stop() before terminating your application then the application may terminate in an inconsistent state. If you neglect to call CamelContext.stop() in a JUnit test then the test may fail due to messages not having had a chance to be fully processed.

CamelTemplate

Camel used to have a class called CamelClient, but this was renamed to be CamelTemplate to be similar to a naming convention used in some other open-source projects, such as the TransactionTemplate and JmsTemplate classes in Spring.
The CamelTemplate class is a thin wrapper around the CamelContext class. It has methods that send a Message or Exchange – both discussed in Section 4.6 ("Message and Exchange")) – to an Endpoint – discussed in Section 4.1 ("Endpoint"). This provides a way to enter messages into source endpoints, so that the messages will move along routes – discussed in Section 4.8 ("Routes, RouteBuilders and Java DSL") – to destination endpoints.

The Meaning of URL, URI, URN and IRI

Some Camel methods take a parameter that is a URI string. Many people know that a URI is "something like a URL" but do not properly understand the relationship between URI and URL, or indeed its relationship with other acronyms such as IRI and URN.
Most people are familiar with URLs (uniform resource locators), such as "http://...", "ftp://...", "mailto:...". Put simply, a URL specifies the location of a resource.
A URI (uniform resource identifier) is a URL or a URN. So, to fully understand what URI means, you need to first understand what is a URN.
URN is an acronym for uniform resource name. There are may "unique identifier" schemes in the world, for example, ISBNs (globally unique for books), social security numbers (unique within a country), customer numbers (unique within a company's customers database) and telephone numbers. Each "unique identifier" scheme has its own notation. A URN is a wrapper for different "unique identifier" schemes. The syntax of a URN is "urn:<scheme-name>:<unique-identifier>". A URN uniquely identifies a resource, such as a book, person or piece of equipment. By itself, a URN does not specify the location of the resource. Instead, it is assumed that a registry provides a mapping from a resource's URN to its location. The URN specification does not state what form a registry takes, but it might be a database, a server application, a wall chart or anything else that is convenient. Some hypothetical examples of URNs are "urn:employee:08765245", "urn:customer:uk:3458:hul8" and "urn:foo:0000-0000-9E59-0000-5E-2". The <scheme-name> ("employee", "customer" and "foo" in these examples) part of a URN implicitly defines how to parse and interpret the <unique-identifier> that follows it. An arbitrary URN is meaningless unless: (1) you know the semantics implied by the <scheme-name>, and (2) you have access to the registry appropriate for the <scheme-name>. A registry does not have to be public or globally accessible. For example, "urn:employee:08765245" might be meaningful only within a specific company.
To date, URNs are not (yet) as popular as URLs. For this reason, URI is widely misused as a synonym for URL.
IRI is an acronym for internationalized resource identifier. An IRI is simply an internationalized version of a URI. In particular, a URI can contain letters and digits in the US-ASCII character set, while a IRI can contain those same letters and digits, and also European accented characters, Greek letters, Chinese ideograms and so on.

Components

Component is confusing terminology; EndpointFactory would have been more appropriate because a Component is a factory for creating Endpoint instances. For example, if a Camel-based application uses several JMS queues then the application will create one instance of the JmsComponent class (which implements the Component interface), and then the application invokes the createEndpoint() operation on this JmsComponent object several times. Each invocation of JmsComponent.createEndpoint() creates an instance of the JmsEndpoint class (which implements the Endpoint interface). Actually, application-level code does not invoke Component.createEndpoint() directly. Instead, application-level code normally invokes CamelContext.getEndpoint(); internally, the CamelContext object finds the desired Component object (as I will discuss shortly) and then invokes createEndpoint() on it.
Consider the following code.

The parameter to getEndpoint() is a URI. The URI prefix (that is, the part before ":") specifies the name of a component. Internally, the CamelContext object maintains a mapping from names of components to Component objects. For the URI given in the above example, the CamelContext object would probably map the pop3 prefix to an instance of the MailComponent class. Then the CamelContext object invokes createEndpoint("pop3://john.smith@mailserv.example.com?password=myPassword") on that MailComponent object. The createEndpoint() operation splits the URI into its component parts and uses these parts to create and configure an Endpoint object.
In the previous paragraph, I mentioned that a CamelContext object maintains a mapping from component names to Component objects. This raises the question of how this map is populated with named Component objects. There are two ways of populating the map. The first way is for application-level code to invoke CamelContext.addComponent(String componentName, Component component). The example below shows a single MailComponent object being registered in the map under 3 different names.

The second (and preferred) way to populate the map of named Component objects in the CamelContext object is to let the CamelContext object perform lazy initialization. This approach relies on developers following a convention when they write a class that implements the Component interface. I illustrate the convention by an example. Let's assume you write a class called com.example.myproject.FooComponent and you want Camel to automatically recognize this by the name "foo". To do this, you have to write a properties file called "META-INF/services/org/apache/camel/component/foo" (without a ".properties" file extension) that has a single entry in it called class, the value of which is the fully-scoped name of your class. This is shown below.

If you want Camel to also recognize the class by the name "bar" then you write another properties file in the same directory called "bar" that has the same contents. Once you have written the properties file(s), you create a jar file that contains the com.example.myproject.FooComponent class and the properties file(s), and you add this jar file to your CLASSPATH. Then, when application-level code invokes createEndpoint("foo:...") on a CamelContext object, Camel will find the "foo"" properties file on the CLASSPATH, get the value of the class property from that properties file, and use reflection APIs to create an instance of the specified class.
As I said in Section 4.1 ("Endpoint"), Camel provides out-of-the-box support for numerous communication technologies. The out-of-the-box support consists of classes that implement the Component interface plus properties files that enable a CamelContext object to populate its map of named Component objects.
Earlier in this section I gave the following example of calling CamelContext.getEndpoint().

When I originally gave that example, I said that the parameter to getEndpoint() was a URI. I said that because the online Camel documentation and the Camel source code both claim the parameter is a URI. In reality, the parameter is restricted to being a URL. This is because when Camel extracts the component name from the parameter, it looks for the first ":", which is a simplistic algorithm. To understand why, recall from Section 4.4 ("The Meaning of URL, URI, URN and IRI") that a URI can be a URL or a URN. Now consider the following calls to getEndpoint.

Camel identifies the components in the above example as "pop3", "jms", "urn" and "urn". It would be more useful if the latter components were identified as "urn:foo" and "urn:bar" or, alternatively, as "foo" and "bar" (that is, by skipping over the "urn:" prefix). So, in practice you must identify an endpoint with a URL (a string of the form "<scheme>:...") rather than with a URN (a string of the form "urn:<scheme>:..."). This lack of proper support for URNs means the you should consider the parameter to getEndpoint() as being a URL rather than (as claimed) a URI.

Message and Exchange

The Message interface provides an abstraction for a single message, such as a request, reply or exception message.
There are concrete classes that implement the Message interface for each Camel-supported communications technology. For example, the JmsMessage class provides a JMS-specific implementation of the Message interface. The public API of the Message interface provides get- and set-style methods to access the message id, body and individual header fields of a messge.
The Exchange interface provides an abstraction for an exchange of messages, that is, a request message and its corresponding reply or exception message. In Camel terminology, the request, reply and exception messages are called in, out and fault messages.
There are concrete classes that implement the Exchange interface for each Camel-supported communications technology. For example, the JmsExchange class provides a JMS-specific implementation of the Exchange interface. The public API of the Exchange interface is quite limited. This is intentional, and it is expected that each class that implements this interface will provide its own technology-specific operations.
Application-level programmers rarely access the Exchange interface (or classes that implement it) directly. However, many classes in Camel are generic types that are instantiated on (a class that implements) Exchange. Because of this, the Exchange interface appears a lot in the generic signatures of classes and methods.

Processor

The Processor interface represents a class that processes a message. The signature of this interface is shown below.

Notice that the parameter to the process() method is an Exchange rather than a Message. This provides flexibility. For example, an implementation of this method initially might call exchange.getIn() to get the input message and process it. If an error occurs during processing then the method can call exchange.setException().
An application-level developer might implement the Processor interface with a class that executes some business logic. However, there are many classes in the Camel library that implement the Processor interface in a way that provides support for a design pattern in the EIP book. For example, ChoiceProcessor implements the message router pattern, that is, it uses a cascading if-then-else statement to route a message from an input queue to one of several output queues. Another example is the FilterProcessor class which discards messages that do not satisfy a stated predicate (that is, condition).

Routes, RouteBuilders and Java DSL

A route is the step-by-step movement of a Message from an input queue, through arbitrary types of decision making (such as filters and routers) to a destination queue (if any). Camel provides two ways for an application developer to specify routes. One way is to specify route information in an XML file. A discussion of that approach is outside the scope of this document. The other way is through what Camel calls a Java DSL (domain-specific language).

Introduction to Java DSL

For many people, the term "domain-specific language" implies a compiler or interpreter that can process an input file containing keywords and syntax specific to a particular domain. This is not the approach taken by Camel. Camel documentation consistently uses the term "Java DSL" instead of "DSL", but this does not entirely avoid potential confusion. The Camel "Java DSL" is a class library that can be used in a way that looks almost like a DSL, except that it has a bit of Java syntactic baggage. You can see this in the example below. Comments afterwards explain some of the constructs used in the example.

The first line in the above example creates an object which is an instance of an anonymous subclass of RouteBuilder with the specified configure() method.
The CamelContext.addRoutes(RouterBuilder builder) method invokes builder.setContext(this) – so the RouteBuilder object knows which CamelContext object it is associated with – and then invokes builder.configure(). The body of configure() invokes methods such as from(), filter(), choice(), when(), isEqualTo(), otherwise() and to().
The RouteBuilder.from(String uri) method invokes getEndpoint(uri) on the CamelContext associated with the RouteBuilder object to get the specified Endpoint and then puts a FromBuilder "wrapper" around this Endpoint. The FromBuilder.filter(Predicate predicate) method creates a FilterProcessor object for the Predicate (that is, condition) object built from the header("foo").isEqualTo("bar") expression. In this way, these operations incrementally build up a Route object (with a RouteBuilder wrapper around it) and add it to the CamelContext object associated with the RouteBuilder.

Critique of Java DSL

The online Camel documentation compares Java DSL favourably against the alternative of configuring routes and endpoints in a XML-based Spring configuration file. In particular, Java DSL is less verbose than its XML counterpart. In addition, many integrated development environments (IDEs) provide an auto-completion feature in their editors. This auto-completion feature works with Java DSL, thereby making it easier for developers to write Java DSL.
However, there is another option that the Camel documentation neglects to consider: that of writing a parser that can process DSL stored in, say, an external file. Currently, Camel does not provide such a DSL parser, and I do not know if it is on the "to do" list of the Camel maintainers. I think that a DSL parser would offer a significant benefit over the current Java DSL. In particular, the DSL would have a syntactic definition that could be expressed in a relatively short BNF form. The effort required by a Camel user to learn how to use DSL by reading this BNF would almost certainly be significantly less than the effort currently required to study the API of the RouterBuilder classes.

Continue Learning about Camel

Return to the main Getting Started page for additional introductory reference information.

CookBook

This document describes various recipes for working with Camel

• Bean Integration describes how to work with beans and Camel in a loosely coupled way so that your beans do not have to depend on any Camel APIs
  • Annotation Based Expression Language binds expressions to method parameters
  • Bean Binding defines which methods are invoked and how the Message is converted into the parameters of the method when it is invoked
  • Bean Injection for injecting Camel related resources into your POJOs
  • Parameter Binding Annotations for extracting various headers, properties or payloads from a Message
  • POJO Consuming for consuming and possibly routing messages from Camel
  • POJO Producing for producing camel messages from your POJOs
  • RecipientList Annotation for creating a Recipient List from a POJO method
  • Using Exchange Pattern Annotations describes how pattern annotations can be used to change the behaviour of method invocations
• Hiding Middleware describes how to avoid your business logic being coupled to any particular middleware APIs allowing you to easily switch from in JVM SEDA to JMS, ActiveMQ, Hibernate, JPA, JDBC, iBatis or JavaSpace etc.
• Visualisation describes how to visualise your Enterprise Integration Patterns to help you understand your routing rules
• Business Activity Monitoring (BAM) for monitoring business processes across systems
• Extract Transform Load (ETL) to load data into systems or databases
• Testing for testing distributed and asynchronous systems using a messaging approach
  • Camel Test for creating test cases using a single Java class for all your configuration and routing
  • Spring Testing uses Spring Test together with either XML or Java Config to dependency inject your test classes
  • Guice uses Guice to dependency inject your test classes
• Templating is a great way to create service stubs to be able to test your system without some back end system.
• Database for working with databases
• Parallel Processing and Ordering on how using parallel processing and SEDA or JMS based load balancing can be achieved.
• Asynchronous Processing in Camel Routes.
• Implementing Virtual Topics on other JMS providers shows how to get the effect of Virtual Topics and avoid issues with JMS durable topics
• Camel Transport for CXF describes how to put the Camel context into the CXF transport layer.
• Fine Grained Control Over a Channel describes how to deliver a sequence of messages over a single channel and then stopping any more messages being sent over that channel. Typically used for sending data over a socket and then closing the socket.
• EventNotifier to log details about all sent Exchanges shows how to let Camels EventNotifier log all sent to endpoint events and how long time it took.
• Loading routes from XML files into an existing CamelContext.
• Using MDC logging with Camel
• Running Camel standalone and have it keep running shows how to keep Camel running when you run it standalone.
• Hazelcast Idempotent Repository Tutorial shows how to avoid to consume duplicated messages in a clustered environment.
• How to use Camel as a HTTP proxy between a client and server shows how to use Camel as a HTTP adapter/proxy between a client and HTTP service.

Bean Integration

Camel supports the integration of beans and POJOs in a number of ways

Annotations

If a bean is defined in Spring XML or scanned using the Spring component scanning mechanism and a <camelContext> is used or a CamelBeanPostProcessor then we process a number of Camel annotations to do various things such as injecting resources or producing, consuming or routing messages.

The following annotations is supported and inject by Camel's CamelBeanPostProcessor

See more details at:

• POJO Consuming to consume and possibly route messages from Camel
• POJO Producing to make it easy to produce camel messages from your POJOs
• DynamicRouter Annotation for creating a Dynamic Router from a POJO method
• RecipientList Annotation for creating a Recipient List from a POJO method
• RoutingSlip Annotation for creating a Routing Slip for a POJO method
• Bean Injection to inject Camel related resources into your POJOs
• Using Exchange Pattern Annotations describes how the pattern annotations can be used to change the behaviour of method invocations with Spring Remoting or POJO Producing

Bean Component

The Bean component allows one to invoke a particular method. Alternately the Bean component supports the creation of a proxy via ProxyHelper to a Java interface; which the implementation just sends a message containing a BeanInvocation to some Camel endpoint.

Spring Remoting

We support a Spring Remoting provider which uses Camel as the underlying transport mechanism. The nice thing about this approach is we can use any of the Camel transport Components to communicate between beans. It also means we can use Content Based Router and the other Enterprise Integration Patterns in between the beans; in particular we can use Message Translator to be able to convert what the on-the-wire messages look like in addition to adding various headers and so forth.

Annotation Based Expression Language

You can also use any of the Languages supported in Camel to bind expressions to method parameters when using Bean Integration. For example you can use any of these annotations:

Example:

Advanced example using @Bean

And an example of using the the @Bean binding annotation, where you can use a POJO where you can do whatever java code you like:

And then we can have a spring bean with the id myCorrelationIdGenerator where we can compute the id.

The POJO MyIdGenerator has one public method that accepts two parameters. However we have also annotated this one with the @Header and @Body annotation to help Camel know what to bind here from the Message from the Exchange being processed.

Of course this could be simplified a lot if you for instance just have a simple id generator. But we wanted to demonstrate that you can use the Bean Binding annotations anywhere.

And finally we just need to remember to have our bean registered in the Spring Registry:

Example using Groovy

In this example we have an Exchange that has a User object stored in the in header. This User object has methods to get some user information. We want to use Groovy to inject an expression that extracts and concats the fullname of the user into the fullName parameter.

Groovy supports GStrings that is like a template where we can insert $ placeholders that will be evaluated by Groovy.

Bean Binding

Bean Binding in Camel defines both which methods are invoked and also how the Message is converted into the parameters of the method when it is invoked.

Choosing the method to invoke

The binding of a Camel Message to a bean method call can occur in different ways, in the following order of importance:

• if the message contains the header CamelBeanMethodName then that method is invoked, converting the body to the type of the method's argument.
  • From Camel 2.8 onwards you can qualify parameter types to select exactly which method to use among overloads with the same name (see below for more details).
  • From Camel 2.9 onwards you can specify parameter values directly in the method option (see below for more details).
• you can explicitly specify the method name in the DSL or when using POJO Consuming or POJO Producing
• if the bean has a method marked with the @Handler annotation, then that method is selected
• if the bean can be converted to a Processor using the Type Converter mechanism, then this is used to process the message. The ActiveMQ component uses this mechanism to allow any JMS MessageListener to be invoked directly by Camel without having to write any integration glue code. You can use the same mechanism to integrate Camel into any other messaging/remoting frameworks.
• if the body of the message can be converted to a BeanInvocation (the default payload used by the ProxyHelper) component - then that is used to invoke the method and pass its arguments
• otherwise the type of the body is used to find a matching method; an error is thrown if a single method cannot be chosen unambiguously.
• you can also use Exchange as the parameter itself, but then the return type must be void.
• if the bean class is private (or package-private), interface methods will be preferred (from Camel 2.9 onwards) since Camel can't invoke class methods on such beans

In cases where Camel cannot choose a method to invoke, an AmbiguousMethodCallException is thrown.

By default the return value is set on the outbound message body. 

Asynchronous processing

From Camel 2.18 onwards you can return a CompletionStage implementation (e.g. a CompletableFuture) to implement asynchronous processing.

Please be sure to properly complete the CompletionStage with the result or exception, including any timeout handling. Exchange processing would wait for completion and would not impose any timeouts automatically. It's extremely useful to monitor Inflight repository for any hanging messages.

Note that completing with "null" won't set outbody message body to null, but would keep message intact. This is useful to support methods that don't modify exchange and return CompletableFuture<Void>. To set body to null, just add Exchange method parameter and directly modify exchange messages.

Examples:

Simple asynchronous processor, modifying message body.

Composite processor that do not modify exchange

Parameter binding

When a method has been chosen for invocation, Camel will bind to the parameters of the method.

The following Camel-specific types are automatically bound:

• org.apache.camel.Exchange
• org.apache.camel.Message
• org.apache.camel.CamelContext
• org.apache.camel.TypeConverter
• org.apache.camel.spi.Registry
• java.lang.Exception

So, if you declare any of these types, they will be provided by Camel. Note that Exception will bind to the caught exception of the Exchange - so it's often usable if you employ a Pojo to handle, e.g., an onException route.

What is most interesting is that Camel will also try to bind the body of the Exchange to the first parameter of the method signature (albeit not of any of the types above). So if, for instance, we declare a parameter as String body, then Camel will bind the IN body to this type. Camel will also automatically convert to the type declared in the method signature.

Let's review some examples:

Below is a simple method with a body binding. Camel will bind the IN body to the body parameter and convert it to a String.

In the following sample we got one of the automatically-bound types as well - for instance, a Registry that we can use to lookup beans.

We can use Exchange as well:

You can also have multiple types:

And imagine you use a Pojo to handle a given custom exception InvalidOrderException - we can then bind that as well:

Notice that we can bind to it even if we use a sub type of java.lang.Exception as Camel still knows it's an exception and can bind the cause (if any exists).

So what about headers and other stuff? Well now it gets a bit tricky - so we can use annotations to help us, or specify the binding in the method name option.
See the following sections for more detail.

Binding Annotations

You can use the Parameter Binding Annotations to customize how parameter values are created from the Message

Examples

For example, a Bean such as:

Or the Exchange example. Notice that the return type must be void when there is only a single parameter of the type org.apache.camel.Exchange:

@Handler

You can mark a method in your bean with the @Handler annotation to indicate that this method should be used for Bean Binding.
This has an advantage as you need not specify a method name in the Camel route, and therefore do not run into problems after renaming the method in an IDE that can't find all its references.

Parameter binding using method option

Available as of Camel 2.9

Camel uses the following rules to determine if it's a parameter value in the method option

• The value is either true or false which denotes a boolean value
• The value is a numeric value such as 123 or 7
• The value is a String enclosed with either single or double quotes
• The value is null which denotes a null value
• It can be evaluated using the Simple language, which means you can use, e.g., body, header.foo and other Simple tokens. Notice the tokens must be enclosed with ${ }.

Any other value is consider to be a type declaration instead - see the next section about specifying types for overloaded methods.

When invoking a Bean you can instruct Camel to invoke a specific method by providing the method name:

Here we tell Camel to invoke the doSomething method - Camel handles the parameters' binding. Now suppose the method has 2 parameters, and the 2nd parameter is a boolean where we want to pass in a true value:

This is now possible in Camel 2.9 onwards:

In the example above, we defined the first parameter using the wild card symbol *, which tells Camel to bind this parameter to any type, and let Camel figure this out. The 2nd parameter has a fixed value of true. Instead of the wildcard symbol we can instruct Camel to use the message body as shown:

The syntax of the parameters is using the Simple expression language so we have to use ${ } placeholders in the body to refer to the message body.

If you want to pass in a null value, then you can explicit define this in the method option as shown below:

Specifying null as a parameter value instructs Camel to force passing a null value.

Besides the message body, you can pass in the message headers as a java.util.Map:

You can also pass in other fixed values besides booleans. For example, you can pass in a String and an integer:

In the example above, we invoke the echo method with two parameters. The first has the content 'World' (without quotes), and the 2nd has the value of 5.
Camel will automatically convert these values to the parameters' types.

Having the power of the Simple language allows us to bind to message headers and other values such as:

You can also use the OGNL support of the Simple expression language. Now suppose the message body is an object which has a method named asXml. To invoke the asXml method we can do as follows:

Instead of using .bean as shown in the examples above, you may want to use .to instead as shown:

Using type qualifiers to select among overloaded methods

Available as of Camel 2.8

If you have a Bean with overloaded methods, you can now specify parameter types in the method name so Camel can match the method you intend to use.
Given the following bean:

Then the MyBean has 2 overloaded methods with the names hello and times. So if we want to use the method which has 2 parameters we can do as follows in the Camel route:We can also use a * as wildcard so we can just say we want to execute the method with 2 parameters we doBy default Camel will match the type name using the simple name, e.g. any leading package name will be disregarded. However if you want to match using the FQN, then specify the FQN type and Camel will leverage that. So if you have a com.foo.MyOrder and you want to match against the FQN, and not the simple name "MyOrder", then follow this example:

Bean Injection

We support the injection of various resources using @EndpointInject or @BeanInject. This can be used to inject

• Endpoint instances which can be used for testing when used with Mock endpoints; see the Spring Testing for an example.
• ProducerTemplate instances for POJO Producing
• client side proxies for POJO Producing which is a simple approach to Spring Remoting

Using @BeanInject

From Camel 2.13 onwards you can inject beans (obtained from the Registry) into your beans such as RouteBuilder classes.

For example to inject a bean named foo, you can enlist the bean in the Registry such as in a Spring XML file:

And then in a Java RouteBuilder class, you can inject the bean using @BeanInject as shown below:

If you omit the name, then Camel does a lookup by type, and injects the bean if there is exactly only one bean of that type enlisted in the Registry.

Parameter Binding Annotations

Annotations can be used to define an Expression or to extract various headers, properties or payloads from a Message when invoking a bean method (see Bean Integration for more detail of how to invoke bean methods) together with being useful to help disambiguate which method to invoke.

If no annotations are used then Camel assumes that a single parameter is the body of the message. Camel will then use the Type Converter mechanism to convert from the expression value to the actual type of the parameter.

The core annotations are as follows

The follow annotations @Headers, @OutHeaders and @Properties binds to the backing java.util.Map so you can alter the content of these maps directly, for instance using the put method to add a new entry. See the OrderService class at Exception Clause for such an example. You can use @Header("myHeader") and @Property("myProperty") to access the backing java.util.Map.

Example

In this example below we have a @Consume consumer (like message driven) that consumes JMS messages from the activemq queue. We use the @Header and @Body parameter binding annotations to bind from the JMSMessage to the method parameters.

In the above Camel will extract the value of Message.getJMSCorrelationID(), then using the Type Converter to adapt the value to the type of the parameter if required - it will inject the parameter value for the correlationID parameter. Then the payload of the message will be converted to a String and injected into the body parameter.

You don't necessarily need to use the @Consume annotation if you don't want to as you could also make use of the Camel DSL to route to the bean's method as well.

Using the DSL to invoke the bean method

Here is another example which does not use POJO Consuming annotations but instead uses the DSL to route messages to the bean method

The routing DSL then looks like this

Here myBean would be looked up in the Registry (such as JNDI or the Spring ApplicationContext), then the body of the message would be used to try figure out what method to call.

If you want to be explicit you can use

And here we have a nifty example for you to show some great power in Camel. You can mix and match the annotations with the normal parameters, so we can have this example with annotations and the Exchange also:

Annotation Based Expression Language

You can also use any of the Languages supported in Camel to bind expressions to method parameters when using Bean Integration. For example you can use any of these annotations:

Example:

Advanced example using @Bean

And an example of using the the @Bean binding annotation, where you can use a POJO where you can do whatever java code you like:

And then we can have a spring bean with the id myCorrelationIdGenerator where we can compute the id.

The POJO MyIdGenerator has one public method that accepts two parameters. However we have also annotated this one with the @Header and @Body annotation to help Camel know what to bind here from the Message from the Exchange being processed.

Of course this could be simplified a lot if you for instance just have a simple id generator. But we wanted to demonstrate that you can use the Bean Binding annotations anywhere.

And finally we just need to remember to have our bean registered in the Spring Registry:

Example using Groovy

In this example we have an Exchange that has a User object stored in the in header. This User object has methods to get some user information. We want to use Groovy to inject an expression that extracts and concats the fullname of the user into the fullName parameter.

Groovy supports GStrings that is like a template where we can insert $ placeholders that will be evaluated by Groovy.

@Consume

To consume a message you use the @Consume annotation to mark a particular method of a bean as being a consumer method. The uri of the annotation defines the Camel Endpoint to consume from.

e.g. lets invoke the onCheese() method with the String body of the inbound JMS message from ActiveMQ on the cheese queue; this will use the Type Converter to convert the JMS ObjectMessage or BytesMessage to a String - or just use a TextMessage from JMS

The Bean Binding is then used to convert the inbound Message to the parameter list used to invoke the method .

What this does is basically create a route that looks kinda like this

Using context option to apply only a certain CamelContext

See the warning above.

You can use the context option to specify which CamelContext the consumer should only apply for. For example:

The consumer above will only be created for the CamelContext that have the context id = camel-1. You set this id in the XML tag:

Using an explicit route

If you want to invoke a bean method from many different endpoints or within different complex routes in different circumstances you can just use the normal routing DSL or the Spring XML configuration file.

For example

which will then look up in the Registry and find the bean and invoke the given bean name. (You can omit the method name and have Camel figure out the right method based on the method annotations and body type).

Use the Bean endpoint

You can always use the bean endpoint

Using a property to define the endpoint

Available as of Camel 2.11

The following annotations @Consume, @Produce, @EndpointInject, now offers a property attribute you can use to define the endpoint as a property on the bean. Then Camel will use the getter method to access the property.

For example

The bean MyService has a property named serviceEndpoint which has getter/setter for the property. Now we want to use the bean for POJO Consuming, and hence why we use @Consume in the onService method. Notice how we use the property = "serviceEndpoint to configure the property that has the endpoint url.

If you define the bean in Spring XML or Blueprint, then you can configure the property as follows:

This allows you to configure the bean using any standard IoC style.

Camel offers a naming convention which allows you to not have to explicit name the property.
Camel uses this algorithm to find the getter method. The method must be a getXXX method.

1. Use the property name if explicit given
2. If no property name was configured, then use the method name
3. Try to get the property with name*Endpoint* (eg with Endpoint as postfix)
4. Try to get the property with the name as is (eg no postfix or postfix)
5. If the property name starts with on then omit that, and try step 3 and 4 again.

So in the example above, we could have defined the @Consume annotation as

Now the property is named 'service' which then would match step 3 from the algorithm, and have Camel invoke the getServiceEndpoint method.

We could also have omitted the property attribute, to make it implicit

Now Camel matches step 5, and loses the prefix on in the name, and looks for 'service' as the property. And because there is a getServiceEndpoint method, Camel will use that.

Which approach to use?

Using the @Consume annotations are simpler when you are creating a simple route with a single well defined input URI.

However if you require more complex routes or the same bean method needs to be invoked from many places then please use the routing DSL as shown above.

There are two different ways to send messages to any Camel Endpoint from a POJO

@EndpointInject

To allow sending of messages from POJOs you can use the @EndpointInject annotation. This will inject a ProducerTemplate so that the bean can participate in message exchanges.

e.g. lets send a message to the foo.bar queue in ActiveMQ at some point

The downside of this is that your code is now dependent on a Camel API, the ProducerTemplate. The next section describes how to remove this

Hiding the Camel APIs from your code using @Produce

We recommend Hiding Middleware APIs from your application code so the next option might be more suitable.
You can add the @Produce annotation to an injection point (a field or property setter) using a ProducerTemplate or using some interface you use in your business logic. e.g.

Here Camel will automatically inject a smart client side proxy at the @Produce annotation - an instance of the MyListener instance. When we invoke methods on this interface the method call is turned into an object and using the Camel Spring Remoting mechanism it is sent to the endpoint - in this case the ActiveMQ endpoint to queue foo; then the caller blocks for a response.

If you want to make asynchronous message sends then use an @InOnly annotation on the injection point.

@RecipientList Annotation

We support the use of @RecipientList on a bean method to easily create a dynamic Recipient List using a Java method.

Simple Example using @Consume and @RecipientList

For example if the above bean is configured in Spring when using a <camelContext> element as follows

then a route will be created consuming from the foo queue on the ActiveMQ component which when a message is received the message will be forwarded to the endpoints defined by the result of this method call - namely the bar and whatnot queues.

How it works

The return value of the @RecipientList method is converted to either a java.util.Collection / java.util.Iterator or array of objects where each element is converted to an Endpoint or a String, or if you are only going to route to a single endpoint then just return either an Endpoint object or an object that can be converted to a String. So the following methods are all valid

Then for each endpoint or URI the message is forwarded a separate copy to that endpoint.

You can then use whatever Java code you wish to figure out what endpoints to route to; for example you can use the Bean Binding annotations to inject parts of the message body or headers or use Expression values on the message.

More Complex Example Using DSL

In this example we will use more complex Bean Binding, plus we will use a separate route to invoke the Recipient List

Notice how we are injecting some headers or expressions and using them to determine the recipients using Recipient List EIP.
See the Bean Integration for more details.

Using Exchange Pattern Annotations

When working with POJO Producing or Spring Remoting you invoke methods which typically by default are InOut for Request Reply. That is there is an In message and an Out for the result. Typically invoking this operation will be synchronous, the caller will block until the server returns a result.

Camel has flexible Exchange Pattern support - so you can also support the Event Message pattern to use InOnly for asynchronous or one way operations. These are often called 'fire and forget' like sending a JMS message but not waiting for any response.

From 1.5 onwards Camel supports annotations for specifying the message exchange pattern on regular Java methods, classes or interfaces.

Specifying InOnly methods

Typically the default InOut is what most folks want but you can customize to use InOnly using an annotation.

The above code shows three methods on an interface; the first two use the default InOut mechanism but the someInOnlyMethod uses the InOnly annotation to specify it as being a oneway method call.

Class level annotations

You can also use class level annotations to default all methods in an interface to some pattern such as

Annotations will also be detected on base classes or interfaces. So for example if you created a client side proxy for

Then the methods inherited from Foo would be InOnly.

Overloading a class level annotation

You can overload a class level annotation on specific methods. A common use case for this is if you have a class or interface with many InOnly methods but you want to just annote one or two methods as InOut

In the above Foo interface the someInOutMethod will be InOut

Using your own annotations

You might want to create your own annotations to represent a group of different bits of metadata; such as combining synchrony, concurrency and transaction behaviour.

So you could annotate your annotation with the @Pattern annotation to default the exchange pattern you wish to use.

For example lets say we want to create our own annotation called @MyAsyncService

Now we can use this annotation and Camel will figure out the correct exchange pattern...

When writing software these days, its important to try and decouple as much middleware code from your business logic as possible.

This provides a number of benefits...

• you can choose the right middleware solution for your deployment and switch at any time
• you don't have to spend a large amount of time learning the specifics of any particular technology, whether its JMS or JavaSpace or Hibernate or JPA or iBatis whatever

For example if you want to implement some kind of message passing, remoting, reliable load balancing or asynchronous processing in your application we recommend you use Camel annotations to bind your services and business logic to Camel Components which means you can then easily switch between things like

• in JVM messaging with SEDA
• using JMS via ActiveMQ or other JMS providers for reliable load balancing, grid or publish and subscribe
• for low volume, but easier administration since you're probably already using a database you could use
  • Hibernate or JPA to use an entity bean / table as a queue
  • iBatis to work with SQL
  • JDBC for raw SQL access
• use JavaSpace

How to decouple from middleware APIs

The best approach when using remoting is to use Spring Remoting which can then use any messaging or remoting technology under the covers. When using Camel's implementation you can then use any of the Camel Components along with any of the Enterprise Integration Patterns.

Another approach is to bind Java beans to Camel endpoints via the Bean Integration. For example using POJO Consuming and POJO Producing you can avoid using any Camel APIs to decouple your code both from middleware APIs and Camel APIs!

Visualisation

 

Camel supports the visualisation of your Enterprise Integration Patterns using the GraphViz DOT files which can either be rendered directly via a suitable GraphViz tool or turned into HTML, PNG or SVG files via the Camel Maven Plugin.

Here is a typical example of the kind of thing we can generate

If you click on the actual generated htmlyou will see that you can navigate from an EIP node to its pattern page, along with getting hover-over tool tips ec.

How to generate

See Camel Dot Maven Goal or the other maven goals Camel Maven Plugin

For OS X users

If you are using OS X then you can open the DOT file using graphviz which will then automatically re-render if it changes, so you end up with a real time graphical representation of the topic and queue hierarchies!

Also if you want to edit the layout a little before adding it to a wiki to distribute to your team, open the DOT file with OmniGraffle then just edit away

Business Activity Monitoring

The Camel BAM module provides a Business Activity Monitoring (BAM) framework for testing business processes across multiple message exchanges on different Endpoint instances.

Consider, for example, a simple system in which you submit Purchase Orders into system A and then receive Invoices from system B. You might want to test that, for a given Purchase Order, you receive a matching Invoice from system B within a specific time period.

How Camel BAM Works

Camel BAM uses a Correlation Identifier on an input message to determine the Process Instance to which it belongs. The process instance is an entity bean which can maintain state for each Activity (where an activity typically maps to a single endpoint - such as the submission of Purchase Orders or the receipt of Invoices).

You can then add rules to be triggered when a message is received on any activity - such as to set time expectations or perform real time reconciliation of values across activities.

Simple Example

The following example shows how to perform some time based rules on a simple business process of 2 activities - A and B - which correspond with Purchase Orders and Invoices in the example above. If you would like to experiment with this scenario, you may edit this Test Case, which defines the activities and rules, and then tests that they work.

As you can see in the above example, we first define two activities, and then rules to specify when we expect them to complete for a process instance and when an error condition should be raised.p. The ProcessBuilder is a RouteBuilder and can be added to any CamelContext.

Complete Example

For a complete example please see the BAM Example, which is part of the standard Camel Examples

Use Cases

In the world of finance, a common requirement is tracking trades. Often a trader will submit a Front Office Trade which then flows through the Middle Office and Back Office through various systems to settle the trade so that money is exchanged. You may wish to test that the front and back office trades match up within a certain time period; if they don't match or a back office trade does not arrive within a required amount of time, you might signal an alarm.

Extract Transform Load (ETL)

The ETL (Extract, Transform, Load) is a mechanism for loading data into systems or databases using some kind of Data Format from a variety of sources; often files then using Pipes and Filters, Message Translator and possible other Enterprise Integration Patterns.

So you could query data from various Camel Components such as File, HTTP or JPA, perform multiple patterns such as Splitter or Message Translator then send the messages to some other Component.

To show how this all fits together, try the ETL Example

Mock Component

Testing of distributed and asynchronous processing is notoriously difficult. The Mock, Test and DataSet endpoints work great with the Camel Testing Framework to simplify your unit and integration testing using Enterprise Integration Patterns and Camel's large range of Components together with the powerful Bean Integration.

The Mock component provides a powerful declarative testing mechanism, which is similar to jMock in that it allows declarative expectations to be created on any Mock endpoint before a test begins. Then the test is run, which typically fires messages to one or more endpoints, and finally the expectations can be asserted in a test case to ensure the system worked as expected.

This allows you to test various things like:

• The correct number of messages are received on each endpoint,
• The correct payloads are received, in the right order,
• Messages arrive on an endpoint in order, using some Expression to create an order testing function,
• Messages arrive match some kind of Predicate such as that specific headers have certain values, or that parts of the messages match some predicate, such as by evaluating an XPath or XQuery Expression.

Note that there is also the Test endpoint which is a Mock endpoint, but which uses a second endpoint to provide the list of expected message bodies and automatically sets up the Mock endpoint assertions. In other words, it's a Mock endpoint that automatically sets up its assertions from some sample messages in a File or database, for example.

URI format

Where someName can be any string that uniquely identifies the endpoint.

You can append query options to the URI in the following format, ?option=value&option=value&...

Options

Simple Example

Here's a simple example of Mock endpoint in use. First, the endpoint is resolved on the context. Then we set an expectation, and then, after the test has run, we assert that our expectations have been met.

You typically always call the assertIsSatisfied() method to test that the expectations were met after running a test.

Camel will by default wait 10 seconds when the assertIsSatisfied() is invoked. This can be configured by setting the setResultWaitTime(millis) method.

Using assertPeriod

Available as of Camel 2.7
When the assertion is satisfied then Camel will stop waiting and continue from the assertIsSatisfied method. That means if a new message arrives on the mock endpoint, just a bit later, that arrival will not affect the outcome of the assertion. Suppose you do want to test that no new messages arrives after a period thereafter, then you can do that by setting the setAssertPeriod method, for example:

Setting expectations

You can see from the javadoc of MockEndpoint the various helper methods you can use to set expectations. The main methods are as follows:

Here's another example:

Adding expectations to specific messages

In addition, you can use the message(int messageIndex) method to add assertions about a specific message that is received.

For example, to add expectations of the headers or body of the first message (using zero-based indexing like java.util.List), you can use the following code:

There are some examples of the Mock endpoint in use in the camel-core processor tests.

Mocking existing endpoints

Available as of Camel 2.7

Camel now allows you to automatically mock existing endpoints in your Camel routes.

Suppose you have the given route below:

You can then use the adviceWith feature in Camel to mock all the endpoints in a given route from your unit test, as shown below:

Notice that the mock endpoints is given the uri mock:<endpoint>, for example mock:direct:foo. Camel logs at INFO level the endpoints being mocked:

Its also possible to only mock certain endpoints using a pattern. For example to mock all log endpoints you do as shown:

The pattern supported can be a wildcard or a regular expression. See more details about this at Intercept as its the same matching function used by Camel.

Mocking existing endpoints using the camel-test component

Instead of using the adviceWith to instruct Camel to mock endpoints, you can easily enable this behavior when using the camel-test Test Kit.
The same route can be tested as follows. Notice that we return "*" from the isMockEndpoints method, which tells Camel to mock all endpoints.
If you only want to mock all log endpoints you can return "log*" instead.

Mocking existing endpoints with XML DSL

If you do not use the camel-test component for unit testing (as shown above) you can use a different approach when using XML files for routes.
The solution is to create a new XML file used by the unit test and then include the intended XML file which has the route you want to test.

Suppose we have the route in the camel-route.xml file:

Then we create a new XML file as follows, where we include the camel-route.xml file and define a spring bean with the class org.apache.camel.impl.InterceptSendToMockEndpointStrategy which tells Camel to mock all endpoints:

Then in your unit test you load the new XML file (test-camel-route.xml) instead of camel-route.xml.

To only mock all Log endpoints you can define the pattern in the constructor for the bean:

Mocking endpoints and skip sending to original endpoint

Available as of Camel 2.10

Sometimes you want to easily mock and skip sending to a certain endpoints. So the message is detoured and send to the mock endpoint only. From Camel 2.10 onwards you can now use the mockEndpointsAndSkip method using AdviceWith or the Test Kit. The example below will skip sending to the two endpoints "direct:foo", and "direct:bar".

The same example using the Test Kit

Limiting the number of messages to keep

Available as of Camel 2.10

The Mock endpoints will by default keep a copy of every Exchange that it received. So if you test with a lot of messages, then it will consume memory.
From Camel 2.10 onwards we have introduced two options retainFirst and retainLast that can be used to specify to only keep N'th of the first and/or last Exchanges.

For example in the code below, we only want to retain a copy of the first 5 and last 5 Exchanges the mock receives.

Using this has some limitations. The getExchanges() and getReceivedExchanges() methods on the MockEndpoint will return only the retained copies of the Exchanges. So in the example above, the list will contain 10 Exchanges; the first five, and the last five.
The retainFirst and retainLast options also have limitations on which expectation methods you can use. For example the expectedXXX methods that work on message bodies, headers, etc. will only operate on the retained messages. In the example above they can test only the expectations on the 10 retained messages.

Testing with arrival times

Available as of Camel 2.7

The Mock endpoint stores the arrival time of the message as a property on the Exchange.

You can use this information to know when the message arrived on the mock. But it also provides foundation to know the time interval between the previous and next message arrived on the mock. You can use this to set expectations using the arrives DSL on the Mock endpoint.

For example to say that the first message should arrive between 0-2 seconds before the next you can do:

You can also define this as that 2nd message (0 index based) should arrive no later than 0-2 seconds after the previous:

You can also use between to set a lower bound. For example suppose that it should be between 1-4 seconds:

You can also set the expectation on all messages, for example to say that the gap between them should be at most 1 second:

See Also

• Configuring Camel
• Component
• Endpoint
• Getting Started

• Spring Testing
• Testing

Testing

Testing is a crucial activity in any piece of software development or integration. Typically Camel Riders use various different technologies wired together in a variety of patterns with different expression languages together with different forms of Bean Integration and Dependency Injection so its very easy for things to go wrong! . Testing is the crucial weapon to ensure that things work as you would expect.

Camel is a Java library so you can easily wire up tests in whatever unit testing framework you use (JUnit 3.x (deprecated), 4.x, or TestNG). However the Camel project has tried to make the testing of Camel as easy and powerful as possible so we have introduced the following features.

Testing mechanisms

The following mechanisms are supported:

In all approaches the test classes look pretty much the same in that they all reuse the Camel binding and injection annotations.

Camel Test Example

Here is the Camel Test example:

Notice how it derives from the Camel helper class CamelTestSupport but has no CDI, Spring or Guice dependency injection configuration but instead overrides the createRouteBuilder() method.

CDI Test example

Here is the CDI Testing example:

You can find more testing patterns illustrated in the camel-example-cdi-test example and the test classes that come with it.

Spring Test with XML Config Example

Here is the Spring Testing example using XML Config:

Notice that we use @DirtiesContext on the test methods to force Spring Testing to automatically reload the CamelContext after each test method - this ensures that the tests don't clash with each other (e.g. one test method sending to an endpoint that is then reused in another test method).

Also notice the use of @ContextConfiguration to indicate that by default we should look for the FilterTest-context.xml on the classpath to configure the test case which looks like this:

Spring Test with Java Config Example

Here is the Spring Testing example using Java Config.

For more information see Spring Java Config.

This is similar to the XML Config example above except that there is no XML file and instead the nested ContextConfig class does all of the configuration; so your entire test case is contained in a single Java class. We currently have to reference by class name this class in the @ContextConfiguration which is a bit ugly. Please vote for SJC-238 to address this and make Spring Test work more cleanly with Spring JavaConfig.

Its totally optional but for the ContextConfig implementation we derive from SingleRouteCamelConfiguration which is a helper Spring Java Config class which will configure the CamelContext for us and then register the RouteBuilder we create.

Since Camel 2.11.0 you can use the CamelSpringJUnit4ClassRunner with CamelSpringDelegatingTestContextLoader like example using Java Config with CamelSpringJUnit4ClassRunner:

Spring Test with XML Config and Declarative Configuration Example

Here is a Camel test support enhanced Spring Testing example using XML Config and pure Spring Test based configuration of the Camel Context:

Notice how a custom test runner is used with the @RunWith annotation to support the features of CamelTestSupport through annotations on the test class. See Spring Testing for a list of annotations you can use in your tests.

Blueprint Test

Here is the Blueprint Testing example using XML Config:

Also notice the use of getBlueprintDescriptors to indicate that by default we should look for the camelContext.xml in the package to configure the test case which looks like this:

Testing endpoints

Camel provides a number of endpoints which can make testing easier.

The main endpoint is the Mock endpoint which allows expectations to be added to different endpoints; you can then run your tests and assert that your expectations are met at the end.

Stubbing out physical transport technologies

If you wish to test out a route but want to avoid actually using a real physical transport (for example to unit test a transformation route rather than performing a full integration test) then the following endpoints can be useful.

Testing existing routes

Camel provides some features to aid during testing of existing routes where you cannot or will not use Mock etc. For example you may have a production ready route which you want to test with some 3rd party API which sends messages into this route.

Camel Test

As a simple alternative to using CDI Testing, Spring Testing or Guice the camel-test module was introduced so you can perform powerful Testing of your Enterprise Integration Patterns easily.

Adding to your pom.xml

To get started using Camel Test you will need to add an entry to your pom.xml

JUnit

TestNG

Available as of Camel 2.8

You might also want to add slf4j and log4j to ensure nice logging messages (and maybe adding a log4j.properties file into your src/test/resources directory).

Writing your test

You firstly need to derive from the class CamelTestSupport (org.apache.camel.test.CamelTestSupport, org.apache.camel.test.junit4.CamelTestSupport, or org.apache.camel.testng.CamelTestSupport for JUnit 3.x, JUnit 4.x, and TestNG, respectively) and typically you will need to override the createRouteBuilder() or createRouteBuilders() method to create routes to be tested.

Here is an example.

Notice how you can use the various Camel binding and injection annotations to inject individual Endpoint objects - particularly the Mock endpoints which are very useful for Testing. Also you can inject producer objects such as ProducerTemplate or some application code interface for sending messages or invoking services.

Features Provided by CamelTestSupport

The various CamelTestSupport classes provide a standard set of behaviors relating to the CamelContext used to host the route(s) under test.  The classes provide a number of methods that allow a test to alter the configuration of the CamelContext used.  The following table describes the available customization methods and the default behavior of tests that are built from a CamelTestSupport class.

JNDI

Camel uses a Registry to allow you to configure Component or Endpoint instances or Beans used in your routes. If you are not using Spring or OSGi then JNDI is used as the default registry implementation.

So you will also need to create a jndi.properties file in your src/test/resources directory so that there is a default registry available to initialise the CamelContext.

Here is an example jndi.properties file

Dynamically assigning ports

Available as of Camel 2.7

Tests that use port numbers will fail if that port is already on use. AvailablePortFinder provides methods for finding unused port numbers at runtime.

Setup CamelContext once per class, or per every test method

Available as of Camel 2.8

The Camel Test kit will by default setup and shutdown CamelContext per every test method in your test class. So for example if you have 3 test methods, then CamelContext is started and shutdown after each test, that is 3 times.

You may want to do this once, to share the CamelContext between test methods, to speedup unit testing. This requires to use JUnit 4! In your unit test method you have to extend the org.apache.camel.test.junit4.CamelTestSupport or the org.apache.camel.test.junit4.CamelSpringTestSupport test class and override the isCreateCamelContextPerClass method and return true as shown in the following example:

See Also

• Testing
• Mock
• Test

Spring Testing

Testing is a crucial part of any development or integration work. The Spring Framework offers a number of features that makes it easy to test while using Spring for Inversion of Control which works with JUnit 3.x, JUnit 4.x, and TestNG.

We can use Spring for IoC and the Camel Mock and Test endpoints to create sophisticated integration/unit tests that are easy to run and debug inside your IDE.  There are three supported approaches for testing with Spring in Camel.

CamelSpringTestSupport

org.apache.camel.test.CamelSpringTestSupport, org.apache.camel.test.junit4.CamelSpringTestSupport, and org.apache.camel.testng.CamelSpringTestSupport extend their non-Spring aware counterparts (org.apache.camel.test.CamelTestSupport, org.apache.camel.test.junit4.CamelTestSupport, and org.apache.camel.testng.CamelTestSupport) and deliver integration with Spring into your test classes.  Instead of instantiating the CamelContext and routes programmatically, these classes rely on a Spring context to wire the needed components together.  If your test extends one of these classes, you must provide the Spring context by implementing the following method.

You are responsible for the instantiation of the Spring context in the method implementation.  All of the features available in the non-Spring aware counterparts from Camel Test are available in your test.

Plain Spring Test

In this approach, your test classes directly inherit from the Spring Test abstract test classes or use the JUnit 4.x test runner provided in Spring Test.  This approach supports dependency injection into your test class and the full suite of Spring Test annotations but does not support the features provided by the CamelSpringTestSupport classes.

Plain Spring Test using JUnit 3.x with XML Config Example

Here is a simple unit test using JUnit 3.x support from Spring Test using XML Config.

Notice that we use @DirtiesContext on the test methods to force Spring Testing to automatically reload the CamelContext after each test method - this ensures that the tests don't clash with each other (e.g. one test method sending to an endpoint that is then reused in another test method).

Also notice the use of @ContextConfiguration to indicate that by default we should look for the FilterTest-context.xml on the classpath to configure the test case which looks like this

This test will load a Spring XML configuration file calledFilterTest-context.xml from the classpath in the same package structure as the FilterTest class and initialize it along with any Camel routes we define inside it, then inject theCamelContextinstance into our test case.

For instance, like this maven folder layout:

Plain Spring Test using JUnit 4.x with Java Config Example

You can completely avoid using an XML configuration file by using Spring Java Config.  Here is a unit test using JUnit 4.x support from Spring Test using Java Config.

This is similar to the XML Config example above except that there is no XML file and instead the nested ContextConfig class does all of the configuration; so your entire test case is contained in a single Java class. We currently have to reference by class name this class in the @ContextConfiguration which is a bit ugly. Please vote for SJC-238 to address this and make Spring Test work more cleanly with Spring JavaConfig.

Plain Spring Test using JUnit 4.0.x Runner with XML Config

You can avoid extending Spring classes by using the SpringJUnit4ClassRunner provided by Spring Test.  This custom JUnit runner means you are free to choose your own class hierarchy while retaining all the capabilities of Spring Test.

Plain Spring Test using JUnit 4.1.x Runner with XML Config

You can avoid extending Spring classes by using the SpringJUnit4ClassRunner provided by Spring Test.  This custom JUnit runner means you are free to choose your own class hierarchy while retaining all the capabilities of Spring Test.

Camel Enhanced Spring Test

Using org.apache.camel.test.junit4.CamelSpringJUnit4ClassRunner runner with the @RunWith annotation or extending org.apache.camel.testng.AbstractCamelTestNGSpringContextTests provides the full feature set of Spring Test with support for the feature set provided in the CamelTestSupport classes.  A number of Camel specific annotations have been developed in order to provide for declarative manipulation of the Camel context(s) involved in the test.  These annotations free your test classes from having to inherit from the CamelSpringTestSupport classes and also reduce the amount of code required to customize the tests.

The following example illustrates the use of the @MockEndpoints annotation in order to setup mock endpoints as interceptors on all endpoints using the Camel Log component and the @DisableJmx annotation to enable JMX which is disabled during tests by default.  Note that we still use the @DirtiesContext annotation to ensure that the CamelContext, routes, and mock endpoints are reinitialized between test methods.

Adding more Mock expectations

If you wish to programmatically add any new assertions to your test you can easily do so with the following. Notice how we use @EndpointInject to inject a Camel endpoint into our code then the Mock API to add an expectation on a specific message.

Further processing the received messages

Sometimes once a Mock endpoint has received some messages you want to then process them further to add further assertions that your test case worked as you expect.

So you can then process the received message exchanges if you like...

Sending and receiving messages

It might be that the Enterprise Integration Patterns you have defined in either Spring XML or using the Java DSL do all of the sending and receiving and you might just work with the Mock endpoints as described above. However sometimes in a test case its useful to explicitly send or receive messages directly.

To send or receive messages you should use the Bean Integration mechanism. For example to send messages inject a ProducerTemplate using the @EndpointInject annotation then call the various send methods on this object to send a message to an endpoint. To consume messages use the @MessageDriven annotation on a method to have the method invoked when a message is received.

See Also

• A real example test case using Mock and Spring along with its Spring XML
• Bean Integration
• Mock endpoint
• Test endpoint

Camel Guice

We have support for Google Guice as a dependency injection framework.

Maven users will need to add the following dependency to their pom.xml for this component:

Dependency Injecting Camel with Guice

The GuiceCamelContext is designed to work nicely inside Guice. You then need to bind it using some Guice Module.

The camel-guice library comes with a number of reusable Guice Modules you can use if you wish - or you can bind the GuiceCamelContext yourself in your own module.

• CamelModule is the base module which binds the GuiceCamelContext but leaves it up you to bind the RouteBuilder instances
• CamelModuleWithRouteTypes extends CamelModule so that in the constructor of the module you specify the RouteBuilder classes or instances to use
• CamelModuleWithMatchingRoutes extends CamelModule so that all bound RouteBuilder instances will be injected into the CamelContext or you can supply an optional Matcher to find RouteBuilder instances matching some kind of predicate.

So you can specify the exact RouteBuilder instances you want

Or inject them all

You can then use Guice in the usual way to inject the route instances or any other dependent objects.

Bootstrapping with JNDI

A common pattern used in J2EE is to bootstrap your application or root objects by looking them up in JNDI. This has long been the approach when working with JMS for example - looking up the JMS ConnectionFactory in JNDI for example.

You can follow a similar pattern with Guice using the GuiceyFruit JNDI Provider which lets you bootstrap Guice from a jndi.properties file which can include the Guice Modules to create along with environment specific properties you can inject into your modules and objects.

If the jndi.properties is conflict with other component, you can specify the jndi properties file name in the Guice Main with option -j or -jndiProperties with the properties file location to let Guice Main to load right jndi properties file.

Configuring Component, Endpoint or RouteBuilder instances

You can use Guice to dependency inject whatever objects you need to create, be it an Endpoint, Component, RouteBuilder or arbitrary bean used within a route.

The easiest way to do this is to create your own Guice Module class which extends one of the above module classes and add a provider method for each object you wish to create. A provider method is annotated with @Provides as follows

You can optionally annotate the method with @JndiBind to bind the object to JNDI at some name if the object is a component, endpoint or bean you wish to refer to by name in your routes.

You can inject any environment specific properties (such as URLs, machine names, usernames/passwords and so forth) from the jndi.properties file easily using the @Named annotation as shown above. This allows most of your configuration to be in Java code which is typesafe and easily refactorable - then leaving some properties to be environment specific (the jndi.properties file) which you can then change based on development, testing, production etc.

Creating multiple RouteBuilder instances per type

It is sometimes useful to create multiple instances of a particular RouteBuilder with different configurations.

To do this just create multiple provider methods for each configuration; or create a single provider method that returns a collection of RouteBuilder instances.

For example

See Also

• there are a number of Examples you can look at to see Guice and Camel being used such as Guice JMS Example
• Guice Maven Plugin for running your Guice based routes via Maven

Templating

When you are testing distributed systems its a very common requirement to have to stub out certain external systems with some stub so that you can test other parts of the system until a specific system is available or written etc.

A great way to do this is using some kind of Template system to generate responses to requests generating a dynamic message using a mostly-static body.

There are a number of templating components included in the Camel distribution you could use

• FreeMarker
• Mustache
• StringTemplate
• Velocity
• XQuery
• XSLT

or the following external Camel components

• Scalate

Example

Here's a simple example showing how we can respond to InOut requests on the My.Queue queue on ActiveMQ with a template generated response. The reply would be sent back to the JMSReplyTo Destination.

If you want to use InOnly and consume the message and send it to another destination you could use

See Also

• Mock for details of mock endpoint testing (as opposed to template based stubs).

Database

Camel can work with databases in a number of different ways. This document tries to outline the most common approaches.

Database endpoints

Camel provides a number of different endpoints for working with databases

• JPA for working with hibernate, openjpa or toplink. When consuming from the endpoints entity beans are read (and deleted/updated to mark as processed) then when producing to the endpoints they are written to the database (via insert/update).
• iBATIS similar to the above but using Apache iBATIS
• JDBC similar though using explicit SQL
• SQL uses spring-jdbc behind the scene for the actual SQL handling. The difference between this component and JDBC component is that in case of SQL the query is a property of the endpoint and it uses message payload as parameters passed to the query

Database pattern implementations

Various patterns can work with databases as follows

• Idempotent Consumer
• Aggregator
• BAM for business activity monitoring

Parallel Processing and Ordering

It is a common requirement to want to use parallel processing of messages for throughput and load balancing, while at the same time process certain kinds of messages in order.

How to achieve parallel processing

You can send messages to a number of Camel Components to achieve parallel processing and load balancing such as

• SEDA for in-JVM load balancing across a thread pool
• ActiveMQ or JMS for distributed load balancing and parallel processing
• JPA for using the database as a poor mans message broker

When processing messages concurrently, you should consider ordering and concurrency issues. These are described below

Concurrency issues

Note that there is no concurrency or locking issue when using ActiveMQ, JMS or SEDA by design; they are designed for highly concurrent use. However there are possible concurrency issues in the Processor of the messages i.e. what the processor does with the message?

For example if a processor of a message transfers money from one account to another account; you probably want to use a database with pessimistic locking to ensure that operation takes place atomically.

Ordering issues

As soon as you send multiple messages to different threads or processes you will end up with an unknown ordering across the entire message stream as each thread is going to process messages concurrently.

For many use cases the order of messages is not too important. However for some applications this can be crucial. e.g. if a customer submits a purchase order version 1, then amends it and sends version 2; you don't want to process the first version last (so that you loose the update). Your Processor might be clever enough to ignore old messages. If not you need to preserve order.

Recommendations

This topic is large and diverse with lots of different requirements; but from a high level here are our recommendations on parallel processing, ordering and concurrency

• for distributed locking, use a database by default, they are very good at it
• to preserve ordering across a JMS queue consider using Exclusive Consumers in the ActiveMQ component
• even better are Message Groups which allows you to preserve ordering across messages while still offering parallelisation via the JMSXGroupID header to determine what can be parallelized
• if you receive messages out of order you could use the Resequencer to put them back together again

A good rule of thumb to help reduce ordering problems is to make sure each single can be processed as an atomic unit in parallel (either without concurrency issues or using say, database locking); or if it can't, use a Message Group to relate the messages together which need to be processed in order by a single thread.

Using Message Groups with Camel

To use a Message Group with Camel you just need to add a header to the output JMS message based on some kind of Correlation Identifier to correlate messages which should be processed in order by a single thread - so that things which don't correlate together can be processed concurrently.

For example the following code shows how to create a message group using an XPath expression taking an invoice's product code as the Correlation Identifier

You can of course use the Xml Configuration if you prefer

Asynchronous Processing

Overview

Camel supports a more complex asynchronous processing model. The asynchronous processors implement the org.apache.camel.AsyncProcessor interface which is derived from the more synchronous org.apache.camel.Processor interface. There are advantages and disadvantages when using asynchronous processing when compared to using the standard synchronous processing model.

Advantages:

• Processing routes that are composed fully of asynchronous processors do not use up threads waiting for processors to complete on blocking calls. This can increase the scalability of your system by reducing the number of threads needed to process the same workload.
• Processing routes can be broken up into SEDA processing stages where different thread pools can process the different stages. This means that your routes can be processed concurrently.

Disadvantages:

• Implementing asynchronous processors is more complex than implementing the synchronous versions.

When to Use

We recommend that processors and components be implemented the more simple synchronous APIs unless you identify a performance of scalability requirement that dictates otherwise. A Processor whose process() method blocks for a long time would be good candidates for being converted into an asynchronous processor.

Interface Details

The AsyncProcessor defines a single process() method which is very similar to it's synchronous Processor.process() brethren.

Here are the differences:

• A non-null AsyncCallback MUST be supplied which will be notified when the exchange processing is completed.
• It MUST not throw any exceptions that occurred while processing the exchange. Any such exceptions must be stored on the exchange's Exception property.
• It MUST know if it will complete the processing synchronously or asynchronously. The method will return true if it does complete synchronously, otherwise it returns false.
• When the processor has completed processing the exchange, it must call the callback.done(boolean sync) method.
• The sync parameter MUST match the value returned by the process() method.

Implementing Processors that Use the AsyncProcessor API

All processors, even synchronous processors that do not implement the AsyncProcessor interface, can be coerced to implement the AsyncProcessor interface. This is usually done when you are implementing a Camel component consumer that supports asynchronous completion of the exchanges that it is pushing through the Camel routes. Consumers are provided a Processor object when created. All Processor object can be coerced to a AsyncProcessor using the following API:

For a route to be fully asynchronous and reap the benefits to lower Thread usage, it must start with the consumer implementation making use of the asynchronous processing API. If it called the synchronous process() method instead, the consumer's thread would be forced to be blocked and in use for the duration that it takes to process the exchange.

It is important to take note that just because you call the asynchronous API, it does not mean that the processing will take place asynchronously. It only allows the possibility that it can be done without tying up the caller's thread. If the processing happens asynchronously is dependent on the configuration of the Camel route.

Normally, the the process call is passed in an inline inner AsyncCallback class instance which can reference the exchange object that was declared final. This allows it to finish up any post processing that is needed when the called processor is done processing the exchange.

Example.

Asynchronous Route Sequence Scenarios

Now that we have understood the interface contract of the AsyncProcessor, and have seen how to make use of it when calling processors, let's looks a what the thread model/sequence scenarios will look like for some sample routes.

The Jetty component's consumers support asynchronous processing through the use of continuations. Suffice to say it can take a HTTP request and pass it to a Camel route for asynchronous processing. If the processing is indeed asynchronous, it uses a Jetty continuation so that the HTTP request is 'parked' and the thread is released. Once the Camel route finishes processing the request, the Jetty component uses the AsyncCallback to tell Jetty to 'un-park' the request. Jetty un-parks the request, the HTTP response returned using the result of the exchange processing.

Notice that the jetty continuations feature is only used "If the processing is indeed async". This is why AsyncProcessor.process() implementations must accurately report if request is completed synchronously or not.

The jhc component's producer allows you to make HTTP requests and implement the AsyncProcessor interface. A route that uses both the jetty asynchronous consumer and the jhc asynchronous producer will be a fully asynchronous route and has some nice attributes that can be seen if we take a look at a sequence diagram of the processing route.

For the route:

The sequence diagram would look something like this:

The diagram simplifies things by making it looks like processors implement the AsyncCallback interface when in reality the AsyncCallback interfaces are inline inner classes, but it illustrates the processing flow and shows how two separate threads are used to complete the processing of the original HTTP request. The first thread is synchronous up until processing hits the jhc producer which issues the HTTP request. It then reports that the exchange processing will complete asynchronously using NIO to get the response back. Once the jhc component has received a full response it uses AsyncCallback.done() method to notify the caller. These callback notifications continue up until it reaches the original Jetty consumer which then un-parks the HTTP request and completes it by providing the response.

Mixing Synchronous and Asynchronous Processors

It is totally possible and reasonable to mix the use of synchronous and asynchronous processors/components. The pipeline processor is the backbone of a Camel processing route. It glues all the processing steps together. It is implemented as an AsyncProcessor and supports interleaving synchronous and asynchronous processors as the processing steps in the pipeline.

Let's say we have two custom asynchronous processors, namely: MyValidator and MyTransformation. Let's say we want to load file from the data/in directory validate them with the MyValidator() processor, transform them into JPA Java objects using MyTransformation and then insert them into the database using the JPA component. Let's say that the transformation process takes quite a bit of time and we want to allocate 20 threads to do parallel transformations of the input files. The solution is to make use of the thread processor. The thread is AsyncProcessor that forces subsequent processing in asynchronous thread from a thread pool.

The route might look like:

The sequence diagram would look something like this:

You would actually have multiple threads executing the second part of the thread sequence.

Staying Synchronous in an AsyncProcessor

Generally speaking you get better throughput processing when you process things synchronously. This is due to the fact that starting up an asynchronous thread and doing a context switch to it adds a little bit of of overhead. So it is generally encouraged that AsyncProcessor's do as much work as they can synchronously. When they get to a step that would block for a long time, at that point they should return from the process call and let the caller know that it will be completing the call asynchronously.

Implementing Virtual Topics on other JMS providers

ActiveMQ supports Virtual Topics since durable topic subscriptions kinda suck (see this page for more detail) mostly since they don't support Competing Consumers.

Most folks want Queue semantics when consuming messages; so that you can support Competing Consumers for load balancing along with things like Message Groups and Exclusive Consumers to preserve ordering or partition the queue across consumers.

However if you are using another JMS provider you can implement Virtual Topics by switching to ActiveMQ or you can use the following Camel pattern.

First here's the ActiveMQ approach.

• send to activemq:topic:VirtualTopic.Orders
• for consumer A consume from activemq:Consumer.A.VirtualTopic.Orders

When using another message broker use the following pattern

• send to jms:Orders
• add this route with a to() for each logical durable topic subscriber
• for consumer A consume from jms:Consumer.A

What's the Camel Transport for CXF

In CXF you offer or consume a webservice by defining its address. The first part of the address specifies the protocol to use. For example address="http://localhost:9000" in an endpoint configuration means your service will be offered using the http protocol on port 9000 of localhost. When you integrate Camel Tranport into CXF you get a new transport "camel". So you can specify address="camel://direct:MyEndpointName" to bind the CXF service address to a camel direct endpoint.

Technically speaking Camel transport for CXF is a component which implements the CXF transport API with the Camel core library. This allows you to easily use Camel's routing engine and integration patterns support together with your CXF services.

Integrate Camel into CXF transport layer

To include the Camel Tranport into your CXF bus you use the CamelTransportFactory. You can do this in Java as well as in Spring.

Setting up the Camel Transport in Spring

You can use the following snippet in your applicationcontext if you want to configure anything special. If you only want to activate the camel transport you do not have to do anything in your application context. As soon as you include the camel-cxf-transport jar (or camel-cxf.jar if your camel version is less than 2.7.x) in your app, cxf will scan the jar and load a CamelTransportFactory for you.

Integrating the Camel Transport in a programmatic way

Camel transport provides a setContext method that you could use to set the Camel context into the transport factory. If you want this factory take effect, you need to register the factory into the CXF bus. Here is a full example for you.

Configure the destination and conduit with Spring

Namespace

The elements used to configure an Camel transport endpoint are defined in the namespace http://cxf.apache.org/transports/camel. It is commonly referred to using the prefix camel. In order to use the Camel transport configuration elements, you will need to add the lines shown below to the beans element of your endpoint's configuration file. In addition, you will need to add the configuration elements' namespace to the xsi:schemaLocation attribute.

The destination element

You configure an Camel transport server endpoint using the camel:destination element and its children. The camel:destination element takes a single attribute, name, that specifies the WSDL port element that corresponds to the endpoint. The value for the name attribute takes the form portQName.camel-destination. The example below shows the camel:destination element that would be used to add configuration for an endpoint that was specified by the WSDL fragment <port binding="widgetSOAPBinding" name="widgetSOAPPort"> if the endpoint's target namespace was http://widgets.widgetvendor.net.

The camel:destination element for Spring has a number of child elements that specify configuration information. They are described below.

The conduit element

You configure a Camel transport client using the camel:conduit element and its children. The camel:conduit element takes a single attribute, name, that specifies the WSDL port element that corresponds to the endpoint. The value for the name attribute takes the form portQName.camel-conduit. For example, the code below shows the camel:conduit element that would be used to add configuration for an endpoint that was specified by the WSDL fragment <port binding="widgetSOAPBinding" name="widgetSOAPPort"> if the endpoint's target namespace was http://widgets.widgetvendor.net.

The camel:conduit element has a number of child elements that specify configuration information. They are described below.

Configure the destination and conduit with Blueprint

From Camel 2.11.x, Camel Transport supports to be configured with Blueprint.

If you are using blueprint, you should use the the namespace http://cxf.apache.org/transports/camel/blueprint and import the schema like the blow.

In blueprint camel:conduit camel:destination only has one camelContextId attribute, they doesn't support to specify the camel context in the camel destination.

Example Using Camel as a load balancer for CXF

This example shows how to use the camel load balancing feature in CXF. You need to load the configuration file in CXF and publish the endpoints on the address "camel://direct:EndpointA" and "camel://direct:EndpointB"

Complete Howto and Example for attaching Camel to CXF

Better JMS Transport for CXF Webservice using Apache Camel 

Introduction

When sending an Exchange to an Endpoint you can either use a Route or a ProducerTemplate. This works fine in many scenarios. However you may need to guarantee that an exchange is delivered to the same endpoint that you delivered a previous exchange on. For example in the case of delivering a batch of exchanges to a MINA socket you may need to ensure that they are all delivered through the same socket connection. Furthermore once the batch of exchanges have been delivered the protocol requirements may be such that you are responsible for closing the socket.

Using a Producer

To achieve fine grained control over sending exchanges you will need to program directly to a Producer. Your code will look similar to:

In the case of using Apache MINA the producer.stop() invocation will cause the socket to be closed.

Tutorials

There now follows the documentation on camel tutorials

We have a number of tutorials as listed below. The tutorials often comes with source code which is either available in the Camel Download or attached to the wiki page.

• OAuth Tutorial
  This tutorial demonstrates how to implement OAuth for a web application with Camel's gauth component. The sample application of this tutorial is also online at http://gauthcloud.appspot.com/

• Tutorial for Camel on Google App Engine
  This tutorial demonstrates the usage of the Camel Components for Google App Engine. The sample application of this tutorial is also online at http://camelcloud.appspot.com/

• Tutorial on Spring Remoting with JMS
  This tutorial is focused on different techniques with Camel for Client-Server communication.

• Report Incident - This tutorial introduces Camel steadily and is based on a real life integration problem
  This is a very long tutorial beginning from the start; its for entry level to Camel. Its based on a real life integration, showing how Camel can be introduced in an existing solution. We do this in baby steps. The tutorial is currently work in progress, so check it out from time to time. The tutorial explains some of the inner building blocks Camel uses under the covers. This is good knowledge to have when you start using Camel on a higher abstract level where it can do wonders in a few lines of routing DSL.

• Using Camel with ServiceMix a tutorial on using Camel inside Apache ServiceMix.

• Better JMS Transport for CXF Webservice using Apache Camel Describes how to use the Camel Transport for CXF to attach a CXF Webservice to a JMS Queue

• Tutorial how to use good old Axis 1.4 with Camel
  This tutorial shows that Camel does work with the good old frameworks such as AXIS that is/was widely used for WebService.

• Tutorial on using Camel in a Web Application
  This tutorial gives an overview of how to use Camel inside Tomcat, Jetty or any other servlet engine

• Tutorial on Camel 1.4 for Integration
  Another real-life scenario. The company sells widgets, with a somewhat unique business process (their customers periodically report what they've purchased in order to get billed). However every customer uses a different data format and protocol. This tutorial goes through the process of integrating (and testing!) several customers and their electronic reporting of the widgets they've bought, along with the company's response.

• Tutorial how to build a Service Oriented Architecture using Camel with OSGI - Updated 20/11/2009
  The tutorial has been designed in two parts. The first part introduces basic concept to create a simple SOA solution using Camel and OSGI and deploy it in a OSGI Server like Apache Felix Karaf and Spring DM Server while the second extends the ReportIncident tutorial part 4 to show How we can separate the different layers (domain, service, ...) of an application and deploy them in separate bundles. The Web Application has also be modified in order to communicate to the OSGI bundles.

• Several of the vendors on the Commercial Camel Offerings page also offer various tutorials, webinars, examples, etc.... that may be useful.

• Examples
  While not actual tutorials you might find working through the source of the various Examples useful.

Tutorial on Spring Remoting with JMS

 

Preface

This tutorial aims to guide the reader through the stages of creating a project which uses Camel to facilitate the routing of messages from a JMS queue to a Spring service. The route works in a synchronous fashion returning a response to the client.

Prerequisites

This tutorial uses Maven to setup the Camel project and for dependencies for artifacts.

Distribution

This sample is distributed with the Camel distribution as examples/camel-example-spring-jms.

About

This tutorial is a simple example that demonstrates more the fact how well Camel is seamless integrated with Spring to leverage the best of both worlds. This sample is client server solution using JMS messaging as the transport. The sample has two flavors of servers and also for clients demonstrating different techniques for easy communication.

The Server is a JMS message broker that routes incoming messages to a business service that does computations on the received message and returns a response.
The EIP patterns used in this sample are:

We use the following Camel components:

Create the Camel Project

Update the POM with Dependencies

First we need to have dependencies for the core Camel jars, spring, jms components, and finally ActiveMQ as the message broker.

As we use spring xml configuration for the ActiveMQ JMS broker we need this dependency:

Writing the Server

Create the Spring Service

For this example the Spring service (our business service) on the server will be a simple multiplier which trebles in the received value.

And the implementation of this service is:Notice that this class has been annotated with the @Service spring annotation. This ensures that this class is registered as a bean in the registry with the given name multiplier.

Define the Camel Routes

This defines a Camel route from the JMS queue named numbers to the Spring bean named multiplier. Camel will create a consumer to the JMS queue which forwards all received messages onto the the Spring bean, using the method named multiply.

Configure Spring

The Spring config file is placed under META-INF/spring as this is the default location used by the Camel Maven Plugin, which we will later use to run our server.
First we need to do the standard scheme declarations in the top. In the camel-server.xml we are using spring beans as the default bean: namespace and springs context:. For configuring ActiveMQ we use broker: and for Camel we of course have camel:. Notice that we don't use version numbers for the camel-spring schema. At runtime the schema is resolved in the Camel bundle. If we use a specific version number such as 1.4 then its IDE friendly as it would be able to import it and provide smart completion etc. See Xml Reference for further details.

We use Spring annotations for doing IoC dependencies and its component-scan features comes to the rescue as it scans for spring annotations in the given package name:Camel will of course not be less than Spring in this regard so it supports a similar feature for scanning of Routes. This is configured as shown below.
Notice that we also have enabled the JMXAgent so we will be able to introspect the Camel Server with a JMX Console.The ActiveMQ JMS broker is also configured in this xml file. We set it up to listen on TCP port 61610.As this examples uses JMS then Camel needs a JMS component that is connected with the ActiveMQ broker. This is configured as shown below:Notice: The JMS component is configured in standard Spring beans, but the gem is that the bean id can be referenced from Camel routes - meaning we can do routing using the JMS Component by just using jms: prefix in the route URI. What happens is that Camel will find in the Spring Registry for a bean with the id="jms". Since the bean id can have arbitrary name you could have named it id="jmsbroker" and then referenced to it in the routing as from="jmsbroker:queue:numbers).to("multiplier");
We use the vm protocol to connect to the ActiveMQ server as its embedded in this application.

Run the Server

The Server is started using the org.apache.camel.spring.Main class that can start camel-spring application out-of-the-box. The Server can be started in several flavors:

• as a standard java main application - just start the org.apache.camel.spring.Main class
• using maven jave:exec
• using camel:run

In this sample as there are two servers (with and without AOP) we have prepared some profiles in maven to start the Server of your choice.
The server is started with:
mvn compile exec:java -PCamelServer

Writing The Clients

This sample has three clients demonstrating different Camel techniques for communication

• CamelClient using the ProducerTemplate for Spring template style coding
• CamelRemoting using Spring Remoting
• CamelEndpoint using the Message Endpoint EIP pattern using a neutral Camel API

Client Using The ProducerTemplate

We will initially create a client by directly using ProducerTemplate. We will later create a client which uses Spring remoting to hide the fact that messaging is being used.

The client will not use the Camel Maven Plugin so the Spring XML has been placed in src/main/resources to not conflict with the server configs.

And the CamelClient source code:

The ProducerTemplate is retrieved from a Spring ApplicationContext and used to manually place a message on the "numbers" JMS queue. The requestBody method will use the exchange pattern InOut, which states that the call should be synchronous, and that the caller expects a response.

Before running the client be sure that both the ActiveMQ broker and the CamelServer are running.

Client Using Spring Remoting

Spring Remoting "eases the development of remote-enabled services". It does this by allowing you to invoke remote services through your regular Java interface, masking that a remote service is being called.

The snippet above only illustrates the different and how Camel easily can setup and use Spring Remoting in one line configurations.

The proxy will create a proxy service bean for you to use to make the remote invocations. The serviceInterface property details which Java interface is to be implemented by the proxy. The serviceUrl defines where messages sent to this proxy bean will be directed. Here we define the JMS endpoint with the "numbers" queue we used when working with Camel template directly. The value of the id property is the name that will be the given to the bean when it is exposed through the Spring ApplicationContext. We will use this name to retrieve the service in our client. I have named the bean multiplierProxy simply to highlight that it is not the same multiplier bean as is being used by CamelServer. They are in completely independent contexts and have no knowledge of each other. As you are trying to mask the fact that remoting is being used in a real application you would generally not include proxy in the name.

And the Java client source code:

Again, the client is similar to the original client, but with some important differences.

1. The Spring context is created with the new camel-client-remoting.xml
2. We retrieve the proxy bean instead of a ProducerTemplate. In a non-trivial example you would have the bean injected as in the standard Spring manner.
3. The multiply method is then called directly. In the client we are now working to an interface. There is no mention of Camel or JMS inside our Java code.

Client Using Message Endpoint EIP Pattern

This client uses the Message Endpoint EIP pattern to hide the complexity to communicate to the Server. The Client uses the same simple API to get hold of the endpoint, create an exchange that holds the message, set the payload and create a producer that does the send and receive. All done using the same neutral Camel API for all the components in Camel. So if the communication was socket TCP based you just get hold of a different endpoint and all the java code stays the same. That is really powerful.

Okay enough talk, show me the code!

Switching to a different component is just a matter of using the correct endpoint. So if we had defined a TCP endpoint as: "mina:tcp://localhost:61610" then its just a matter of getting hold of this endpoint instead of the JMS and all the rest of the java code is exactly the same.

Run the Clients

The Clients is started using their main class respectively.

• as a standard java main application - just start their main class
• using maven jave:exec

In this sample we start the clients using maven:
mvn compile exec:java -PCamelClient
mvn compile exec:java -PCamelClientRemoting
mvn compile exec:java -PCamelClientEndpoint

Also see the Maven pom.xml file how the profiles for the clients is defined.

Using the Camel Maven Plugin

The Camel Maven Plugin allows you to run your Camel routes directly from Maven. This negates the need to create a host application, as we did with Camel server, simply to start up the container. This can be very useful during development to get Camel routes running quickly.

All that is required is a new plugin definition in your Maven POM. As we have already placed our Camel config in the default location (camel-server.xml has been placed in META-INF/spring/) we do not need to tell the plugin where the route definitions are located. Simply run mvn camel:run.

Using Camel JMX

Camel has extensive support for JMX and allows us to inspect the Camel Server at runtime. As we have enabled the JMXAgent in our tutorial we can fire up the jconsole and connect to the following service URI: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi/camel. Notice that Camel will log at INFO level the JMX Connector URI:

In the screenshot below we can see the route and its performance metrics:

See Also

• Spring Remoting with JMS Example on Amin Abbaspour's Weblog

Tutorial - camel-example-reportincident

Introduction

Creating this tutorial was inspired by a real life use-case I discussed over the phone with a colleague. He was working at a client whom uses a heavy-weight integration platform from a very large vendor. He was in talks with developer shops to implement a new integration on this platform. His trouble was the shop tripled the price when they realized the platform of choice. So I was wondering how we could do this integration with Camel. Can it be done, without tripling the cost .

This tutorial is written during the development of the integration. I have decided to start off with a sample that isn't Camel's but standard Java and then plugin Camel as we goes. Just as when people needed to learn Spring you could consume it piece by piece, the same goes with Camel.

The target reader is person whom hasn't experience or just started using Camel.

Motivation for this tutorial

I wrote this tutorial motivated as Camel lacked an example application that was based on the web application deployment model. The entire world hasn't moved to pure OSGi deployments yet.

The use-case

The goal is to allow staff to report incidents into a central administration. For that they use client software where they report the incident and submit it to the central administration. As this is an integration in a transition phase the administration should get these incidents by email whereas they are manually added to the database. The client software should gather the incident and submit the information to the integration platform that in term will transform the report into an email and send it to the central administrator for manual processing.

The figure below illustrates this process. The end users reports the incidents using the client applications. The incident is sent to the central integration platform as webservice. The integration platform will process the incident and send an OK acknowledgment back to the client. Then the integration will transform the message to an email and send it to the administration mail server. The users in the administration will receive the emails and take it from there.

In EIP patterns

We distill the use case as EIP patterns:

Parts

This tutorial is divided into sections and parts:

Section A: Existing Solution, how to slowly use Camel

Part 1 - This first part explain how to setup the project and get a webservice exposed using Apache CXF. In fact we don't touch Camel yet.

Part 2 - Now we are ready to introduce Camel piece by piece (without using Spring or any XML configuration file) and create the full feature integration. This part will introduce different Camel's concepts and How we can build our solution using them like :

• CamelContext
• Endpoint, Exchange & Producer
• Components : Log, File

Part 3 - Continued from part 2 where we implement that last part of the solution with the event driven consumer and how to send the email through the Mail component.

Section B: The Camel Solution

Part 4 - We now turn into the path of Camel where it excels - the routing.
Part 5 - Is about how embed Camel with Spring and using CXF endpoints directly in Camel
Part 6 - Showing a alternative solution primarily using XML instead of Java code

Links

• Introduction
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

Part 1

Prerequisites

This tutorial uses the following frameworks:

• Maven 3.0.4
• Apache Camel 2.10.0
• Apache CXF 2.6.1
• Spring 3.0.7

Note: The sample project can be downloaded, see the resources section.

Initial Project Setup

We want the integration to be a standard .war application that can be deployed in any web container such as Tomcat, Jetty or even heavy weight application servers such as WebLogic or WebSphere. There fore we start off with the standard Maven webapp project that is created with the following long archetype command:

Notice that the groupId etc. doens't have to be org.apache.camel it can be com.mycompany.whatever. But I have used these package names as the example is an official part of the Camel distribution.

Then we have the basic maven folder layout. We start out with the webservice part where we want to use Apache CXF for the webservice stuff. So we add this to the pom.xml

Developing the WebService

As we want to develop webservice with the contract first approach we create our .wsdl file. As this is a example we have simplified the model of the incident to only include 8 fields. In real life the model would be a bit more complex, but not to much.

We put the wsdl file in the folder src/main/webapp/WEB-INF/wsdl and name the file report_incident.wsdl.

CXF wsdl2java

Then we integration the CXF wsdl2java generator in the pom.xml so we have CXF generate the needed POJO classes for our webservice contract.
However at first we must configure maven to live in the modern world of Java 1.6 so we must add this to the pom.xml

And then we can add the CXF wsdl2java code generator that will hook into the compile goal so its automatic run all the time:

You are now setup and should be able to compile the project. So running the mvn compile should run the CXF wsdl2java and generate the source code in the folder &{basedir}/target/generated/src/main/java that we specified in the pom.xml above. Since its in the target/generated/src/main/java maven will pick it up and include it in the build process.

Configuration of the web.xml

Next up is to configure the web.xml to be ready to use CXF so we can expose the webservice.
As Spring is the center of the universe, or at least is a very important framework in today's Java land we start with the listener that kick-starts Spring. This is the usual piece of code:

And then we have the CXF part where we define the CXF servlet and its URI mappings to which we have chosen that all our webservices should be in the path /webservices/

Then the last piece of the puzzle is to configure CXF, this is done in a spring XML that we link to fron the web.xml by the standard Spring contextConfigLocation property in the web.xml

We have named our CXF configuration file cxf-config.xml and its located in the root of the classpath. In Maven land that is we can have the cxf-config.xml file in the src/main/resources folder. We could also have the file located in the WEB-INF folder for instance <param-value>/WEB-INF/cxf-config.xml</param-value>.

Getting rid of the old jsp world

The maven archetype that created the basic folder structure also created a sample .jsp file index.jsp. This file src/main/webapp/index.jsp should be deleted.

Configuration of CXF

The cxf-config.xml is as follows:

The configuration is standard CXF and is documented at the Apache CXF website.

The 3 import elements is needed by CXF and they must be in the file.

Noticed that we have a spring bean reportIncidentEndpoint that is the implementation of the webservice endpoint we let CXF expose.
Its linked from the jaxws element with the implementator attribute as we use the # mark to identify its a reference to a spring bean. We could have stated the classname directly as implementor="org.apache.camel.example.reportincident.ReportIncidentEndpoint" but then we lose the ability to let the ReportIncidentEndpoint be configured by spring.
The address attribute defines the relative part of the URL of the exposed webservice. wsdlLocation is an optional parameter but for persons like me that likes contract-first we want to expose our own .wsdl contracts and not the auto generated by the frameworks, so with this attribute we can link to the real .wsdl file. The last stuff is needed by CXF as you could have several services so it needs to know which this one is. Configuring these is quite easy as all the information is in the wsdl already.

Implementing the ReportIncidentEndpoint

Phew after all these meta files its time for some java code so we should code the implementor of the webservice. So we fire up mvn compile to let CXF generate the POJO classes for our webservice and we are ready to fire up a Java editor.

You can use mvn idea:idea or mvn eclipse:eclipse to create project files for these editors so you can load the project. However IDEA has been smarter lately and can load a pom.xml directly.

As we want to quickly see our webservice we implement just a quick and dirty as it can get. At first beware that since its jaxws and Java 1.5 we get annotations for the money, but they reside on the interface so we can remove them from our implementations so its a nice plain POJO again:

We just output the person that invokes this webservice and returns a OK response. This class should be in the maven source root folder src/main/java under the package name org.apache.camel.example.reportincident. Beware that the maven archetype tool didn't create the src/main/java folder, so you should create it manually.

To test if we are home free we run mvn clean compile.

Running our webservice

Now that the code compiles we would like to run it inside a web container, for this purpose we make use of Jetty which we will bootstrap using it's plugin org.mortbay.jetty:maven-jetty-plugin:

Notice: We make use of the Jetty version being defined inside the Camel's Parent POM.

So to see if everything is in order we fire up jetty with mvn jetty:run and if everything is okay you should be able to access http://localhost:8080.
Jetty is smart that it will list the correct URI on the page to our web application, so just click on the link. This is smart as you don't have to remember the exact web context URI for your application - just fire up the default page and Jetty will help you.

So where is the damn webservice then? Well as we did configure the web.xml to instruct the CXF servlet to accept the pattern /webservices/* we should hit this URL to get the attention of CXF: http://localhost:8080/camel-example-reportincident/webservices.

 

Hitting the webservice

Now we have the webservice running in a standard .war application in a standard web container such as Jetty we would like to invoke the webservice and see if we get our code executed. Unfortunately this isn't the easiest task in the world - its not so easy as a REST URL, so we need tools for this. So we fire up our trusty webservice tool SoapUI and let it be the one to fire the webservice request and see the response.

Using SoapUI we sent a request to our webservice and we got the expected OK response and the console outputs the System.out so we are ready to code.

 

Remote Debugging

Okay a little sidestep but wouldn't it be cool to be able to debug your code when its fired up under Jetty? As Jetty is started from maven, we need to instruct maven to use debug mode.
Se we set the MAVEN_OPTS environment to start in debug mode and listen on port 5005.

Then you need to restart Jetty so its stopped with ctrl + c. Remember to start a new shell to pickup the new environment settings. And start jetty again.

Then we can from our IDE attach a remote debugger and debug as we want.
First we configure IDEA to attach to a remote debugger on port 5005:

 

Then we set a breakpoint in our code ReportIncidentEndpoint and hit the SoapUI once again and we are breaked at the breakpoint where we can inspect the parameters:

 

Adding a unit test

Oh so much hard work just to hit a webservice, why can't we just use an unit test to invoke our webservice? Yes of course we can do this, and that's the next step.
First we create the folder structure src/test/java and src/test/resources. We then create the unit test in the src/test/java folder.

Here we have a plain old JUnit class. As we want to test webservices we need to start and expose our webservice in the unit test before we can test it. And JAXWS has pretty decent methods to help us here, the code is simple as:

The Endpoint class is the javax.xml.ws.Endpoint that under the covers looks for a provider and in our case its CXF - so its CXF that does the heavy lifting of exposing out webservice on the given URL address. Since our class ReportIncidentEndpointImpl implements the interface ReportIncidentEndpoint that is decorated with all the jaxws annotations it got all the information it need to expose the webservice. Below is the CXF wsdl2java generated interface:

Next up is to create a webservice client so we can invoke our webservice. For this we actually use the CXF framework directly as its a bit more easier to create a client using this framework than using the JAXWS style. We could have done the same for the server part, and you should do this if you need more power and access more advanced features.

So now we are ready for creating a unit test. We have the server and the client. So we just create a plain simple unit test method as the usual junit style:

Now we are nearly there. But if you run the unit test with mvn test then it will fail. Why!!! Well its because that CXF needs is missing some dependencies during unit testing. In fact it needs the web container, so we need to add this to our pom.xml.

Well what is that, CXF also uses Jetty for unit test - well its just shows how agile, embedable and popular Jetty is.

So lets run our junit test with, and it reports:

Yep thats it for now. We have a basic project setup.

End of part 1

Thanks for being patient and reading all this more or less standard Maven, Spring, JAXWS and Apache CXF stuff. Its stuff that is well covered on the net, but I wanted a full fledged tutorial on a maven project setup that is web service ready with Apache CXF. We will use this as a base for the next part where we demonstrate how Camel can be digested slowly and piece by piece just as it was back in the times when was introduced and was learning the Spring framework that we take for granted today.

#Resources

• Apache CXF user guide

Links

• Introduction
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

Part 2

Adding Camel

In this part we will introduce Camel so we start by adding Camel to our pom.xml:

That's it, only one dependency for now.

Now we turn towards our webservice endpoint implementation where we want to let Camel have a go at the input we receive. As Camel is very non invasive its basically a .jar file then we can just grap Camel but creating a new instance of DefaultCamelContext that is the hearth of Camel its context.

In fact we create a constructor in our webservice and add this code:

Logging the "Hello World"

Here at first we want Camel to log the givenName and familyName parameters we receive, so we add the LogComponent with the key log. And we must start Camel before its ready to act.

Then we change the code in the method that is invoked by Apache CXF when a webservice request arrives. We get the name and let Camel have a go at it in the new method we create sendToCamel:

Next is the Camel code. At first it looks like there are many code lines to do a simple task of logging the name - yes it is. But later you will in fact realize this is one of Camels true power. Its concise API. Hint: The same code can be used for any component in Camel.

Okay there are code comments in the code block above that should explain what is happening. We run the code by invoking our unit test with maven mvn test, and we should get this log line:

Write to file - easy with the same code style

Okay that isn't to impressive, Camel can log Well I promised that the above code style can be used for any component, so let's store the payload in a file. We do this by adding the file component to the Camel context

And then we let camel write the payload to the file after we have logged, by creating a new method sendToCamelFile. We want to store the payload in filename with the incident id so we need this parameter also:

And then the code that is 99% identical. We have change the URI configuration when we create the endpoint as we pass in configuration parameters to the file component.
And then we need to set the output filename and this is done by adding a special header to the exchange. That's the only difference:

After running our unit test again with mvn test we have a output file in the target folder:

Fully java based configuration of endpoints

In the file example above the configuration was URI based. What if you want 100% java setter based style, well this is of course also possible. We just need to cast to the component specific endpoint and then we have all the setters available:

That's it. Now we have used the setters to configure the FileEndpoint that it should store the file in the folder target/subfolder. Of course Camel now stores the file in the subfolder.

Lessons learned

Okay I wanted to demonstrate how you can be in 100% control of the configuration and usage of Camel based on plain Java code with no hidden magic or special XML or other configuration files. Just add the camel-core.jar and you are ready to go.

You must have noticed that the code for sending a message to a given endpoint is the same for both the log and file, in fact any Camel endpoint. You as the client shouldn't bother with component specific code such as file stuff for file components, jms stuff for JMS messaging etc. This is what the Message Endpoint EIP pattern is all about and Camel solves this very very nice - a key pattern in Camel.

Reducing code lines

Now that you have been introduced to Camel and one of its masterpiece patterns solved elegantly with the Message Endpoint its time to give productive and show a solution in fewer code lines, in fact we can get it down to 5, 4, 3, 2 .. yes only 1 line of code.

The key is the ProducerTemplate that is a Spring'ish xxxTemplate based producer. Meaning that it has methods to send messages to any Camel endpoints. First of all we need to get hold of such a template and this is done from the CamelContext

Now we can use template for sending payloads to any endpoint in Camel. So all the logging gabble can be reduced to:

And the same goes for the file, but we must also send the header to instruct what the output filename should be:

Reducing even more code lines

Well we got the Camel code down to 1-2 lines for sending the message to the component that does all the heavy work of wring the message to a file etc. But we still got 5 lines to initialize Camel.

This can also be reduced. All the standard components in Camel is auto discovered on-the-fly so we can remove these code lines and we are down to 3 lines.

Okay back to the 3 code lines:

Later will we see how we can reduce this to ... in fact 0 java code lines. But the 3 lines will do for now.

Message Translation

Okay lets head back to the over goal of the integration. Looking at the EIP diagrams at the introduction page we need to be able to translate the incoming webservice to an email. Doing so we need to create the email body. When doing the message translation we could put up our sleeves and do it manually in pure java with a StringBuilder such as:

But as always it is a hardcoded template for the mail body and the code gets kinda ugly if the mail message has to be a bit more advanced. But of course it just works out-of-the-box with just classes already in the JDK.

Lets use a template language instead such as Apache Velocity. As Camel have a component for Velocity integration we will use this component. Looking at the Component List overview we can see that camel-velocity component uses the artifactId camel-velocity so therefore we need to add this to the pom.xml

And now we have a Spring conflict as Apache CXF is dependent on Spring 2.0.8 and camel-velocity is dependent on Spring 2.5.5. To remedy this we could wrestle with the pom.xml with excludes settings in the dependencies or just bring in another dependency camel-spring:

In fact camel-spring is such a vital part of Camel that you will end up using it in nearly all situations - we will look into how well Camel is seamless integration with Spring in part 3. For now its just another dependency.

We create the mail body with the Velocity template and create the file src/main/resources/MailBody.vm. The content in the MailBody.vm file is:

Letting Camel creating the mail body and storing it as a file is as easy as the following 3 code lines:

What is impressive is that we can just pass in our POJO object we got from Apache CXF to Velocity and it will be able to generate the mail body with this object in its context. Thus we don't need to prepare anything before we let Velocity loose and generate our mail body. Notice that the template method returns a object with out response. This object contains the mail body as a String object. We can cast to String if needed.

If we run our unit test with mvn test we can in fact see that Camel has produced the file and we can type its content:

First part of the solution

What we have seen here is actually what it takes to build the first part of the integration flow. Receiving a request from a webservice, transform it to a mail body and store it to a file, and return an OK response to the webservice. All possible within 10 lines of code. So lets wrap it up here is what it takes:

Okay I missed by one, its in fact only 9 lines of java code and 2 fields.

End of part 2

I know this is a bit different introduction to Camel to how you can start using it in your projects just as a plain java .jar framework that isn't invasive at all. I took you through the coding parts that requires 6 - 10 lines to send a message to an endpoint, buts it's important to show the Message Endpoint EIP pattern in action and how its implemented in Camel. Yes of course Camel also has to one liners that you can use, and will use in your projects for sending messages to endpoints. This part has been about good old plain java, nothing fancy with Spring, XML files, auto discovery, OGSi or other new technologies. I wanted to demonstrate the basic building blocks in Camel and how its setup in pure god old fashioned Java. There are plenty of eye catcher examples with one liners that does more than you can imagine - we will come there in the later parts.

Okay part 3 is about building the last pieces of the solution and now it gets interesting since we have to wrestle with the event driven consumer.
Brew a cup of coffee, tug the kids and kiss the wife, for now we will have us some fun with the Camel. See you in part 3.

Links

• Introduction
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

Part 3

Recap

Lets just recap on the solution we have now:

This completes the first part of the solution: receiving the message using webservice, transform it to a mail body and store it as a text file.
What is missing is the last part that polls the text files and send them as emails. Here is where some fun starts, as this requires usage of the Event Driven Consumer EIP pattern to react when new files arrives. So lets see how we can do this in Camel. There is a saying: Many roads lead to Rome, and that is also true for Camel - there are many ways to do it in Camel.

Adding the Event Driven Consumer

We want to add the consumer to our integration that listen for new files, we do this by creating a private method where the consumer code lives. We must register our consumer in Camel before its started so we need to add, and there fore we call the method addMailSenderConsumer in the constructor below:

The consumer needs to be consuming from an endpoint so we grab the endpoint from Camel we want to consume. It's file://target/subfolder. Don't be fooled this endpoint doesn't have to 100% identical to the producer, i.e. the endpoint we used in the previous part to create and store the files. We could change the URL to include some options, and to make it more clear that it's possible we setup a delay value to 10 seconds, and the first poll starts after 2 seconds. This is done by adding ?consumer.delay=10000&consumer.initialDelay=2000 to the URL.

When we have the endpoint we can create the consumer (just as in part 1 where we created a producer}. Creating the consumer requires a Processor where we implement the java code what should happen when a message arrives. To get the mail body as a String object we can use the getBody method where we can provide the type we want in return.

Sending the email is still left to be implemented, we will do this later. And finally we must remember to start the consumer otherwise its not active and won't listen for new files.

Before we test it we need to be aware that our unit test is only catering for the first part of the solution, receiving the message with webservice, transforming it using Velocity and then storing it as a file - it doesn't test the Event Driven Consumer we just added. As we are eager to see it in action, we just do a common trick adding some sleep in our unit test, that gives our Event Driven Consumer time to react and print to System.out. We will later refine the test:

We run the test with mvn clean test and have eyes fixed on the console output.
During all the output in the console, we see that our consumer has been triggered, as we want.

Sending the email

Sending the email requires access to a SMTP mail server, but the implementation code is very simple:

And just invoke the method from our consumer:

Unit testing mail

For unit testing the consumer part we will use a mock mail framework, so we add this to our pom.xml:

Then we prepare our integration to run with or without the consumer enabled. We do this to separate the route into the two parts:

• receive the webservice, transform and save mail file and return OK as repose
• the consumer that listen for mail files and send them as emails

So we change the constructor code a bit:

Then remember to change the ReportIncidentEndpointTest to pass in false in the ReportIncidentEndpointImpl constructor.
And as always run mvn clean test to be sure that the latest code changes works.

Adding new unit test

We are now ready to add a new unit test that tests the consumer part so we create a new test class that has the following code structure:

As we want to test the consumer that it can listen for files, read the file content and send it as an email to our mailbox we will test it by asserting that we receive 1 mail in our mailbox and that the mail is the one we expect. To do so we need to grab the mailbox with the mockmail API. This is done as simple as:

How do we trigger the consumer? Well by creating a file in the folder it listen for. So we could use plain java.io.File API to create the file, but wait isn't there an smarter solution? ... yes Camel of course. Camel can do amazing stuff in one liner codes with its ProducerTemplate, so we need to get a hold of this baby. We expose this template in our ReportIncidentEndpointImpl but adding this getter:

Then we can use the template to create the file in one code line:

Then we just need to wait a little for the consumer to kick in and do its work and then we should assert that we got the new mail. Easy as just:

The final class for the unit test is:

End of part 3

Okay we have reached the end of part 3. For now we have only scratched the surface of what Camel is and what it can do. We have introduced Camel into our integration piece by piece and slowly added more and more along the way. And the most important is: you as the developer never lost control. We hit a sweet spot in the webservice implementation where we could write our java code. Adding Camel to the mix is just to use it as a regular java code, nothing magic. We were in control of the flow, we decided when it was time to translate the input to a mail body, we decided when the content should be written to a file. This is very important to not lose control, that the bigger and heavier frameworks tend to do. No names mentioned, but boy do developers from time to time dislike these elephants. And Camel is no elephant.

I suggest you download the samples from part 1 to 3 and try them out. It is great basic knowledge to have in mind when we look at some of the features where Camel really excel - the routing domain language.

From part 1 to 3 we touched concepts such as::

• Endpoint
• URI configuration
• Consumer
• Producer
• Event Driven Consumer
• Component
• CamelContext
• ProducerTemplate
• Processor
• Type Converter

Links

• Introduction
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

Part 4

Introduction

This section is about regular Camel. The examples presented here in this section is much more in common of all the examples we have in the Camel documentation.

Routing

Camel is particular strong as a light-weight and agile routing and mediation framework. In this part we will introduce the routing concept and how we can introduce this into our solution.
Looking back at the figure from the Introduction page we want to implement this routing. Camel has support for expressing this routing logic using Java as a DSL (Domain Specific Language). In fact Camel also has DSL for XML and Scala. In this part we use the Java DSL as its the most powerful and all developers know Java. Later we will introduce the XML version that is very well integrated with Spring.

Before we jump into it, we want to state that this tutorial is about Developers not loosing control. In my humble experience one of the key fears of developers is that they are forced into a tool/framework where they loose control and/or power, and the possible is now impossible. So in this part we stay clear with this vision and our starting point is as follows:

• We have generated the webservice source code using the CXF wsdl2java generator and we have our ReportIncidentEndpointImpl.java file where we as a Developer feels home and have the power.

So the starting point is:

Yes we have a simple plain Java class where we have the implementation of the webservice. The cursor is blinking at the WE ARE HERE block and this is where we feel home. More or less any Java Developers have implemented webservices using a stack such as: Apache AXIS, Apache CXF or some other quite popular framework. They all allow the developer to be in control and implement the code logic as plain Java code. Camel of course doesn't enforce this to be any different. Okay the boss told us to implement the solution from the figure in the Introduction page and we are now ready to code.

RouteBuilder

RouteBuilder is the hearth in Camel of the Java DSL routing. This class does all the heavy lifting of supporting EIP verbs for end-users to express the routing. It does take a little while to get settled and used to, but when you have worked with it for a while you will enjoy its power and realize it is in fact a little language inside Java itself. Camel is the only integration framework we are aware of that has Java DSL, all the others are usually only XML based.

As an end-user you usually use the RouteBuilder as of follows:

• create your own Route class that extends RouteBuilder
• implement your routing DSL in the configure method

So we create a new class ReportIncidentRoutes and implement the first part of the routing:

What to notice here is the configure method. Here is where all the action is. Here we have the Java DSL langauge, that is expressed using the fluent builder syntax that is also known from Hibernate when you build the dynamic queries etc. What you do is that you can stack methods separating with the dot.

In the example above we have a very common routing, that can be distilled from pseudo verbs to actual code with:

• from A to B
• From Endpoint A To Endpoint B
• from("endpointA").to("endpointB")
• from("direct:start").to("velocity:MailBody.vm");

from("direct:start") is the consumer that is kick-starting our routing flow. It will wait for messages to arrive on the direct queue and then dispatch the message.
to("velocity:MailBody.vm") is the producer that will receive a message and let Velocity generate the mail body response.

So what we have implemented so far with our ReportIncidentRoutes RouteBuilder is this part of the picture:

Adding the RouteBuilder

Now we have our RouteBuilder we need to add/connect it to our CamelContext that is the hearth of Camel. So turning back to our webservice implementation class ReportIncidentEndpointImpl we add this constructor to the code, to create the CamelContext and add the routes from our route builder and finally to start it.

Okay how do you use the routes then? Well its just as before we use a ProducerTemplate to send messages to Endpoints, so we just send to the direct:start endpoint and it will take it from there.
So we implement the logic in our webservice operation:

Notice that we get the producer template using the createProducerTemplate method on the CamelContext. Then we send the input parameters to the direct:start endpoint and it will route it to the velocity endpoint that will generate the mail body. Since we use direct as the consumer endpoint (=from) and its a synchronous exchange we will get the response back from the route. And the response is of course the output from the velocity endpoint.

We have now completed this part of the picture:

Unit testing

Now is the time we would like to unit test what we got now. So we call for camel and its great test kit. For this to work we need to add it to the pom.xml

After adding it to the pom.xml you should refresh your Java Editor so it pickups the new jar. Then we are ready to create out unit test class.
We create this unit test skeleton, where we extend this class ContextTestSupport

ContextTestSupport is a supporting unit test class for much easier unit testing with Apache Camel. The class is extending JUnit TestCase itself so you get all its glory. What we need to do now is to somehow tell this unit test class that it should use our route builder as this is the one we gonna test. So we do this by implementing the createRouteBuilder method.

That is easy just return an instance of our route builder and this unit test will use our routes.

We then code our unit test method that sends a message to the route and assert that its transformed to the mail body using the Velocity template.

Adding the File Backup

The next piece of puzzle that is missing is to store the mail body as a backup file. So we turn back to our route and the EIP patterns. We use the Pipes and Filters pattern here to chain the routing as:

Notice that we just add a 2nd .to on the newline. Camel will default use the Pipes and Filters pattern here when there are multi endpoints chained liked this. We could have used the pipeline verb to let out stand out that its the Pipes and Filters pattern such as:

But most people are using the multi .to style instead.

We re-run out unit test and verifies that it still passes:

But hey we have added the file producer endpoint and thus a file should also be created as the backup file. If we look in the target/subfolder we can see that something happened.
On my humble laptop it created this folder: target\subfolder\ID-claus-acer. So the file producer create a sub folder named ID-claus-acer what is this? Well Camel auto generates an unique filename based on the unique message id if not given instructions to use a fixed filename. In fact it creates another sub folder and name the file as: target\subfolder\ID-claus-acer\3750-1219148558921\1-0 where 1-0 is the file with the mail body. What we want is to use our own filename instead of this auto generated filename. This is archived by adding a header to the message with the filename to use. So we need to add this to our route and compute the filename based on the message content.

Setting the filename

For starters we show the simple solution and build from there. We start by setting a constant filename, just to verify that we are on the right path, to instruct the file producer what filename to use. The file producer uses a special header FileComponent.HEADER_FILE_NAME to set the filename.

What we do is to send the header when we "kick-start" the routing as the header will be propagated from the direct queue to the file producer. What we need to do is to use the ProducerTemplate.sendBodyAndHeader method that takes both a body and a header. So we change out webservice code to include the filename also:

However we could also have used the route builder itself to configure the constant filename as shown below:

But Camel can be smarter and we want to dynamic set the filename based on some of the input parameters, how can we do this?
Well the obvious solution is to compute and set the filename from the webservice implementation, but then the webservice implementation has such logic and we want this decoupled, so we could create our own POJO bean that has a method to compute the filename. We could then instruct the routing to invoke this method to get the computed filename. This is a string feature in Camel, its Bean binding. So lets show how this can be done:

Using Bean Language to compute the filename

First we create our plain java class that computes the filename, and it has 100% no dependencies to Camel what so ever.

The class is very simple and we could easily create unit tests for it to verify that it works as expected. So what we want now is to let Camel invoke this class and its generateFilename with the input parameters and use the output as the filename. Pheeeww is this really possible out-of-the-box in Camel? Yes it is. So lets get on with the show. We have the code that computes the filename, we just need to call it from our route using the Bean Language:

Notice that we use the bean language where we supply the class with our bean to invoke. Camel will instantiate an instance of the class and invoke the suited method. For completeness and ease of code readability we add the method name as the 2nd parameter

Then other developers can understand what the parameter is, instead of null.

Now we have a nice solution, but as a sidetrack I want to demonstrate the Camel has other languages out-of-the-box, and that scripting language is a first class citizen in Camel where it etc. can be used in content based routing. However we want it to be used for the filename generation.

Whatever worked for you we have now implemented the backup of the data files:

Sending the email

What we need to do before the solution is completed is to actually send the email with the mail body we generated and stored as a file. In the previous part we did this with a File consumer, that we manually added to the CamelContext. We can do this quite easily with the routing.

The last 3 lines of code does all this. It adds a file consumer from("file://target/subfolder"), sets the mail subject, and finally send it as an email.

The DSL is really powerful where you can express your routing integration logic.
So we completed the last piece in the picture puzzle with just 3 lines of code.

We have now completed the integration:

Conclusion

We have just briefly touched the routing in Camel and shown how to implement them using the fluent builder syntax in Java. There is much more to the routing in Camel than shown here, but we are learning step by step. We continue in part 5. See you there.

Links

• Introduction
• Part 1
• Part 2
• Part 3
• Part 4
• Part 5
• Part 6

Better JMS Transport for CXF Webservice using Apache Camel

Configuring JMS in Apache CXF before Version 2.1.3 is possible but not really easy or nice. This article shows how to use Apache Camel to provide a better JMS Transport for CXF.

Update: Since CXF 2.1.3 there is a new way of configuring JMS (Using the JMSConfigFeature). It makes JMS config for CXF as easy as with Camel. Using Camel for JMS is still a good idea if you want to use the rich feature of Camel for routing and other Integration Scenarios that CXF does not support.

You can find the original announcement for this Tutorial and some additional info on Christian Schneider´s Blog

So how to connect Apache Camel and CXF

The best way to connect Camel and CXF is using the Camel transport for CXF. This is a camel module that registers with cxf as a new transport. It is quite easy to configure.

This bean registers with CXF and provides a new transport prefix camel:// that can be used in CXF address configurations. The bean references a bean cxf which will be already present in your config. The other refrenceis a camel context. We will later define this bean to provide the routing config.

How is JMS configured in Camel

In camel you need two things to configure JMS. A ConnectionFactory and a JMSComponent. As ConnectionFactory you can simply set up the normal Factory your JMS provider offers or bind a JNDI ConnectionFactory. In this example we use the ConnectionFactory provided by ActiveMQ.

Then we set up the JMSComponent. It offers a new transport prefix to camel that we simply call jms. If we need several JMSComponents we can differentiate them by their name.

You can find more details about the JMSComponent at the Camel Wiki. For example you find the complete configuration options and a JNDI sample there.

Setting up the CXF client

We will configure a simple CXF webservice client. It will use stub code generated from a wsdl. The webservice client will be configured to use JMS directly. You can also use a direct: Endpoint and do the routing to JMS in the Camel Context.

We explicitly configure serviceName and endpointName so they are not read from the wsdl. The names we use are arbitrary and have no further function but we set them to look nice. The serviceclass points to the service interface that was generated from the wsdl. Now the important thing is address. Here we tell cxf to use the camel transport, use the JmsComponent who registered the prefix "jms" and use the queue "CustomerService".

Setting up the CamelContext

As we do not need additional routing an empty CamelContext bean will suffice.

Running the Example

• Download the example project here

• Follow the readme.txt

Conclusion

As you have seen in this example you can use Camel to connect services to JMS easily while being able to also use the rich integration features of Apache Camel.

Tutorial using Axis 1.4 with Apache Camel

Prerequisites

This tutorial uses Maven 2 to setup the Camel project and for dependencies for artifacts.

Distribution

This sample is distributed with the Camel 1.5 distribution as examples/camel-example-axis.

Introduction

Apache Axis is/was widely used as a webservice framework. So in line with some of the other tutorials to demonstrate how Camel is not an invasive framework but is flexible and integrates well with existing solution.

We have an existing solution that exposes a webservice using Axis 1.4 deployed as web applications. This is a common solution. We use contract first so we have Axis generated source code from an existing wsdl file. Then we show how we introduce Spring and Camel to integrate with Axis.

This tutorial uses the following frameworks:

• Maven 2.0.9
• Apache Camel 1.5.0
• Apache Axis 1.4
• Spring 2.5.5

Setting up the project to run Axis

This first part is about getting the project up to speed with Axis. We are not touching Camel or Spring at this time.

Maven 2

Axis dependencies is available for maven 2 so we configure our pom.xml as:

Then we need to configure maven to use Java 1.5 and the Axis maven plugin that generates the source code based on the wsdl file:

wsdl

We use the same .wsdl file as the Tutorial-Example-ReportIncident and copy it to src/main/webapp/WEB-INF/wsdl

Configuring Axis

Okay we are now setup for the contract first development and can generate the source file. For now we are still only using standard Axis and not Spring nor Camel. We still need to setup Axis as a web application so we configure the web.xml in src/main/webapp/WEB-INF/web.xml as:

The web.xml just registers Axis servlet that is handling the incoming web requests to its servlet mapping. We still need to configure Axis itself and this is done using its special configuration file server-config.wsdd. We nearly get this file for free if we let Axis generate the source code so we run the maven goal:

The tool will generate the source code based on the wsdl and save the files to the following folder:

This is standard Axis and so far no Camel or Spring has been touched. To implement our webservice we will add our code, so we create a new class AxisReportIncidentService that implements the port type interface where we can implement our code logic what happens when the webservice is invoked.

Now we need to configure Axis itself and this is done using its server-config.wsdd file. We nearly get this for for free from the auto generated code, we copy the stuff from deploy.wsdd and made a few modifications:

The globalConfiguration and transport is not in the deploy.wsdd file so you gotta write that yourself. The service is a 100% copy from deploy.wsdd. Axis has more configuration to it than shown here, but then you should check the Axis documentation.

What we need to do now is important, as we need to modify the above configuration to use our webservice class than the default one, so we change the classname parameter to our class AxisReportIncidentService:

Running the Example

Now we are ready to run our example for the first time, so we use Jetty as the quick web container using its maven command:

Then we can hit the web browser and enter this URL: http://localhost:8080/camel-example-axis/services and you should see the famous Axis start page with the text And now... Some Services.

Clicking on the .wsdl link shows the wsdl file, but what. It's an auto generated one and not our original .wsdl file. So we need to fix this ASAP and this is done by configuring Axis in the server-config.wsdd file:

We do this by adding the wsdlFile tag in the service element where we can point to the real .wsdl file.

Integrating Spring

First we need to add its dependencies to the pom.xml.

Spring is integrated just as it would like to, we add its listener to the web.xml and a context parameter to be able to configure precisely what spring xml files to use:

Next is to add a plain spring XML file named axis-example-context.xml in the src/main/resources folder.

The spring XML file is currently empty. We hit jetty again with mvn jetty:run just to make sure Spring was setup correctly.

Using Spring

We would like to be able to get hold of the Spring ApplicationContext from our webservice so we can get access to the glory spring, but how do we do this? And our webservice class AxisReportIncidentService is created and managed by Axis we want to let Spring do this. So we have two problems.

We solve these problems by creating a delegate class that Axis creates, and this delegate class gets hold on Spring and then gets our real webservice as a spring bean and invoke the service.

First we create a new class that is 100% independent from Axis and just a plain POJO. This is our real service.

So now we need to get from AxisReportIncidentService to this one ReportIncidentService using Spring. Well first of all we add our real service to spring XML configuration file so Spring can handle its lifecycle:

And then we need to modify AxisReportIncidentService to use Spring to lookup the spring bean id="incidentservice" and delegate the call. We do this by extending the spring class org.springframework.remoting.jaxrpc.ServletEndpointSupport so the refactored code is:

To see if everything is okay we run mvn jetty:run.

In the code above we get hold of our service at each request by looking up in the application context. However Spring also supports an init method where we can do this once. So we change the code to:

So now we have integrated Axis with Spring and we are ready for Camel.

Integrating Camel

Again the first step is to add the dependencies to the maven pom.xml file:

Now that we have integrated with Spring then we easily integrate with Camel as Camel works well with Spring.

We choose to integrate Camel in the Spring XML file so we add the camel namespace and the schema location:

CamelContext

CamelContext is the heart of Camel its where all the routes, endpoints, components, etc. is registered. So we setup a CamelContext and the spring XML files looks like:

Store a file backup

We want to store the web service request as a file before we return a response. To do this we want to send the file content as a message to an endpoint that produces the file. So we need to do two steps:

• configure the file backup endpoint
• send the message to the endpoint

The endpoint is configured in spring XML so we just add it as:

In the CamelContext we have defined our endpoint with the id backup and configured it use the URL notation that we know from the internet. Its a file scheme that accepts a context and some options. The contest is target and its the folder to store the file. The option is just as the internet with ? and & for subsequent options. We configure it to not append, meaning than any existing file will be overwritten. See the File component for options and how to use the camel file endpoint.

Next up is to be able to send a message to this endpoint. The easiest way is to use a ProducerTemplate. A ProducerTemplate is inspired by Spring template pattern with for instance JmsTemplate or JdbcTemplate in mind. The template that all the grunt work and exposes a simple interface to the end-user where he/she can set the payload to send. Then the template will do proper resource handling and all related issues in that regard. But how do we get hold of such a template? Well the CamelContext is able to provide one. This is done by configuring the template on the camel context in the spring XML as:

Then we can expose a ProducerTemplate property on our service with a setter in the Java code as:

And then let Spring handle the dependency inject as below:

Now we are ready to use the producer template in our service to send the payload to the endpoint. The template has many sendXXX methods for this purpose. But before we send the payload to the file endpoint we must also specify what filename to store the file as. This is done by sending meta data with the payload. In Camel metadata is sent as headers. Headers is just a plain Map<String, Object>. So if we needed to send several metadata then we could construct an ordinary HashMap and put the values in there. But as we just need to send one header with the filename Camel has a convenient send method sendBodyAndHeader so we choose this one.

The template in the code above uses 4 parameters:

• the endpoint name, in this case the id referring to the endpoint defined in Spring XML in the camelContext element.
• the payload, can be any kind of object
• the key for the header, in this case a Camel keyword to set the filename
• and the value for the header

Running the example

We start our integration with maven using mvn jetty:run. Then we open a browser and hit http://localhost:8080. Jetty is so smart that it display a frontpage with links to the deployed application so just hit the link and you get our application. Now we hit append /services to the URL to access the Axis frontpage. The URL should be http://localhost:8080/camel-example-axis/services.

You can then test it using a web service test tools such as SoapUI.
Hitting the service will output to the console

And there should be a file in the target subfolder.

Unit Testing

We would like to be able to unit test our ReportIncidentService class. So we add junit to the maven dependency:

And then we create a plain junit testcase for our service class.

Then we can run the test with maven using: mvn test. But we will get a failure:

What is the problem? Well our service uses a CamelProducer (the template) to send a message to the file endpoint so the message will be stored in a file. What we need is to get hold of such a producer and inject it on our service, by calling the setter.

Since Camel is very light weight and embedable we are able to create a CamelContext and add the endpoint in our unit test code directly. We do this to show how this is possible:

So now we are ready to set the ProducerTemplate on our service, and we get a hold of that baby from the CamelContext as:

And this time when we run the unit test its a success:

We would like to test that the file exists so we add these two lines to our test method:

Smarter Unit Testing with Spring

The unit test above requires us to assemble the Camel pieces manually in java code. What if we would like our unit test to use our spring configuration file axis-example-context.xml where we already have setup the endpoint. And of course we would like to test using this configuration file as this is the real file we will use. Well hey presto the xml file is a spring ApplicationContext file and spring is able to load it, so we go the spring path for unit testing. First we add the spring-test jar to our maven dependency:

And then we refactor our unit test to be a standard spring unit class. What we need to do is to extend AbstractJUnit38SpringContextTests instead of TestCase in our unit test. Since Spring 2.5 embraces annotations we will use one as well to instruct what our xml configuration file is located:

What we must remember to add is the classpath: prefix as our xml file is located in src/main/resources. If we omit the prefix then Spring will by default try to locate the xml file in the current package and that is org.apache.camel.example.axis. If the xml file is located outside the classpath you can use file: prefix instead. So with these two modifications we can get rid of all the setup and teardown code we had before and now we will test our real configuration.

The last change is to get hold of the producer template and now we can just refer to the bean id it has in the spring xml file:

So we get hold of it by just getting it from the spring ApplicationContext as all spring users is used to do:

Now our unit test is much better, and a real power of Camel is that is fits nicely with Spring and you can use standard Spring'ish unit test to test your Camel applications as well.

Unit Test calling WebService

What if you would like to execute a unit test where you send a webservice request to the AxisReportIncidentService how do we unit test this one? Well first of all the code is merely just a delegate to our real service that we have just tested, but nevertheless its a good question and we would like to know how. Well the answer is that we can exploit that fact that Jetty is also a slim web container that can be embedded anywhere just as Camel can. So we add this to our pom.xml:

Then we can create a new class AxisReportIncidentServiceTest to unit test with Jetty. The code to setup Jetty is shown below with code comments:

Now we just need to send the incident as a webservice request using Axis. So we add the following code:

And now we have an unittest that sends a webservice request using good old Axis.

Annotations

Both Camel and Spring has annotations that can be used to configure and wire trivial settings more elegantly. Camel has the endpoint annotation @EndpointInjected that is just what we need. With this annotation we can inject the endpoint into our service. The annotation takes either a name or uri parameter. The name is the bean id in the Registry. The uri is the URI configuration for the endpoint. Using this you can actually inject an endpoint that you have not defined in the camel context. As we have defined our endpoint with the id backup we use the name parameter.

Camel is smart as @EndpointInjected supports different kinds of object types. We like the ProducerTemplate so we just keep it as it is.
Since we use annotations on the field directly we do not need to set the property in the spring xml file so we change our service bean:

Running the unit test with mvn test reveals that it works nicely.

And since we use the @EndpointInjected that refers to the endpoint with the id backup directly we can loose the template tag in the xml, so its shorter:

And the final touch we can do is that since the endpoint is injected with concrete endpoint to use we can remove the "backup" name parameter when we send the message. So we change from:

To without the name:

Then we avoid to duplicate the name and if we rename the endpoint name then we don't forget to change it in the code also.

The End

This tutorial hasn't really touched the one of the key concept of Camel as a powerful routing and mediation framework. But we wanted to demonstrate its flexibility and that it integrates well with even older frameworks such as Apache Axis 1.4.

Check out the other tutorials on Camel and the other examples.

Note that the code shown here also applies to Camel 1.4 so actually you can get started right away with the released version of Camel. As this time of writing Camel 1.5 is work in progress.

See Also

• Tutorials
• Examples

Tutorial on using Camel in a Web Application

Camel has been designed to work great with the Spring framework; so if you are already a Spring user you can think of Camel as just a framework for adding to your Spring XML files.

So you can follow the usual Spring approach to working with web applications; namely to add the standard Spring hook to load a /WEB-INF/applicationContext.xml file. In that file you can include your usual Camel XML configuration.

Step1: Edit your web.xml

To enable spring add a context loader listener to your /WEB-INF/web.xml file

This will cause Spring to boot up and look for the /WEB-INF/applicationContext.xml file.

Step 2: Create a /WEB-INF/applicationContext.xml file

Now you just need to create your Spring XML file and add your camel routes or configuration.

For example

Then boot up your web application and you're good to go!

Hints and Tips

If you use Maven to build your application your directory tree will look like this...

You should update your Maven pom.xml to enable WAR packaging/naming like this...

To enable more rapid development we highly recommend the jetty:run maven plugin.

Please refer to the help for more information on using jetty:run - but briefly if you add the following to your pom.xml

Then you can run your web application as follows

Then Jetty will also monitor your target/classes directory and your src/main/webapp directory so that if you modify your spring XML, your web.xml or your java code the web application will be restarted, re-creating your Camel routes.

If your unit tests take a while to run, you could miss them out when running your web application via

Tutorial Business Partners

Background and Introduction

Business Background

So there's a company, which we'll call Acme. Acme sells widgets, in a fairly unusual way. Their customers are responsible for telling Acme what they purchased. The customer enters into their own systems (ERP or whatever) which widgets they bought from Acme. Then at some point, their systems emit a record of the sale which needs to go to Acme so Acme can bill them for it. Obviously, everyone wants this to be as automated as possible, so there needs to be integration between the customer's system and Acme.

Sadly, Acme's sales people are, technically speaking, doormats. They tell all their prospects, "you can send us the data in whatever format, using whatever protocols, whatever. You just can't change once it's up and running."

The result is pretty much what you'd expect. Taking a random sample of 3 customers:

• Customer 1: XML over FTP
• Customer 2: CSV over HTTP
• Customer 3: Excel via e-mail

Now on the Acme side, all this has to be converted to a canonical XML format and submitted to the Acme accounting system via JMS. Then the Acme accounting system does its stuff and sends an XML reply via JMS, with a summary of what it processed (e.g. 3 line items accepted, line item #2 in error, total invoice $123.45). Finally, that data needs to be formatted into an e-mail, and sent to a contact at the customer in question ("Dear Joyce, we received an invoice on 1/2/08. We accepted 3 line items totaling $123.45, though there was an error with line items #2 [invalid quantity ordered]. Thank you for your business. Love, Acme.").

So it turns out Camel can handle all this:

• Listen for HTTP, e-mail, and FTP files
• Grab attachments from the e-mail messages
• Convert XML, XLS, and CSV files to a canonical XML format
• read and write JMS messages
• route based on company ID
• format e-mails using Velocity templates
• send outgoing e-mail messages

Tutorial Background

This tutorial will cover all that, plus setting up tests along the way.

Before starting, you should be familiar with:

• Camel concepts including the CamelContext, Routes, Components and Endpoints, and Enterprise Integration Patterns
• Configuring Camel with the XML or Java DSL

You'll learn:

• How to set up a Maven build for a Camel project
• How to transform XML, CSV, and Excel data into a standard XML format with Camel
  • How to write POJOs (Plain Old Java Objects), Velocity templates, and XSLT stylesheets that are invoked by Camel routes for message transformation
• How to configure simple and complex Routes in Camel, using either the XML or the Java DSL format
• How to set up unit tests that load a Camel configuration and test Camel routes
• How to use Camel's Data Formats to automatically convert data between Java objects and XML, CSV files, etc.
• How to send and receive e-mail from Camel
• How to send and receive JMS messages from Camel
• How to use Enterprise Integration Patterns including Message Router and Pipes and Filters
  • How to use various languages to express content-based routing rules in Camel
• How to deal with Camel messages, headers, and attachments

You may choose to treat this as a hands-on tutorial, and work through building the code and configuration files yourself. Each of the sections gives detailed descriptions of the steps that need to be taken to get the components and routes working in Camel, and takes you through tests to make sure they are working as expected.

But each section also links to working copies of the source and configuration files, so if you don't want the hands-on approach, you can simply review and/or download the finished files.

High-Level Diagram

Here's more or less what the integration process looks like.

First, the input from the customers to Acme:

And then, the output from Acme to the customers:

Tutorial Tasks

To get through this scenario, we're going to break it down into smaller pieces, implement and test those, and then try to assemble the big scenario and test that.

Here's what we'll try to accomplish:

1. Create a Maven build for the project
2. Get sample files for the customer Excel, CSV, and XML input
3. Get a sample file for the canonical XML format that Acme's accounting system uses
4. Create an XSD for the canonical XML format
5. Create JAXB POJOs corresponding to the canonical XSD
6. Create an XSLT stylesheet to convert the Customer 1 (XML over FTP) messages to the canonical format
7. Create a unit test to ensure that a simple Camel route invoking the XSLT stylesheet works
8. Create a POJO that converts a List<List<String>> to the above JAXB POJOs
   • Note that Camel can automatically convert CSV input to a List of Lists of Strings representing the rows and columns of the CSV, so we'll use this POJO to handle Customer 2 (CSV over HTTP)
9. Create a unit test to ensure that a simple Camel route invoking the CSV processing works
10. Create a POJO that converts a Customer 3 Excel file to the above JAXB POJOs (using POI to read Excel)
11. Create a unit test to ensure that a simple Camel route invoking the Excel processing works
12. Create a POJO that reads an input message, takes an attachment off the message, and replaces the body of the message with the attachment
    • This is assuming for Customer 3 (Excel over e-mail) that the e-mail contains a single Excel file as an attachment, and the actual e-mail body is throwaway
13. Build a set of Camel routes to handle the entire input (Customer -> Acme) side of the scenario.
14. Build unit tests for the Camel input.
15. TODO: Tasks for the output (Acme -> Customer) side of the scenario

Let's Get Started!

Step 1: Initial Maven build

We'll use Maven for this project as there will eventually be quite a few dependencies and it's nice to have Maven handle them for us. You should have a current version of Maven (e.g. 2.0.9) installed.

You can start with a pretty empty project directory and a Maven POM file, or use a simple JAR archetype to create one.

Here's a sample POM. We've added a dependency on camel-core, and set the compile version to 1.5 (so we can use annotations):

Step 2: Get Sample Files

You can make up your own if you like, but here are the "off the shelf" ones. You can save yourself some time by downloading these to src/test/resources in your Maven project.

• Customer 1 (XML): input-customer1.xml
• Customer 2 (CSV): input-customer2.csv
• Customer 3 (Excel): input-customer3.xls
• Canonical Acme XML Request: canonical-acme-request.xml
• Canonical Acme XML Response: TODO

If you look at these files, you'll see that the different input formats use different field names and/or ordering, because of course the sales guys were totally OK with that. Sigh.

Step 3: XSD and JAXB Beans for the Canonical XML Format

Here's the sample of the canonical XML file:

If you're ambitions, you can write your own XSD (XML Schema) for files that look like this, and save it to src/main/xsd.

Solution: If not, you can download mine, and save that to save it to src/main/xsd.

Generating JAXB Beans

Down the road we'll want to deal with the XML as Java POJOs. We'll take a moment now to set up those XML binding POJOs. So we'll update the Maven POM to generate JAXB beans from the XSD file.

We need a dependency:

And a plugin configured:

That should do it (it automatically looks for XML Schemas in src/main/xsd to generate beans for). Run mvn install and it should emit the beans into target/generated-sources/jaxb. Your IDE should see them there, though you may need to update the project to reflect the new settings in the Maven POM.

Step 4: Initial Work on Customer 1 Input (XML over FTP)

To get a start on Customer 1, we'll create an XSLT template to convert the Customer 1 sample file into the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the XSLT template is valid and can be run safely in Camel.

Create an XSLT template

Start with the Customer 1 sample input. You want to create an XSLT template to generate XML like the canonical XML sample above – an invoice element with line-item elements (one per item in the original XML document). If you're especially clever, you can populate the current date and order total elements too.

Solution: My sample XSLT template isn't that smart, but it'll get you going if you don't want to write one of your own.

Create a unit test

Here's where we get to some meaty Camel work. We need to:

• Set up a unit test
• That loads a Camel configuration
• That has a route invoking our XSLT
• Where the test sends a message to the route
• And ensures that some XML comes out the end of the route

The easiest way to do this is to set up a Spring context that defines the Camel stuff, and then use a base unit test class from Spring that knows how to load a Spring context to run tests against. So, the procedure is:

Set Up a Skeletal Camel/Spring Unit Test

1. Add dependencies on Camel-Spring, and the Spring test JAR (which will automatically bring in JUnit 3.8.x) to your POM:
2. Create a new unit test class in src/test/java/your-package-here, perhaps called XMLInputTest.java
3. Make the test extend Spring's AbstractJUnit38SpringContextTests class, so it can load a Spring context for the test
4. Create a Spring context configuration file in src/test/resources, perhaps called XMLInputTest-context.xml
5. In the unit test class, use the class-level @ContextConfiguration annotation to indicate that a Spring context should be loaded
   • By default, this looks for a Context configuration file called TestClassName-context.xml in a subdirectory corresponding to the package of the test class. For instance, if your test class was org.apache.camel.tutorial.XMLInputTest, it would look for org/apache/camel/tutorial/XMLInputTest-context.xml
   • To override this default, use the locations attribute on the @ContextConfiguration annotation to provide specific context file locations (starting each path with a / if you don't want it to be relative to the package directory). My solution does this so I can put the context file directly in src/test/resources instead of in a package directory under there.
6. Add a CamelContext instance variable to the test class, with the @Autowired annotation. That way Spring will automatically pull the CamelContext out of the Spring context and inject it into our test class.
7. Add a ProducerTemplate instance variable and a setUp method that instantiates it from the CamelContext. We'll use the ProducerTemplate later to send messages to the route.
8. Put in an empty test method just for the moment (so when we run this we can see that "1 test succeeded")
9. Add the Spring <beans> element (including the Camel Namespace) with an empty <camelContext> element to the Spring context, like this:

Test it by running mvn install and make sure there are no build errors. So far it doesn't test much; just that your project and test and source files are all organized correctly, and the one empty test method completes successfully.

Solution: Your test class might look something like this:

• src/test/java/org/apache/camel/tutorial/XMLInputTest.java
• src/test/resources/XMLInputTest-context.xml (same as just above)

Flesh Out the Unit Test

So now we're going to write a Camel route that applies the XSLT to the sample Customer 1 input file, and makes sure that some XML output comes out:

1. Save the input-customer1.xml file to src/test/resources
2. Save your XSLT file (created in the previous step) to src/main/resources
3. Write a Camel Route, either right in the Spring XML, or using the Java DSL (in another class under src/test/java somewhere). This route should use the Pipes and Filters integration pattern to:
   1. Start from the endpoint direct:start (which lets the test conveniently pass messages into the route)
   2. Call the endpoint xslt:YourXSLTFile.xsl (to transform the message with the specified XSLT template)
   3. Send the result to the endpoint mock:finish (which lets the test verify the route output)
4. Add a test method to the unit test class that:
   1. Get a reference to the Mock endpoint mock:finish using code like this:
   2. Set the expectedMessageCount on that endpoint to 1
   3. Get a reference to the Customer 1 input file, using code like this:
   4. Send that InputStream as a message to the direct:start endpoint, using code like this:
      Note that we can send the sample file body in several formats (File, InputStream, String, etc.) but in this case an InputStream is pretty convenient.
   5. Ensure that the message made it through the route to the final endpoint, by testing all configured Mock endpoints like this:
   6. If you like, inspect the final message body using some code like finish.getExchanges().get(0).getIn().getBody().
      • If you do this, you'll need to know what format that body is – String, byte array, InputStream, etc.
5. Run your test with mvn install and make sure the build completes successfully.

Solution: Your finished test might look something like this:

• src/test/java/org/apache/camel/tutorial/XMLInputTest.java
• For XML Configuration:
  • src/test/resources/XMLInputTest-context.xml
• Or, for Java DSL Configuration:
  • src/test/resources/XMLInputTest-dsl-context.xml
  • src/test/java/org/apache/camel/tutorial/routes/XMLInputTestRoute.java

Step 5: Initial Work on Customer 2 Input (CSV over HTTP)

To get a start on Customer 2, we'll create a POJO to convert the Customer 2 sample CSV data into the JAXB POJOs representing the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the CSV conversion and JAXB handling is valid and can be run safely in Camel.

Create a CSV-handling POJO

To begin with, CSV is a known data format in Camel. Camel can convert a CSV file to a List (representing rows in the CSV) of Lists (representing cells in the row) of Strings (the data for each cell). That means our POJO can just assume the data coming in is of type List<List<String>>, and we can declare a method with that as the argument.

Looking at the JAXB code in target/generated-sources/jaxb, it looks like an Invoice object represents the whole document, with a nested list of LineItemType objects for the line items. Therefore our POJO method will return an Invoice (a document in the canonical XML format).

So to implement the CSV-to-JAXB POJO, we need to do something like this:

1. Create a new class under src/main/java, perhaps called CSVConverterBean.
2. Add a method, with one argument of type List<List<String>> and the return type Invoice
   • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
3. In the method, the logic should look roughly like this:
   1. Create a new Invoice, using the method on the generated ObjectFactory class
   2. Loop through all the rows in the incoming CSV (the outer List)
   3. Skip the first row, which contains headers (column names)
   4. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values (the Strings in the inner List) and put them into the correct fields of the LineItemType
         • Not all of the values will actually go into the line item in this example
         • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
         • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means using a SimpleDateFormat to parse the date and setting that date on a GregorianCalendar
      3. Add the line item to the invoice
   5. Populate the partner ID, date of receipt, and order total on the Invoice
   6. Throw any exceptions out of the method, so Camel knows something went wrong
   7. Return the finished Invoice

Solution: Here's an example of what the CSVConverterBean might look like.

Create a unit test

Start with a simple test class and test Spring context like last time, perhaps based on the name CSVInputTest:

Now the meaty part is to flesh out the test class and write the Camel routes.

1. Update the Maven POM to include CSV Data Format support:
2. Write the routes (right in the Spring XML context, or using the Java DSL) for the CSV conversion process, again using the Pipes and Filters pattern:
   1. Start from the endpoint direct:CSVstart (which lets the test conveniently pass messages into the route). We'll name this differently than the starting point for the previous test, in case you use the Java DSL and put all your routes in the same package (which would mean that each test would load the DSL routes for several tests.)
   2. This time, there's a little preparation to be done. Camel doesn't know that the initial input is a CSV, so it won't be able to convert it to the expected List<List<String>> without a little hint. For that, we need an unmarshal transformation in the route. The unmarshal method (in the DSL) or element (in the XML) takes a child indicating the format to unmarshal; in this case that should be csv.
   3. Next invoke the POJO to transform the message with a bean:CSVConverter endpoint
   4. As before, send the result to the endpoint mock:finish (which lets the test verify the route output)
   5. Finally, we need a Spring <bean> element in the Spring context XML file (but outside the <camelContext> element) to define the Spring bean that our route invokes. This Spring bean should have a name attribute that matches the name used in the bean endpoint (CSVConverter in the example above), and a class attribute that points to the CSV-to-JAXB POJO class you wrote above (such as, org.apache.camel.tutorial.CSVConverterBean). When Spring is in the picture, any bean endpoints look up Spring beans with the specified name.
3. Write a test method in the test class, which should look very similar to the previous test class:
   1. Get the MockEndpoint for the final endpoint, and tell it to expect one message
   2. Load the Partner 2 sample CSV file from the ClassPath, and send it as the body of a message to the starting endpoint
   3. Verify that the final MockEndpoint is satisfied (that is, it received one message) and examine the message body if you like
      • Note that we didn't marshal the JAXB POJOs to XML in this test, so the final message should contain an Invoice as the body. You could write a simple line of code to get the Exchange (and Message) from the MockEndpoint to confirm that.
4. Run this new test with mvn install and make sure it passes and the build completes successfully.

Solution: Your finished test might look something like this:

• src/test/java/org/apache/camel/tutorial/CSVInputTest.java
• For XML Configuration:
  • src/test/resources/CSVInputTest-context.xml
• Or, for Java DSL Configuration:
  • src/test/resources/CSVInputTest-dsl-context.xml
  • src/test/java/org/apache/camel/tutorial/routes/CSVInputTestRoute.java

Step 6: Initial Work on Customer 3 Input (Excel over e-mail)

To get a start on Customer 3, we'll create a POJO to convert the Customer 3 sample Excel data into the JAXB POJOs representing the canonical XML format, write a small Camel route to test it, and build that into a unit test. If we get through this, we can be pretty sure that the Excel conversion and JAXB handling is valid and can be run safely in Camel.

Create an Excel-handling POJO

Camel does not have a data format handler for Excel by default. We have two options – create an Excel DataFormat (so Camel can convert Excel spreadsheets to something like the CSV List<List<String>> automatically), or create a POJO that can translate Excel data manually. For now, the second approach is easier (if we go the DataFormat route, we need code to both read and write Excel files, whereas otherwise read-only will do).

So, we need a POJO with a method that takes something like an InputStream or byte[] as an argument, and returns in Invoice as before. The process should look something like this:

1. Update the Maven POM to include POI support:
2. Create a new class under src/main/java, perhaps called ExcelConverterBean.
3. Add a method, with one argument of type InputStream and the return type Invoice
   • You may annotate the argument with @Body to specifically designate it as the body of the incoming message
4. In the method, the logic should look roughly like this:
   1. Create a new Invoice, using the method on the generated ObjectFactory class
   2. Create a new HSSFWorkbook from the InputStream, and get the first sheet from it
   3. Loop through all the rows in the sheet
   4. Skip the first row, which contains headers (column names)
   5. For the other rows:
      1. Create a new LineItemType (using the ObjectFactory again)
      2. Pick out all the cell values and put them into the correct fields of the LineItemType (you'll need some data type conversion logic)
         • Not all of the values will actually go into the line item in this example
         • You may hardcode the column ordering based on the sample data file, or else try to read it dynamically from the headers in the first line
         • Note that you'll need to use a JAXB DatatypeFactory to create the XMLGregorianCalendar values that JAXB uses for the date fields in the XML – which probably means setting the date from a date cell on a GregorianCalendar
      3. Add the line item to the invoice
   6. Populate the partner ID, date of receipt, and order total on the Invoice
   7. Throw any exceptions out of the method, so Camel knows something went wrong
   8. Return the finished Invoice

Solution: Here's an example of what the ExcelConverterBean might look like.

Create a unit test

The unit tests should be pretty familiar now. The test class and context for the Excel bean should be quite similar to the CSV bean.

1. Create the basic test class and corresponding Spring Context XML configuration file
2. The XML config should look a lot like the CSV test, except:
   • Remember to use a different start endpoint name if you're using the Java DSL and not use separate packages per test
   • You don't need the unmarshal step since the Excel POJO takes the raw InputStream from the source endpoint
   • You'll declare a <bean> and endpoint for the Excel bean prepared above instead of the CSV bean
3. The test class should look a lot like the CSV test, except use the right input file name and start endpoint name.

Solution: Your finished test might look something like this:

• src/test/java/org/apache/camel/tutorial/ExcelInputTest.java
• For XML Configuration:
  • src/test/resources/ExcelInputTest-context.xml
• Or, for Java DSL Configuration:
  • src/test/resources/ExcelInputTest-dsl-context.xml
  • src/test/java/org/apache/camel/tutorial/routes/ExcelInputTestRoute.java

Step 7: Put this all together into Camel routes for the Customer Input

With all the data type conversions working, the next step is to write the real routes that listen for HTTP, FTP, or e-mail input, and write the final XML output to an ActiveMQ queue. Along the way these routes will use the data conversions we've developed above.

So we'll create 3 routes to start with, as shown in the diagram back at the beginning:

1. Accept XML orders over FTP from Customer 1 (we'll assume the FTP server dumps files in a local directory on the Camel machine)
2. Accept CSV orders over HTTP from Customer 2
3. Accept Excel orders via e-mail from Customer 3 (we'll assume the messages are sent to an account we can access via IMAP)

...

Step 8: Create a unit test for the Customer Input Routes

Languages Supported Appendix

To support flexible and powerful Enterprise Integration Patterns Camel supports various Languages to create an Expression or Predicate within either the Routing Domain Specific Language or the Xml Configuration. The following languages are supported

Bean Language

The purpose of the Bean Language is to be able to implement an Expression or Predicate using a simple method on a bean. The bean name is resolved using a Registry, such as the Spring ApplicationContext, then a method is invoked to evaluate the Expression or Predicate. If no method name is provided then one is chosen using the rules for Bean Binding; using the type of the message body and using any annotations on the bean methods.

The Bean Binding rules are used to bind the Message Exchange to the method parameters; so you can annotate the bean to extract headers or other expressions such as XPath or XQuery from the message.

Using Bean Expressions in Java

Using Bean Expressions in Spring XML

Writing the Expression Bean

The bean in the above examples is just any old Java Bean with a method called isGoldCustomer() that returns some object that is easily converted to a boolean value in this case, as its used as a predicate.

Example:

We can also use the Bean Integration annotations.

Example:

or

So you can bind parameters of the method to the Exchange, the Message or individual headers, properties, the body or other expressions.

Non-Registry Beans

The Bean Language also supports invoking beans that isn't registered in the Registry. This is usable for quickly to invoke a bean from Java DSL where you don't need to register the bean in the Registry such as the Spring ApplicationContext. Camel can instantiate the bean and invoke the method if given a class or invoke an already existing instance.

Example:

The 2nd parameter isGoldCustomer is an optional parameter to explicit set the method name to invoke. If not provided Camel will try to invoke the most suitable method. If case of ambiguity Camel will thrown an Exception. In these situations the 2nd parameter can solve this problem. Also the code is more readable if the method name is provided. The 1st parameter can also be an existing instance of a Bean such as:

In Camel 2.2: you can avoid the BeanLanguage and have it just as:

Which also can be done in a bit shorter and nice way:

Other Examples

We have some test cases you can look at if it'll help

• MethodFilterTest is a JUnit test case showing the Java DSL use of the bean expression being used in a filter
• aggregator.xml is a Spring XML test case for the Aggregator which uses a bean method call to test for the completion of the aggregation.

Dependencies

The Bean language is part of camel-core.

Constant Expression Language

The Constant Expression Language is really just a way to specify constant strings as a type of expression.

Example usage

The setHeader element of the Spring DSL can utilize a constant expression like:

In this case, the Message coming from the seda:a Endpoint will have theHeader header set to the constant value the value.

And the same example using Java DSL:

Dependencies

The Constant language is part of camel-core.

EL

Camel supports the unified JSP and JSF Expression Language via the JUEL to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

For example you could use EL inside a Message Filter in XML

You could also use slightly different syntax, e.g. if the header name is not a valid identifier:

You could use EL to create an Predicate in a Message Filter or as an Expression for a Recipient List

Variables

Samples

You can use EL dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamiliyName method then you can construct the syntax as follows:

Dependencies

To use EL in your camel routes you need to add the a dependency on camel-juel which implements the EL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Otherwise you'll also need to include JUEL.

Header Expression Language

The Header Expression Language allows you to extract values of named headers.

Example usage

The recipientList element of the Spring DSL can utilize a header expression like:

In this case, the list of recipients are contained in the header 'myHeader'.

And the same example in Java DSL:

And with a slightly different syntax where you use the builder to the fullest (i.e. avoid using parameters but using stacked operations, notice that header is not a parameter but a stacked method call)

Dependencies

The Header language is part of camel-core.

JXPath

Camel supports JXPath to allow XPath expressions to be used on beans in an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use JXPath to create an Predicate in a Message Filter or as an Expression for a Recipient List.

You can use XPath expressions directly using smart completion in your IDE as follows

Variables

Options

Using XML configuration

If you prefer to configure your routes in your Spring XML file then you can use JXPath expressions as follows

Examples

Here is a simple example using a JXPath expression as a predicate in a Message Filter

JXPath injection

You can use Bean Integration to invoke a method on a bean and use various languages such as JXPath to extract a value from the message and bind it to a method parameter.

For example

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

Dependencies

To use JXpath in your camel routes you need to add the a dependency on camel-jxpath which implements the JXpath language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Otherwise, you'll also need Commons JXPath.

Mvel

Camel allows Mvel to be used as an Expression or Predicate the DSL or Xml Configuration.

You could use Mvel to create an Predicate in a Message Filter or as an Expression for a Recipient List

You can use Mvel dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamiliyName method then you can construct the syntax as follows:

Variables

Samples

For example you could use Mvel inside a Message Filter in XML

And the sample using Java DSL:

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

Dependencies

To use Mvel in your camel routes you need to add the a dependency on camel-mvel which implements the Mvel language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

OGNL

Camel allows OGNL to be used as an Expression or Predicate the DSL or Xml Configuration.

You could use OGNL to create an Predicate in a Message Filter or as an Expression for a Recipient List

You can use OGNL dot notation to invoke operations. If you for instance have a body that contains a POJO that has a getFamilyName method then you can construct the syntax as follows:

Variables

Samples

For example you could use OGNL inside a Message Filter in XML

And the sample using Java DSL:

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

Dependencies

To use OGNL in your camel routes you need to add the a dependency on camel-ognl which implements the OGNL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Otherwise, you'll also need OGNL

Property Expression Language

The Property Expression Language allows you to extract values of named exchange properties.

 

Example usage

The recipientList element of the Spring DSL can utilize a property expression like:

In this case, the list of recipients are contained in the property 'myProperty'.

And the same example in Java DSL:

And with a slightly different syntax where you use the builder to the fullest (i.e. avoid using parameters but using stacked operations, notice that property is not a parameter but a stacked method call)

Dependencies

The Property language is part of camel-core.

Scripting Languages

Camel supports a number of scripting languages which can be used to create an Expression or Predicate via the standard JSR 223 which is a standard part of Java 6.

The following scripting languages are integrated into the DSL:

However any JSR 223 scripting language can be used using the generic DSL methods.

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

See Also

• Languages
• DSL
• Xml Configuration

BeanShell

Camel supports BeanShell among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a BeanShell expression use the following Java code:

Or the something like this in your Spring XML:

You could follow the examples above to create an Predicate in a Message Filter or as an Expression for a Recipient List

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

JavaScript

Camel supports JavaScript/ECMAScript among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a JavaScript expression use the following Java code

For example you could use the javaScript function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use JavaScript to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

And a Spring DSL sample as well:

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Groovy

Camel supports Groovy among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Groovy expression use the following Java code

For example you could use the groovy function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Dependency

You should add the camel-groovy dependeny when using Groovy language with Camel. The generic camel-script is not optimized for best Groovy experience, and hence you should add camel-groovy as dependency.

Customizing Groovy Shell

Sometimes you may need to use custom GroovyShell instance in your Groovy expressions. To provide custom GroovyShell, add implementation of the org.apache.camel.language.groovy.GroovyShellFactory SPI interface to your Camel registry. For example after adding the following bean to your Spring context...

...Camel will use your custom GroovyShell instance (containing your custom static imports), instead of the default one.

Example

And the Spring DSL:

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Python

Camel supports Python among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Python expression use the following Java code

For example you could use the python function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use Python to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

And a Spring DSL sample as well:

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

PHP

Camel supports PHP among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a PHP expression use the following Java code

For example you could use the php function to create an Predicate in a Message Filter or as an Expression for a Recipient List

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Ruby

Camel supports Ruby among other Scripting Languages to allow an Expression or Predicate to be used in the DSL or Xml Configuration.

To use a Ruby expression use the following Java code

For example you could use the ruby function to create an Predicate in a Message Filter or as an Expression for a Recipient List

Example

In the sample below we use Ruby to create a Predicate use in the route path, to route exchanges from admin users to a special queue.

And a Spring DSL sample as well:

ScriptContext Options

 

See Scripting Languages for the list of languages with explicit DSL support.

Passing Additional Arguments to the ScriptingEngine

Available from Camel 2.8

You can provide additional arguments to the ScriptingEngine using a header on the Camel message with the key CamelScriptArguments.

Example:

Using Properties Function

Available from Camel 2.9

If you need to use the Properties component from a script to lookup property placeholders, then its a bit cumbersome to do so. For example, to set a header name myHeader with a value from a property placeholder, whose key is taken from a header named foo.

From Camel 2.9: you can now use the properties function and the same example is simpler:

Loading Script From External Resource

Available from Camel 2.11

You can externalize the script and have Camel load it from a resource such as classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location e.g. to refer to a file on the classpath you can do:

How to Get the Result from Multiple Statements Script

Available from Camel 2.14

The script engine's eval method returns a null when it runs a multi-statement script. However, Camel can look up the value of a script's result by using the key result from the value set. When writing a multi-statement script set the value of the result variable as the script return value.

 

Dependencies

To use scripting languages in your camel routes you need to add the a dependency on camel-script which integrates the JSR-223 scripting engine.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Simple Expression Language

The Simple Expression Language was a really simple language when it was created, but has since grown more powerful. It is primarily intended for being a really small and simple language for evaluating Expressions and Predicates without requiring any new dependencies or knowledge of XPath; so it is ideal for testing in camel-core. The idea was to cover 95% of the common use cases when you need a little bit of expression based script in your Camel routes.

However for much more complex use cases you are generally recommended to choose a more expressive and powerful language such as:

• SpEL
• Mvel
• Groovy
• JavaScript
• EL
• OGNL
• one of the supported Scripting Languages

The simple language uses ${body} placeholders for complex expressions where the expression contains constant literals.

Deprecated: The ${ } placeholders can be omitted if the expression starts with the token, or if the token is only itself.

To get the body of the in message: body, or in.body or ${body}.

A complex expression must use ${ } placeholders, such as: Hello ${in.header.name} how are you?.

You can have multiple functions in the same expression: "Hello ${in.header.name} this is ${in.header.me} speaking". However you can not nest functions in Camel 2.8.x or older e.g., having another ${ } placeholder in an existing, is not allowed. From Camel 2.9 you can nest functions.

Variables

OGNL expression support

Available as of Camel 2.3

The Simple and Bean language now supports a Camel OGNL notation for invoking beans in a chain like fashion. Suppose the Message IN body contains a POJO which has a getAddress() method.

Then you can use Camel OGNL notation to access the address object:

Camel understands the shorthand names for accessors, but you can invoke any method or use the real name such as:

You can also use the null safe operator (?.) to avoid NPE if for example the body does not have an address

It is also possible to index in Map or List types, so you can do:

To assume the body is Map based and lookup the value with foo as key, and invoke the getName method on that value.

You can access the Map or List objects directly using their key name (with or without dots) :

Suppose there was no value with the key foo then you can use the null safe operator to avoid the NPE as shown:

You can also access List types, for example to get lines from the address you can do:

There is a special last keyword which can be used to get the last value from a list.

And to get the 2nd last you can subtract a number, so we can use last-1 to indicate this:

And the 3rd last is of course:

And you can call the size method on the list with

From Camel 2.11.1 we added support for the length field for Java arrays as well, eg:

And yes you can combine this with the operator support as shown below:

Operator Support

The parser is limited to only support a single operator. To enable it the left value must be enclosed in ${ }.

The syntax is:

Where the rightValue can be a String literal enclosed in ' ', null, a constant value or another expression enclosed in ${}.

Camel will automatically type convert the rightValue type to the leftValue type, so it is possible to for example, convert a string into a numeric so you can use > comparison for numeric values.

The following operators are supported:

And the following unary operators can be used:

And the following logical operators can be used to group expressions:

The syntax for AND is:

And the syntax for OR is:

Some examples:

And a bit more advanced example where the right value is another expression,

And an example with contains, testing if the title contains the word Camel

And an example with regex, testing if the number header is a four digit value:

And finally an example if the header equals any of the values in the list. Each element must be separated by comma, and no space around. This also works for numbers etc, as Camel will convert each element into the type of the left hand side.

And for all the last three we also support the negate test using not:

And you can test if the type is a certain instance, e.g., for instance a String:

We have added a shorthand for all java.lang types so you can write it as:

Ranges are also supported. The range interval requires numbers and both from and end are inclusive. For instance to test whether a value is between 100 and 199:

Notice we use .. in the range without spaces. It is based on the same syntax as Groovy.

From Camel 2.9 onwards the range value must be in single quotes:

Using and / or

If you have two expressions you can combine them with the and or or operator.

For instance:

And of course the or is also supported. The sample would be:

Note: currently and or or can only be used once in a simple language expression. This might change in the future. So you cannot do:

Samples

In the Spring XML sample below we filter based on a header value:

The Simple language can be used for the predicate test above in the Message Filter pattern, where we test if the in message has a foo header (a header with the key foo exists). If the expression evaluates to true then the message is routed to the mock:fooOrders endpoint, otherwise it is lost in the deep blue sea .

The same example in Java DSL:

You can also use the simple language for simple text concatenations such as:

Notice that we must use ${ } placeholders in the expression now to allow Camel to parse it correctly.

And this sample uses the date command to output current date.

And in the sample below we invoke the bean language to invoke a method on a bean to be included in the returned string:

Where orderIdGenerator is the id of the bean registered in the Registry. If using Spring then it is the Spring bean id.

If we want to declare which method to invoke on the order id generator bean we must prepend .method name such as below where we invoke the generateId method.

We can use the ?method=methodname option that we are familiar with the Bean component itself:

And from Camel 2.3 you can also convert the body to a given type, for example to ensure that it is a String you can do:

There are a few types which have a shorthand notation, so we can use String instead of java.lang.String. These are: byte[], String, Integer, Long. All other types must use their FQN name, e.g. org.w3c.dom.Document.

It is also possible to lookup a value from a header Map in Camel 2.3:

In the code above we lookup the header with name type and regard it as a java.util.Map and we then lookup with the key gold and return the value. If the header is not convertible to Map an exception is thrown. If the header with name type does not exist null is returned.

From Camel 2.9 you can nest functions, such as shown below:

Referring to Constants or Enums

Available as of Camel 2.11

Suppose you have an enum for customers

And in a Content Based Router we can use the Simple language to refer to this enum, to check the message which enum it matches.

Using New Lines or Tabs in XML DSLs

Available as of Camel 2.9.3

From Camel 2.9.3 it is easier to specify new lines or tabs in XML DSLs as you can escape the value now

Leading and Trailing Whitespace Handling

Available as of Camel 2.10.0

From Camel 2.10.0, the trim attribute of the expression can be used to control whether the leading and trailing whitespace characters are removed or preserved. The default value is true, which removes the whitespace characters.

Setting the Result Type

Available as of Camel 2.8

You can now provide a result type to the Simple expression, which means the result of the evaluation will be converted to the desired type. This is most usable to define types such as boolean's, integer's, etc.

For example to set a header as a boolean type you can do:

And in XML DSL

Changing Function Start and End Tokens

Available as of Camel 2.9.1

You can configure the function start and end tokens - ${ } using the setters changeFunctionStartToken and changeFunctionEndToken on SimpleLanguage, using Java code. From Spring XML you can define a <bean> tag with the new changed tokens in the properties as shown below:

In the example above we use [] as the changed tokens. Notice by changing the start/end token you change those in all the Camel applications which share the same camel-core on their classpath. For example in an OSGi server this may affect many applications, where as a Web Application as a WAR file it only affects the Web Application.

Loading Script from External Resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as: classpath:, file:, or http:. This is done using the following syntax: resource:scheme:location, e.g., to refer to a file on the classpath you can do:

Setting Spring beans to Exchange properties

Available as of Camel 2.6

You can set a spring bean into an exchange property as shown below:

Dependencies

The Simple language is part of camel-core.

File Expression Language

The File Expression Language is an extension to the Simple language, adding file related capabilities. These capabilities are related to common use cases working with file path and names. The goal is to allow expressions to be used with the File and FTP components for setting dynamic file patterns for both consumer and producer.

Syntax

This language is an extension to the Simple language so the Simple syntax applies also. So the table below only lists the additional. By contrast to the Simple language, the File Language also supports the use of Constant expressions to enter a fixed filename, for example.

All the file tokens use the same expression name as the method on the java.io.File object. For example: file:absolute refers to the java.io.File.getAbsolute() method.

File Token Example

Relative Paths

We have a java.io.File handle for the file hello.txt in the following relative directory: .\filelanguage\test. And we configure our endpoint to use this starting directory .\filelanguage.

The file tokens returned are:

Absolute Paths

We have a java.io.File handle for the file hello.txt in the following absolute directory: \workspace\camel\camel-core\target\filelanguage\test. And we configure out endpoint to use the absolute starting directory: \workspace\camel\camel-core\target\filelanguage.

The file tokens return are:

Examples

You can enter a fixed Constant expression such as myfile.txt:

Lets assume we use the file consumer to read files and want to move the read files to backup folder with the current date as a sub folder. This can be achieved using an expression like:

relative folder names are also supported so suppose the backup folder should be a sibling folder then you can append .. as:

As this is an extension to the Simple language we have access to all the goodies from this language also, so in this use case we want to use the in.header.type as a parameter in the dynamic expression:

If you have a custom Date you want to use in the expression then Camel supports retrieving dates from the message header.

And finally we can also use a bean expression to invoke a POJO class that generates some String output (or convertible to String) to be used:

And of course all this can be combined in one expression where you can use the File Language, Simple and the Bean language in one combined expression. This is pretty powerful for those common file path patterns.

Using Spring's PropertyPlaceholderConfigurer with the File Component

In Camel you can use the File Language directly from the Simple language which makes a Content Based Router easier to do in Spring XML, where we can route based on file extensions as shown below:

If you use the fileName option on the File endpoint to set a dynamic filename using the File Language then make sure you use the alternative syntax (available from Camel 2.5) to avoid clashing with Spring's PropertyPlaceholderConfigurer.

Notice how we use the $simple{} syntax in the toEndpoint above. If you don't do this, they will clash and Spring will throw an exception:

Dependencies

The File language is part of camel-core.

SQL Language

The SQL support is added by JoSQL and is primarily used for performing SQL queries on in-memory objects. If you prefer to perform actual database queries then check out the JPA component.

To use SQL in your camel routes you need to add the a dependency on camel-josql which implements the SQL language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Camel supports SQL to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use SQL to create an Predicate in a Message Filter or as an Expression for a Recipient List.

And the spring DSL:

Variables

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

XPath

Camel supports XPath to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use XPath to create an Predicate in a Message Filter or as an Expression for a Recipient List.

Namespaces

You can easily use namespaces with XPath expressions using the Namespaces helper class.

Variables

Variables in XPath is defined in different namespaces. The default namespace is http://camel.apache.org/schema/spring.

Camel will resolve variables according to either:

• namespace given
• no namespace given

Namespace Given

If the namespace is given then Camel is instructed exactly what to return. However when resolving either IN or OUT Camel will try to resolve a header with the given local part first, and return it. If the local part has the value body then the body is returned instead.

No Namespace Given

If there is no namespace given then Camel resolves only based on the local part. Camel will try to resolve a variable in the following steps:

• From variables that has been set using the variable(name, value) fluent builder.
• From message.in.header if there is a header with the given key.
• From exchange.properties if there is a property with the given key.

Functions

Camel adds the following XPath functions that can be used to access the exchange:

Here's an example showing some of these functions in use.

And the new functions introduced in Camel 2.5:

Using XML Configuration

If you prefer to configure your routes in your Spring XML file then you can use XPath expressions as follows

Notice how we can reuse the namespace prefixes, foo in this case, in the XPath expression for easier namespace based XPath expressions! See also this discussion on the mailinglist about using your own namespaces with XPath.

Setting the Result Type

The XPath expression will return a result type using native XML objects such as org.w3c.dom.NodeList. But many times you want a result type to be a String. To do this you have to instruct the XPath which result type to use.

In Java DSL:

In Spring DSL you use the resultType attribute to provide a fully qualified classname:

In @XPath:
Available as of Camel 2.1

Where we use the XPath function concat to prefix the order name with foo-. In this case we have to specify that we want a String as result type so the concat function works.

Using XPath on Headers

Available as of Camel 2.11

Some users may have XML stored in a header. To apply an XPath statement to a header's value you can do this by defining the headerName attribute.

In XML DSL:

And in Java DSL you specify the headerName as the 2nd parameter as shown:

Examples

Here is a simple example using an XPath expression as a predicate in a Message Filter

If you have a standard set of namespaces you wish to work with and wish to share them across many different XPath expressions you can use the NamespaceBuilder as shown in this exampleIn this sample we have a choice construct. The first choice evaulates if the message has a header key type that has the value Camel. The 2nd choice evaluates if the message body has a name tag <name> which values is Kong.
If neither is true the message is routed in the otherwise block:And the spring XML equivalent of the route:

XPath Injection

You can use Bean Integration to invoke a method on a bean and use various languages such as XPath to extract a value from the message and bind it to a method parameter.

The default XPath annotation has SOAP and XML namespaces available. If you want to use your own namespace URIs in an XPath expression you can use your own copy of the XPath annotation to create whatever namespace prefixes you want to use.

e.g., cut and paste upper code to your own project in a different package and/or annotation name then add whatever namespace prefix/URIs you want in scope when you use your annotation on a method parameter. Then when you use your annotation on a method parameter all the namespaces you want will be available for use in your XPath expression.

Example:

Using XPathBuilder Without an Exchange

Available as of Camel 2.3

You can now use the org.apache.camel.builder.XPathBuilder without the need for an Exchange. This comes handy if you want to use it as a helper to do custom XPath evaluations. It requires that you pass in a CamelContext since a lot of the moving parts inside the XPathBuilder requires access to the Camel Type Converter and hence why CamelContext is needed.

For example you can do something like this:

This will match the given predicate.

You can also evaluate for example as shown in the following three examples:

Evaluating with a String result is a common requirement and thus you can do it a bit simpler:

Using Saxon with XPathBuilder

Available as of Camel 2.3

You need to add camel-saxon as dependency to your project. It's now easier to use Saxon with the XPathBuilder which can be done in several ways as shown below. Where as the latter ones are the easiest ones.

Using a factory

Using the object model
The easy one

Setting a Custom XPathFactory Using System Property

Available as of Camel 2.3

Camel now supports reading the JVM system property javax.xml.xpath.XPathFactory that can be used to set a custom XPathFactory to use.

This unit test shows how this can be done to use Saxon instead:

Camel will log at INFO level if it uses a non default XPathFactory such as:

To use Apache Xerces you can configure the system property:

Enabling Saxon from Spring DSL

Available as of Camel 2.10

Similarly to Java DSL, to enable Saxon from Spring DSL you have three options:

Specifying the factory

Specifying the object model

Shortcut

Namespace Auditing to Aid Debugging

Available as of Camel 2.10

A large number of XPath-related issues that users frequently face are linked to the usage of namespaces. You may have some misalignment between the namespaces present in your message and those that your XPath expression is aware of or referencing. XPath predicates or expressions that are unable to locate the XML elements and attributes due to namespaces issues may simply look like "they are not working", when in reality all there is to it is a lack of namespace definition.

Namespaces in XML are completely necessary, and while we would love to simplify their usage by implementing some magic or voodoo to wire namespaces automatically, truth is that any action down this path would disagree with the standards and would greatly hinder interoperability.

Therefore, the utmost we can do is assist you in debugging such issues by adding two new features to the XPath Expression Language and are thus accessible from both predicates and expressions.

Logging the Namespace Context of Your XPath Expression/Predicate

Every time a new XPath expression is created in the internal pool, Camel will log the namespace context of the expression under the org.apache.camel.builder.xml.XPathBuilder logger. Since Camel represents Namespace Contexts in a hierarchical fashion (parent-child relationships), the entire tree is output in a recursive manner with the following format:

Any of these options can be used to activate this logging:

1. Enable TRACE logging on the org.apache.camel.builder.xml.XPathBuilder logger, or some parent logger such as org.apache.camel or the root logger.
2. Enable the logNamespaces option as indicated in Auditing Namespaces, in which case the logging will occur on the INFO level.

Auditing namespaces

Camel is able to discover and dump all namespaces present on every incoming message before evaluating an XPath expression, providing all the richness of information you need to help you analyse and pinpoint possible namespace issues. To achieve this, it in turn internally uses another specially tailored XPath expression to extract all namespace mappings that appear in the message, displaying the prefix and the full namespace URI(s) for each individual mapping.

Some points to take into account:

• The implicit XML namespace (xmlns:xml="http://www.w3.org/XML/1998/namespace") is suppressed from the output because it adds no value.
• Default namespaces are listed under the DEFAULT keyword in the output.
• Keep in mind that namespaces can be remapped under different scopes. Think of a top-level 'a' prefix which in inner elements can be assigned a different namespace, or the default namespace changing in inner scopes. For each discovered prefix, all associated URIs are listed.

You can enable this option in Java DSL and Spring DSL.

Java DSL:

Spring DSL:

The result of the auditing will be appear at the INFO level under the org.apache.camel.builder.xml.XPathBuilder logger and will look like the following:

Loading Script from External Resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as: classpath:, file: or http:.
This is done using the following syntax: resource:scheme:location, e.g., to refer to a file on the classpath you can do:

Dependencies

The XPath language is part of camel-core.

XQuery

Camel supports XQuery to allow an Expression or Predicate to be used in the DSL or Xml Configuration. For example you could use XQuery to create an Predicate in a Message Filter or as an Expression for a Recipient List.

Options

Examples

You can also use functions inside your query, in which case you need an explicit type conversion (or you will get a org.w3c.dom.DOMException: HIERARCHY_REQUEST_ERR) by passing the Class as a second argument to the xquery() method.

Variables

The IN message body will be set as the contextItem. Besides this these Variables is also added as parameters:

Using XML configuration

If you prefer to configure your routes in your Spring XML file then you can use XPath expressions as follows

Notice how we can reuse the namespace prefixes, foo in this case, in the XPath expression for easier namespace based XQuery expressions!

When you use functions in your XQuery expression you need an explicit type conversion which is done in the xml configuration via the @type attribute:

Using XQuery as transformation

We can do a message translation using transform or setBody in the route, as shown below:

Notice that xquery will use DOMResult by default, so if we want to grab the value of the person node, using text() we need to tell xquery to use String as result type, as shown:

 

Using XQuery as an endpoint

Sometimes an XQuery expression can be quite large; it can essentally be used for Templating. So you may want to use an XQuery Endpoint so you can route using XQuery templates.

The following example shows how to take a message of an ActiveMQ queue (MyQueue) and transform it using XQuery and send it to MQSeries.

Examples

Here is a simple example using an XQuery expression as a predicate in a Message Filter

This example uses XQuery with namespaces as a predicate in a Message Filter

Learning XQuery

XQuery is a very powerful language for querying, searching, sorting and returning XML. For help learning XQuery try these tutorials

• Mike Kay's XQuery Primer
• the W3Schools XQuery Tutorial

You might also find the XQuery function reference useful

Loading script from external resource

Available as of Camel 2.11

You can externalize the script and have Camel load it from a resource such as "classpath:", "file:", or "http:".
This is done using the following syntax: "resource:scheme:location", eg to refer to a file on the classpath you can do:

Dependencies

To use XQuery in your camel routes you need to add the a dependency on camel-saxon which implements the XQuery language.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Data Format Appendix

Data Format

Camel supports a pluggable DataFormat to allow messages to be marshalled to and from binary or text formats to support a kind of Message Translator.

The following data formats are currently supported:

• Standard JVM object marshalling
  • Serialization
  • String

• Object marshalling
  • Avro
  • Boon
  • Hessian
  • JSON
  • Protobuf
  • YAML

• Object/XML marshalling
  • Castor
  • JAXB
  • XmlBeans
  • XStream
  • JiBX
  • Jackson XML

• Object/XML/Webservice marshalling
  • SOAP

• Direct JSON / XML marshalling
  • XmlJson

• Flat data structure marshalling
  • BeanIO
  • Bindy
  • CSV
  • EDI
  • Flatpack DataFormat
  • uniVocity-parsers formats

• Domain specific marshalling
  • HL7 DataFormat

• Compression
  • GZip data format
  • Zip DataFormat
  • Zip File DataFormat
  • LZF Data Format
  • Tar DataFormat

• Security
  • Crypto
  • PGP
  • XMLSecurity DataFormat

• Misc.
  • Base64
  • Custom DataFormat - to use your own custom implementation
  • MIME-Multipart
  • RSS
  • TidyMarkup
  • Syslog
  • ICal
  • Barcode - to read and generate barcodes (QR-Code, PDF417, ...)

And related is the following:

• DataFormat Component for working with Data Formats as if it was a regular Component supporting Endpoints and URIs.
• Dozer Type Conversion using Dozer for type converting POJOs

Unmarshalling

If you receive a message from one of the Camel Components such as File, HTTP or JMS you often want to unmarshal the payload into some bean so that you can process it using some Bean Integration or perform Predicate evaluation and so forth. To do this use the unmarshal word in the DSL in Java or the Xml Configuration.

For example

The above uses a named DataFormat of jaxb which is configured with a number of Java package names. You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file.

You can also use the DSL itself to define the data format as you use it. For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to ActiveMQ

Marshalling

Marshalling is the opposite of unmarshalling, where a bean is marshalled into some binary or textual format for transmission over some transport via a Camel Component. Marshalling is used in the same way as unmarshalling above; in the DSL you can use a DataFormat instance, you can configure the DataFormat dynamically using the DSL or you can refer to a named instance of the format in the Registry.

The following example unmarshals via serialization then marshals using a named JAXB data format to perform a kind of Message Translator

Using Spring XML

This example shows how to configure the data type just once and reuse it on multiple routes

You can also define reusable data formats as Spring beans

Serialization

Serialization is a Data Format which uses the standard Java Serialization mechanism to unmarshal a binary payload into Java objects or to marshal Java objects into a binary blob.
For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to ActiveMQ

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

JAXB

JAXB is a Data Format which uses the JAXB2 XML marshalling standard which is included in Java 6 to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

Using the Java DSL

For example the following uses a named DataFormat of jaxb which is configured with a number of Java package names to initialize the JAXBContext.

You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g.

Using Spring XML

The following example shows how to use JAXB to unmarshal using Spring configuring the jaxb data type

This example shows how to configure the data type just once and reuse it on multiple routes.

Partial marshalling/unmarshalling

This feature is new to Camel 2.2.0.
JAXB 2 supports marshalling and unmarshalling XML tree fragments. By default JAXB looks for @XmlRootElement annotation on given class to operate on whole XML tree. This is useful but not always - sometimes generated code does not have @XmlRootElement annotation, sometimes you need unmarshall only part of tree.
In that case you can use partial unmarshalling. To enable this behaviours you need set property partClass. Camel will pass this class to JAXB's unmarshaler.

For marshalling you have to add partNamespace attribute with QName of destination namespace. Example of Spring DSL you can find above.

Fragment

This feature is new to Camel 2.8.0.
JaxbDataFormat has new property fragment which can set the the Marshaller.JAXB_FRAGMENT encoding property on the JAXB Marshaller. If you don't want the JAXB Marshaller to generate the XML declaration, you can set this option to be true. The default value of this property is false.

Ignoring the NonXML Character

This feature is new to Camel 2.2.0.
JaxbDataFromat supports to ignore the NonXML Character, you just need to set the filterNonXmlChars property to be true, JaxbDataFormat will replace the NonXML character with " " when it is marshaling or unmarshaling the message. You can also do it by setting the Exchange property Exchange.FILTER_NON_XML_CHARS.

This feature has been tested with Woodstox 3.2.9 and Sun JDK 1.6 StAX implementation.

New for Camel 2.12.1
JaxbDataFormat now allows you to customize the XMLStreamWriter used to marshal the stream to XML. Using this configuration, you can add your own stream writer to completely remove, escape, or replace non-xml characters.

The following example shows using the Spring DSL and also enabling Camel's NonXML filtering:

Working with the ObjectFactory

If you use XJC to create the java class from the schema, you will get an ObjectFactory for you JAXB context. Since the ObjectFactory uses JAXBElement to hold the reference of the schema and element instance value, jaxbDataformat will ignore the JAXBElement by default and you will get the element instance value instead of the JAXBElement object form the unmarshaled message body.
If you want to get the JAXBElement object form the unmarshaled message body, you need to set the JaxbDataFormat object's ignoreJAXBElement property to be false.

Setting encoding

You can set the encoding option to use when marshalling. Its the Marshaller.JAXB_ENCODING encoding property on the JAXB Marshaller.
You can setup which encoding to use when you declare the JAXB data format. You can also provide the encoding in the Exchange property Exchange.CHARSET_NAME. This property will overrule the encoding set on the JAXB data format.

In this Spring DSL we have defined to use iso-8859-1 as the encoding:

Controlling namespace prefix mapping

Available as of Camel 2.11

When marshalling using JAXB or SOAP then the JAXB implementation will automatic assign namespace prefixes, such as ns2, ns3, ns4 etc. To control this mapping, Camel allows you to refer to a map which contains the desired mapping.

Notice this requires having JAXB-RI 2.1 or better (from SUN) on the classpath, as the mapping functionality is dependent on the implementation of JAXB, whether its supported.

For example in Spring XML we can define a Map with the mapping. In the mapping file below, we map SOAP to use soap as prefix. While our custom namespace "http://www.mycompany.com/foo/2" is not using any prefix.

To use this in JAXB or SOAP you refer to this map, using the namespacePrefixRef attribute as shown below. Then Camel will lookup in the Registry a java.util.Map with the id "myMap", which was what we defined above.

Schema validation

Available as of Camel 2.11

The JAXB Data Format supports validation by marshalling and unmarshalling from/to XML. Your can use the prefix classpath:, file:* or *http: to specify how the resource should by resolved. You can separate multiple schema files by using the ',' character.

Using the Java DSL, you can configure it in the following way:

You can do the same using the XML DSL:

Camel will create and pool the underling SchemaFactory instances on the fly, because the SchemaFactory shipped with the JDK is not thread safe.
However, if you have a SchemaFactory implementation which is thread safe, you can configure the JAXB data format to use this one:

Schema Location

Available as of Camel 2.14

The JAXB Data Format supports to specify the SchemaLocation when marshaling the XML. 

Using the Java DSL, you can configure it in the following way:

You can do the same using the XML DSL:

Marshal data that is already XML

Available as of Camel 2.14.1

Dependencies

To use JAXB in your camel routes you need to add the a dependency on camel-jaxb which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

XmlBeans

XmlBeans is a Data Format which uses the XmlBeans library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

Dependencies

To use XmlBeans in your camel routes you need to add the dependency on camel-xmlbeans which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

XStream

XStream is a Data Format which uses the XStream library to marshal and unmarshal Java objects to and from XML.

To use XStream in your camel routes you need to add the a dependency on camel-xstream which implements this data format.

Maven users will need to add the following dependency to their pom.xml for this component:

Using the Java DSL

If you would like to configure the XStream instance used by the Camel for the message transformation, you can simply pass a reference to that instance on the DSL level.

XMLInputFactory and XMLOutputFactory

The XStream library uses the javax.xml.stream.XMLInputFactory and javax.xml.stream.XMLOutputFactory, you can control which implementation of this factory should be used.

The Factory is discovered using this algorithm:
1. Use the javax.xml.stream.XMLInputFactory , javax.xml.stream.XMLOutputFactory system property.
2. Use the lib/xml.stream.properties file in the JRE_HOME directory.
3. Use the Services API, if available, to determine the classname by looking in the META-INF/services/javax.xml.stream.XMLInputFactory, META-INF/services/javax.xml.stream.XMLOutputFactory files in jars available to the JRE.
4. Use the platform default XMLInputFactory,XMLOutputFactory instance.

How to set the XML encoding in Xstream DataFormat?

From Camel 2.2.0, you can set the encoding of XML in Xstream DataFormat by setting the Exchange's property with the key Exchange.CHARSET_NAME, or setting the encoding property on Xstream from DSL or Spring config.

Setting the type permissions of Xstream DataFormat

In Camel, one can always use its own processing step in the route to filter and block certain XML documents to be routed to the XStream's unmarhall step. From Camel 2.16.1, 2.15.5, you can set XStream's type permissions to automatically allow or deny the instantiation of certain types.

The default type permissions setting used by Camel denies all types except for those from java.lang and java.util packages. This setting can be changed by setting System property org.apache.camel.xstream.permissions. Its value is a string of comma-separated permission terms, each representing a type being allowed or denied, depending on whether the term is prefixed with '+' (note '+' may be omitted) or with '-', respectively.

Each term may contain a wildcard character '*'. For example, value "-*,java.lang.*,java.util.*" indicates denying all types except for java.lang.* and java.util.* classes. Setting this value to an empty string "" reverts to the default XStream's type permissions handling which denies certain blacklisted classes and allow others.

The type permissions setting can be extended at an individual XStream DataFormat instance by setting its type permissions property.

CSV

The CSV Data Format uses Apache Commons CSV to handle CSV payloads (Comma Separated Values) such as those exported/imported by Excel.

As of Camel 2.15.0, it now uses the Apache Commons CSV 1.1 which is based on a completely different set of options.

Available options until Camel 2.15

Available options as of Camel 2.15

Marshalling a Map to CSV

The component allows you to marshal a Java Map (or any other message type that can be converted in a Map) into a CSV payload.

Unmarshalling a CSV message into a Java List

Unmarshalling will transform a CSV messsage into a Java List with CSV file lines (containing another List with all the field values).

An example: we have a CSV file with names of persons, their IQ and their current activity.

We can now use the CSV component to unmarshal this file:

The resulting message will contain a List<List<String>> like...

Marshalling a List<Map> to CSV

Available as of Camel 2.1

If you have multiple rows of data you want to be marshalled into CSV format you can now store the message payload as a List<Map<String, Object>> object where the list contains a Map for each row.

File Poller of CSV, then unmarshaling

Given a bean which can handle the incoming data...

... your route then looks as follows

Marshaling with a pipe as delimiter

 

 

Using autogenColumns, configRef and strategyRef attributes inside XML DSL

Available as of Camel 2.9.2 / 2.10 and deleted for Camel 2.15

You can customize the CSV Data Format to make use of your own CSVConfig and/or CSVStrategy. Also note that the default value of the autogenColumns option is true. The following example should illustrate this customization.

Using skipFirstLine option while unmarshaling

Available as of Camel 2.10 and deleted for Camel 2.15

You can instruct the CSV Data Format to skip the first line which contains the CSV headers. Using the Spring/XML DSL:

Or the Java DSL:

Unmarshaling with a pipe as delimiter

Using the Spring/XML DSL:

Or the Java DSL:

Dependencies

To use CSV in your Camel routes you need to add a dependency on camel-csv, which implements this data format.

If you use Maven you can just add the following to your pom.xml, substituting the version number for the latest and greatest release (see the download page for the latest versions).

The String Data Format is a textual based format that supports encoding.

Options

Marshal

In this example we marshal the file content to String object in UTF-8 encoding.

Unmarshal

In this example we unmarshal the payload from the JMS queue to a String object using UTF-8 encoding, before its processed by the newOrder processor.

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

HL7 DataFormat

The HL7 component ships with a HL7 data format that can be used to marshal or unmarshal HL7 model objects.

• marshal = from Message to byte stream (can be used when responding using the HL7 MLLP codec)
• unmarshal = from byte stream to Message (can be used when receiving streamed data from the HL7 MLLP

To use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder:

In the sample above, the HL7 is marshalled from a HAPI Message object to a byte stream and put on a JMS queue.
The next example is the opposite:

Here we unmarshal the byte stream into a HAPI Message object that is passed to our patient lookup service.

 

There is a shorthand syntax in Camel for well-known data formats that are commonly used.
Then you don't need to create an instance of the HL7DataFormat object:

 

 

EDI DataFormat

We encourage end users to look at the Smooks which supports EDI and Camel natively.

Flatpack DataFormat

The Flatpack component ships with the Flatpack data format that can be used to format between fixed width or delimited text messages to a List of rows as Map.

• marshal = from List<Map<String, Object>> to OutputStream (can be converted to String)
• unmarshal = from java.io.InputStream (such as a File or String) to a java.util.List as an org.apache.camel.component.flatpack.DataSetList instance.
  The result of the operation will contain all the data. If you need to process each row one by one you can split the exchange, using Splitter.

Notice: The Flatpack library does currently not support header and trailers for the marshal operation.

Options

The data format has the following options:

Usage

To use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder:

The sample above will read files from the order/in folder and unmarshal the input using the Flatpack configuration file INVENTORY-Delimited.pzmap.xml that configures the structure of the files. The result is a DataSetList object we store on the SEDA queue.

In the code above we marshal the data from a Object representation as a List of rows as Maps. The rows as Map contains the column name as the key, and the the corresponding value. This structure can be created in Java code from e.g. a processor. We marshal the data according to the Flatpack format and convert the result as a String object and store it on a JMS queue.

Dependencies

To use Flatpack in your camel routes you need to add the a dependency on camel-flatpack which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

JSON

JSON is a Data Format to marshal and unmarshal Java objects to and from JSON.

For JSON to object marshalling, Camel provides integration with three popular JSON libraries:

• The XStream library and Jettsion
• The Jackson library
• Camel 2.10: The GSon library

Every library requires adding the special camel component (see "Dependency..." paragraphs further down). By default Camel uses the XStream library.

Using JSON data format with the XStream library

Using JSON data format with the Jackson library

Using JSON data format with the GSON library

Using JSON in Spring DSL

When using Data Format in Spring DSL you need to declare the data formats first. This is done in the DataFormats XML tag.

And then you can refer to this id in the route:

Excluding POJO fields from marshalling

As of Camel 2.10
When marshalling a POJO to JSON you might want to exclude certain fields from the JSON output. With Jackson you can use JSON views to accomplish this. First create one or more marker classes.

Use the marker classes with the @JsonView annotation to include/exclude certain fields. The annotation also works on getters.Finally use the Camel JacksonDataFormat to marshall the above POJO to JSON.Note that the height field is missing in the resulting JSON:

The GSON library supports a similar feature through the notion of ExclusionStrategies:

The GsonDataFormat accepts an ExclusionStrategy in its constructor:The line above will exclude fields annotated with @ExcludeAge when marshalling to JSON.

Configuring field naming policy

Available as of Camel 2.11

The GSON library supports specifying policies and strategies for mapping from json to POJO fields. A common naming convention is to map json fields using lower case with underscores.

We may have this JSON string

Which we want to map to a POJO that has getter/setters as

Then we can configure the org.apache.camel.component.gson.GsonDataFormat in a Spring XML files as shown below. Notice we use fieldNamingPolicy property to set the field mapping. This property is an enum from GSon com.google.gson.FieldNamingPolicy which has a number of pre defined mappings. If you need full control you can use the property FieldNamingStrategy and implement a custom com.google.gson.FieldNamingStrategy where you can control the mapping.

And use it in Camel routes by referring to its bean id as shown:

Include/Exclude fields using the jsonView attribute with JacksonDataFormat

Available as of Camel 2.12

As an example of using this attribute you can instead of:

Directly specify your JSON view inside the Java DSL as:

And the same in XML DSL:

Setting serialization include option for Jackson marshal

Available as of Camel 2.13.3/2.14

If you want to marshal a pojo to JSON, and the pojo has some fields with null values. And you want to skip these null values, then you need to set either an annotation on the pojo, 

But this requires you to include that annotation in your pojo source code. You can also configure the Camel JsonDataFormat to set the include option, as shown below:

Or from XML DSL you configure this as

Unmarshalling from json to POJO with dynamic class name

Available as of Camel 2.14

If you use jackson to unmarshal json to POJO, then you can now specify a header in the message that indicate which class name to unmarshal to.
The header has key CamelJacksonUnmarshalType if that header is present in the message, then Jackson will use that as FQN for the POJO class to unmarshal the json payload as. Notice that behavior is enabled out of the box from Camel 2.14 onwards. 

 For JMS end users there is the JMSType header from the JMS spec that indicates that also. To enable support for JMSType you would need to turn that on, on the jackson data format as shown:

Or from XML DSL you configure this as

Unmarshalling from json to List<Map> or List<pojo>

Available as of Camel 2.14

If you are using Jackson to unmarshal json to a list of map/pojo, you can now specify this by setting useList="true" or use the org.apache.camel.component.jackson.ListJacksonDataFormat. For example with Java you can do as shown below:

And if you use XML DSL then you configure to use list using useList attribute as shown below:

And you can specify the pojo type also

Using custom Jackson ObjectMapper

Available as of Camel 2.17

You can use custom Jackson ObjectMapper instance, can be configured as shown below.

Where myMapper is the id of the custom instance that Camel will lookup in the Registry

Using custom Jackson modules

Available as of Camel 2.15

You can use custom Jackson modules by specifying the class names of those using the moduleClassNames option as shown below.

When using moduleClassNames then the custom jackson modules are not configured, by created using default constructor and used as-is. If a custom module needs any custom configuration, then an instance of the module can be created and configured, and then use modulesRefs to refer to the module as shown below:

 Multiple modules can be specified separated by comma, such as moduleRefs="myJacksonModule,myOtherModule"

Enabling or disable features using Jackson

Available as of Camel 2.15

Jackson has a number of features you can enable or disable, which its ObjectMapper uses. For example to disable failing on unknown properties when marshalling, you can configure this using the disableFeatures:

You can disable multiple features by separating the values using comma. The values for the features must be the name of the enums from Jackson from the following enum classes

• com.fasterxml.jackson.databind.SerializationFeature
• com.fasterxml.jackson.databind.DeserializationFeature
• com.fasterxml.jackson.databind.MapperFeature

To enable a feature use the enableFeatures options instead.

From Java code you can use the type safe methods from camel-jackson module:

Converting Maps to POJO using Jackson

Available since Camel 2.16. Jackson ObjectMapper can be used to convert maps to POJO objects. Jackson component comes with the data converter that can be used to convert java.util.Map instance to non-String, non-primitive and non-Number objects.

If there is a single ObjectMapper instance available in the Camel registry, it will used by the converter to perform the conversion. Otherwise the default mapper will be used.  

Formatted JSON marshalling (pretty-printing)

Available as of Camel 2.16

Using the prettyPrint option one can output a well formatted JSON while marshalling:

And in Java DSL:

Please note that as of Camel 2.16 there’re 5 different overloaded json() DSL methods which support the prettyPrint option in combination with other settings for JsonLibrary, unmarshalType, jsonView etc. 

 

Integrating Jackson with Camel's TypeConverters

 

Available as of Camel 2.17

The camel-jackson module allows to integrate Jackson as a Type Converter in the Camel registry. This works in similar ways that camel-jaxb integrates with the type converter as well. However camel-jackson must be explicit enabled, which is done by setting some options on the CamelContext properties, as shown below:

The camel-jackson type converter integrates with JAXB which means you can annotate POJO class with JAXB annotations that Jackson can leverage. 

Dependencies for XStream

To use JSON in your camel routes you need to add the a dependency on camel-xstream which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Dependencies for Jackson

To use JSON in your camel routes you need to add the a dependency on camel-jackson which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Dependencies for GSON

To use JSON in your camel routes you need to add the a dependency on camel-gson which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

The Zip Data Format is a message compression and de-compression format. Messages marshalled using Zip compression can be unmarshalled using Zip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads. It facilitates more optimal use of network bandwidth while incurring a small cost in order to compress and decompress payloads at the endpoint.

Options

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing zip compression Deflater.BEST_COMPRESSION and send it an ActiveMQ queue called MY_QUEUE.

Alternatively if you would like to use the default setting you could send it as

Unmarshal

In this example we unmarshal a zipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnZippedMessageProcessor. Note that the compression Level employed during the marshalling should be identical to the one employed during unmarshalling to avoid errors.

Dependencies

This data format is provided in camel-core so no additional dependencies are needed.

TidyMarkup

TidyMarkup is a Data Format that uses the TagSoup to tidy up HTML. It can be used to parse ugly HTML and return it as pretty wellformed HTML.

TidyMarkup only supports the unmarshal operation as we really don't want to turn well formed HTML into ugly HTML

Java DSL Example

An example where the consumer provides some HTML

Spring XML Example

The following example shows how to use TidyMarkup to unmarshal using Spring

Dependencies

To use TidyMarkup in your camel routes you need to add the a dependency on camel-tagsoup which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Bindy

The goal of this component is to allow the parsing/binding of non-structured data (or to be more precise non-XML data)
to/from Java Beans that have binding mappings defined with annotations. Using Bindy, you can bind data from sources such as :

• CSV records,
• Fixed-length records,
• FIX messages,
• or almost any other non-structured data

to one or many Plain Old Java Object (POJO). Bindy converts the data according to the type of the java property. POJOs can be linked together with one-to-many relationships available in some cases. Moreover, for data type like Date, Double, Float, Integer, Short, Long and BigDecimal, you can provide the pattern to apply during the formatting of the property.

For the BigDecimal numbers, you can also define the precision and the decimal or grouping separators.

Decimal* = Double, Integer, Float, Short, Long

To work with camel-bindy, you must first define your model in a package (e.g. com.acme.model) and for each model class (e.g. Order, Client, Instrument, ...) add the required annotations (described hereafter) to the Class or field.

Annotations

The annotations created allow to map different concept of your model to the POJO like :

• Type of record (csv, key value pair (e.g. FIX message), fixed length ...),
• Link (to link object in another object),
• DataField and their properties (int, type, ...),
• KeyValuePairField (for key = value format like we have in FIX financial messages),
• Section (to identify header, body and footer section),
• OneToMany

This section will describe them :

1. CsvRecord

The CsvRecord annotation is used to identified the root class of the model. It represents a record = a line of a CSV file and can be linked to several children model classes.

case 1 : separator = ','

The separator used to segregate the fields in the CSV record is ',' :

10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

case 2 : separator = ';'

Compare to the previous case, the separator here is ';' instead of ',' :

10; J; Pauline; M; XD12345678; Fortis Dynamic 15/15; 2500; USD; 08-01-2009

case 3 : separator = '|'

Compare to the previous case, the separator here is '|' instead of ';' :

10| J| Pauline| M| XD12345678| Fortis Dynamic 15/15| 2500| USD| 08-01-2009

case 4 : separator = '\",\"'
Applies for Camel 2.8.2 or older

When the field to be parsed of the CSV record contains ',' or ';' which is also used as separator, we whould find another strategy
to tell camel bindy how to handle this case. To define the field containing the data with a comma, you will use simple or double quotes
as delimiter (e.g : '10', 'Street 10, NY', 'USA' or "10", "Street 10, NY", "USA").
Remark : In this case, the first and last character of the line which are a simple or double quotes will removed by bindy

"10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009"

From Camel 2.8.3/2.9 or never bindy will automatic detect if the record is enclosed with either single or double quotes and automatic remove those quotes when unmarshalling from CSV to Object. Therefore do not include the quotes in the separator, but simple do as below:

"10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009"

Notice that if you want to marshal from Object to CSV and use quotes, then you need to specify which quote character to use, using the quote attribute on the @CsvRecord as shown below:

case 5 : separator & skipfirstline

The feature is interesting when the client wants to have in the first line of the file, the name of the data fields :

order id, client id, first name, last name, isin code, instrument name, quantity, currency, date

To inform bindy that this first line must be skipped during the parsing process, then we use the attribute :

case 6 : generateHeaderColumns

To add at the first line of the CSV generated, the attribute generateHeaderColumns must be set to true in the annotation like this :

As a result, Bindy during the unmarshaling process will generate CSV like this :

order id, client id, first name, last name, isin code, instrument name, quantity, currency, date
10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009

case 7 : carriage return

If the platform where camel-bindy will run is not Windows but Macintosh or Unix, than you can change the crlf property like this. Three values are available : WINDOWS, UNIX or MAC

Additionally, if for some reason you need to add a different line ending character, you can opt to specify it using the crlf parameter. In the following example, we can end the line with a comma followed by the newline character:

case 8 : isOrdered

Sometimes, the order to follow during the creation of the CSV record from the model is different from the order used during the parsing. Then, in this case, we can use the attribute isOrdered = true to indicate this in combination with attribute 'position' of the DataField annotation.

Remark : pos is used to parse the file, stream while positions is used to generate the CSV

2. Link

The link annotation will allow to link objects together.

e.g : If the model Class Client is linked to the Order class, then use annotation Link in the Order class like this :

AND for the class Client :

3. DataField

The DataField annotation defines the property of the field. Each datafield is identified by its position in the record, a type (string, int, date, ...) and optionally of a pattern

case 1 : pos

This parameter/attribute represents the position of the field in the csv record

As you can see in this example the position starts at '1' but continues at '5' in the class Order. The numbers from '2' to '4' are defined in the class Client (see here after).

case 2 : pattern

The pattern allows to enrich or validates the format of your data

case 3 : precision

The precision is helpful when you want to define the decimal part of your number

case 4 : Position is different in output

The position attribute will inform bindy how to place the field in the CSV record generated. By default, the position used corresponds to the position defined with the attribute 'pos'. If the position is different (that means that we have an asymetric processus comparing marshaling from unmarshaling) than we can use 'position' to indicate this.

Here is an example

case 5 : required

If a field is mandatory, simply use the attribute 'required' setted to true

If this field is not present in the record, than an error will be raised by the parser with the following information :

Some fields are missing (optional or mandatory), line :

case 6 : trim

If a field has leading and/or trailing spaces which should be removed before they are processed, simply use the attribute 'trim' setted to true

case 7 : defaultValue

If a field is not defined then uses the value indicated by the defaultValue attribute

4. FixedLengthRecord

The FixedLengthRecord annotation is used to identified the root class of the model. It represents a record = a line of a file/message containing data fixed length formatted and can be linked to several children model classes. This format is a bit particular beause data of a field can be aligned to the right or to the left.
When the size of the data does not fill completely the length of the field, we can then add 'padd' characters.

case 1 : Simple fixed length record

This simple example shows how to design the model to parse/format a fixed message

10A9PaulineMISINXD12345678BUYShare2500.45USD01-08-2009

case 2 : Fixed length record with alignment and padding

This more elaborated example show how to define the alignment for a field and how to assign a padding character which is ' ' here''

10A9 PaulineM ISINXD12345678BUYShare2500.45USD01-08-2009

case 3 : Field padding

Sometimes, the default padding defined for record cannnot be applied to the field as we have a number format where we would like to padd with '0' instead of ' '. In this case, you can use in the model the attribute paddingField to set this value.

10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009

case 4: Fixed length record with delimiter

Fixed-length records sometimes have delimited content within the record. The firstName and lastName fields are delimited with the '^' character in the following example:

10A9Pauline^M^ISINXD12345678BUYShare000002500.45USD01-08-2009

case 5 : Fixed length record with record-defined field length

Occasionally a fixed-length record may contain a field that define the expected length of another field within the same record. In the following example the length of the instrumentNumber field value is defined by the value of instrumentNumberLen field in the record.

10A9Pauline^M^ISIN10XD12345678BUYShare000002500.45USD01-08-2009

case 6 : Fixed length record with header and footer

Bindy will discover fixed-length header and footer records that are configured as part of the model – provided that the annotated classes exist either in the same package as the primary @FixedLengthRecord class, or within one of the configured scan packages. The following text illustrates two fixed-length records that are bracketed by a header record and footer record.

101-08-2009
10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009
10A9 RichN ISINXD12345678BUYShare000002700.45USD01-08-2009
9000000002

case 7 : Skipping content when parsing a fixed length record. (Camel 2.11.1)

It is common to integrate with systems that provide fixed-length records containing more information than needed for the target use case. It is useful in this situation to skip the declaration and parsing of those fields that we do not need. To accomodate this, Bindy will skip forward to the next mapped field within a record if the 'pos' value of the next declared field is beyond the cursor position of the last parsed field. Using absolute 'pos' locations for the fields of interest (instead of ordinal values) causes Bindy to skip content between two fields.

Similarly, it is possible that none of the content beyond some field is of interest. In this case, you can tell Bindy to skip parsing of everything beyond the last mapped field by setting the ignoreTrailingChars property on the @FixedLengthRecord declaration.

5. Message

The Message annotation is used to identified the class of your model who will contain key value pairs fields. This kind of format is used mainly in Financial Exchange Protocol Messages (FIX). Nevertheless, this annotation can be used for any other format where data are identified by keys. The key pair values are separated each other by a separator which can be a special character like a tab delimitor (unicode representation : \u0009) or a start of heading (unicode representation : \u0001)

case 1 : separator = 'u0001'

The separator used to segregate the key value pair fields in a FIX message is the ASCII '01' character or in unicode format '\u0001'. This character must be escaped a second time to avoid a java runtime error. Here is an example :

8=FIX.4.1 9=20 34=1 35=0 49=INVMGR 56=BRKR 1=BE.CHM.001 11=CHM0001-01 22=4 ...

and how to use the annotation

6. KeyValuePairField

The KeyValuePairField annotation defines the property of a key value pair field. Each KeyValuePairField is identified by a tag (= key) and its value associated, a type (string, int, date, ...), optionaly a pattern and if the field is required

case 1 : tag

This parameter represents the key of the field in the message

case 2 : Different position in output

If the tags/keys that we will put in the FIX message must be sorted according to a predefine order, then use the attribute 'position' of the annotation @KeyValuePairField

7. Section

In FIX message of fixed length records, it is common to have different sections in the representation of the information : header, body and section. The purpose of the annotation @Section is to inform bindy about which class of the model represents the header (= section 1), body (= section 2) and footer (= section 3)

Only one attribute/parameter exists for this annotation.

case 1 : Section

A. Definition of the header section

B. Definition of the body section

C. Definition of the footer section

8. OneToMany

The purpose of the annotation @OneToMany is to allow to work with a List<?> field defined a POJO class or from a record containing repetitive groups.

The relation OneToMany ONLY WORKS in the following cases :

• Reading a FIX message containing repetitive groups (= group of tags/keys)
• Generating a CSV with repetitive data

case 1 : Generating CSV with repetitive data

Here is the CSV output that we want :

Claus,Ibsen,Camel in Action 1,2010,35
Claus,Ibsen,Camel in Action 2,2012,35
Claus,Ibsen,Camel in Action 3,2013,35
Claus,Ibsen,Camel in Action 4,2014,35

Remark : the repetitive data concern the title of the book and its publication date while first, last name and age are common

and the classes used to modeling this. The Author class contains a List of Book.

Very simple isn't it !!!

case 2 : Reading FIX message containing group of tags/keys

Here is the message that we would like to process in our model :

"8=FIX 4.19=2034=135=049=INVMGR56=BRKR"
"1=BE.CHM.00111=CHM0001-0158=this is a camel - bindy test"
"22=448=BE000124567854=1"
"22=548=BE000987654354=2"
"22=648=BE000999999954=3"
"10=220"

tags 22, 48 and 54 are repeated

and the code

Using the Java DSL

The next step consists in instantiating the DataFormat bindy class associated with this record type and providing Java package name(s) as parameter.

For example the following uses the class BindyCsvDataFormat (who correspond to the class associated with the CSV record type) which is configured with "com.acme.model"
package name to initialize the model objects configured in this package.

Setting locale

Bindy supports configuring the locale on the dataformat, such as 

Or to use the platform default locale then use "default" as the locale name. Notice this requires Camel 2.14/2.13.3/2.12.5.

for older releases you can set it using Java code as shown

Unmarshaling

Alternatively, you can use a named reference to a data format which can then be defined in your Registry e.g. your Spring XML file:

The Camel route will pick-up files in the inbox directory, unmarshall CSV records into a collection of model objects and send the collection
to the route referenced by 'handleOrders'.

The collection returned is a List of Map objects. Each Map within the list contains the model objects that were marshalled out of each line of the CSV. The reason behind this is that each line can correspond to more than one object. This can be confusing when you simply expect one object to be returned per line.

Each object can be retrieve using its class name.

Assuming that you want to extract a single Order object from this map for processing in a route, you could use a combination of a Splitter and a Processor as per the following:

Take care of the fact that Bindy uses CHARSET_NAME property or the CHARSET_NAME header as define in the Exchange interface to do a characterset conversion of the inputstream received for unmarshalling. In some producers (e.g. file-endpoint) you can define a characterset. The characterset conversion can already been done by this producer. Sometimes you need to remove this property or header from the exchange before sending it to the unmarshal. If you don't remove it the conversion might be done twice which might lead to unwanted results.

Marshaling

To generate CSV records from a collection of model objects, you create the following route :

Using Spring XML

This is really easy to use Spring as your favorite DSL language to declare the routes to be used for camel-bindy. The following example shows two routes where the first will pick-up records from files, unmarshal the content and bind it to their model. The result is then send to a pojo (doing nothing special) and place them into a queue.

The second route will extract the pojos from the queue and marshal the content to generate a file containing the csv record. The example above is for using Camel 2.16 onwards.

Dependencies

To use Bindy in your camel routes you need to add the a dependency on camel-bindy which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

XMLSecurity Data Format

The XMLSecurity Data Format facilitates encryption and decryption of XML payloads at the Document, Element, and Element Content levels (including simultaneous multi-node encryption/decryption using XPath). To sign messages using the XML Signature specification, please see the Camel XML Security component.

The encryption capability is based on formats supported using the Apache XML Security (Santuario) project. Symmetric encryption/decryption is currently supported using Triple-DES and AES (128, 192, and 256) encryption formats. Additional formats can be easily added later as needed. This capability allows Camel users to encrypt/decrypt payloads while being dispatched or received along a route.

Available as of Camel 2.9
The XMLSecurity Data Format supports asymmetric key encryption. In this encryption model a symmetric key is generated and used to perform XML content encryption or decryption. This "content encryption key" is then itself encrypted using an asymmetric encryption algorithm that leverages the recipient's public key as the "key encryption key". Use of an asymmetric key encryption algorithm ensures that only the holder of the recipient's private key can access the generated symmetric encryption key. Thus, only the private key holder can decode the message. The XMLSecurity Data Format handles all of the logic required to encrypt and decrypt the message content and encryption key(s) using asymmetric key encryption.

The XMLSecurity Data Format also has improved support for namespaces when processing the XPath queries that select content for encryption. A namespace definition mapping can be included as part of the data format configuration. This enables true namespace matching, even if the prefix values in the XPath query and the target xml document are not equivalent strings.

Basic Options

Asymmetric Encryption Options

These options can be applied in addition to relevant the Basic options to use asymmetric key encryption.

Key Cipher Algorithm

As of Camel 2.12.0, the default Key Cipher Algorithm is now XMLCipher.RSA_OAEP instead of XMLCipher.RSA_v1dot5. Usage of XMLCipher.RSA_v1dot5 is discouraged due to various attacks. Requests that use RSA v1.5 as the key cipher algorithm will be rejected unless it has been explicitly configured as the key cipher algorithm.

Marshal

In order to encrypt the payload, the marshal processor needs to be applied on the route followed by the secureXML() tag.

Unmarshal

In order to decrypt the payload, the unmarshal processor needs to be applied on the route followed by the secureXML() tag.

Examples

Given below are several examples of how marshalling could be performed at the Document, Element, and Content levels.

Full Payload encryption/decryption

Partial Payload Content Only encryption/decryption

Partial Multi Node Payload Content Only encryption/decryption

Partial Payload Content Only encryption/decryption with choice of passPhrase(password)

Partial Payload Content Only encryption/decryption with passPhrase(password) and Algorithm

Partial Payload Content with Namespace support

Java DSL

Spring XML

A namespace prefix that is defined as part of the camelContext definition can be re-used in context within the data format secureTag attribute of the secureXML element.

Asymmetric Key Encryption

Spring XML Sender

Spring XML Recipient

Dependencies

This data format is provided within the camel-xmlsecurity component.

The GZip Data Format is a message compression and de-compression format. It uses the same deflate algorithm that is used in Zip DataFormat, although some additional headers are provided. This format is produced by popular gzip/gunzip tool. Messages marshalled using GZip compression can be unmarshalled using GZip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads or when you read messages previously comressed using gzip tool.

Options

There are no options provided for this data format.

Marshal

In this example we marshal a regular text/XML payload to a compressed payload employing gzip compression format and send it an ActiveMQ queue called MY_QUEUE.

Unmarshal

In this example we unmarshal a gzipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnGZippedMessageProcessor.

Dependencies

This data format is provided in camel-core so no additional dependencies is needed.

Castor

Available as of Camel 2.1

Castor is a Data Format which uses the Castor XML library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload.

As usually you can use either Java DSL or Spring XML to work with Castor Data Format.

Using the Java DSL

For example the following uses a named DataFormat of Castor which uses default Castor data binding features.

If you prefer to use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g.

If you want to override default mapping schema by providing a mapping file you can set it as follows.

Also if you want to have more control on Castor Marshaller and Unmarshaller you can access them as below.

Using Spring XML

The following example shows how to use Castor to unmarshal using Spring configuring the castor data type

This example shows how to configure the data type just once and reuse it on multiple routes. You have to set the <castor> element directly in <camelContext>.

Options

Castor supports the following options

Dependencies

To use Castor in your camel routes you need to add the a dependency on camel-castor which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

Protobuf - Protocol Buffers

"Protocol Buffers - Google's data interchange format"

Camel provides a Data Format to serialse between Java and the Protocol Buffer protocol. The project's site details why you may wish to choose this format over xml. Protocol Buffer is language-neutral and platform-neutral, so messages produced by your Camel routes may be consumed by other language implementations.

API Site
Protobuf Implementation
Protobuf Java Tutorial

Protobuf overview

This quick overview of how to use Protobuf. For more detail see the complete tutorial

Defining the proto format

The first step is to define the format for the body of your exchange. This is defined in a .proto file as so:

Generating Java classes

The Protobuf SDK provides a compiler which will generate the Java classes for the format we defined in our .proto file. You can run the compiler for any additional supported languages you require.

protoc --java_out=. ./addressbook.proto

This will generate a single Java class named AddressBookProtos which contains inner classes for Person and AddressBook. Builders are also implemented for you. The generated classes implement com.google.protobuf.Message which is required by the serialisation mechanism. For this reason it important that only these classes are used in the body of your exchanges. Camel will throw an exception on route creation if you attempt to tell the Data Format to use a class that does not implement com.google.protobuf.Message. Use the generated builders to translate the data from any of your existing domain classes.

Java DSL

You can use create the ProtobufDataFormat instance and pass it to Camel DataFormat marshal and unmarsha API like this.

Or use the DSL protobuf() passing the unmarshal default instance or default instance class name like this.

Spring DSL

The following example shows how to use Castor to unmarshal using Spring configuring the protobuf data type

Dependencies

To use Protobuf in your camel routes you need to add the a dependency on camel-protobuf which implements this data format.

If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions).

SOAP DataFormat

Available as of Camel 2.3

SOAP is a Data Format which uses JAXB2 and JAX-WS annotations to marshal and unmarshal SOAP payloads. It provides the basic features of Apache CXF without need for the CXF Stack.

ElementNameStrategy

An element name strategy is used for two purposes. The first is to find a xml element name for a given object and soap action when marshaling the object into a SOAP message. The second is to find an Exception class for a given soap fault name.

If you have generated the web service stub code with cxf-codegen or a similar tool then you probably will want to use the ServiceInterfaceStrategy. In the case you have no annotated service interface you should use QNameStrategy or TypeNameStrategy.

Using the Java DSL

The following example uses a named DataFormat of soap which is configured with the package com.example.customerservice to initialize the JAXBContext. The second parameter is the ElementNameStrategy. The route is able to marshal normal objects as well as exceptions. (Note the below just sends a SOAP Envelope to a queue. A web service provider would actually need to be listening to the queue for a SOAP call to actually occur, in which case it would be a one way SOAP request. If you need request reply then you should look at the next example.)

Using SOAP 1.2

Available as of Camel 2.11

When using XML DSL there is a version attribute you can set on the <soapjaxb> element.

And in the Camel route

Multi-part Messages

Available as of Camel 2.8.1

Multi-part SOAP messages are supported by the ServiceInterfaceStrategy. The ServiceInterfaceStrategy must be initialized with a service interface definition that is annotated in accordance with JAX-WS 2.2 and meets the requirements of the Document Bare style. The target method must meet the following criteria, as per the JAX-WS specification: 1) it must have at most one in or in/out non-header parameter, 2) if it has a return type other than void it must have no in/out or out non-header parameters, 3) if it it has a return type of void it must have at most one in/out or out non-header parameter.

The ServiceInterfaceStrategy should be initialized with a boolean parameter that indicates whether the mapping strategy applies to the request parameters or response parameters.

Multi-part Request

The payload parameters for a multi-part request are initiazlied using a BeanInvocation object that reflects the signature of the target operation. The camel-soap DataFormat maps the content in the BeanInvocation to fields in the SOAP header and body in accordance with the JAX-WS mapping when the marshal() processor is invoked.

Multi-part Response

A multi-part soap response may include an element in the soap body and will have one or more elements in the soap header. The camel-soap DataFormat will unmarshall the element in the soap body (if it exists) and place it onto the body of the out message in the exchange. Header elements will not be marshaled into their JAXB mapped object types. Instead, these elements are placed into the camel out message header org.apache.camel.dataformat.soap.UNMARSHALLED_HEADER_LIST. The elements will appear either as element instance values, or as JAXBElement values, depending upon the setting for the ignoreJAXBElement property. This property is inherited from camel-jaxb.

You can also have the camel-soap DataFormate ignore header content all-together by setting the ignoreUnmarshalledHeaders value to true.

Holder Object mapping

JAX-WS specifies the use of a type-parameterized javax.xml.ws.Holder object for In/Out and Out parameters. A Holder object may be used when building the BeanInvocation, or you may use an instance of the parameterized-type directly. The camel-soap DataFormat marshals Holder values in accordance with the JAXB mapping for the class of the Holder's value. No mapping is provided for Holder objects in an unmarshalled response.

Examples

Webservice client

The following route supports marshalling the request and unmarshalling a response or fault.

The below snippet creates a proxy for the service interface and makes a SOAP call to the above route.

Webservice Server

Using the following route sets up a webservice server that listens on jms queue customerServiceQueue and processes requests using the class CustomerServiceImpl. The customerServiceImpl of course should implement the interface CustomerService. Instead of directly instantiating the server class it could be defined in a spring context as a regular bean.

Dependencies

To use the SOAP dataformat in your camel routes you need to add the following dependency to your pom.

Crypto

Available as of Camel 2.3
PGP Available as of Camel 2.9

The Crypto Data Format integrates the Java Cryptographic Extension into Camel, allowing simple and flexible encryption and decryption of messages using Camel's familiar marshall and unmarshal formatting mechanism. It assumes marshalling to mean encryption to cyphertext and unmarshalling to mean decryption back to the original plaintext. This data format implements only symmetric (shared-key) encryption and decyption.

Options

Basic Usage

At its most basic all that is required to encrypt/decrypt an exchange is a shared secret key. If one or more instances of the Crypto data format are configured with this key the format can be used to encrypt the payload in one route (or part of one) and decrypted in another. For example, using the Java DSL as follows:

In Spring the dataformat is configured first and then used in routes

Specifying the Encryption Algorithm

Changing the algorithm is a matter of supplying the JCE algorithm name. If you change the algorithm you will need to use a compatible key.

A list of the available algorithms in Java 7 is available via the Java Cryptography Architecture Standard Algorithm Name Documentation.

Specifying an Initialization Vector

Some crypto algorithms, particularly block algorithms, require configuration with an initial block of data known as an Initialization Vector. In the JCE this is passed as an AlgorithmParameterSpec when the Cipher is initialized. To use such a vector with the CryptoDataFormat you can configure it with a byte[] containing the required data e.g.

or with spring, suppling a reference to a byte[]The same vector is required in both the encryption and decryption phases. As it is not necessary to keep the IV a secret, the DataFormat allows for it to be inlined into the encrypted data and subsequently read out in the decryption phase to initialize the Cipher. To inline the IV set the /oinline flag.or with spring.For more information of the use of Initialization Vectors, consult

• http://en.wikipedia.org/wiki/Initialization_vector
• http://www.herongyang.com/Cryptography/
• http://en.wikipedia.org/wiki/Block_cipher_modes_of_operation

Hashed Message Authentication Codes (HMAC)

To avoid attacks against the encrypted data while it is in transit the CryptoDataFormat can also calculate a Message Authentication Code for the encrypted exchange contents based on a configurable MAC algorithm. The calculated HMAC is appended to the stream after encryption. It is separated from the stream in the decryption phase. The MAC is recalculated and verified against the transmitted version to insure nothing was tampered with in transit.For more information on Message Authentication Codes see http://en.wikipedia.org/wiki/HMAC

or with spring.By default the HMAC is calculated using the HmacSHA1 mac algorithm though this can be easily changed by supplying a different algorithm name. See here for how to check what algorithms are available through the configured security providersor with spring.

Supplying Keys Dynamically

When using a Recipient list or similar EIP the recipient of an exchange can vary dynamically. Using the same key across all recipients may neither be feasible or desirable. It would be useful to be able to specify keys dynamically on a per exchange basis. The exchange could then be dynamically enriched with the key of its target recipient before being processed by the data format. To facilitate this the DataFormat allow for keys to be supplied dynamically via the message headers below

• CryptoDataFormat.KEY "CamelCryptoKey"

or with spring.

PGP Message

The PGP Data Formater can create and decrypt/verify PGP Messages of the following PGP packet structure (entries in brackets are optional and ellipses indicate repetition, comma represents  sequential composition, and vertical bar separates alternatives):

    Public Key Encrypted Session Key ..., Symmetrically Encrypted Data | Sym. Encrypted and Integrity Protected Data, (Compressed Data,) (One Pass Signature ...,) Literal Data, (Signature ...,)

Since Camel 2.16.0 the Compressed Data packet is optional, before it was mandatory.

 

PGPDataFormat Options

PGPDataFormat Message Headers

You can override the PGPDataFormat options by applying below headers into message dynamically.

 

Encrypting with PGPDataFormat

The following sample uses the popular PGP format for encrypting/decrypting files using the Bouncy Castle Java libraries:

The following sample performs signing + encryption, and then signature verification + decryption. It uses the same keyring for both signing and encryption, but you can obviously use different keys:Or using Spring:

To work with the previous example you need the following

• A public keyring file which contains the public keys used to encrypt the data
• A private keyring file which contains the keys used to decrypt the data
• The keyring password

Managing your keyring

To manage the keyring, I use the command line tools, I find this to be the simplest approach in managing the keys. There are also Java libraries available from http://www.bouncycastle.org/java.html if you would prefer to do it that way.

1. Install the command line utilities on linux

2. Create your keyring, entering a secure password

3. If you need to import someone elses public key so that you can encrypt a file for them.

4. The following files should now exist and can be used to run the example

PGP Decrypting/Verifying of Messages Encrypted/Signed by Different Private/Public Keys

Since Camel 2.12.2.

A PGP Data Formater can decrypt/verify messages which have been encrypted by different public keys or signed by different private keys. Just, provide the corresponding private keys in the secret keyring, the corresponding public keys in the public keyring, and the passphrases in the passphrase accessor.

• The functionality is especially useful to support the key exchange. If you want to exchange the private key for decrypting you can accept for a period of time messages which are either encrypted with the old or new corresponding public key. Or if the sender wants to exchange his signer private key, you can accept for a period of time, the old or new signer key.
• Technical background: The PGP encrypted data contains a Key ID of the public key which was used to encrypt the data. This Key ID can be used to locate the private key in the secret keyring to decrypt the data. The same mechanism is also used to locate the public key for verifying a signature. Therefore you no longer must specify User IDs for the unmarshaling.

Restricting the Signer Identities during PGP Signature Verification

Since Camel 2.12.3.

If you verify a signature you not only want to verify the correctness of the signature but you also want check that the signature comes from a certain identity or a specific set of identities. Therefore it is possible to restrict the number of public keys from the public keyring which can be used for the verification of a signature.  

• If the PGP content has several signatures the verification is successful as soon as one signature can be verified.
• If you do not want to restrict the signer identities for verification then do not specify the signature key User IDs. In this case all public keys in the public keyring are taken into account.

Several Signatures in One PGP Data Format

Since Camel 2.12.3.

The PGP specification allows that one PGP data format can contain several signatures from different keys. Since Camel 2.13.3 it is possible to create such kind of PGP content via specifying signature User IDs which relate to several private keys in the secret keyring.

Support of Sub-Keys and Key Flags in PGP Data Format Marshaler

Since Camel 2.12.3.
An OpenPGP V4 key can have a primary key and sub-keys. The usage of the keys is indicated by the so called Key Flags. For example, you can have a primary key with two sub-keys; the primary key shall only be used for certifying other keys (Key Flag 0x01), the first sub-key  shall only be used for signing (Key Flag 0x02), and the second sub-key shall only be used for encryption (Key Flag 0x04 or 0x08). The PGP Data Format marshaler takes into account these Key Flags of the primary key and sub-keys in order to determine the right key for signing and encryption. This is necessary because the primary key and its sub-keys have the same User IDs.

Support of Custom Key Accessors

Since Camel 2.13.0.
You can implement custom key accessors for encryption/signing. The above PGPDataFormat class selects in a certain predefined way the keys which should be used for signing/encryption or verifying/decryption. If you have special requirements how your keys should be selected you should use the PGPKeyAccessDataFormat class instead and implement the interfaces PGPPublicKeyAccessor and PGPSecretKeyAccessor as beans. There are default implementations DefaultPGPPublicKeyAccessor and DefaultPGPSecretKeyAccessor which cache the keys, so that not every time the keyring is parsed when the processor is called.

PGPKeyAccessDataFormat has the same options as PGPDataFormat except password, keyFileName, encryptionKeyRing, signaturePassword, signatureKeyFileName, and signatureKeyRing.

Dependencies

To use the Crypto dataformat in your camel routes you need to add the following dependency to your pom.

See Also

• Data Format
• Crypto (Digital Signatures)
• http://www.bouncycastle.org/java.html

Syslog DataFormat

Available as of Camel 2.6

The syslog dataformat is used for working with RFC3164 and RFC5424 messages.

This component supports the following:

• UDP consumption of syslog messages
• Agnostic data format using either plain String objects or SyslogMessage model objects.
• Type Converter from/to SyslogMessage and String
• Integration with the camel-mina component.
• Integration with the camel-netty component.
• Camel 2.14: Encoder and decoder for the camel-netty component.
• Camel 2.14: Support for RFC5424 also.

Maven users will need to add the following dependency to their pom.xml for this component:

RFC3164 Syslog protocol

Syslog uses the user datagram protocol (UDP) 1 as its underlying transport layer mechanism.
The UDP port that has been assigned to syslog is 514.

To expose a Syslog listener service we reuse the existing camel-mina component or camel-netty where we just use the Rfc3164SyslogDataFormat to marshal and unmarshal messages. Notice that from Camel 2.14 onwards the syslog dataformat is renamed to SyslogDataFormat.

RFC5424 Syslog protocol

Available as of Camel 2.14

To expose a Syslog listener service we reuse the existing camel-mina component or camel-netty where we just use the SyslogDataFormat to marshal and unmarshal messages

Exposing a Syslog listener

In our Spring XML file, we configure an endpoint to listen for udp messages on port 10514, note that in netty we disable the defaultCodec, this
will allow a fallback to a NettyTypeConverter and delivers the message as an InputStream:

The same route using camel-mina

Sending syslog messages to a remote destination

See Also

• Configuring Camel
• Component
• Endpoint
• Getting Started