Wednesday, August 12, 2009

Apache Camel: The File Componenet

A very common task for so-called batch processes, is to read some files in some directory, so some processing for these files (whereas it might be required to do some data instrumentation with the data from various sources), and store the data in the database.

Here's a file source endpoint definition:

String uri = "file:files" +
"?initialDelay=3000" +
"&delay=1000" +
"&useFixedDelay=true" +
"&include=.*[.]xml" +
"&move=backup/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.xml" +

So consider this route:


With file endpoint definition above, it means, that with consume the files from some directory called "files", the initial delay for the files polling is 3 seconds, and we will poll with fixed intervals - 1 second, filtering out non-xml files.

In addition to that, if a file is processed successfully, it is being moved to a "backup" directory appending a time stamp in the file's name.

A new feature is a moveFailure attribute, which in case of failure in someProcessor or in the target endpoint ("someBean") allows you to handle the failed files, e.g. renaming or moving to another directory. At the time of writing the feature is available in snapshot version of Camel.


Claus Ibsen said...

Hi Anton

Nice little wrapup of the move options on the file component.

And credit to you for bring out in the light we needed the new movedFailed option.

Unknown said...

Thx Claus!

I have also a solution for the synchronization problem between the routes in my next post...

Disqus for Code Impossible