java - Spring Batch - Is it possible to have a dynamic column in FlatFileReader?

Question

I'm dealing with many CSVs files that don't have a fixed header/column, saying that I can get file1.csv with 10 column and file2.csv with 50 column.

I can't know in advance the number of column that I'll have, I can't create a specific job for each file type, my input will be a black box: bunch of CSV that will have an X number of column from 10 to infinite.

As I want to use Spring Batch to auto import these CSVs, I want to know if it is possible? I know that I have to get a fixed number of column because of the processor and the fact that I need to serialize my data into a POJO before sending it back to a writer.

Could my processor serialize an Array? beside sending one simple Object, can I get an Array of Object and in the end of my job I'll will have an Array of an Array of Object?

What do you think?

Thanks

Answer 1

Create your own LineTokenizer implementation. The DelimitedLineTokenizer expects a predefined number of columns. If you create your own, you can be as dynamic as you want. You can read more about the LineTokenizer in the documentation here: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/file/transform/LineTokenizer.html