|
CSVReader
A simple example that shows several ways to read a csv file
Table of contents
IntroductionConsider you have the following CSV file: Holger;Schmidt;35 Max;Mustermann;17 Lisa;Stein;19 Each line represents a person. Each person has a first name, a last name and an age. Nothing exciting this far... It is very easy to parse this csv file using jCSV. It comes with a fully configured CSVReader that does (mostly) all of the work. Parse the dataThe first thing to do, is to read the csv file and translate it into java objects, so that the data can ba used in your program. There are currently three ways to do this:
Parse the data as a String arrayA straightforward approach would be to read the file line by line. Each line would be split at ';', so we would receive an String array containing the three data parts. The default CSVParser configuration follows this approach, you can obtain this parser using the static factory method CSVReaderBuilder.newDefaultReader(Reader reader). An example code might look like this: Reader reader = new FileReader("persons.csv");
CSVReader<String[]> csvPersonReader = CSVReaderBuilder.newDefaultReader(reader);
List<String[]> persons = csvPersonReader.readAll();You see, it's quite easy to read a CSV file and use the data in your program. Create Java objects using the CSVEntryParserBut it would be a lot easier to use a List of Person objects in your program. jCSV can achieve this for you, it can convert each row into a java object. All we have to do is to tell jCSV how to convert a row from the csv file into a Person object. For this purpose exists an interface called CSVEntryParser: public interface CSVEntryParser<E> {
public E parseEntry(String... data);
}The String array represents the columns of the csv file. The method parseEntry converts this String array to an object of the class E, E will be Person in our example. So all you have to do, is to provide an appropiate CSVEntryParser, a straightforward implementation might look like this: public class PersonEntryParser implements CSVEntryParser<Person> {
public Person parseEntry(String... data) {
String firstname = data[0];
String lastname = data[1];
int age = Integer.parseInt(data[2]);
return new Person(firstname, lastname, age);
}
}Once you have written this entry parser, you can use the CSVReaderBuilder to provide your CSVEntryParser implementation: Reader reader = new FileReader("persons.csv");
CSVReader<Person> csvPersonReader = new CSVReaderBuilder<Person>(reader).entryParser(new PersonEntryParser()).build();
List<Person> persons = csvPersonReader.readAll();That's it, you have completely parsed a csv file in just 4 lines! Create Java objects using annotationsInstead of implementing an entry parser, you can instruct jCSV with an annotated class, that specifies how to build the Person objects. All we have to do is to map the columns from the csv file to the class's properties. This mapping is done via annotations in the class that we would like to build. The annotation we use for this purpose is MapToColumn, it comes with two parameter:
The annotated Person class may look like this: public class Person {
@MapToColumn(column=0)
private String firstName;
@MapToColumn(column=1)
private String lastName;
@MapToColumn(column=2)
private int age;
// getter, equals, toString, ...
}This code should be self explaining if you look at the csv file above. You now have to provide this information to jCSV via the AnnotationEntryParser: Reader reader = new FileReader("persons.csv");
ValueProcessorProvider provider = new ValueProcessorProvider();
CSVEntryParser<Person> entryParser = new AnnotationEntryParser<Person>(Person.class, provider);
CSVReader<Person> csvPersonReader = new CSVReaderBuilder<Person>(reader).entryParser(entryParser).build();
List<Person> persons = csvPersonReader.readAll();Thats all. There is no need to implement an entry parser anymore. jCSV uses reflection to set the values, so there are a few drawbacks if you choose to parse your csv using annotations:
Value processorsjCSV uses value processors to convert the String data to the needed type. As you can see in the example above, the third column is mapped to an int property. jCSV now uses the IntegerProcessor to convert the String into an Integer. There are several default value processors, such as primitives (and its wrapper types), a StringProcessor and a DateProcessor that uses the default SimpleDateFormat. If these value processors are not enough, you can simply add your own value processors. Consider you have a column that holds an address and should be converted to an Address object. The value processor for this type might look look this: public class AddressProcessor implements ValueProcessor<Address> {
@Override
public Address processColumn(String value) {
Address address = toAddress(value);
return address;
}
}To publish this value processor, simply call: ValueProcessorProvider vpp = new ValueProcessorProvider(); vpp.registerValueProcessor(Address.class, new AddressProcessor()); However, in the most cases, you do not have to provide your own ValueProcessor. The default implementations of the primitives, it's wrapper classes, String and Date should be enough. Read the entriesAfter configuring the CSVReader, you can access the data. As you may have seen above, one apprach is to read the whole data into a List. This can be done, calling the readAll() method. List<Person> persons = csvReader.readAll(); If you have a large csv file, you possibly can not read the whole csv file into your memory. Therefore, there is the possbility to incrementaly read the records of the file. The CSVReader interfaces extends the Iterable interface, so you can simply iterate over the records using an Iterator: Iterator<Person> it = csvPersonReader.iterator();
while (it.hasNext()) {
Person p = it.next();
// ...
}Another option is to use the for each loop: for (Person p : csvPersonReader) {
/// ...
}Examples | |
a predefined DateProcessor? taking a SimpleDateFormat? would be handy. however, using static ValueProcessorProvider? is quite ugly, it implies that all parser instances share common singleton-like configuration. in case I have different date formats in varouis locales, my DateProcessor? looks ugly with a if-statement cascade :( I'd prefer to configure each parser instance separately.
Hi again, I've already made the ValueProcessorProvider? non-final in version 1.4.0, that really was ugly... ;) The AnnotationEntryParser? now takes a ValueProcessorProvider? instance. I hope that I can finalize that version soon and release it. http://code.google.com/p/jcsv/source/browse/trunk/src/main/java/com/googlecode/jcsv/reader/internal/AnnotationEntryParser.java
The DateProcessor? sound like a good enhancement. This Processor will be implemented in version 1.4.0!
If you have any additional ideas, please let me know :)
reasAll -> readAll
Danke für den Hinweis, ist korrigiert.
Hi I have found this project very useful. A couple of things that caused me a bit of a problem though were that if I annotate a class with @MapToColumn? (column = 0) etc but then extend the class the superclass annotations and members don't get populated (all nulls). What am I missing here. Also I have some csv files that have more than one header row is there a way to overide the number of header rows? so that I can simply call get header with the number of header rows to read? Cheers
if i come across some bad values in a csv line and want to continue to the next line from within a custom CSVEntryParser i'd expect to return null, but this prevents the remainder of the file from being parsed. :-(
is this project still active ?
Hi seanjones1966, I do not have much time right now, but I'm planning to release Version 1.4 in the next few weeks (depends on my free time). There will be some bugfixes and feature requests in that version.
I have multiple CSV files like persons, computers, animals etc. I want to build a single function for all these CSV files.
Reader reader = new FileReader?(path);
ValueProcessorProvider? provider = new ValueProcessorProvider?(); CSVEntryParser<?> entryParser = new AnnotationEntryParser?<Person>(Person.class, provider); CSVReader<?> csvPersonReader = new CSVReaderBuilder<Person>(reader).entryParser(entryParser).build();
can u tell me how to correctly implement the above function? List<Person> persons = csvPersonReader.readAll();
how to parse Date Formate frm csv to java
Please please please PLEASE Show the example "Create Java objects using annotations" into a real java file, I cannot make it work :( I got "can not instantiate class dataFromCsv.DataFromCsv?$Person" Error even with the constructor public Person(){}
Abiyu: You might have to resolve the CSVReader<..> templated type with "com.googlecode.jcsv.reader.CSVReader". That's how I got it working: