Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I know this is trivial issue, but I just feel this can be more elegant.

So I need to write/read data files for my program, lets say they are CSV for now. I can implement the format as I see fit, but I may have need to change that format later. The simply thing to do is something like

out.write(For.getValue()+","+bar.getMinValue()+","+fi.toString());

This is easy to write, but obviously is guilty of hard coding and the general 'magic number' issue. The format is hard-coded, requires parsing of the code to figure out the file format, and changing the format requires changing multiple methods.

I could instead have my constants specifying the location that I want each variable to be saved in the CSV file to remove some of the 'magic numbers'; then save/load into the an array at the location specified by the constants:

int FOO_LOCATION=0;
int BAR_MIN_VAL_LOCATION=1;
int FI_LOCATION=2
int NUM_ARGUMENTS=3;

String[] outputArguments=new String[NUM_ARGUMENTS];
outputArguments[FOO_LOCATION] = foo.getValue();
outputArgumetns[BAR_MIN_VAL_LOCATION] = bar.getMinValue();
outptArguments[FI_LOCATOIN==fi.toString();

writeAsCSV(outputArguments);

But this is...extremely verbose and still a bit ugly. It makes it easy to see the format of existing CSV and to swap the location of variables within the file easily. However, if I decide to add an extra value to the csv I need to not only add a new constant, but also modify the read and write methods to add the logic that actually saves/reads the argument from the array; I still have to hunt down every method using these variables and change them by hand!

If I use Java enums I can clean this up slightly, but the real issue is still present. Short of some sort of functional programming (and java's inner classes are too ugly to be considered functional) I still have no obvious way of clearly expressing what variable is associated with each constant short of writing (and maintaining) it in the read/write methods. For instance I still need to write somewhere that the FOO_LOCATION specifies the location of foo.getValue().

It seems as if there should be a prettier, easier to maintain, manner for approaching this?

Incidentally, I'm working in java at the moment, however, I am interested conceptually about the design approach regardless of language. Some library in java that does all the work for me is definitely welcome (though it may prove more hassle to get permission to add it to the codebase then to just write something by hand quickly), but what I'm really asking is more about how to write elegant code if you had to do this by hand.

share|improve this question
    
Sounds like you have a not-stupid reason to use an interface. And yes, stop assigning significance to array position. That's just yucky. –  Erik Reppen Oct 30 '13 at 23:42
4  
Does it have to be CSV? A self describing and flexible format like JSON or XML would be far easier to extend/modify/parse. Either way, an object mapper like Jackson databind might make your life easier (and it'll write to CSV, JSON or XML). –  tom Oct 30 '13 at 23:50
    
unless i am misunderstanding the question, why not use a header row to define the columns and use one of the many Java CSV parsers to read/write the files? Then you can have any combination of columns in any order, when reading & writing the header row tells the parser the field layout/format of each file. –  james Oct 31 '13 at 2:47
    
I actually do think CSV makes more sense here. The stuff I'm writing out is very simple, and we want it to be easy to parse AND read. XML is too hard to parse for non-geeks. JSON would make more sense, but I still feel for as simple as these files are JSON risks being too verbos to the point of hurting ability of human to parse by hand. Plus...the format is actually forced to be WINI by external programs, I just didn't want to confused the question by explaining that since WINI is nearly the same as writing CSV in this case... –  dsollen Oct 31 '13 at 16:06
add comment

6 Answers

up vote 4 down vote accepted

Regardless of language, if you don't want to hard code the values, then one will need some sort of meta data that describes how the data will be mapped, formatted, and output.

In this example, it could be called a mapper or formatter.

Map/format files (XML, JSON, or simliar) would describe how the data would be formatted and written. Your application would read in the map/format file and use it to create the output.

Then your application could format the CSV anyway you see fit, without a programming change. One could also extend this to flat or fixed formats and XML as well. Then your code is generic as it uses the mapping meta data to create the CSV file.

For the CSV case, at a high level one would need to describe:

  • Name and Order of fields
  • Delimiter (Sometimes comma is not used)
  • Whether or not to include a header
  • Whether or not to include quotes around the data

Just as a side note, there is a time difference to develop the two applications. Hardcoded field values are much faster to develop, although as you have pointed out less elegant. But if you need to get something done quickly, the aproach is OK.

Developing something more generic would take more time up front, but if your producing a lot of CSV formatted files of different format, in the long run one would get ROI on it.

With this approach, one can also write some sort of nice "GUI" for business analysts to use to create the map files so developers will be less involved in the overall process.

share|improve this answer
add comment

I am interested conceptually about the design approach regardless of language

For low-level problems like this one, there is no "design approach regardless of language". For Java, one answer is to use annotations to mark the fields that should be written to the CSV. Then a single write function can use reflection to find the annotated fields and write them to the CSV. Alternatively, you could work with the existing Java serialization facilities and write all the fields that are not transient.

For deeper understanding, read the code for an existing serialization library. You might want to start with a simple one like openCSV.

share|improve this answer
    
All language features are specific answers to general questions. "Writing files" is one such general question. –  DougM Oct 31 '13 at 12:53
    
I disagree. There is definately a design approach that would allow for more elegant/generic code. –  Jon Raynor Oct 31 '13 at 15:29
add comment

Use varargs...

public void write(Object... args);

... and call it like this:

out.write(new Object { For.getValue(), bar.getMinValue(), fi.toString() });
share|improve this answer
add comment

If you dont need csv, please look into using JAX-B to write them as XML instead of writing them as CSV, if it is for local file storage. It's clutterless: just annotate the classes that will contain the information, and use a Marshaller. It's what I'd do anyway. In case you're interested: http://www.oracle.com/technetwork/articles/javase/index-140168.html

On the other hand, you could have a generic method to marshall into csv, like this:

public String marshallCsv(String[][] data,boolean includeFirstLine){
    String delimeter = "\"";
    String separator = ",";
    StringBuilder csv = new StringBuilder("");
    int i =0;
    if(!includeFirstLine){
        i=1;
    }
    for(;i<data.length;i++){//if includeFirstLine is true, start from index 1
         for(j=0;j<data[i].length;j++){
             csv.append(delimeter+data[i][j]+delimeter);
             if(j<data[i].length){
                  csv.append(separator);
             }
             csv.append("\n");
         }
    }
    return csv.toString();
}

If a certain field is omitted, fill it with a blank string. You can then have your objects implement a method, say "toContentStringArray()" which returns the relevant fields as Strings of an array.

For finer grained control, there are many things you can do, but I like crazy: you can have an overload of toContentStringArray which received an array of Strings naming the fields you want to output, then use reflections to get those fields and omit the others as blank strings.

For better results, make sure you have a sanitizeString(String cleanable,String[] forbiddenStrings), so that you escape the strings you dont want in the csv. For instance, you want to escape all the "," so your fields aren't broken in half.

But trust me, this gets messy. I'd definitely go to JAX-B.

share|improve this answer
    
Out of curiosity - why do you suggest XML over .csv? I am currently parsing data from NOAA, which is the National Oceanic and Atmospheric Administration, and the data they output via the SOAP protocol is XML. It took me a couple hours to write a parser to make sense of the data, and my output for that data is csv. To me, csv seems much easier to manage to, for example, put into Excel or databases. Seems entirely opposite of what you are suggesting. –  theGreenCabbage Oct 31 '13 at 15:29
    
I have to agree with green cabbage. The data I'm writing is simple and stright forward, there isn't the...complexity of XML. the same fields always, no missing fields etc. I want something easily human readable by end users and quick to parse. It seems CSV is preferable for the human readability and parsing. Though I do agree JAX-B looks great for more complicated situations then my simple use-case –  dsollen Oct 31 '13 at 15:51
    
I also agree with the other comments. I've got nothing against XML, but XML is for structured data. If all you need is a flat file for tabular data, you should use CSV (or TSV or PSV or an equivalent). Every time a user dumps CSV data into Excel or their own database, you've eliminated one support call. –  Aaronaught Nov 4 '13 at 20:54
    
I suggested it because it is already done, and would be easy to work with if what you needed was a quick solution. In any case, the same design used by JAX-B can be applied to any type of class-to-structured data marshaling, such as Class-To-DSV, which is your case... so either an answer or a good start for you to find your own. –  iajrz Nov 5 '13 at 4:05
add comment

Label your fields in the first row

When it comes to writing out data, the cardinal sin is to assume that you and only you will ever want to read it in. Such an assumption leads to omitting the second most important part of your file, the identification of your data.

You can switch from CSV to self-describing formats such as XML or JSON, or you can simply implement the proper convention of including a label for each field in the first row. With this convention, it becomes easy for your input engine to map incoming data to whatever internal structure is appropriate, and your output engine is freed from having to especially care about it column order.

The particulars of how you decide which fields to include will vary by language, but that's programming.

share|improve this answer
    
When you say label your fields in the first row for .csv, do you mean: name, age, sex | doug, 20, m et cetera? –  theGreenCabbage Oct 31 '13 at 15:31
    
That's nice, but none of it really answers the question. –  Aaronaught Nov 4 '13 at 20:51
    
Look at Wikipedia, which has a descent example. The last entry in "basic rules and examples" has a clear illustration, with the first row listing the labels of the three fields for the file. –  DougM Nov 4 '13 at 22:30
    
@Aaronaught: How does "label your fields in the first row" not answer "how can I elegantly avoid hard coding the format of a CSV file"? –  DougM Nov 4 '13 at 22:33
    
That ought to be obvious if you actually read the question. At no point did it ask anything about the file format or column order. It's asking how to write clean code for a CSV export. –  Aaronaught Nov 4 '13 at 22:59
show 1 more comment

Use metadata.

I generally associate a CSV file with a separate resource file containing format version and listing the fields. In this way one can develop the quality at the same spot: checking the number of fields, field types and all, in one generic piece of code.

And it is a tiny bit more declarative as an enum.

In fact, on that level one is not interested in specific business processing of data, giving special meanings to fields. So why creating specific coding fluff. As soon as the data arrives on the business side in DTOs or so, the enum (for instance) has its final use, and is further dead: you have built two layers, where only one would be needed to be maintained.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.