Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Given the following assumptions:

  • The amount of available data storage is highly limited
  • All data is international
  • As much storage as possible must be preserved since it will be used for something else

Is there any widely accepted standard or best practice which defines how much storage should be allocated for those items?

Note: I'm not looking for assumptions but for facts like "97.5% of all names fit into 70 bytes using UTF-8" and "An international address must at least contain field1, field2 and field3 of size1, size2 and size3".

Edit: I'm looking for the fields/lengths to use in the user interface. Not a technique to store it efficiently.

share|improve this question
    
If all data is international, many chances are that their UTF-8 representations will be wider than 2 bytes, so you might actually end up saving space by choosing UTF-16 encoding. You should check this. –  Mike Nakis Jan 1 '12 at 16:20
    
It may help you to look at postal authority standards in your specific geography/country. –  Emmad Kareem Jan 1 '12 at 21:23
    
@Emmad i'm looking for something that works across all countries –  mibollma Jan 2 '12 at 13:11
    
stackoverflow.com/questions/20958/… –  JeffO Jan 2 '12 at 15:07

2 Answers 2

Coming at it from a slightly different perspective, since your goal is to store things as compactly as possible, avoid using fixed width fields.

To pull from your own example:

97.5% of all names fit into 70 bytes using UTF-8

You shouldn't have a fixed 70 byte allocation for the name.

Instead, have a fixed allocation for the entire record - say, 200 bytes - then store all the data in that one record with a delimiter between fields.

For example:
Title/Given/Middle/Family/KnownAs/Born
Mr/William/Henry/Gates/1955-10-28

This is a classic space/time tradeoff as searching such a structure will be slower.

One key advantage of this structure is that it allows for some fields to be unusually long without compromising storage. So, if someone's street address happened to require 70 characters, or if their full name took 90, you can accomodate this as long as the same person didn't have both.

share|improve this answer
1  
Why stop at dynamic sizing for record elements. It is entirely possible to have variable sized records (e.g. Just use a different delmiter for end of record). This is a trivial problme if speed performance is not a major consideration, and gets interesting while being entirely solvable even if it is. Theres not enough informtion in the question to suggest what solution to use though. –  mattnz Jan 2 '12 at 7:21
    
I think this doesn't translate well into the user interface though: "Error shorten any field". I'm looking for the size limits and fields in the user interface. Once they are defined it is true that additional space might be saved using your approach. –  mibollma Jan 2 '12 at 13:21

If the space is limited, then the overhead of skipping elements one-by-one wouldn't be so important.

Thus, you can construct a tree, made of lists of length-prefixed elements. You can encode the lengths as single bytes, and for records meant to be larger than what this allows, you can shift the value to the left (that wouldn't waste because it's compensated by the smaller field).

Also, if you want to simplify your code further, it might make sense to embed a flag in the size field indicating if the record is raw or a list. That would make trivial to parse the whole data set (either incrementally or on a single pass).

That would be much more efficient than the fixed-length statistical approach.

Plus, you might want to consider applying some form of compression to the whole thing to save some more space.

share|improve this answer
    
This is already too far ahead. First i'm looking for fields and lengths suitable to work across all countries. Once they are defined we could talk about an efficient way to store them... although i would probably use JSON, Protocol Buffer or MessagePack anyway. –  mibollma Jan 2 '12 at 13:28
    
But why would you want to limit this on the UI? –  Ismael Jan 2 '12 at 18:07
    
Assuming it is unlimited. How do you suggest to tell the user if the sum of his inputs exceeds the combined storage limit? I can't think of any user friendly way. –  mibollma Jan 2 '12 at 19:31
    
Why? it's very simple, anyone can interpret something like "not enough space left, try cutting down at least N characters". It doesn't needs to be more specific than that... –  Ismael Jan 2 '12 at 20:43
1  
Or, a much better approach, put a small counter somewhere in the border of the screen. It can be a small color coded counter, even if it doesn't has any caption, everyone will deduce the purpose as soon as they start writing. –  Ismael Jan 2 '12 at 20:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.