Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

4
votes
2answers
433 views

Deduplication of complex records / Similarity Detection

I'm working on a project that involves records with fairly large numbers of fields (~15-20) and I'm trying to figure out a good way to implement deduplication. Essentially the records are people along ...
1
vote
1answer
213 views

Consolidating hotels data from various booking sites with different IDs or reference

In one of my projects, I have data for hotels, and other booking sites are able to book this hotel. For example: Hotel A - Booking (ID = 4002), Expedia (ID = 123), Priceline (ID = 147) The three ...
0
votes
0answers
36 views

Program design with multiple data sets, algorithms and results [duplicate]

I have a question regarding the design of my software. The question has already been posted by me at ...
1
vote
3answers
266 views

Program design with multiple data sets, algorithms and results

I have a question regarding the design of my software. There are four different data sets and three consecutive algorithms operating on the data sets and on the results from the previous algorithm ...
2
votes
1answer
380 views

Storing graphs with json and db

I have a Rails 4 app having to do with graphs. These graphs need to be stored and I'm not sure of the best way to do it. Firstly, the app has a SQLite db in development and PostgreSQL in production. I ...
9
votes
2answers
3k views

Data Synchronization in mobile apps - multiple devices, multiple users

I'm looking into building my first mobile app. One of the core features of the application is that multiple devices/users will have access to the same data -- and all of them will have CRUD rights. I ...
0
votes
1answer
218 views

Version-control friendly, append-only file format: “binary” or “text”?

I'm designing an append-only ("journaled") file format, and I'd like it to be friendly towards being stored in VCS (git/Mercurial/...). On one hand, I'd like if after each change, only delta ...
4
votes
5answers
386 views

Should I encrypt files saved by my program

I'm trying to decide on what format I should save data in, and hopefully this question will help me come to a decision. The program I'm writing uses 'databases' in which word=meaning (not a ...
2
votes
2answers
205 views

Why do CSV file formats normally use quoting instead of escaping?

There are a lot of variants of the CSV "standard" (or lack thereof). I've never personally see any that use an escape character (like \) instead of surrounding each field with double quotes. Instead ...
1
vote
1answer
224 views

Best approach for near real time message passing from server to client across router?

I am developing an application which must run on up to thousands of client systems that I control (Linux) which are connected behind common NAT devices to public Internet. Each of these client ...
2
votes
3answers
285 views

Storing data in 2 different systems/database structures

My company is an online retailer, and we use an homebrew ecommerce platform. In the last year or so, we've partnered with a accounting software vendor, as our homebrew solution was unable to produce ...
1
vote
1answer
104 views

Are there any standard for exchanging rules rather than data?

Just out of curiosity. There are many open standards for data exchange, as well as standards for exchanging code (jsonp, for example). But are there any standards for exchanging formulas/rules, for ...
1
vote
2answers
835 views

Radix exchange sort implementation?

I need a little help understanding the implementation of radix exchange sort algorithm. Radix exchange (I don't know if it's the exact name in english based literature) is a lot similar to quicksort, ...
1
vote
2answers
2k views

Dataset with coordinates of borders of countries [closed]

I am creating an application that visually displays world regions, e.g. to place markers within an administrative region. Does a dataset exist with geometrical or geographical (long/lat) descriptions ...
2
votes
1answer
2k views

Why retrieves a native app data faster than one with PhoneGap?

I know that an app developed with PhoneGap is slower than a native one because at PhoneGap the whole code must be interpreted at runtime into a native one. But could someone explain or has a good ...
1
vote
0answers
190 views

Single write, multiple read of stateful objects

Scenario: One data input feed (call this Source) Multiple objects (call these Layer1) read this Source, and analyze the feed. This analysis is stateful. Layer1 objects have configuration parameters. ...
0
votes
1answer
152 views

The best approach to building a mobile service that's also available when the connection drops?

Hopefully this question will be a good suit for P.SE since it is a whiteboard-level 'how can this be done' conceptual question. I'm trying to build a product that needs to be available to someone on ...
0
votes
0answers
116 views

Matching two lists of data with different conventions

I have two lists of data, A and B. These lists are themselves aggregated from multiple sources, and contain typos, abbreviations not found in the other, and also lack a 1-1 mapping, but will never ...
2
votes
2answers
111 views

Looking for session/event analytics analysis tool [closed]

I have data that is basically made up of a set of sessions filled with events. For example: Session1 - [Event1] [Event2] [Event3]...[Event64] Session2 - [Event1] [Event3] [Event4]...[Event64] The ...
-1
votes
1answer
477 views

Data aggregation like polyvore.com [closed]

I'm creating a website that is going to aggregate women shoes from across the web into one website. Kinda like polyvore.com does with all their products but mine is only going to be shoes. Which ...
0
votes
2answers
696 views

Design pattern for data privileges

I'm designing a financial system that should grant access to data based on roles and privileges. For example, a manager can see the financial transaction of users under his domain but not information ...
1
vote
4answers
772 views

Best Practices for Maintaining Uppercase vs Proper Case Data

We have several years of customer data that has been collected over time. I'd guess that 95% of the data (Names, Business Names & Addresses) is in all UPPERCASE format. The data entry tool used ...
3
votes
1answer
239 views

How to process an endless XML data stream

There is an endless data stream of XML messages (and "heartbeats"), that I receive via a telnet connection and through a site-to-site VPN IPsec tunnel. I'm still pondering. What is the best/most ...
8
votes
3answers
242 views

Is a single object to be preferred over multiple variables?

It was quite hard to put what I meant into a title, but it's easy to put into code. C++ Is this int offset_x = 10; int offset_y = 40; ... element.move(offset_x, offset_y); To be preferred over ...
3
votes
2answers
133 views

What's a good format for documenting hundreds of data feeds?

Several governments are starting to publish open data: datasets generated by the goverment, made freely available for citizens to use for value-added app development, analysis, and feedback. For ...
7
votes
2answers
557 views

Why build data models in a dynamic language?

Background This question stems out of an architectural discussion we had at work. For the current project we are using Python and an object oriented database. There are input services that we ...
4
votes
2answers
258 views

Design strategies for storing and validating serial numbers

We are writing software to track Foo Widgets. Each Foo Widget has a serial number. The serial number is an 32-character alphanumeric string. The string is separated into five sets. Each set is ...
1
vote
1answer
168 views

Storing 'sensitive' data in settings file

I'm writing a small utility in AutoIt that connects to Twitter. I would like to store the username and password in the programs setting file, but I know that it needs to be encrypted obviously. ...
1
vote
1answer
52 views

Multiple Display of a Document and persist each viewer change in a single document

i have a scenario where i have to make available a document for viewing and editing to multiple users, the users can edit and view the document at the same time. Now the problem arises here if for ...
1
vote
0answers
165 views

How to organize integration tests and test data for different environments when using Maven Failsafe

I have started to separate my unit tests from my integration tests by using Maven Surefire and Failsafe. Now I'm wondering what good ways there are to deal with separate environments. Take for ...
6
votes
1answer
573 views

Terminology: Really Delete vs set IsDeleted (“soft delete”) flag

I have a site where a regular user can delete records, where what happens is that an IsDeleted flag gets set. On administrative pages, a privileged user can delete records where an actual SQL DELETE ...
1
vote
0answers
123 views

What will be the better way for data retrieval on application that needs to handle limited amount of data? [duplicate]

Just moved this question from Stack Overflow. Since, adding my code snippets itself would make this question really long. Instead, I am pretty interested in knowing a better ways for data retrieval on ...
2
votes
1answer
496 views

How best to merge/sort/page through tons of JSON arrays?

Here's the scenario: Say you have millions of JSON documents stored as text files. Each JSON document is an array of "activity" objects, each of which contain a "created_datetime" attribute. What is ...
1
vote
2answers
194 views

Referential integrity in a database where tuples are not physically deleted

Many modern Relational Database Management Systems automatically support referential integrity, i.e. when you try to delete a tuple which has a reference (in the form of foreign key, for example), the ...
2
votes
1answer
241 views

Pattern for mobile app data synchronisation using HTTP over 3G networks

I have a mobile app built using jquerymobile, PhoneGap and some custom CSS and JavaScript. The data objects are received in JSON and used to display different forms to the user. For example { id: 1, ...
0
votes
1answer
142 views

Design patterns to avoiding breaking the SRP while performing heavy data logging

A class that performs both computations and data logging seems to have at least two responsibilities. Given a system for which the specifications require heavy data logging, what kind of design ...
12
votes
8answers
900 views

Are there examples of non CRUD approaches?

I'm a programmer but also have worked as an archivist. As archivist it's a lot about keeping data. I often get into arguments with colleagues when it comes to operations on data. I don't like the U ...
5
votes
4answers
510 views

Am I sending large amounts of data sensibly?

I am about to design a video conversion service, that is scalable on the conversion side. The architecture is as follows: Webpage for video upload When done, a message gets sent out to one of ...
1
vote
3answers
301 views

Inter-process and inter-thread data sharing [closed]

I know that operating systems facilitate inter-process and inter-thread data sharing. I want to know about the mechanisms used to facilitate such sharing. I read about "pipes". What are the other ...
4
votes
1answer
248 views

Isolating test data in acceptance tests

I'm looking for guidance on how to keep my acceptance tests isolated. Right now the issue I'm having with being able to run the tests in parallel is the database records that are manipulated in the ...
0
votes
3answers
191 views

I need some help creating a non-binary tree (or some other data structure that will better solve my problem)

I have about ten lists of numbers and some strings. Each list has about <= 30K lines. Each line on a list has a distinct number. I need to build an efficient way of finding all the lines in each ...
5
votes
4answers
291 views

Can data classes contain methods for validation?

OK, say I have a data class for a user: public class User { private String firstName; private String lastName; private long personCode; private Date birthDate; private Gender gender; private String ...
-1
votes
2answers
201 views

Can non-IT people learn and take advantage of regular expressions? [closed]

Often times, not-IT people has to deal with massive text data, clean it, filter it, modify it. Often times normal office tools like Excel lack the tools to make complex search and replace operations ...
1
vote
1answer
69 views

Enforcing Constraints Upon Data Documents of Various Formats

This seems like the sort of problem that must have been solved elegantly long ago, but I haven't the foggiest how to google it and find it. Suppose you're maintaining a large legacy system, which has ...
9
votes
5answers
633 views

What's the proper term for a function inverse to a constructor - to unwrap a value from a data type?

Edit: I'm rephrasing the question a bit. Apparently I caused some confusion because I didn't realize that the term destructor is used in OOP for something quite different - it's a function invoked ...
3
votes
2answers
728 views

Having a generic data type for a database table column, is it “good” practice?

I'm working on a PHP project where some object (class member) may contain different data type. For example : class Property { private $_id; // (PK) private $_ref_id; // the object ...
9
votes
8answers
657 views

When does 'optimizing code' == 'structuring data'?

A recent article by ycombinator lists a comment with principles of a great programmer. #7. Good programmer: I optimize code. Better programmer: I structure data. Best programmer: What's the ...
2
votes
3answers
141 views

Having two sets of infrequently changing data or converting on the fly

I am currently working on a small project in which I work with some "static data", which is extracted from one source. The data might change from time to time (up to once a week). I don't need all of ...
0
votes
2answers
400 views

How should I track approval workflow when users at every security level can create a request?

I am writing a new application that allows users to enter requests. Once a request is entered, it must follow an approval workflow to be finally approved by a user the highest security level. So, ...
0
votes
2answers
3k views

Passing the data of a certain page to another page. Windows Phone 7

The user will fill out information in page 1, and the data will be seen in the other page listed in a listbox. Is it advisable to use data binding? I've been studying/researching about data binding ...