Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

3
votes
2answers
175 views

What's a good format for documenting hundreds of data feeds?

Several governments are starting to publish open data: datasets generated by the goverment, made freely available for citizens to use for value-added app development, analysis, and feedback. For ...
7
votes
2answers
685 views

Why build data models in a dynamic language?

Background This question stems out of an architectural discussion we had at work. For the current project we are using Python and an object oriented database. There are input services that we ...
4
votes
2answers
390 views

Design strategies for storing and validating serial numbers

We are writing software to track Foo Widgets. Each Foo Widget has a serial number. The serial number is an 32-character alphanumeric string. The string is separated into five sets. Each set is ...
1
vote
1answer
270 views

Storing 'sensitive' data in settings file

I'm writing a small utility in AutoIt that connects to Twitter. I would like to store the username and password in the programs setting file, but I know that it needs to be encrypted obviously. ...
1
vote
1answer
55 views

Multiple Display of a Document and persist each viewer change in a single document

i have a scenario where i have to make available a document for viewing and editing to multiple users, the users can edit and view the document at the same time. Now the problem arises here if for ...
1
vote
0answers
216 views

How to organize integration tests and test data for different environments when using Maven Failsafe

I have started to separate my unit tests from my integration tests by using Maven Surefire and Failsafe. Now I'm wondering what good ways there are to deal with separate environments. Take for ...
6
votes
1answer
827 views

Terminology: Really Delete vs set IsDeleted (“soft delete”) flag

I have a site where a regular user can delete records, where what happens is that an IsDeleted flag gets set. On administrative pages, a privileged user can delete records where an actual SQL DELETE ...
1
vote
0answers
123 views

What will be the better way for data retrieval on application that needs to handle limited amount of data? [duplicate]

Just moved this question from Stack Overflow. Since, adding my code snippets itself would make this question really long. Instead, I am pretty interested in knowing a better ways for data retrieval on ...
2
votes
1answer
620 views

How best to merge/sort/page through tons of JSON arrays?

Here's the scenario: Say you have millions of JSON documents stored as text files. Each JSON document is an array of "activity" objects, each of which contain a "created_datetime" attribute. What is ...
1
vote
2answers
213 views

Referential integrity in a database where tuples are not physically deleted

Many modern Relational Database Management Systems automatically support referential integrity, i.e. when you try to delete a tuple which has a reference (in the form of foreign key, for example), the ...
2
votes
1answer
331 views

Pattern for mobile app data synchronisation using HTTP over 3G networks

I have a mobile app built using jquerymobile, PhoneGap and some custom CSS and JavaScript. The data objects are received in JSON and used to display different forms to the user. For example { id: 1, ...
2
votes
1answer
380 views

Design patterns to avoiding breaking the SRP while performing heavy data logging

A class that performs both computations and data logging* seems to have at least two responsibilities. Given a system for which the specifications require heavy data logging, what kind of design ...
13
votes
8answers
1k views

Are there examples of non CRUD approaches?

I'm a programmer but also have worked as an archivist. As archivist it's a lot about keeping data. I often get into arguments with colleagues when it comes to operations on data. I don't like the U ...
5
votes
4answers
627 views

Am I sending large amounts of data sensibly?

I am about to design a video conversion service, that is scalable on the conversion side. The architecture is as follows: Webpage for video upload When done, a message gets sent out to one of ...
2
votes
3answers
417 views

Inter-process and inter-thread data sharing [closed]

I know that operating systems facilitate inter-process and inter-thread data sharing. I want to know about the mechanisms used to facilitate such sharing. I read about "pipes". What are the other ...
4
votes
2answers
395 views

Isolating test data in acceptance tests

I'm looking for guidance on how to keep my acceptance tests isolated. Right now the issue I'm having with being able to run the tests in parallel is the database records that are manipulated in the ...
0
votes
3answers
204 views

I need some help creating a non-binary tree (or some other data structure that will better solve my problem)

I have about ten lists of numbers and some strings. Each list has about <= 30K lines. Each line on a list has a distinct number. I need to build an efficient way of finding all the lines in each ...
5
votes
4answers
345 views

Can data classes contain methods for validation?

OK, say I have a data class for a user: public class User { private String firstName; private String lastName; private long personCode; private Date birthDate; private Gender gender; private String ...
1
vote
1answer
69 views

Enforcing Constraints Upon Data Documents of Various Formats

This seems like the sort of problem that must have been solved elegantly long ago, but I haven't the foggiest how to google it and find it. Suppose you're maintaining a large legacy system, which has ...
10
votes
5answers
861 views

What's the proper term for a function inverse to a constructor - to unwrap a value from a data type?

Edit: I'm rephrasing the question a bit. Apparently I caused some confusion because I didn't realize that the term destructor is used in OOP for something quite different - it's a function invoked ...
4
votes
2answers
926 views

Having a generic data type for a database table column, is it “good” practice?

I'm working on a PHP project where some object (class member) may contain different data type. For example : class Property { private $_id; // (PK) private $_ref_id; // the object ...
9
votes
8answers
670 views

When does 'optimizing code' == 'structuring data'?

A recent article by ycombinator lists a comment with principles of a great programmer. #7. Good programmer: I optimize code. Better programmer: I structure data. Best programmer: What's the ...
2
votes
3answers
144 views

Having two sets of infrequently changing data or converting on the fly

I am currently working on a small project in which I work with some "static data", which is extracted from one source. The data might change from time to time (up to once a week). I don't need all of ...
0
votes
2answers
567 views

How should I track approval workflow when users at every security level can create a request?

I am writing a new application that allows users to enter requests. Once a request is entered, it must follow an approval workflow to be finally approved by a user the highest security level. So, ...
0
votes
2answers
4k views

Passing the data of a certain page to another page. Windows Phone 7

The user will fill out information in page 1, and the data will be seen in the other page listed in a listbox. Is it advisable to use data binding? I've been studying/researching about data binding ...
4
votes
1answer
422 views

Data service API design

We're designing a data service that will store a large number of statistics about stocks, including historical data. The client will be a web application, which will need to pull various bits of data ...
1
vote
1answer
412 views

Developing an analytics's system processing large amounts of data - where to start

Imagine you're writing some sort of Web Analytics system - you're recording raw page hits along with some extra things like tagging cookies etc and then producing stats such as Which pages got most ...
2
votes
2answers
122 views

What is the best strategy for transforming unicode strings into filenames?

I have a bunch (thousands) of resources in an RDF/XML file. I am writing a certain subset of the resources to files -- one file for each, and I'm using the resource's title property as the file name. ...
1
vote
1answer
567 views

Countour Mapping API

I have done some searching on Google, but haven't come up with much as of yet. I am wanting to take a set of point data, which I had previously been using to create weighted points for a heat map ...
9
votes
5answers
21k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
1
vote
2answers
568 views

Almost Realtime Data and Web application

I have a computer that is recording 100 different data points into an OPC server. I've written a simple OPC client that can read all of this data. I have a front-end website on a different network ...
0
votes
3answers
313 views

From where do financial firms obtain the stock data to analyze [closed]

I've been thinking about the subject and I've always wondered, from where do they obtain the data that is analized in applications, I've looked at Nasdaq's website and they do not seem to have any ...
0
votes
1answer
534 views

How to get a list of valid addresses?

I need to get a list of valid addresses in 4 or 5 clusters (cities) in both the UK and US. It's to generate sample data for an application that will do geolocation and searching. At the moment I'm ...
23
votes
6answers
17k views

Why the static data members have to be defined outside the class separately in C++ (unlike Java)?

class A { static int foo () {} // ok static int x; // <--- needed to be defined separately in .cpp file }; I don't see a need of having A::x defined separately in a .cpp file (or same file ...
2
votes
4answers
237 views

Data migration to a single shared SQL table

We have three internal web applications we use at my company. Unfortunately none of my predecessors had the foresight of sharing tables amongst those apps. It has become necessary to share some ...
2
votes
4answers
2k views

Data Structures usage and motivational aspects

For long student life I was always wondering why there are so many of them yet there seems to be lack of usage at all in many of them. The opinion didn't really change when I got a job. We have ...
2
votes
1answer
146 views

Ensure data entered by users adjusts to what actually is

This is more of a usability issue I'd say, looking of suggestions on it. I have a system that depends on the data entered by the users to be correct, as in the address exists, the name is correct, ...
1
vote
1answer
159 views

Approach for housekeeping production data

If you are going write a set of program to housekeep the production data, how can you ensure your program will work correctly? We cannot housekeep the data based on their creation time. We would ...
2
votes
2answers
560 views

How to handle encryption key conflicts when synchronizing data?

Assume that there is data that gets synchronized between several devices. The data is protected with a symmetric encryption algorithm and a key. The key is stored on each device and encrypted with a ...
3
votes
3answers
326 views

Designing access to file-based “database”

It happened frequently that I have to provide access to a bunch of files organized in a directory tree according to some (sometimes loosely specified) rules. My standard pattern is to provide a ...
3
votes
2answers
277 views

Getting and maintaining data from a large number of sources on the web

For the sake of an exercice I want to aggregate price listings with prices coming from a series of web sites in a structured form (XML, JSON...). If I have control on both sides how can I go about ...
3
votes
3answers
414 views

Where can I locate business data to use in my application?

This question talks about any and all free public raw data which appeared to have valuable pieces but nothing that really provides what I am looking for. Instead of using a socially defined listing ...
8
votes
3answers
2k views

How do Data Scientists Use Python? [closed]

Do you know how Data Scientists use Python and if it requires data, where I can get sample data to practice with? I'm currently learning the language and plan on applying for Data Scientists jobs, so ...
5
votes
1answer
215 views

What are graph datastores better at doing than other datastores and why?

Where previously there was only relational databases, the data store world is now rife with options like Key-Value, Document, and Graph datastores. Unfortunately, every datastore likes to show how it ...
41
votes
12answers
3k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states in between? Wouldn't it be more natural to store floats like ...
0
votes
2answers
173 views

Optimal storage space calculation for name, address, phone, fax and email data

Given the following assumptions: The amount of available data storage is highly limited All data is international As much storage as possible must be preserved since it will be used for something ...
19
votes
1answer
364 views

How should HTML data formats be applied in everyday situations?

Given Google's shift towards a stronger focus on page markup data, how do the data formats used in Schema.org work alongside those for Microformats? How do these (and other specs) compliment each ...
3
votes
3answers
4k views

Where can I find a large dataset of tweets?

I'm trying some machine learning with Twitter, and I was wondering if there were any pre-collected just massive tweet data that anyone knew about before I went and just used the API (which has a ...
2
votes
1answer
197 views

How do you define, organize, and document your data?

Please feel free to suggest a better name or way of explaining this question - I wasn't quite sure what to call it. The situation is one that may just come under the heading of "be organized" but I ...
1
vote
1answer
419 views

How much data validation is too much? [duplicate]

Possible Duplicate: Data input validation - Where? How much? I'm a new PHP developer and am into Powershell quite a bit but this question is language agnostic. I've been questioning my code ...