Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

5
votes
6answers
473 views

Algorithm for determining transactions among weekly data series?

I'm trying to develop a small reporting tool (with sqlite backend). I can best describe this tool as a "transaction" ledger. What I'm trying to do is keep track of "transactions" from weekly data ...
4
votes
2answers
457 views

Synchronization between two systems using MongoDB as changelog

We are developing two related system. One of them (A) will be installed in our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
1
vote
2answers
92 views

Handling Production Seed Data

We have a requirement for a new feature which requires some seed data to be present in the database (basically some default values) for the feature to function correctly. We have this in 2 different ...
1
vote
2answers
167 views

Designing validation classes

This is a follow-up of this question. I decided to implement what the accepted answer suggests, however, I realized I also needed individual public methods for each member the entity to be validated ...
1
vote
2answers
158 views

What design patterns exist for large dataset manipulation within MVC for a rich client data grid?

I want to feed an HTML5 data grid with large data sets that the client will want to manipulate in different ways, e.g. aggregation, grouping, pivoting. The data comes from different web services and ...
-1
votes
1answer
116 views

What is the proper or recommended way to save String[]'s, Array's, ArrayList's and int[]'s to Android?

I've looked up this question many many times but keep seeing different answers, what is the BEST way to save array's? It seems ludicrously infantile that there is no standard or easy way to save ...
-2
votes
1answer
33 views

Need a solution for data filter problem

Problem statement: I have a input with multiple key value pairs and a list tasks stored in database with filter object. As an output I need a list of tasks matching the input values based on task's ...
5
votes
0answers
2k views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: Once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: ...
2
votes
0answers
661 views

using flat file vs sqlite db for frequent data grabbing in a cron job

I have some automated task to be done using certain types of logs generated on the server end of each day. Log files contains specific machine identification "MAC" that is to be re-written in to ...
2
votes
0answers
299 views

How to organize integration tests and test data for different environments when using Maven Failsafe

I have started to separate my unit tests from my integration tests by using Maven Surefire and Failsafe. Now I'm wondering what good ways there are to deal with separate environments. Take for ...
1
vote
0answers
31 views

Criteria for choosing which common data formats to support for tabular data in a simple ML library

What criteria does one consider when choosing which data formats a library for doing some machine learning task should support for reading/writing tabular (i.e. non-hierarchical) data? I found a ...
1
vote
0answers
84 views

Data Store Design for NxN Data Aggregation

I am trying to come up with a theoretical solution to an NxN problem for data aggregation and storage. As an example I have a huge amount of data that comes in via a stream. The stream sends the data ...
1
vote
0answers
105 views

Saving and retrieving multiple cached json/txt data files

I am working on some numerical programming and need to generate the results for a model given a variety of input parameters. Since the model takes a while to run, I was planning to save the data to ...
1
vote
0answers
97 views

Using a distributed system as an alternative to disk storage

So, one of the differences between "main memory" and disk storage is durability -- the data on hard disks will stay accessible and unaffected even if the host machine crashes or loses power. One can ...
1
vote
0answers
218 views

What is the difference between a single version of the truth and a single source of the truth when dealing with data?

According to Wikipedia a single version of the truth is described as "is a technical concept describing the data warehousing ideal of having either a single centralised database, or at least a ...
1
vote
0answers
37 views

Manual repetitive conversion between file types

Let's say i have an interest in file conversions, but everything should be made by hand and i have multiple output formats (say: csv and excel). Once i get contacted by a client, i have to link ...
1
vote
0answers
775 views

What are the bad points of using Core Data for iOS like an ORM

I just starting to use Core Data for my iOS app and I am thinking about how use Core Data for my needs. What are the features of my app: The user look for products proposed from a catalog. The user ...
1
vote
0answers
248 views

Single write, multiple read of stateful objects

Scenario: One data input feed (call this Source) Multiple objects (call these Layer1) read this Source, and analyze the feed. This analysis is stateful. Layer1 objects have configuration parameters. ...
1
vote
0answers
142 views

Matching two lists of data with different conventions

I have two lists of data, A and B. These lists are themselves aggregated from multiple sources, and contain typos, abbreviations not found in the other, and also lack a 1-1 mapping, but will never ...
0
votes
0answers
25 views

What is needed server side in order to feed excel pivot users?

Our company has a lot of different business service "silos". They come with their client software, but they do not interop or offer cross domain searches. It is hard to compile the costs of a project ...
0
votes
0answers
30 views

Storing mass user-files

I am thinking about the concepts of a web-application in which users can upload files to the server. I have multiple questions about the storage of these files. Imagine a service with 10,000+ users ...
0
votes
0answers
37 views

Automated data processing in BPMN/BPEL?

In order to run complex simulations, I need to do preprocessing of data from various data sources. This is done by a bunch of Postgresql scripts. However, having only these is unsatisfactory, because ...
0
votes
0answers
25 views

Verify That Filters on a Dataset are Exclusive

Suppose I have a data table of, say, cars with a bunch of properties for each record. Example: |Vin|Year|Make|Model|Color|Mileage| I want to allow users to define (arbitrarily complex) filters which ...
0
votes
0answers
78 views

Will this data mining approach work? Is it a good idea?

I need to extract fields like the document number, date, and invoice amount from a bunch of .csv files, which I believe are referred to as "unstructured text." I have some labeled input files and will ...
0
votes
0answers
165 views

Whats the best way to implement DataMapper design pater and lazy loading (ghost object)

I'm doing a small project and I wanted to keep it simple to mantain. It is a renting management software. To keep it clean I decided to implement the data mapper patern, to keep the domain objects in ...
0
votes
0answers
71 views

How to deal with requests that don't conform to the existing data structure?

The system I'm working on has a list of assets that belong to different customers of the company. This data is saved on an SQL database. Removing irrelevant data, let's say the table is like this: +-...
0
votes
0answers
44 views

Efficiently storing and parsing client's browser details

Do there exist browser fingerprinting API's that instead of running browser information (e.g. the user agent, installed plugins, screen resolution, system fonts etc.) through a hash function, instead ...
0
votes
0answers
84 views

Converting large files in python

I have a few files that are ~64GB in size that I think I would like to convert to hdf5 format. I was wondering what the best approach for doing so would be? Reading line-by-line seems to take more ...
0
votes
0answers
38 views

What data should QR-Code contain to serve both my own and external QR-Code reader

My webapplication uses codes (in form of http://www.url.com/code/105820582020204), that I want to encode as QR-Code. I assume that such generated QR-Code will be scanned by both people using my mobile ...
-2
votes
0answers
53 views

Text file - editing in C++

I have some lines of text in a word doc. Each line has the format of an article, like: Author(s). Title. Journal, number of the Journal (Year), pages I want to put this data in Excel but in way such ...