Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

1
vote
1answer
27 views

Consistency for disk write in a model object graph

You're writing an app where the model (as in MVC) is an object graph. Occasionally, you want to save this object graph to disk. What are some best practices to ensure that the write is consistent? (...
1
vote
2answers
167 views

Designing validation classes

This is a follow-up of this question. I decided to implement what the accepted answer suggests, however, I realized I also needed individual public methods for each member the entity to be validated ...
1
vote
2answers
92 views

Handling Production Seed Data

We have a requirement for a new feature which requires some seed data to be present in the database (basically some default values) for the feature to function correctly. We have this in 2 different ...
-2
votes
1answer
33 views

Need a solution for data filter problem

Problem statement: I have a input with multiple key value pairs and a list tasks stored in database with filter object. As an output I need a list of tasks matching the input values based on task's ...
-3
votes
1answer
84 views

What is 16 bit word's equivalent in decimal?

Question: For a 16 bit word with 6 bits for an opcode How many different instructions could I fit into the instruction set? What is the largest number that I could use as data? ...
15
votes
4answers
2k views

How should I store readonly data to deploy with my application?

I'm developing a desktop application, and this application requires some information to run, but it doesn't change any of this information (the data must be loaded on every execution of the app, but ...
-2
votes
0answers
19 views

VBA accessing files [closed]

I want to be able to access workbooks that are contained inside a folder. If I have a number of workbooks inside a folder, is there a way to pull the data that exist inside the workbooks? My goal is ...
-1
votes
2answers
97 views

better way to load data for millions of users

I am working on a Spring/JSF app in which i have 1 scheduler to load user's sleep activity from fitbit. Right now what i do is that whenever the scheduler kicks in, I load ALL users from mongoDB and ...
1
vote
2answers
158 views

What design patterns exist for large dataset manipulation within MVC for a rich client data grid?

I want to feed an HTML5 data grid with large data sets that the client will want to manipulate in different ways, e.g. aggregation, grouping, pivoting. The data comes from different web services and ...
5
votes
6answers
473 views

Algorithm for determining transactions among weekly data series?

I'm trying to develop a small reporting tool (with sqlite backend). I can best describe this tool as a "transaction" ledger. What I'm trying to do is keep track of "transactions" from weekly data ...
4
votes
2answers
441 views

Synchronization between two systems using MongoDB as changelog

We are developing two related system. One of them (A) will be installed in our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
7
votes
2answers
1k views

Terminology: Really Delete vs set IsDeleted (“soft delete”) flag

I have a site where a regular user can delete records, where what happens is that an IsDeleted flag gets set. On administrative pages, a privileged user can delete records where an actual SQL DELETE ...
-1
votes
1answer
125 views

Which file format should I use as an alternative to my current Excel worksheet? [closed]

My .NET application does its work and then creates a report (an Excel worksheet). That report contains thousands of short text strings [KEY - VALUE]. So Excel worksheets suites very well, one column ...
3
votes
1answer
136 views

What is the correct way to test performance of methods?

I made a few classes for processing and validating data in xlsx and csv. However, I am curious in testing how fast the methods I wrote in C# for the heavy lifting are. Here's what I currently am ...
3
votes
6answers
2k views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
6
votes
2answers
110 views

Seeking benefits of both SQL and binary files in a data storage scheme

I'm looking for some input on how to keep certain benefits of both a binary file and a SQL database in an interesting data storage problem. I have a legacy custom binary file format that is ...
3
votes
1answer
51 views

versioning data schema in android internal storage

I am using Android's internal storage to hold data for my application and I am trying to find a good way to handle reading data of an older schema. For example, let's say I have serialized and written ...
-1
votes
1answer
116 views

What is the proper or recommended way to save String[]'s, Array's, ArrayList's and int[]'s to Android?

I've looked up this question many many times but keep seeing different answers, what is the BEST way to save array's? It seems ludicrously infantile that there is no standard or easy way to save ...
0
votes
0answers
25 views

What is needed server side in order to feed excel pivot users?

Our company has a lot of different business service "silos". They come with their client software, but they do not interop or offer cross domain searches. It is hard to compile the costs of a project ...
6
votes
1answer
187 views

Is there a standard way of cleaning data files?

Having spent quite a lot of time working in software companies that deal in medium to large data sets, I've seen that a choke point in their processes is often prepping data files for loading into ...
6
votes
3answers
218 views

Fuzzy algorithm for grouping data

I have data that indicates which topics each student has got right/wrong in an assessment.: | Student | Topic 1 | Topic 2 | Topic 3 | Topic 4 | Topic 5 | ... (perhaps 30 topics) | 1 | T | ...
-3
votes
1answer
41 views

How to download data from a web page

I wonder if there is a way that I can get the data from a website automatically. Here's a sample link: http://www.trademap.org/CompaniesList.aspx?nvpm=1|704||||03|||4|1|1|2|3|1|1|1|1 You can ...
3
votes
2answers
163 views

Correct way for storing data for runtime use in stand-alone applications?

I am working on a project that involves a "team builder" type application, if you will using C#. For the sake of simplicity, let us say it involves the user creating a "Team." There are three teams ...
4
votes
2answers
630 views

Why do binary files load quicker than alphanumeric text files?

I've noticed that when I load/store large data files in a binary format, the program runs much faster than if I load data from an ASCII encoded file. Why is this the case? The data in my case is ...
-1
votes
1answer
150 views

What is the faster way to save and browse objects using vectors +/or maps? (c++)

I'm doing a object Garbage class using c++: #include "Object.h" class ObjectGarbage { public: ObjectGarbage(); ~ObjectGarbage(); void Call_All_Objects() { //(...
3
votes
2answers
6k views

How many bytes can I fit into a QR code, with low error correction?

I need to fit at least 300 bytes (or much much more) into a QR code and think I can do this by mapping each byte into the associated ISO/IEC 8859-1 character located here. Since each byte (1-255) ...
3
votes
5answers
455 views

What is the best way to save data of a C++ Program ? Binary vs JSON serialization

We are implementing a CAD program (in C++, Qt) where we have interdependant classes : The smallest brick is the Pattern, it is just a distribution of points. Then we have Layouts which contain ...
1
vote
2answers
56 views

Name for Keeping Multiple Copies of a Variable

There is a bad practice in software engineering where a single piece of data is held by value in multiple places. For example, in an FPS, the location of the player could be held by the AI class, the ...
4
votes
1answer
99 views

“Reactive programming is programming with asynchronous data streams.” is this true?

I was reading about Reactive programming, and came across this article The introduction to Reactive Programming you've been missing with the quote I put in the question title: Reactive programming ...
2
votes
1answer
72 views

Vector input for Artificial Neural Network?

My problem is fairly complex, so here is a similar, simplified example. Let's say you wanted to predict the Miles Per Gallon (fuel consumption) of a car, based on the following information: you have ...
25
votes
2answers
15k views

Data Synchronization in mobile apps - multiple devices, multiple users

I'm looking into building my first mobile app. One of the core features of the application is that multiple devices/users will have access to the same data -- and all of them will have CRUD rights. I ...
6
votes
2answers
314 views

Data validation: separated class or not?

When I have a lot of data that needs to be validated, should I create a new class for the sole purpose of validation or should I stick with in-method validation? My particular example contemplates a ...
1
vote
0answers
31 views

Criteria for choosing which common data formats to support for tabular data in a simple ML library

What criteria does one consider when choosing which data formats a library for doing some machine learning task should support for reading/writing tabular (i.e. non-hierarchical) data? I found a ...
-1
votes
1answer
57 views

How costly is DB data transfer as compared to data storage?

I've seen a number of answers (1, 2, 3) where they state that storage is cheap. So when designing a DB structure, it shouldn't be a main concern, or at least it doesn't rank very high in the list of ...
2
votes
4answers
214 views

What separates text files from binary files, considering they are stored in binary?

There are text files and there are image, video, and music files. Why are image, video, and music files considered binary files, and why are text files not?
3
votes
3answers
218 views

How to test data based on SQL queries?

I'm trying to set up a test framework for (SQL) datasets that get updated daily. I want to write assertions that involve comparisons between two tables–for example, the sum of column visitors in the ...
0
votes
1answer
22 views

What relationship describes two different abstractions of the same dataset

I am wondering if there is a specific term for when multiple representations are data equivalent. Meaning that you can transfer the data from one representation to the other without any loss of data. ...
65
votes
6answers
5k views

Should temporary files be saved to /tmp or the current working directory?

I have a program that needs to generate temporary files. It is written for cluster machines. If I saved those files to a system-wide temporary directory (eg: /tmp), some users complained the program ...
1
vote
1answer
30 views

File structure for storing pixels that changed from one image to another

Have two images with the same dimensions. I want to create a file that contains information on what pixels are different between the two images. The file should also tell me the value of the new ...
0
votes
0answers
30 views

Storing mass user-files

I am thinking about the concepts of a web-application in which users can upload files to the server. I have multiple questions about the storage of these files. Imagine a service with 10,000+ users ...
0
votes
0answers
37 views

Automated data processing in BPMN/BPEL?

In order to run complex simulations, I need to do preprocessing of data from various data sources. This is done by a bunch of Postgresql scripts. However, having only these is unsatisfactory, because ...
0
votes
1answer
105 views

Best practice for settings for PHP class for code generation [closed]

I would like to have a class creating any code from given options and output it in a chosen way. I have written class for generation of simple JavaScript code for such cases where is needed to ...
3
votes
1answer
99 views

PHP OOP, structuring my objects properly

I have an architecture similar to this: User (Name, Email, ID...) ---- Studios (Name, Location, Description...) ---------- Videos (Title, Description, URL, Length, Views...) Simply put, a user can ...
1
vote
2answers
102 views

Best practice for creating defined versions in git

In my company we have a 'production pipeline', which is a bunch of code which we use to produce data, with the data later shipped to customers. The code is still being developed, bugs get fixed etc. ...
2
votes
2answers
75 views

How to perform summary analytics on production databases?

I want to use production data to generate "user activity" metrics. My thought is to set up a cron job that regularly reads production databases to summarize user activity and then stores the results ...
0
votes
3answers
4k views

Passing the data of a certain page to another page. Windows Phone 7

The user will fill out information in page 1, and the data will be seen in the other page listed in a listbox. Is it advisable to use data binding? I've been studying/researching about data binding ...
3
votes
1answer
60 views

Interactive review of data in modal before inserting in db

I'm looking for a conceptual approach to a problem. I'm building a web app that imports data from a source I don't control. The data quality is pretty high, but occasionally, the data does not make ...
7
votes
3answers
532 views

Microservices and isolated data storage models [duplicate]

I have been looking at Microservices for a while now. The concept is not new but it's communicated in a lightweight manner. So, I am very excited about this. However, there is a question that I am ...
1
vote
1answer
68 views

BigData: time-based word count

What we are trying to do: We're trying to build a system that will count the number of unique entries for a certain timeframe. It's working ok until the data grows up or the timeframe increase, then ...
4
votes
2answers
52 views

AutoIncrement Integer Column,Guid or PGuid, which to choose?

Guid has been a Global Unique Identification. Why does "PGuid" (Global Unique Identification) appear? 1) Is Guid NOT PERMENT for unique? (I sometimes wonder whether Guid is REALLY PERMENT generator ...