Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

0
votes
1answer
38 views

File validation rules

I have an application that can accept CSV files to run some operations. The files look like: CREATE USER:username,last_name,first_name,age user1,Smith,John,23 user2,Poppins,Mary,257 There are a ...
1
vote
0answers
60 views

How to store data that is recorded with different frequency?

I'm writing a metallurgical web application, that must store data (electrical parameters) recorded at different frequencies. For example: six parameters recorded with frequency of one measure per ...
-1
votes
1answer
28 views

can i use json file as a module in node.js [closed]

i have a json module which contains empty containers like this { "files":{ "rootNeeded":[], "folders":[], "files":[], "images":[], "text":[], "unknown":[] }, } and i wonder if i can push ...
2
votes
3answers
61 views

Data format for binary data transfer

I have to make an oscilloscope. Ideally, the oscilloscope connects to a probe. This probe should be another program, and could get the data locally or from a network. This data are numbers, floats ...
0
votes
1answer
79 views

Best Approach for Config File for CSV Parsing [on hold]

I'm writing my own CSV File parser using C#. Now I would like to provide some deep validation based off a config file. This config file will describe: How many columns to expect in the file. If to ...
0
votes
1answer
46 views

Is there a detriment to using SQLite over a pList for persistence?

I'm making a fairly simple application and I've got a settings menu that stores times and some boolean values. I have the application storing and retrieving data already but I started to wonder if ...
0
votes
0answers
55 views

Abstracting the data in a relational database. [duplicate]

While working with data, which is in a relational database I came up with this issue. At first, I was just writing helper functions to retrieve A SPECIFIC data I needed for the current problem but it ...
0
votes
1answer
124 views

How much to encapsulate when objects being dealt with are pretty similar

As I have been answering questions related to object-oriented design and how to go about reducing the amount of classes to make sure that there is not a lot of "class clutter" and code repetition I ...
0
votes
1answer
176 views

Is a linked list an actual data structure?

I hear conflicting sides from this topic. I was under the impression that a linked list was a way to implement a data structure (stack, queue), not actually a data structure.
0
votes
0answers
56 views

How to deal with requests that don't conform to the existing data structure?

The system I'm working on has a list of assets that belong to different customers of the company. This data is saved on an SQL database. Removing irrelevant data, let's say the table is like this: ...
1
vote
1answer
92 views

Running an algorithm against a large dataset

We have a python application which does a DFS search against data in a database. Right now, that data is small so I can hold everything in a python container but the data is going to grow and there ...
1
vote
0answers
25 views

How to use an Address Gazatteer in an application that contains addresses [closed]

Say I had access to the post office address file (http://en.wikipedia.org/wiki/Postcode_Address_File) and wanted to cleanse about one million addresses. I am trying to think of an "algorithm" to do ...
1
vote
1answer
114 views

ER Diagram Multiplicity (UML)

Schema: #=PK Employee (#NIN, fName, lName, address, DOB, sex, salary, taxCode, deptNo) Department (#deptNo, deptName, managerNIN, businessAreaNo, regionNo) Project (#projNo, projName, contractPrice, ...
4
votes
2answers
98 views

Is it a good idea to have a database row that represents an unknown value within a system [closed]

We have two systems, System A imports a list of sports events from system B, it only wants to import the events we will be providing live data for, the filter used to hide events that we are not ...
1
vote
1answer
87 views

Finding a solution for interleaving multiple data messages into one network stream (in QT framework)

I need to find a solution for high throughput network data interleaving in my QT framework based windows application. My application is based on server/client architecture. The client connects to a ...
3
votes
1answer
55 views

Bottleneck performance in ECS

I've been looking into building an entity-component-system. Basically, an entity is just an id wrapped around a struct, components are data belonging to that entity(and reference said id), and systems ...
1
vote
1answer
85 views

Order data for a fictional online store

I'm developing a warehouse automation system for a client that operates an online store. I've already made a domain model and translated it into CREATE TABLE statements for an SQL database. But to ...
1
vote
1answer
61 views

Does cleaning data before saving fall under the term “data sanitization”?

I understand that "data sanitization" mainly deals with cleaning user-entered data to prevent harmful effects (e.g. SQL injection), but what about other forms of cleaning data? E.g., trimming ...
1
vote
0answers
56 views

What is the difference between a single version of the truth and a single source of the truth when dealing with data?

According to Wikipedia a single version of the truth is described as "is a technical concept describing the data warehousing ideal of having either a single centralised database, or at least a ...
2
votes
1answer
188 views

Matching algorithms in context of data quality problems

I work for an organization that has lots of databases containing person information. The data quality is poor. One case was a surname I found like this (this is the worst-case scenario): Mark ...
1
vote
0answers
29 views

Manual repetitive conversion between file types

Let's say i have an interest in file conversions, but everything should be made by hand and i have multiple output formats (say: csv and excel). Once i get contacted by a client, i have to link ...
1
vote
1answer
147 views

Synchronization between two systems using MongoDB as changelog

We are developing two related system. One of them (A) will be installed in our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
2
votes
3answers
779 views

What should I do in C++ when implementing a container class: storing objects by value or by reference?

I'm new to C++, coming from Java. In Java, all variables (except for primitives) are essentially pointers. They hold the address of whatever they're 'holding'. So any Java data structure stores it's ...
31
votes
4answers
2k views

Should test data be checked into version control?

I'm writing some test code for a feature which processes PDF files. The basic idea behind the tests is that I point them towards some PDFs I've selected specially, they process them and I check that ...
-1
votes
1answer
121 views

Is there a set of standards for data validation rules? [closed]

I'm trying to develop a generic data validation engine for web forms. Basically, the idea is to write a library that loads a JSON "validation schema", and then generates the appropriate server-side ...
0
votes
1answer
98 views

Strategy for storing mixed value and reference types without boxing

I'm looking for a way to store a large quantity of individual data values, with the following constraints. Assume that the types are bool, int32, double, decimal, string and blob (byte array). The ...
2
votes
2answers
176 views

Static Test-Data in Helper Class: Getter vs Public Constant

Currently I am in process of writing large batches of Unit-Tests in the Project I am working on. Additionally we recently Introduced GUI-Testing. This project is a relatively simple webapp, on basis ...
2
votes
1answer
683 views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
0
votes
0answers
25 views

An efficient + extendable way to store/reference a normalized set of “causations”?

I'm looking for a good way to define and store references to a normalized set of "causations". I have a well-defined set of reasons (short, descriptive strings) why certain objects are created and ...
0
votes
1answer
139 views

Lazy-initialization vs preprocessing in image-based iOS app

I am making an iOS app which contains museum gallery: user can look through exhibits and then select info about it, etc. On the exhibit screen I have two ScrollViews that are different, but are being ...
1
vote
2answers
209 views

How do you deal with clients asking for manual data edits in the database? [duplicate]

I've recently started working on a legacy application that frankly doesn't do all that it should. It's lacking a lot of features, has barely any administration capacities and doesn't check half the ...
3
votes
2answers
208 views

Why do binary files load quicker than alphanumeric text files?

I've noticed that when I load/store large data files in a binary format, the program runs much faster than if I load data from an ASCII encoded file. Why is this the case? The data in my case is ...
1
vote
1answer
1k views

How many bytes can I fit into a QR code, with low error correction?

I need to fit at least 300 bytes (or much much more) into a QR code and think I can do this by mapping each byte into the associated ISO/IEC 8859-1 character located here. Since each byte (1-255) ...
1
vote
3answers
328 views

Passing data between hundreds of objects in java [closed]

Currently, I'm working with a group on building a model. This model simulates interactions between many "agents" in a region. Agents can be any entity such as a city, a farmer, a business etc. Each ...
0
votes
0answers
347 views

Python Compare Values from 2 Columns from two separate CSV files

I am using Python. I have two columns(in separate CSVs) that contain an ID tag. Ideally, I would like to add on columns of information from one CSV to the other whenever the ID tag matches. For the ...
0
votes
1answer
67 views

Full signed part of two complement long [closed]

I am reverse engineering a data file (from a synthesizer). It seems that e.g. -1 is written as FF FF FF FF, so in four bytes. while +1 is written as 00 00 01 (--), so only three bytes. Is something ...
1
vote
4answers
352 views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
-1
votes
1answer
87 views

Choosing a file format to storage a simple text data [closed]

My .NET application makes its work and then creating a report (an Excel worksheet). The report holds thousands of short text strings [KEY - VALUE], so excel worksheets suites very well, one column ...
5
votes
1answer
161 views

Is there a standard way of cleaning data files?

Having spent quite a lot of time working in software companies that deal in medium to large data sets, I've seen that a choke point in their processes is often prepping data files for loading into ...
1
vote
1answer
23 views

Format for getting clear directions on data paramters from users

As part of my job, I regularly get ad-hoc requests from users for snapshots of our very large database. However, there's no fixed format for delivering these and they usually come through in the form ...
0
votes
0answers
40 views

Efficiently storing and parsing client's browser details

Do there exist browser fingerprinting API's that instead of running browser information (e.g. the user agent, installed plugins, screen resolution, system fonts etc.) through a hash function, instead ...
0
votes
0answers
72 views

Converting large files in python

I have a few files that are ~64GB in size that I think I would like to convert to hdf5 format. I was wondering what the best approach for doing so would be? Reading line-by-line seems to take more ...
2
votes
0answers
1k views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: Once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: ...
1
vote
0answers
27 views

versioning data schema in android internal storage

I am using Android's internal storage to hold data for my application and I am trying to find a good way to handle reading data of an older schema. For example, let's say I have serialized and written ...
1
vote
0answers
518 views

What are the bad points of using Core Data for iOS like an ORM

I just starting to use Core Data for my iOS app and I am thinking about how use Core Data for my needs. What are the features of my app: The user look for products proposed from a catalog. The user ...
0
votes
1answer
86 views

Persisting type in Java - should I be using flags?

Is there some specific piece of advice around persist values that indicate type? Specifically I want to know if I should persist a flag/state indicating the type in persistence or not. I get the ...
2
votes
0answers
362 views

using flat file vs sqlite db for frequent data grabbing in a cron job

I have some automated task to be done using certain types of logs generated on the server end of each day. Log files contains specific machine identification "MAC" that is to be re-written in to ...
0
votes
2answers
116 views

How to design this better?

I'm developing a system using .NET which will be used by multiple users. Because of that, I need to identify on the database which data belongs to each user. Explaining with an example, imagine I have ...
0
votes
3answers
105 views

Looking for a certain datatype if it exists

I'm looking for the kind of a certain datatypes. The main idea of this datatype is to describe a productionline containing several steps. Each step should be represented as a "vertex" containing an ...
4
votes
1answer
58 views

Deleting Content from Service [closed]

I'm not quite sure what the real practice is in the programming community. But when a user deletes content from your service (ie. deleting a status update, a photo of theirs, a comment they made). ...