Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

0
votes
0answers
21 views

Is there a detriment to using SQLite over a pList for persistence?

I'm making a fairly simple application and I've got a settings menu that stores times and some boolean values. I have the application storing and retrieving data already but I started to wonder if ...
0
votes
0answers
53 views

Abstracting the data in a relational database. [duplicate]

While working with data, which is in a relational database I came up with this issue. At first, I was just writing helper functions to retrieve A SPECIFIC data I needed for the current problem but it ...
0
votes
1answer
117 views

How much to encapsulate when objects being dealt with are pretty similar

As I have been answering questions related to object-oriented design and how to go about reducing the amount of classes to make sure that there is not a lot of "class clutter" and code repetition I ...
0
votes
1answer
173 views

Is a linked list an actual data structure?

I hear conflicting sides from this topic. I was under the impression that a linked list was a way to implement a data structure (stack, queue), not actually a data structure.
0
votes
0answers
52 views

How to deal with requests that don't conform to the existing data structure?

The system I'm working on has a list of assets that belong to different customers of the company. This data is saved on an SQL database. Removing irrelevant data, let's say the table is like this: ...
1
vote
1answer
91 views

Running an algorithm against a large dataset

We have a python application which does a DFS search against data in a database. Right now, that data is small so I can hold everything in a python container but the data is going to grow and there ...
1
vote
0answers
22 views

How to use an Address Gazatteer in an application that contains addresses [closed]

Say I had access to the post office address file (http://en.wikipedia.org/wiki/Postcode_Address_File) and wanted to cleanse about one million addresses. I am trying to think of an "algorithm" to do ...
1
vote
0answers
44 views

ER Diagram Multiplicity (UML)

Schema: #=PK Employee (#NIN, fName, lName, address, DOB, sex, salary, taxCode, deptNo) Department (#deptNo, deptName, managerNIN, businessAreaNo, regionNo) Project (#projNo, projName, contractPrice, ...
4
votes
2answers
98 views

Is it a good idea to have a database row that represents an unknown value within a system [closed]

We have two systems, System A imports a list of sports events from system B, it only wants to import the events we will be providing live data for, the filter used to hide events that we are not ...
1
vote
1answer
66 views

Finding a solution for interleaving multiple data messages into one network stream (in QT framework)

I need to find a solution for high throughput network data interleaving in my QT framework based windows application. My application is based on server/client architecture. The client connects to a ...
3
votes
1answer
53 views

Bottleneck performance in ECS

I've been looking into building an entity-component-system. Basically, an entity is just an id wrapped around a struct, components are data belonging to that entity(and reference said id), and systems ...
2
votes
0answers
33 views

Order data for a fictional online store

I'm developing a warehouse automation system for a client that operates an online store. I've already made a domain model and translated it into CREATE TABLE statements for an SQL database. But to ...
1
vote
1answer
61 views

Does cleaning data before saving fall under the term “data sanitization”?

I understand that "data sanitization" mainly deals with cleaning user-entered data to prevent harmful effects (e.g. SQL injection), but what about other forms of cleaning data? E.g., trimming ...
1
vote
0answers
51 views

What is the difference between a single version of the truth and a single source of the truth when dealing with data?

According to Wikipedia a single version of the truth is described as "is a technical concept describing the data warehousing ideal of having either a single centralised database, or at least a ...
2
votes
1answer
172 views

Matching algorithms in context of data quality problems

I work for an organization that has lots of databases containing person information. The data quality is poor. One case was a surname I found like this (this is the worst-case scenario): Mark ...
1
vote
0answers
29 views

Manual repetitive conversion between file types

Let's say i have an interest in file conversions, but everything should be made by hand and i have multiple output formats (say: csv and excel). Once i get contacted by a client, i have to link ...
1
vote
1answer
130 views

Synchronization between two systems using MongoDB as changelog

We are developing two related system. One of them (A) will be installed in our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
2
votes
3answers
660 views

What should I do in C++ when implementing a container class: storing objects by value or by reference?

I'm new to C++, coming from Java. In Java, all variables (except for primitives) are essentially pointers. They hold the address of whatever they're 'holding'. So any Java data structure stores it's ...
31
votes
4answers
2k views

Should test data be checked into version control?

I'm writing some test code for a feature which processes PDF files. The basic idea behind the tests is that I point them towards some PDFs I've selected specially, they process them and I check that ...
-1
votes
1answer
99 views

Is there a set of standards for data validation rules? [closed]

I'm trying to develop a generic data validation engine for web forms. Basically, the idea is to write a library that loads a JSON "validation schema", and then generates the appropriate server-side ...
0
votes
1answer
91 views

Strategy for storing mixed value and reference types without boxing

I'm looking for a way to store a large quantity of individual data values, with the following constraints. Assume that the types are bool, int32, double, decimal, string and blob (byte array). The ...
2
votes
2answers
155 views

Static Test-Data in Helper Class: Getter vs Public Constant

Currently I am in process of writing large batches of Unit-Tests in the Project I am working on. Additionally we recently Introduced GUI-Testing. This project is a relatively simple webapp, on basis ...
2
votes
1answer
572 views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
0
votes
0answers
23 views

An efficient + extendable way to store/reference a normalized set of “causations”?

I'm looking for a good way to define and store references to a normalized set of "causations". I have a well-defined set of reasons (short, descriptive strings) why certain objects are created and ...
0
votes
1answer
133 views

Lazy-initialization vs preprocessing in image-based iOS app

I am making an iOS app which contains museum gallery: user can look through exhibits and then select info about it, etc. On the exhibit screen I have two ScrollViews that are different, but are being ...
1
vote
2answers
207 views

How do you deal with clients asking for manual data edits in the database? [duplicate]

I've recently started working on a legacy application that frankly doesn't do all that it should. It's lacking a lot of features, has barely any administration capacities and doesn't check half the ...
3
votes
2answers
195 views

Why do binary files load quicker than alphanumeric text files?

I've noticed that when I load/store large data files in a binary format, the program runs much faster than if I load data from an ASCII encoded file. Why is this the case? The data in my case is ...
1
vote
1answer
814 views

How many bytes can I fit into a QR code, with low error correction?

I need to fit at least 300 bytes (or much much more) into a QR code and think I can do this by mapping each byte into the associated ISO/IEC 8859-1 character located here. Since each byte (1-255) ...
1
vote
3answers
328 views

Passing data between hundreds of objects in java [closed]

Currently, I'm working with a group on building a model. This model simulates interactions between many "agents" in a region. Agents can be any entity such as a city, a farmer, a business etc. Each ...
1
vote
0answers
330 views

Python Compare Values from 2 Columns from two separate CSV files

I am using Python. I have two columns(in separate CSVs) that contain an ID tag. Ideally, I would like to add on columns of information from one CSV to the other whenever the ID tag matches. For the ...
0
votes
1answer
67 views

Full signed part of two complement long [closed]

I am reverse engineering a data file (from a synthesizer). It seems that e.g. -1 is written as FF FF FF FF, so in four bytes. while +1 is written as 00 00 01 (--), so only three bytes. Is something ...
1
vote
4answers
299 views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
-1
votes
1answer
87 views

Choosing a file format to storage a simple text data [closed]

My .NET application makes its work and then creating a report (an Excel worksheet). The report holds thousands of short text strings [KEY - VALUE], so excel worksheets suites very well, one column ...
5
votes
1answer
159 views

Is there a standard way of cleaning data files?

Having spent quite a lot of time working in software companies that deal in medium to large data sets, I've seen that a choke point in their processes is often prepping data files for loading into ...
1
vote
1answer
22 views

Format for getting clear directions on data paramters from users

As part of my job, I regularly get ad-hoc requests from users for snapshots of our very large database. However, there's no fixed format for delivering these and they usually come through in the form ...
0
votes
0answers
40 views

Efficiently storing and parsing client's browser details

Do there exist browser fingerprinting API's that instead of running browser information (e.g. the user agent, installed plugins, screen resolution, system fonts etc.) through a hash function, instead ...
0
votes
0answers
70 views

Converting large files in python

I have a few files that are ~64GB in size that I think I would like to convert to hdf5 format. I was wondering what the best approach for doing so would be? Reading line-by-line seems to take more ...
2
votes
0answers
1k views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: Once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: ...
1
vote
0answers
25 views

versioning data schema in android internal storage

I am using Android's internal storage to hold data for my application and I am trying to find a good way to handle reading data of an older schema. For example, let's say I have serialized and written ...
1
vote
0answers
494 views

What are the bad points of using Core Data for iOS like an ORM

I just starting to use Core Data for my iOS app and I am thinking about how use Core Data for my needs. What are the features of my app: The user look for products proposed from a catalog. The user ...
0
votes
1answer
85 views

Persisting type in Java - should I be using flags?

Is there some specific piece of advice around persist values that indicate type? Specifically I want to know if I should persist a flag/state indicating the type in persistence or not. I get the ...
2
votes
0answers
343 views

using flat file vs sqlite db for frequent data grabbing in a cron job

I have some automated task to be done using certain types of logs generated on the server end of each day. Log files contains specific machine identification "MAC" that is to be re-written in to ...
0
votes
2answers
116 views

How to design this better?

I'm developing a system using .NET which will be used by multiple users. Because of that, I need to identify on the database which data belongs to each user. Explaining with an example, imagine I have ...
0
votes
3answers
105 views

Looking for a certain datatype if it exists

I'm looking for the kind of a certain datatypes. The main idea of this datatype is to describe a productionline containing several steps. Each step should be represented as a "vertex" containing an ...
4
votes
1answer
58 views

Deleting Content from Service [closed]

I'm not quite sure what the real practice is in the programming community. But when a user deletes content from your service (ie. deleting a status update, a photo of theirs, a comment they made). ...
0
votes
1answer
50 views

Place measurement data in database (access)

In a software I use records of measurement data. Each record contains basic information like a name, some parameters and the actual measurement data in the form of three arrays of double values that ...
4
votes
1answer
365 views

Should reflection be part of design?

I'm currently designing a somewhat large program that will involve the simulation of math/physics models and collection of data (I have not implemented any code yet). One of the main problems I'm ...
0
votes
1answer
134 views

scipy.stats.percentileofscore() wrong?

i'm a data scientist and python noob, but I seem to have found a bug with scipy.stats.percentileofscore. it may be "by design" but please bear with me. if you have an array, you will get an upper ...
2
votes
3answers
281 views

Building presentation layer directly on data layer

My application is currently quite a mess and I'm trying to untangle it all. My current idea is that I should have 3 separate layers: data layer that knows how to keep stuff in database (and handle ...
3
votes
2answers
187 views

What is a scalable and practical way to search existence of a group of strings in a huge file

Context: I built an app which generates around 1000 domain names based on user input. I need to check if they are available or not by matching against a huge zone file of parsed domain names which is ...