Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

0
votes
0answers
23 views

Manual repetitive conversion between file types

Let's say i have an interest in file conversions, but everything should be made by hand and i have multiple output formats (say: csv and excel). Once i get contacted by a client, i have to link ...
1
vote
1answer
40 views

Synchronization between two systems using MongoDB as changelog

We are developing two related system. One of them (A) will be installed in our customers' machines. The remaining (B) will be used by my organization. Each system has its own database (relational) ...
2
votes
3answers
334 views

What should I do in C++ when implementing a container class: storing objects by value or by reference?

I'm new to C++, coming from Java. In Java, all variables (except for primitives) are essentially pointers. They hold the address of whatever they're 'holding'. So any Java data structure stores it's ...
29
votes
4answers
2k views

Should test data be checked into version control?

I'm writing some test code for a feature which processes PDF files. The basic idea behind the tests is that I point them towards some PDFs I've selected specially, they process them and I check that ...
-1
votes
1answer
55 views

Is there a set of standards for data validation rules? [closed]

I'm trying to develop a generic data validation engine for web forms. Basically, the idea is to write a library that loads a JSON "validation schema", and then generates the appropriate server-side ...
0
votes
1answer
60 views

Strategy for storing mixed value and reference types without boxing

I'm looking for a way to store a large quantity of individual data values, with the following constraints. Assume that the types are bool, int32, double, decimal, string and blob (byte array). The ...
2
votes
2answers
98 views

Static Test-Data in Helper Class: Getter vs Public Constant

Currently I am in process of writing large batches of Unit-Tests in the Project I am working on. Additionally we recently Introduced GUI-Testing. This project is a relatively simple webapp, on basis ...
2
votes
1answer
224 views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
0
votes
0answers
19 views

An efficient + extendable way to store/reference a normalized set of “causations”?

I'm looking for a good way to define and store references to a normalized set of "causations". I have a well-defined set of reasons (short, descriptive strings) why certain objects are created and ...
0
votes
1answer
60 views

Lazy-initialization vs preprocessing in image-based iOS app

I am making an iOS app which contains museum gallery: user can look through exhibits and then select info about it, etc. On the exhibit screen I have two ScrollViews that are different, but are being ...
1
vote
2answers
184 views

How do you deal with clients asking for manual data edits in the database? [duplicate]

I've recently started working on a legacy application that frankly doesn't do all that it should. It's lacking a lot of features, has barely any administration capacities and doesn't check half the ...
2
votes
2answers
166 views

Why do binary files load quicker than alphanumeric text files?

I've noticed that when I load/store large data files in a binary format, the program runs much faster than if I load data from an ASCII encoded file. Why is this the case? The data in my case is ...
1
vote
1answer
201 views

How many bytes can I fit into a QR code, with low error correction?

I need to fit at least 300 bytes (or much much more) into a QR code and think I can do this by mapping each byte into the associated ISO/IEC 8859-1 character located here. Since each byte (1-255) ...
1
vote
3answers
318 views

Passing data between hundreds of objects in java [closed]

Currently, I'm working with a group on building a model. This model simulates interactions between many "agents" in a region. Agents can be any entity such as a city, a farmer, a business etc. Each ...
0
votes
0answers
256 views

Python Compare Values from 2 Columns from two separate CSV files

I am using Python. I have two columns(in separate CSVs) that contain an ID tag. Ideally, I would like to add on columns of information from one CSV to the other whenever the ID tag matches. For the ...
0
votes
1answer
64 views

Full signed part of two complement long [closed]

I am reverse engineering a data file (from a synthesizer). It seems that e.g. -1 is written as FF FF FF FF, so in four bytes. while +1 is written as 00 00 01 (--), so only three bytes. Is something ...
1
vote
4answers
206 views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
-1
votes
1answer
77 views

Choosing a file format to storage a simple text data [closed]

My .NET application makes its work and then creating a report (an Excel worksheet). The report holds thousands of short text strings [KEY - VALUE], so excel worksheets suites very well, one column ...
5
votes
1answer
157 views

Is there a standard way of cleaning data files?

Having spent quite a lot of time working in software companies that deal in medium to large data sets, I've seen that a choke point in their processes is often prepping data files for loading into ...
1
vote
1answer
22 views

Format for getting clear directions on data paramters from users

As part of my job, I regularly get ad-hoc requests from users for snapshots of our very large database. However, there's no fixed format for delivering these and they usually come through in the form ...
0
votes
0answers
38 views

Efficiently storing and parsing client's browser details

Do there exist browser fingerprinting API's that instead of running browser information (e.g. the user agent, installed plugins, screen resolution, system fonts etc.) through a hash function, instead ...
0
votes
0answers
64 views

Converting large files in python

I have a few files that are ~64GB in size that I think I would like to convert to hdf5 format. I was wondering what the best approach for doing so would be? Reading line-by-line seems to take more ...
0
votes
0answers
868 views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: Once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: ...
0
votes
0answers
20 views

versioning data schema in android internal storage

I am using Android's internal storage to hold data for my application and I am trying to find a good way to handle reading data of an older schema. For example, let's say I have serialized and written ...
1
vote
0answers
315 views

What are the bad points of using Core Data for iOS like an ORM

I just starting to use Core Data for my iOS app and I am thinking about how use Core Data for my needs. What are the features of my app: The user look for products proposed from a catalog. The user ...
0
votes
1answer
84 views

Persisting type in Java - should I be using flags?

Is there some specific piece of advice around persist values that indicate type? Specifically I want to know if I should persist a flag/state indicating the type in persistence or not. I get the ...
1
vote
0answers
222 views

using flat file vs sqlite db for frequent data grabbing in a cron job

I have some automated task to be done using certain types of logs generated on the server end of each day. Log files contains specific machine identification "MAC" that is to be re-written in to ...
0
votes
2answers
113 views

How to design this better?

I'm developing a system using .NET which will be used by multiple users. Because of that, I need to identify on the database which data belongs to each user. Explaining with an example, imagine I have ...
0
votes
3answers
103 views

Looking for a certain datatype if it exists

I'm looking for the kind of a certain datatypes. The main idea of this datatype is to describe a productionline containing several steps. Each step should be represented as a "vertex" containing an ...
4
votes
1answer
55 views

Deleting Content from Service [closed]

I'm not quite sure what the real practice is in the programming community. But when a user deletes content from your service (ie. deleting a status update, a photo of theirs, a comment they made). ...
0
votes
1answer
32 views

Place measurement data in database (access)

In a software I use records of measurement data. Each record contains basic information like a name, some parameters and the actual measurement data in the form of three arrays of double values that ...
4
votes
1answer
327 views

Should reflection be part of design?

I'm currently designing a somewhat large program that will involve the simulation of math/physics models and collection of data (I have not implemented any code yet). One of the main problems I'm ...
0
votes
1answer
107 views

scipy.stats.percentileofscore() wrong?

i'm a data scientist and python noob, but I seem to have found a bug with scipy.stats.percentileofscore. it may be "by design" but please bear with me. if you have an array, you will get an upper ...
2
votes
3answers
210 views

Building presentation layer directly on data layer

My application is currently quite a mess and I'm trying to untangle it all. My current idea is that I should have 3 separate layers: data layer that knows how to keep stuff in database (and handle ...
3
votes
2answers
177 views

What is a scalable and practical way to search existence of a group of strings in a huge file

Context: I built an app which generates around 1000 domain names based on user input. I need to check if they are available or not by matching against a huge zone file of parsed domain names which is ...
0
votes
0answers
38 views

modelling data / UI models in relation with the Data Layer

I have a user interface in Asp.Net MVC, which would for example need to load a list of products to show. Let us assume this calls ProductRepository.GetProducts() . In the data layer, I have a class ...
0
votes
2answers
62 views

Test Data in a Distributed System

A question that has been vexing me lately has been about how to effectively test (end-to-end) features in a distributed system. Particuarly, how to effectively manage (through time) test data for ...
0
votes
0answers
27 views

What data should QR-Code contain to serve both my own and external QR-Code reader

My webapplication uses codes (in form of http://www.url.com/code/105820582020204), that I want to encode as QR-Code. I assume that such generated QR-Code will be scanned by both people using my mobile ...
4
votes
3answers
273 views

Big Data: Can it be pre-processed?

My question is about "big data". Basically, big data involves the analysis of a large amount of data to make meaningful insights from it. I would like to know: Whether or not large amounts of data ...
0
votes
2answers
77 views

Data sharing across processes [closed]

I have a C# .NET app that downloads a users Tweets using LinqToTwitter. This is for TV broadcast - the client wants to show Tweets on air. We have two other apps that need to access these tweets. One ...
0
votes
1answer
65 views

Using MVC strictly with DAAB?

I am the only one at my company that is familiar with MVC and they are getting more pressure to modernize and switch to MVC for future projects, so I was tasked to create a template to use as a base ...
3
votes
1answer
89 views

Algorithm that determines relevant attributes in ordered list

Suppose I have a list of objects sorted by a person based on that person's subjective taste. Let's assume that this person is being somewhat consistent in their sorting and that there are quantifiable ...
0
votes
1answer
204 views

Design pattern for configuring objects

Context In a Cocoa application I am developing, I need to load a data model that is defined in plist files. So, I have something like: root (Array) ---- item (Dictionary) -------- name : "objects" ...
1
vote
1answer
252 views

Is it ethical or even legal to track user data/metrics using IP [closed]

This one is a bit tough. It is publicly available information, is it not? But it's can also be used maliciously. So is it legal/ethical to store and/or gather information on users based on IP? It's ...
1
vote
3answers
234 views

Starting with BI / Data Analysis [closed]

We have a bunch of data about our customers and we'd like to learn more about it. Writing queries to pull data out is not a problem, and we're doing that right now, but we'd like to take it to the ...
1
vote
4answers
358 views

Data duplication vs Encapsulating. Which design to use?

The problem that I face is how to combine encapsulating and optimal memory use. I can't show you my code and therefore explain it on extensive (I hope) example. Let's say we need to have a database ...
5
votes
1answer
140 views

Version control and data provenance in charts, slides, and marketing materials that derive from code ouput

I develop as part of a small team that mostly does research and statistics stuff. But from the output of our code, other teams often create promotional materials, slides, presentations, etc. We run ...
2
votes
1answer
952 views

AngularJS dealing with large data sets (Strategy)

I am working on developing a personal temperature logging viewer based on my rasppi curl'ing data into my web server's api. Temperatures are taken every 2 seconds and I can have several temperature ...
0
votes
2answers
72 views

Cleaning a dataset of song data - what sort of problem is this?

I have a set of data about songs. Each entry is a line of text which includes the artist name, song title, and some extra text. Some entries are only "extra text". My goal is to resolve as many ...
-2
votes
1answer
66 views

Turning a problem into a data

OK, I have an app that I'm creating but I'm just really not sure about how to approach the problem. The idea is fairly simple. I'm just not sure how to wrap it in a data model (or even if I should). ...