Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

0
votes
1answer
50 views

Full signed part of two complement long [on hold]

I am reverse engineering a data file (from a synthesizer). It seems that e.g. -1 is written as FF FF FF FF, so in four bytes. while +1 is written as 00 00 01 (--), so only three bytes. Is something ...
1
vote
4answers
173 views

Best datatype to store a ternary, or three-state variable

Disclaimer: I know that datatypes are a little bit subjective to which scripting/programming language you are using, I like to write in Python as a matter of preference; though I am happy to hear ...
-2
votes
0answers
32 views

Explaination Required ASP.NET MVC Complex Views [closed]

I am new to MVC framework. below are my questions. as I'm new to this framework, Please give a detail info how to achieve it. 1) Is it possible to retrieve all the data from these tables through ...
-1
votes
1answer
59 views

Choosing a file format to storage a simple text data [closed]

My .NET application makes its work and then creating a report (an Excel worksheet). The report holds thousands of short text strings [KEY - VALUE], so excel worksheets suites very well, one column ...
5
votes
1answer
143 views

Is there a standard way of cleaning data files?

Having spent quite a lot of time working in software companies that deal in medium to large data sets, I've seen that a choke point in their processes is often prepping data files for loading into ...
1
vote
1answer
21 views

Format for getting clear directions on data paramters from users

As part of my job, I regularly get ad-hoc requests from users for snapshots of our very large database. However, there's no fixed format for delivering these and they usually come through in the form ...
0
votes
0answers
33 views

Efficiently storing and parsing client's browser details

Do there exist browser fingerprinting API's that instead of running browser information (e.g. the user agent, installed plugins, screen resolution, system fonts etc.) through a hash function, instead ...
0
votes
0answers
53 views

Converting large files in python

I have a few files that are ~64GB in size that I think I would like to convert to hdf5 format. I was wondering what the best approach for doing so would be? Reading line-by-line seems to take more ...
0
votes
2answers
72 views

Looking for a certain datatype if it exists

I'm looking for the kind of a certain datatypes. The main idea of this datatype is to describe a productionline containing several steps. Each step should be represented as a "vertex" containing an ...
0
votes
2answers
53 views

Test Data in a Distributed System

A question that has been vexing me lately has been about how to effectively test (end-to-end) features in a distributed system. Particuarly, how to effectively manage (through time) test data for ...
1
vote
2answers
1k views

Dataset with coordinates of borders of countries [closed]

I am creating an application that visually displays world regions, e.g. to place markers within an administrative region. Does a dataset exist with geometrical or geographical (long/lat) descriptions ...
0
votes
0answers
160 views

How to parse multipart field/file data separately?

I want to parse a multipart form twice: Once to grab the incoming fields, and later to process the file upload. I am trying to maintain a proper separation of concerns within my Node app: ...
0
votes
0answers
16 views

versioning data schema in android internal storage

I am using Android's internal storage to hold data for my application and I am trying to find a good way to handle reading data of an older schema. For example, let's say I have serialized and written ...
1
vote
0answers
96 views

What are the bad points of using Core Data for iOS like an ORM

I just starting to use Core Data for my iOS app and I am thinking about how use Core Data for my needs. What are the features of my app: The user look for products proposed from a catalog. The user ...
0
votes
1answer
78 views

Persisting type in Java - should I be using flags?

Is there some specific piece of advice around persist values that indicate type? Specifically I want to know if I should persist a flag/state indicating the type in persistence or not. I get the ...
0
votes
2answers
110 views

How to design this better?

I'm developing a system using .NET which will be used by multiple users. Because of that, I need to identify on the database which data belongs to each user. Explaining with an example, imagine I have ...
1
vote
0answers
106 views

using flat file vs sqlite db for frequent data grabbing in a cron job

I have some automated task to be done using certain types of logs generated on the server end of each day. Log files contains specific machine identification "MAC" that is to be re-written in to ...
39
votes
12answers
3k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states in between? Wouldn't it be more natural to store floats like ...
4
votes
1answer
52 views

Deleting Content from Service [closed]

I'm not quite sure what the real practice is in the programming community. But when a user deletes content from your service (ie. deleting a status update, a photo of theirs, a comment they made). ...
0
votes
1answer
23 views

Place measurement data in database (access)

In a software I use records of measurement data. Each record contains basic information like a name, some parameters and the actual measurement data in the form of three arrays of double values that ...
4
votes
1answer
305 views

Should reflection be part of design?

I'm currently designing a somewhat large program that will involve the simulation of math/physics models and collection of data (I have not implemented any code yet). One of the main problems I'm ...
0
votes
1answer
65 views

scipy.stats.percentileofscore() wrong?

i'm a data scientist and python noob, but I seem to have found a bug with scipy.stats.percentileofscore. it may be "by design" but please bear with me. if you have an array, you will get an upper ...
2
votes
3answers
165 views

Building presentation layer directly on data layer

My application is currently quite a mess and I'm trying to untangle it all. My current idea is that I should have 3 separate layers: data layer that knows how to keep stuff in database (and handle ...
3
votes
2answers
170 views

What is a scalable and practical way to search existence of a group of strings in a huge file

Context: I built an app which generates around 1000 domain names based on user input. I need to check if they are available or not by matching against a huge zone file of parsed domain names which is ...
1
vote
2answers
456 views

Radix exchange sort implementation?

I need a little help understanding the implementation of radix exchange sort algorithm. Radix exchange (I don't know if it's the exact name in english based literature) is a lot similar to quicksort, ...
0
votes
0answers
30 views

modelling data / UI models in relation with the Data Layer

I have a user interface in Asp.Net MVC, which would for example need to load a list of products to show. Let us assume this calls ProductRepository.GetProducts() . In the data layer, I have a class ...
4
votes
3answers
258 views

Big Data: Can it be pre-processed?

My question is about "big data". Basically, big data involves the analysis of a large amount of data to make meaningful insights from it. I would like to know: Whether or not large amounts of data ...
0
votes
0answers
26 views

What data should QR-Code contain to serve both my own and external QR-Code reader

My webapplication uses codes (in form of http://www.url.com/code/105820582020204), that I want to encode as QR-Code. I assume that such generated QR-Code will be scanned by both people using my mobile ...
3
votes
7answers
301 views

make-like build tools for data? [closed]

Make is a standard tools for building software. But make decides whether a target needs to be regenerated by comparing file modification times. Are there any proven, preferably small tools that ...
0
votes
2answers
77 views

Data sharing across processes [closed]

I have a C# .NET app that downloads a users Tweets using LinqToTwitter. This is for TV broadcast - the client wants to show Tweets on air. We have two other apps that need to access these tweets. One ...
0
votes
0answers
14 views

Article about “painting” coorelation between complex data [closed]

I am trying to get an understanding of quite a big data set. I have 20 different inputs parameters and 10 different outputs. The control algorithm is made and seems to work, but now I need to ...
2
votes
1answer
698 views

AngularJS dealing with large data sets (Strategy)

I am working on developing a personal temperature logging viewer based on my rasppi curl'ing data into my web server's api. Temperatures are taken every 2 seconds and I can have several temperature ...
0
votes
1answer
53 views

Using MVC strictly with DAAB?

I am the only one at my company that is familiar with MVC and they are getting more pressure to modernize and switch to MVC for future projects, so I was tasked to create a template to use as a base ...
3
votes
1answer
82 views

Algorithm that determines relevant attributes in ordered list

Suppose I have a list of objects sorted by a person based on that person's subjective taste. Let's assume that this person is being somewhat consistent in their sorting and that there are quantifiable ...
0
votes
1answer
172 views

Design pattern for configuring objects

Context In a Cocoa application I am developing, I need to load a data model that is defined in plist files. So, I have something like: root (Array) ---- item (Dictionary) -------- name : "objects" ...
5
votes
1answer
129 views

Version control and data provenance in charts, slides, and marketing materials that derive from code ouput

I develop as part of a small team that mostly does research and statistics stuff. But from the output of our code, other teams often create promotional materials, slides, presentations, etc. We run ...
7
votes
1answer
328 views

Does current evidence support the adoption of Contextual over Canonical Data Models?

The "canonical" idea is pervasive in software; patterns like Canonical Model, Canonical Schema, Canonical Data Model and so on, seem to come up again and again in development. Like many developers, ...
1
vote
4answers
320 views

Data duplication vs Encapsulating. Which design to use?

The problem that I face is how to combine encapsulating and optimal memory use. I can't show you my code and therefore explain it on extensive (I hope) example. Let's say we need to have a database ...
1
vote
1answer
199 views

Is it ethical or even legal to track user data/metrics using IP [closed]

This one is a bit tough. It is publicly available information, is it not? But it's can also be used maliciously. So is it legal/ethical to store and/or gather information on users based on IP? It's ...
1
vote
3answers
147 views

Starting with BI / Data Analysis [closed]

We have a bunch of data about our customers and we'd like to learn more about it. Writing queries to pull data out is not a problem, and we're doing that right now, but we'd like to take it to the ...
1
vote
1answer
154 views

Consolidating hotels data from various booking sites with different IDs or reference

In one of my projects, I have data for hotels, and other booking sites are able to book this hotel. For example: Hotel A - Booking (ID = 4002), Expedia (ID = 123), Priceline (ID = 147) The three ...
0
votes
0answers
83 views

Searching in a large dataset

I am working on a model of social interactions in mice. I have mice and boxes and a simulation that outputs which mouse stays in which box during which time period. The problem is how to obtain, in ...
0
votes
2answers
71 views

Cleaning a dataset of song data - what sort of problem is this?

I have a set of data about songs. Each entry is a line of text which includes the artist name, song title, and some extra text. Some entries are only "extra text". My goal is to resolve as many ...
-2
votes
1answer
64 views

Turning a problem into a data

OK, I have an app that I'm creating but I'm just really not sure about how to approach the problem. The idea is fairly simple. I'm just not sure how to wrap it in a data model (or even if I should). ...
2
votes
1answer
467 views

How best to merge/sort/page through tons of JSON arrays?

Here's the scenario: Say you have millions of JSON documents stored as text files. Each JSON document is an array of "activity" objects, each of which contain a "created_datetime" attribute. What is ...
7
votes
2answers
2k views

Data Synchronization in mobile apps - multiple devices, multiple users

I'm looking into building my first mobile app. One of the core features of the application is that multiple devices/users will have access to the same data -- and all of them will have CRUD rights. I ...
1
vote
0answers
122 views

What will be the better way for data retrieval on application that needs to handle limited amount of data? [duplicate]

Just moved this question from Stack Overflow. Since, adding my code snippets itself would make this question really long. Instead, I am pretty interested in knowing a better ways for data retrieval on ...
0
votes
2answers
174 views

Perfomance of 8 bit operations on 64 bit architechture

I am usually a Python / Database programmer, and I am considering using C for a problem. I have a set of sequences, 8 characters long with 4 possible characters. My problem involves combining sets of ...
7
votes
3answers
235 views

Are there best practices for checking for errors in business logic for DB refreshes? [closed]

I'm working for a company with very large databases which all refresh overnight, ready for users to interrogate in the morning. These over night loads essentially truncate all the tables, and reload ...
1
vote
3answers
239 views

Program design with multiple data sets, algorithms and results

I have a question regarding the design of my software. There are four different data sets and three consecutive algorithms operating on the data sets and on the results from the previous algorithm ...