Data are values of qualitative or quantitative variables, belonging to a set of items. Data in computing (or data processing) are represented in a structure, often tabular (represented by rows and columns), a tree (a set of nodes with parent-children relationship) or a graph structure (a set of ...

learn more… | top users | synonyms

1
vote
1answer
58 views

Best practice for settings for PHP class for code generation

I would like to have a class create any code from given options and output it in a chosen way. Background I have written class for generation of simple JavaScript code for such cases where is needed ...
-4
votes
0answers
21 views

How would one go about creating a bi-directional sync between a CRM and a website?

I am a business user that would like to understand in layman's terms how this works. I would like some of my users to update their information in our CRM system (Salesforce.com) and other users to ...
3
votes
1answer
55 views

Interactive review of data in modal before inserting in db

I'm looking for a conceptual approach to a problem. I'm building a web app that imports data from a source I don't control. The data quality is pretty high, but occasionally, the data does not make ...
5
votes
3answers
181 views

Microservices and isolated data storage models [duplicate]

I have been looking at Microservices for a while now. The concept is not new but it's communicated in a lightweight manner. So, I am very excited about this. However, there is a question that I am ...
1
vote
1answer
49 views

BigData: time-based word count

What we are trying to do: We're trying to build a system that will count the number of unique entries for a certain timeframe. It's working ok until the data grows up or the timeframe increase, then ...
2
votes
2answers
244 views

Data Layer vs Business Layer

I think I'm in the minority on this but would be curious to see other perspectives. A lot of times I see people talk about the Data Layer like having functions like UpdateCustomer() or Summarize(). I ...
-1
votes
1answer
118 views

What is the difference between size and length? [closed]

It seems that the terms size and length are used interchangeably to describe how many bits, bytes or octets some data occupies, i.e. a length field in a data header is said to indicate the size of the ...
6
votes
1answer
148 views

Querying large amount of data for parallel processing [closed]

I have a dataset containing list of users (around 50M). Each user has an email address, name, and some more data columns. I want to send a weekly email to those users, and the content of the email ...
0
votes
1answer
46 views

Better way to transform data [closed]

Context Imagine complicated service that provides info about bank products, fees and so on. Imagine complicated UI that deals with it. UI and Service data formats are changed 1-2 times/month because ...
2
votes
0answers
65 views

Dealing with big data [closed]

I am on a project dealing with a lot of data in the form of images and videos (Data related to wind engineering). My requirement is to build a predictive algorithm based on the data I have. I have ...
4
votes
2answers
46 views

AutoIncrement Integer Column,Guid or PGuid, which to choose?

Guid has been a Global Unique Identification. Why does "PGuid" (Global Unique Identification) appear? 1) Is Guid NOT PERMENT for unique? (I sometimes wonder whether Guid is REALLY PERMENT generator ...
7
votes
3answers
742 views

How to deal with constructors in large data classes [duplicate]

Several times now I have come across the situations where you have some kind of settings class that simply contains a mass of data. Often these classes are simply not valid without at least most of ...
2
votes
2answers
151 views

Design pattern for fetching data in chunks

I am creating Qt application that uses database with huge amount of data to draw some charts. Fetching data from the database is time consuming, so is blocking the application thread or worker thread ...
0
votes
0answers
24 views

Verify That Filters on a Dataset are Exclusive

Suppose I have a data table of, say, cars with a bunch of properties for each record. Example: |Vin|Year|Make|Model|Color|Mileage| I want to allow users to define (arbitrarily complex) filters which ...
2
votes
3answers
170 views

Design pattern for large amounts of overflowing data?

Our current queues publish messages that consumed by 3rd party services with rate limits. Currently the messages are retried with exponential back-off. However there could be cases where data is ...
1
vote
3answers
46 views

Find likely candidates for visited locations along gps track

I'm creating a visualization tool for workout data, specializing on bicycle rides. I'm looking for ways to generate more value from the data that was recorded. Since there usually is a GPS track ...
1
vote
0answers
72 views

Data Store Design for NxN Data Aggregation

I am trying to come up with a theoretical solution to an NxN problem for data aggregation and storage. As an example I have a huge amount of data that comes in via a stream. The stream sends the data ...
2
votes
3answers
79 views

Best practise for databinding when a non-data object is included with the data

I'm using .NET with Windows Forms, but I believe there may be similar concerns with other toolkits. Suppose we have a list of objects of a certain type. Using an example to make the following easier ...
3
votes
3answers
159 views

Storing great amounts of data

I was brainstorming and I found quite an interesting topic I couldn't come up with a solution for myself. How does storing excessively large amounts of data work? I mean physically, no matter how big ...
-1
votes
1answer
177 views

process large dataset c# [closed]

I have an application that process portfolios. I must aggregate 8 years of returns of aproximatelly 8K securities. (about 2016 doubles (8 years of data) for 2 dates (for comparison) 4032 for 8K ...
-2
votes
1answer
179 views

3D visualization of moment using accelerometer on Android device [closed]

Is it possible to reconstruct the movement of an Android device in 3D space using data from accelerometer? For example, I may draw a circle with my hand in the air, holding an Android device. Is ...
2
votes
2answers
142 views

How to handle multiple models that relate to a single model

First of, sorry for the vague title. In our website framework, we have modal classes to represent our database tables, that hold a rows data. I'm currently working on a stat style page, and I'm ...
1
vote
1answer
61 views

Send more data, or do client processing?

I have a conceptual problem that I want feedback on. First and foremost, the layout of my system. It contains a server, with many clients connected to it. The clients get fed data from the server at ...
5
votes
6answers
390 views

Algorithm for determining transactions among weekly data series?

I'm trying to develop a small reporting tool (with sqlite backend). I can best describe this tool as a "transaction" ledger. What I'm trying to do is keep track of "transactions" from weekly data ...
1
vote
0answers
73 views

Saving and retrieving multiple cached json/txt data files

I am working on some numerical programming and need to generate the results for a model given a variety of input parameters. Since the model takes a while to run, I was planning to save the data to ...
2
votes
2answers
309 views

How do you sync changes to the data in your database? [closed]

I am new to a team developing a very large web application. Prior to my start, the team had 0 automation in place and just did everything by hand. This was possible because they were only a team of 2. ...
1
vote
0answers
79 views

Using a distributed system as an alternative to disk storage

So, one of the differences between "main memory" and disk storage is durability -- the data on hard disks will stay accessible and unaffected even if the host machine crashes or loses power. One can ...
4
votes
1answer
88 views

What method should I use to populate a tree as I'm consuming data that is contained within the tree?

I have a tree that consists of 5 types of nodes - V, W, X, Y, and Z. V represents the root node, and a given tree has exactly one element of type V. A V contains one or more Ws, a W contains one or ...
59
votes
14answers
8k views

Are data type declarators like “int” and “char” stored in RAM when a C program executes?

When a C program is running, the data is stored on the heap or the stack. The values are stored in RAM addresses. But what about the type indicators (e.g., int or char)? Are they also stored? ...
0
votes
3answers
1k views

Is it bad to store data in JSON form or should I use mySQL? [closed]

I've been building an application and the whole thing is built around a simple data storage system. Create an array. Convert it into JSON. Add <?php die() at the beginning to prevent access. ...
1
vote
2answers
157 views

Avoid data manipulation by user

I have a C# program (could be any programming language) that saves data to a file on a memory (hard disk, USB drive, etc.). The program uses this data for monitoring its operation time, but it could ...
4
votes
2answers
328 views

Is this data model a list or tree?

Using python syntax with below environment diagram, teacher taught us that there are 11 trees(orange boundary) in this diagram, including leaf. It has been taught here, lists are represented as a ...
4
votes
3answers
407 views

Saving to Registry vs. CustomFile [closed]

I've been using the usual My.Settings... method when it came to saving settings for my program but I was just wondering, If I wanted to save program settings into the Registry and load those settings, ...
4
votes
4answers
143 views

Approaches for storing and analysing large amounts of time-based data

I've been asked to develop a "telemetry" application that records data generated by a hardware device, which I read every 100ms. There are approx 250 data points (32-bit values), but only a subset ...
0
votes
0answers
76 views

Will this data mining approach work? Is it a good idea?

I need to extract fields like the document number, date, and invoice amount from a bunch of .csv files, which I believe are referred to as "unstructured text." I have some labeled input files and will ...
0
votes
0answers
67 views

Simple Path to Route Algorithm

I'm currently conducting research into subway paths, but stand in front of a problem, with no programming knowledge and an extremely short time-limit. Probably the question I am asking is trivial, ...
1
vote
1answer
73 views

Benefits of this data format

I'm integrating with a bluetooth blood pressure monitor, and I've come across one of the most bizarre data formats I've ever seen. The systolic, diastolic and pulse rate (all ints, should all be ...
0
votes
0answers
135 views

Whats the best way to implement DataMapper design pater and lazy loading (ghost object)

I'm doing a small project and I wanted to keep it simple to mantain. It is a renting management software. To keep it clean I decided to implement the data mapper patern, to keep the domain objects in ...
1
vote
2answers
186 views

One Enum vs Multiple Enums

I am developing an application where a user submits a mission and other users accept the mission. Pretty simple. I want to keep a track of the mission progress status and store it into a database. ...
0
votes
0answers
16 views

One Enum vs Multiple Enums [duplicate]

I am developing an application where a user submits a mission and other users accept the mission. Pretty simple. I want to keep a track of the mission progress status and store it into a database. ...
1
vote
1answer
94 views

File validation rules

I have an application that can accept CSV files to run some operations. The files look like: CREATE USER:username,last_name,first_name,age user1,Smith,John,23 user2,Poppins,Mary,257 There are a ...
4
votes
1answer
260 views

How to store data that is recorded with different frequency?

I'm writing a metallurgical web application, that must store data (electrical parameters) recorded at different frequencies. For example: six parameters recorded with frequency of one measure per ...
2
votes
3answers
319 views

Data format for binary data transfer

I have to make an oscilloscope. Ideally, the oscilloscope connects to a probe. This probe should be another program, and could get the data locally or from a network. This data are numbers, floats ...
1
vote
2answers
134 views

Is there a detriment to using SQLite over a pList for persistence?

I'm making a fairly simple application and I've got a settings menu that stores times and some boolean values. I have the application storing and retrieving data already but I started to wonder if ...
0
votes
0answers
60 views

Abstracting the data in a relational database. [duplicate]

While working with data, which is in a relational database I came up with this issue. At first, I was just writing helper functions to retrieve A SPECIFIC data I needed for the current problem but it ...
0
votes
1answer
177 views

How much to encapsulate when objects being dealt with are pretty similar

As I have been answering questions related to object-oriented design and how to go about reducing the amount of classes to make sure that there is not a lot of "class clutter" and code repetition I ...
1
vote
1answer
218 views

Is a linked list an actual data structure?

I hear conflicting sides from this topic. I was under the impression that a linked list was a way to implement a data structure (stack, queue), not actually a data structure.
0
votes
0answers
65 views

How to deal with requests that don't conform to the existing data structure?

The system I'm working on has a list of assets that belong to different customers of the company. This data is saved on an SQL database. Removing irrelevant data, let's say the table is like this: ...
1
vote
1answer
131 views

Running an algorithm against a large dataset

We have a python application which does a DFS search against data in a database. Right now, that data is small so I can hold everything in a python container but the data is going to grow and there ...
2
votes
1answer
213 views

ER Diagram Multiplicity (UML)

Schema: #=PK Employee (#NIN, fName, lName, address, DOB, sex, salary, taxCode, deptNo) Department (#deptNo, deptName, managerNIN, businessAreaNo, regionNo) Project (#projNo, projName, contractPrice, ...