Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

1
vote
2answers
157 views

Random access (read/write) in datastructures

Certain datastructures, like Python's Dictionary, are unordered/random read/written. As programming in python is iterative (and programming in general is?), how do these unordered datastructures ...
1
vote
0answers
187 views

Most suitable data structure for in-memory string -> int conversion for variable length strings

My project involves validating and normalising email addresses in this format [userpart]@[domainpart].[tld] After syntactic validation of the address, [tld] is checked to exist, otherwise the ...
7
votes
4answers
837 views

How do we keep dependent data structures up to date?

Suppose you have a parse tree, an abstract syntax tree, and a control flow graph, each one logically derived from the one before. In principle it is easy to construct each graph given the parse tree, ...
0
votes
2answers
366 views

PHP SplFixedArray How To Deal With billions of indexes?

I've built a flatfile db system that will work like mini-noSQL system and it split db to smaller files, and I did a SplFixedArray wrapper for it with a method for searching usage built on preg_grep. ...
0
votes
1answer
197 views

Which data structure for representing possible navigations between a game's screens?

I am trying to find an appropriate data structure for representing available navigations between a game's screens. Using a linked list, a node can only have one node after it : inappropriate. Using ...
5
votes
1answer
267 views

Can anyone explain to me what problem Core Data solves? [closed]

Core Data seems to add a needless layer of complexity. If you want to save data created natively by the user in an app why not just use an object and then write the data all to SQLite or back to a ...
1
vote
1answer
79 views

Data structure supporting the following operations

I'm looking for a data structure for working with a set of data which is most efficient to do the following : Check whether an item has been categorized or not. (The categorized and uncategorized ...
-2
votes
3answers
155 views

what would be the best data structure for my situation [closed]

I have a list of unique string. I don't want to destroy this string i need it.I only need to find out weather an given string S exist in this unique list.What would be the best data structure for this ...
0
votes
2answers
136 views

How should I structure my database to gain maximum efficiently in this scenario?

I'm developing a PHP script that analyzes the web traffic of my clients websites. By placing a link to a javascript on the clients website (think of Google Analyses), my script harvests information ...
6
votes
3answers
3k views

What is the best way to store a table in C++

I'm programming a decision tree in C++ using a slightly modified version of the C4.5 algorithm. Each node represents an attribute or a column of your data set and it has a children per possible value ...
1
vote
2answers
3k views

Is it wise to store a big lump of json on a database row

I have this project which stores product details from amazon into the database. Just to give you an idea on how big it is: [{"title":"Genetic Engineering (Opposing ...
0
votes
1answer
1k views

I need to create an employee schedule/appointments program, but I don't know how to set it up [closed]

I work for a gym. I am the programmer, its just me, nobody else! Gets really frustrating when there's nobody to bounce conceptual ideas off of. I'm getting rid of our archaic paper binder systems for ...
0
votes
4answers
130 views

Data Structure for Small Number of Agents in a Relatively Big 2D World

I'm working on a project where we will implement a kind of world simulation where there is a square 2D world. Agents live on this world and make decisions like moving or replicating themselves based ...
1
vote
6answers
5k views

Serializing Data Structures in C [closed]

I've recently read three separate books on algorithms and data structures, tcp/ip socket programming, and programming with memory. The book about memory briefly discussed the topic of serializing data ...
3
votes
2answers
128 views

Reconstructing a tree from depth information

I have a data structure that describes a tree. Nodes are ordered, with "depth" information, and knowing the order and the depth the tree can be reconstructed. I'm not entirely sure how to describe it ...
3
votes
1answer
776 views

Implementing basic data structures in programming interviews [closed]

I have been preparing to do my first technical interview in a month, and I have a question about implementing common data structures like stacks or linked lists. I plan to do the interview in either ...
2
votes
1answer
1k views

Treating a 1D data structure as 2D grid

Hopefully this is a good question. I am working with a native class that represents a 2D image as a 1D array. If you want to change one pixel, for example, you need to now how to derive the index from ...
-1
votes
3answers
2k views

Why are algorithms and data structures important? [duplicate]

It seems like a lot of university-level computer science programs and programmer job interviews focus heavily on algorithms and data structures. I'm curious as to why universities and employers put so ...
-1
votes
2answers
479 views

Importance of data structures in modern S/W development [closed]

During 1990s when there were no advanced frameworks/paradigms available for S/W development , knowledge on data structures was critical... which I can comprehend. But nowadays, for most of the ...
1
vote
1answer
207 views

Inventory / Stock in multiple locations

I am working on a warehouse management system (WMS) that needs to support having stock in multiple locations. Could be in a different building, could be stored in n* places in a building (quick ...
1
vote
1answer
335 views

Immutable Data Structure For Time Series Data

I'm wondering what the best data structure (for storing data on disk) is for storing immutable time-series data (99% of the data is truly immutable, the 1% is metadata that is separate from the ...
0
votes
2answers
89 views

How to properly structure a collection of immutable constants

I'm in the design stages of a program that based upon input received, would select a particular category, and action within that category. The category, the action within in, and the value(s) related ...
2
votes
1answer
159 views

Separating validation from persistence in models?

Models in an MVC application gets kind of hard to test when validation of data and persistence of data is baked together in the model. I would like to test those separately to prevent doing ...
7
votes
3answers
1k views

Architecturally speaking, does a database abstraction layer, such as Microsoft's Entity Framework, void the need for a separate Data Access Layer?

The way it was For years, I have organized my software solutions as such: Data Access Layer (DAL) to abstract the business of accessing data Business Logic Layer (BLL) to apply business rules to ...
2
votes
3answers
503 views

What arguments can I use to justify the use of either XML or JSON to store and transmit object data?

My project lead considers both of these approaches to be unnecessary overhead. I have been involved in and have witnessed a lot of talk around XML vs JSON, but this is the first time that I have heard ...
9
votes
1answer
460 views

Optimal Data Structure for our own API

I'm in the early stages of writing an Emacs major mode for the Stack Exchange network; if you use Emacs regularly, this will benefit you in the end. In order to minimize the number of calls made to ...
2
votes
4answers
294 views

Generic term for “objects” vs “fundamental types”?

What are the exact terms to call data types with a logic structure (like C structures, C++ or Java objects) versus fundamental data types (like numeric types, characters, booleans...) independently of ...
3
votes
2answers
760 views

Collection interfaces in C#, coming from Java

In Java, I'm used to declaring collections using the most-abstract interface possible and then constructing them using the concrete implementation that makes sense at the time. It usually looks ...
3
votes
1answer
438 views

Pure C Vector Implementation

I am implementing a vector in C. I am doing this for the fun of programming, for the fun of learning, and for the use of the data structure in later projects. This is not homework. My question is ...
3
votes
1answer
137 views

Is there a name for a randomized heap?

I've made a container which functions like a Binary Heap in terms of insertions and popping the root element. The key difference is that the comparison steps are all replaced with a random call for ...
0
votes
2answers
97 views

By the definition of a Tree(ADT), is it “wrong” for a node to add a sibling?

The title pretty much says it all. I'm just having a little computer science exercise and am building a Tree. I want a node to be able to add a sibling which requires one to first get a reference to ...
0
votes
3answers
164 views

Best data structure for representing English verb forms

I need to come up with a data structure to keep information about English verb forms. In most cases a verb can be in one of 4 forms: base, present participle, past participle and past simple, for ...
1
vote
1answer
115 views

Is there a name for this tree variant?

So I've created (and I'm sure someone else has come up with this before - I'm not claiming to have invented it) a data structure which consists of a binary search tree, only instead of each node ...
0
votes
1answer
341 views

Comparing performance of different C++ self-made tree data structures

I have two tree data-structures that I made in C++ , and they give a response for range queries/point queries in that program. I am using Ubuntu. I need help in understanding the way to compare the ...
9
votes
2answers
800 views

Uses of persistent data structures in non-functional languages

Languages that are purely functional or near-purely functional benefit from persistent data structures because they are immutable and fit well with the stateless style of functional programming. But ...
-1
votes
2answers
2k views

What do I need to know about Data Structures and Algorithms in the “real” world [closed]

I just finished the data structures and algorithms course in school , I took it during the summer so 6wks course vs a 16 wk course during the regular semester. So not only was the course hard but it ...
-1
votes
2answers
79 views

Store a tree with near next neighbours

I would to find an efficient structure (speed and volume) to store nodes and their neighbourhood. My input is build out of stings in the following format ./X/Y.log where X ∈ [0,359] and Y ...
6
votes
3answers
2k views

Data classes: getters and setters or different method design

I've been trying to design an interface for a data class I'm writing. This class stores styles for characters, for example whether the character is bold, italic or underlined. But also the font-size ...
4
votes
1answer
2k views

Homogeneous vs. heterogeneous AST representation

What are the reasons to choose a homogeneous vs. a heterogeneous AST representation for implementing a complex domain-specific programming language? Just to be very clear about what I'm asking, here ...
0
votes
2answers
402 views

Separation of Concerns, Data Access Layer

I was thinking about this earlier today and figured I would get some input on the matter. When I develop applications I would usually have the Data Access Layer on another project, incase it could be ...
5
votes
2answers
2k views

Optimal way to store 18 billion key, value pairs [closed]

I have around 200 million new objects coming in, and a 90 day retention policy, so that leaves me with 18 billion records to be stored in the form of key-value pairs. Key and value both will be a ...
2
votes
2answers
758 views

Same constructor signature with different semantics?

Occasionally, I'm confronted with the problem that I have to provide some constructors in order to initialize objects with different sets of data. These sets can be mathematically transformed into ...
3
votes
3answers
87 views

How should I manage short-term user preferences, to simplify data cleanup?

When rolling out a new feature, sometimes it makes sense to notify the end user what it is, and give them an opportunity to provide feedback whether the software should get them to 'try something new' ...
5
votes
1answer
669 views

Where to learn graph theory applications [closed]

I am completely new to designing algorithms with graphs. I am following CLRS and other video lectures in Youtube, notably from IIT/MIT. They are pretty good, and I currently have decent idea about ...
21
votes
4answers
1k views

What is the definition of “Big Data”?

Is there one? All the definitions I can find describe the size, complexity / variety or velocity of the data. Wikipedia's definition is the only one I've found with an actual number Big data ...
3
votes
4answers
470 views

What's the difference between a stream and a queue?

What's the difference between a stream and a queue? They both have the concept of an ordered set of elements, but tend to have different implementations and a different vocabulary of ...
5
votes
1answer
1k views

Building a Graph Editor - How to create a data driven graph

I am developing a graph-editor that uses drag and drop to build hierarchical graphs (containing nodes and links). Each node in the graph should be linked to a table in our database (SQL Server). I did ...
2
votes
6answers
3k views

Heap overflow vs stack overflow

So as a general rule to avoid a stack overflow, big objects should be allocated to the heap (correct me if I am wrong). But, since the heap and the stack expand towards each other, wouldn't this cause ...
0
votes
1answer
418 views

Priority queue for Kruskal's algorithm with running time O(E lg V)

I'm reviewing my notes on Kruskal's algorithm and have a question about getting the runtime down to O(E lg V). By using a PQ with edges and a boolean array of which vertices we have added to our tree ...
1
vote
1answer
2k views

How to model/structure data that contains parent/child relationships for the same entity?

I think it is just my thinking that is off here. Suppose I have two entities: question and choice. A question can be a parent or child, or both (in relation to other questions). It seems that this ...