Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

5
votes
1answer
897 views

Where to learn graph theory applications [closed]

I am completely new to designing algorithms with graphs. I am following CLRS and other video lectures in Youtube, notably from IIT/MIT. They are pretty good, and I currently have decent idea about ...
5
votes
7answers
1k views

Priority list of tasks stored in a database

I am trying to think of the best way to do the following: I have a list of tasks stored in the database. A task has a priority assigned to it. You can change the priority of a task to reorder the ...
5
votes
2answers
1k views

Creating huge decision tree

I'm to write an event correlator. A fundamental part of the system will be a decision tree that recognizes the origin of the fault basing on recorded states and log files. Often many accidents will ...
5
votes
2answers
4k views

Optimal way to store 18 billion key, value pairs [closed]

I have around 200 million new objects coming in, and a 90 day retention policy, so that leaves me with 18 billion records to be stored in the form of key-value pairs. Key and value both will be a ...
5
votes
3answers
387 views

Hashing growth strategy

What is a good growth strategy for hash tables? If the number of elements exceeds the number of buckets, I increase the number of buckets with the following formula: n = int(n * 1.618033988749895) | ...
5
votes
2answers
1k views

Data structure for pattern matching

Let's say you have an input file with many entries like these: date, ticker, open, high, low, close, <and some other values> And you want to execute a pattern matching routine on the ...
5
votes
2answers
303 views

Data structures for storing finger/stylus movements in drawing application?

I have a general question about creating a drawing application, the language could be C++ or ObjectiveC with OpenGL. I would like to hear what are the best methods and practices for storing strokes ...
5
votes
3answers
165 views

Maintaining a list of nodes with challenging requirements

This is a question regarding the design of a collection of ordered nodes which have some requirements that I am struggling to satisfy. In the problem area that I am dealing with we have routes which ...
5
votes
1answer
144 views

Bloom filters or similar, but with no false positive

To improve some lookups, I am considering the use of Bloom Filters. But in my use-case, the most probable outcome is that the element do exists in the target set. Bloom filters can have false ...
5
votes
2answers
900 views

Efficient Data Structure for holding a graph

Problem Link - http://opc.iarcs.org.in/index.php/problems/NUMTRIPLE In my opinion, the problem can be solved by a data structure, that shows how each number is connected to another, and via recursion ...
5
votes
1answer
2k views

Building a Graph Editor - How to create a data driven graph

I am developing a graph-editor that uses drag and drop to build hierarchical graphs (containing nodes and links). Each node in the graph should be linked to a table in our database (SQL Server). I did ...
5
votes
3answers
376 views

Generating Deep Arrays: Shallow to Deep, Deep to Shallow or Bad idea?

I'm working on an array structure that will be used as the data source for a report template in a web app. The data comes from relatively complex SQL queries that return one or many rows as one ...
5
votes
4answers
266 views

Users creating instances from a template - is there a software pattern that corresponds to this use case?

I am looking into designing a system where users build a template item and then create many instances based on this template.They then need to be able customise each instance but I also need for them ...
4
votes
5answers
503 views

Is my understanding of abstract datatypes correct?

After reading lots of pages on various sites, I came to the conclusion that abstract datatype helps to separate the use of datastructure from its implementation. We can easily map the data into a ...
4
votes
2answers
423 views

Is there any particular reason for the use of lists over queues in functional programming languages?

Most functional programming languages such as Scheme and Haskell use lists as their main data structure. Queues are identical to lists, except for the fact appending to the end - not to the begin - ...
4
votes
2answers
2k views

Is there a way to handle nested Collections more elegantly?

My question is rather a design question. In my program I got to a data structure that looks something like this: private ConcurrentHashMap<A, ConcurrentHashMap<B, ConcurrentHashMap<Integer, ...
4
votes
2answers
4k views

What is the advantage of using map datastructure?

I am a member of some online coding website and today when I submitted a solution in C++, I saw there were much better answers in C++ itself which took considerably less time. The solutions were open ...
4
votes
2answers
310 views

Are bloom filters actually faster than hashes, even taking in account cache?

Bloom filters look really great when you consider you can determine if an Int is in a set with 99% certainty in constant time. But so can hashes, with the only difference that, in a hash, most of the ...
4
votes
4answers
282 views

Why use an enum to determine node type in a parse tree?

The Sun Compiler Tree API uses an interface called Tree as the parent of all the different kinds of parse tree nodes (assignments, if-statements, class declarations, etc.). A part of this interface is ...
4
votes
3answers
6k views

How can I practice implementing data structures and algorithms? [duplicate]

I just finished learning the basics on data structures and algorithms using Python and C, completing some of the exercises from the book I read. I feel that it is necessary to implement some of these ...
4
votes
3answers
2k views

Data Oriented Design - impractical with more than 1-2 structure “members”?

The usual example of Data Oriented Design is with the Ball structure: struct Ball { float Radius; float XYZ[3]; }; and then they make some algorithm that iterates a std::vector<Ball> ...
4
votes
4answers
439 views

Building a route creator

Ok, already up front, I'm going to tell you, that this is a bonus task for Data structure course I'm taking. That should take care of all the questions whether or not this is for a homework. Route ...
4
votes
2answers
1k views

Which design better when use foreign key instead of a string to store a list of id

I'm building online examination system. I have designed to table, Question and GeneralExam. The table GeneralExam contains info about the exam like name, description, duration,... Now I would like to ...
4
votes
2answers
491 views

Data Structures: What are some common examples of problems where “buffers” come into action?

I was just wondering if there were some "standard" examples that everyone uses as a basis for explaining the nature of a problem that requires the use of a buffer. What are some well-known problems in ...
4
votes
2answers
257 views

Why Do B+ Trees Have a Minimum Occupancy of 50%?

As far as I can tell, the basic operations (add, delete, search) on a B+ Tree would function the same if minimum occupancy was 1, or 1/3, or any other function on node size. All the sources available ...
4
votes
3answers
983 views

Is it conceivable to have millions of lists of data in memory in Python?

I have over the last 30 days been developing a Python application that utilizes a MySQL database of information (specifically about Norwegian addresses) to perform address validation and correction. ...
4
votes
4answers
243 views

Useful Pre-requisites for Operating Systems course?

How important is knowledge of data structures and algorithms for an Operating Systems and Systems Programming course where we will be required to design and implement a kernel of a real-time operating ...
4
votes
1answer
218 views

Why define struct in variable?

Consider the following program: Many people when they want to use a struct, they create a new variable as: struct structureName variableName While it works when you just define it as: ...
4
votes
2answers
198 views

Decimal vs. Integer; Given a fixed range of values, which is preferable for accurate computation?

After getting into a "heated discussion" with someone, I figured I'd ask this question for the sake of posterity. I'm willing to be corrected if my assumption is incorrect but I'd like to hear a ...
4
votes
3answers
671 views

How Immutable Sets are Manipulated

When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead. I know ...
4
votes
2answers
3k views

Nested Enum type in C++ or C#?

I've come across a recurring issue in a few of my recent projects in which I find myself using enums to represent state, or type, or something else, and I need to check against a few conditions. Some ...
4
votes
4answers
592 views

Appropriate data structure for deeply nested data?

I can't figure out what would be an appropriate data structure for my problem. I need to write a script that will count some events. Input is a list of events (thousands of them). Each event is an ...
4
votes
3answers
755 views

Is a tree with nodes that have reference to parent still a tree?

If we make reference to the parent for each node in a tree, do we still have a tree (by definition) anymore? Wikipedia definition is: In computer science, a tree is a widely used abstract data ...
4
votes
2answers
512 views

Fast set indexing data structure for superset retrieval

I am given a set of sets: {{a,b}, {a,b,c}, {a,c}, {a,c,f}} I would like to have a data structure to index those sets such that the following "lookup" is executed fast: find all supersets of a given ...
4
votes
3answers
225 views

Picking powers of two for sizes/limits

A habit I've observed among programmers, and a habit I sometimes subconsciously exhibit myself, is to pick powers of two (or powers of two minus one) when defining a database schema, a data buffer, ...
4
votes
1answer
2k views

Homogeneous vs. heterogeneous AST representation

What are the reasons to choose a homogeneous vs. a heterogeneous AST representation for implementing a complex domain-specific programming language? Just to be very clear about what I'm asking, here ...
4
votes
2answers
979 views

Patterns for a tree of persistent data with multiple storage options?

I have a real-world problem which I'll try to abstract into an illustrative example. So imagine I have data objects in a tree, where parent objects can access children, and children can access ...
4
votes
1answer
512 views

Amazon Kindle - Whispersync implementation?

For those who are not aware of Kindle's whispersync, here is how it works (from amazon.com): "...Whispersync synchronizes the bookmarks and furthest page read among devices registered to the same ...
4
votes
1answer
338 views

Website with test data files for specific algorithms/data structures?

Is there any website, like SPOJ and Project Euler, with the test data files available for specific algorithms/data structures? I know it's a fun challenge to solve those problems on your own, but I ...
4
votes
1answer
114 views

How to store data that is recorded with different frequency?

I'm writing a metallurgical web application, that must store data (electrical parameters) recorded at different frequencies. For example: six parameters recorded with frequency of one measure per ...
4
votes
2answers
150 views

Incremental and partial drawing of the large graphs

I have the following problem: I'm developing a software for data visualization where I get a graph structure and represent it in 3D space. So far, I've been using force-based algorithms to draw graphs ...
4
votes
2answers
102 views

Is it a good idea to have a database row that represents an unknown value within a system [closed]

We have two systems, System A imports a list of sports events from system B, it only wants to import the events we will be providing live data for, the filter used to hide events that we are not ...
4
votes
1answer
322 views

Using macros to implement a generic vector (dynamic array) in C. Is this a good idea?

So far I have only done personal projects at home. I hope to get involved in some open source project some time next year. The languages I that have been using the most are C and C++. I have used both ...
4
votes
1answer
124 views

Merge directed acyclic graphs minimizing number of nodes

I have some DAGs (directed acylic graphs) and I want to merge them in order to minimize the number of nodes (we could say that every node has a cost, while edges are free). These four different DAGs ...
4
votes
1answer
235 views

Designing storage service data structure for decoupled models sharing same data

Surely most of you remember the Norton Commander application where similar (sometimes the same) data is displayed in separate decoupled views. I'm building a web application that follows the same ...
4
votes
1answer
817 views

Which data structure to represent a triangular undirected graph

I was wanting to create a graph similar to this in C++: The one I will implement will have more edges and vertices, but will use this triangle style. I was wanting to create something that I could ...
3
votes
6answers
1k views

Why isn't the line count in Visual Studio zero-based?

This just struck me as an oversight by Microsoft. Since arrays and other data-structures within the .NET framework begin from zero (zero-based) why don't we have a line 0 within the code view in ...
3
votes
6answers
2k views

How did programmers implement Linked list ideas before the object oriented paradigm?

Linked lists, as far as I have seen, are largely implemented using object-oriented ideas. (having an object that holds some information and the address of the next link). How were Linked-lists ...
3
votes
4answers
1k views

What do you call an iterator that returns the current, previous and next siblings of each node of a list? [closed]

What do you call an iterator that given a list [a, b, c], returns an object of the form { prev, curr, next } for each iteration? e.g, (? === undefined) { prev: ?, curr: a, next: b } { prev: a, curr: ...
3
votes
8answers
1k views

Strategy/algorithm to divide pot to chips

I want to split poker pot to chips. Example: Pot = $17.500. I have the endless piles of chips. I have the following types of chips: ChipsTypes = [$1, $5, $10, $25, $100, $500, $1.000, $5.000, ...