Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

5
votes
2answers
916 views

How do scalable bloom filters work?

I was reading up on scalable bloom filters and could not understand how each time a constituent bloom filters fills up, a new bloom filter with larger size is added. The elements that contributed to ...
5
votes
2answers
803 views

Efficient Data Structure for holding a graph

Problem Link - http://opc.iarcs.org.in/index.php/problems/NUMTRIPLE In my opinion, the problem can be solved by a data structure, that shows how each number is connected to another, and via recursion ...
5
votes
3answers
160 views

Maintaining a list of nodes with challenging requirements

This is a question regarding the design of a collection of ordered nodes which have some requirements that I am struggling to satisfy. In the problem area that I am dealing with we have routes which ...
5
votes
1answer
1k views

Building a Graph Editor - How to create a data driven graph

I am developing a graph-editor that uses drag and drop to build hierarchical graphs (containing nodes and links). Each node in the graph should be linked to a table in our database (SQL Server). I did ...
5
votes
3answers
355 views

Generating Deep Arrays: Shallow to Deep, Deep to Shallow or Bad idea?

I'm working on an array structure that will be used as the data source for a report template in a web app. The data comes from relatively complex SQL queries that return one or many rows as one ...
5
votes
4answers
263 views

Users creating instances from a template - is there a software pattern that corresponds to this use case?

I am looking into designing a system where users build a template item and then create many instances based on this template.They then need to be able customise each instance but I also need for them ...
4
votes
5answers
464 views

Is my understanding of abstract datatypes correct?

After reading lots of pages on various sites, I came to the conclusion that abstract datatype helps to separate the use of datastructure from its implementation. We can easily map the data into a ...
4
votes
2answers
399 views

Is there any particular reason for the use of lists over queues in functional programming languages?

Most functional programming languages such as Scheme and Haskell use lists as their main data structure. Queues are identical to lists, except for the fact appending to the end - not to the begin - ...
4
votes
2answers
2k views

Is there a way to handle nested Collections more elegantly?

My question is rather a design question. In my program I got to a data structure that looks something like this: private ConcurrentHashMap<A, ConcurrentHashMap<B, ConcurrentHashMap<Integer, ...
4
votes
2answers
200 views

Are bloom filters actually faster than hashes, even taking in account cache?

Bloom filters look really great when you consider you can determine if an Int is in a set with 99% certainty in constant time. But so can hashes, with the only difference that, in a hash, most of the ...
4
votes
2answers
393 views

Overriding GetHashCode in a mutable struct - What NOT to do?

I am using the XNA Framework to make a learning project. It has a Point struct which exposes an X and Y value; for the purpose of optimization, it breaks the rules for proper struct design, since its ...
4
votes
1answer
2k views

Treating a 1D data structure as 2D grid

Hopefully this is a good question. I am working with a native class that represents a 2D image as a 1D array. If you want to change one pixel, for example, you need to now how to derive the index from ...
4
votes
3answers
5k views

How can I practice implementing data structures and algorithms? [duplicate]

I just finished learning the basics on data structures and algorithms using Python and C, completing some of the exercises from the book I read. I feel that it is necessary to implement some of these ...
4
votes
4answers
426 views

Building a route creator

Ok, already up front, I'm going to tell you, that this is a bonus task for Data structure course I'm taking. That should take care of all the questions whether or not this is for a homework. Route ...
4
votes
3answers
708 views

Data Oriented Design - impractical with more than 1-2 structure “members”?

The usual example of Data Oriented Design is with the Ball structure: struct Ball { float Radius; float XYZ[3]; }; and then they make some algorithm that iterates a std::vector<Ball> ...
4
votes
2answers
1k views

Why does Python use hash table to implement dict, but not Red-Black Tree? [closed]

Why does Python use hash table to implement dict, but not Red-Black Tree? What is the key? Performance?
4
votes
2answers
951 views

Which design better when use foreign key instead of a string to store a list of id

I'm building online examination system. I have designed to table, Question and GeneralExam. The table GeneralExam contains info about the exam like name, description, duration,... Now I would like to ...
4
votes
2answers
437 views

Data Structures: What are some common examples of problems where “buffers” come into action?

I was just wondering if there were some "standard" examples that everyone uses as a basis for explaining the nature of a problem that requires the use of a buffer. What are some well-known problems in ...
4
votes
3answers
766 views

Is it conceivable to have millions of lists of data in memory in Python?

I have over the last 30 days been developing a Python application that utilizes a MySQL database of information (specifically about Norwegian addresses) to perform address validation and correction. ...
4
votes
3answers
727 views

Is wikipedia a valuable resource for studying data structures? (can we call it complete?)

Can I depend on wikipedia to learn data structures fully using the list of data structures http://en.wikipedia.org/wiki/List_of_data_structures and the links they refer to? The same question for ...
4
votes
4answers
239 views

Useful Pre-requisites for Operating Systems course?

How important is knowledge of data structures and algorithms for an Operating Systems and Systems Programming course where we will be required to design and implement a kernel of a real-time operating ...
4
votes
1answer
180 views

Why define struct in variable?

Consider the following program: Many people when they want to use a struct, they create a new variable as: struct structureName variableName While it works when you just define it as: ...
4
votes
2answers
187 views

Decimal vs. Integer; Given a fixed range of values, which is preferable for accurate computation?

After getting into a "heated discussion" with someone, I figured I'd ask this question for the sake of posterity. I'm willing to be corrected if my assumption is incorrect but I'd like to hear a ...
4
votes
3answers
564 views

How Immutable Sets are Manipulated

When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead. I know ...
4
votes
2answers
980 views

Creating huge decision tree

I'm to write an event correlator. A fundamental part of the system will be a decision tree that recognizes the origin of the fault basing on recorded states and log files. Often many accidents will ...
4
votes
4answers
481 views

Appropriate data structure for deeply nested data?

I can't figure out what would be an appropriate data structure for my problem. I need to write a script that will count some events. Input is a list of events (thousands of them). Each event is an ...
4
votes
3answers
475 views

Is a tree with nodes that have reference to parent still a tree?

If we make reference to the parent for each node in a tree, do we still have a tree (by definition) anymore? Wikipedia definition is: In computer science, a tree is a widely used abstract data ...
4
votes
2answers
459 views

Fast set indexing data structure for superset retrieval

I am given a set of sets: {{a,b}, {a,b,c}, {a,c}, {a,c,f}} I would like to have a data structure to index those sets such that the following "lookup" is executed fast: find all supersets of a given ...
4
votes
3answers
224 views

Picking powers of two for sizes/limits

A habit I've observed among programmers, and a habit I sometimes subconsciously exhibit myself, is to pick powers of two (or powers of two minus one) when defining a database schema, a data buffer, ...
4
votes
1answer
2k views

Homogeneous vs. heterogeneous AST representation

What are the reasons to choose a homogeneous vs. a heterogeneous AST representation for implementing a complex domain-specific programming language? Just to be very clear about what I'm asking, here ...
4
votes
2answers
863 views

Patterns for a tree of persistent data with multiple storage options?

I have a real-world problem which I'll try to abstract into an illustrative example. So imagine I have data objects in a tree, where parent objects can access children, and children can access ...
4
votes
1answer
322 views

Website with test data files for specific algorithms/data structures?

Is there any website, like SPOJ and Project Euler, with the test data files available for specific algorithms/data structures? I know it's a fun challenge to solve those problems on your own, but I ...
4
votes
1answer
2k views

How good is The Algorithm Design Manual 2nd edition compared to the 1st edition for a beginner?

I'm looking for a good book on algorithms and data structures for a beginner. I was reading some reviews of these two books: Introduction to Algoritms and The Algorithm Design Manual. Introduction to ...
4
votes
2answers
146 views

Incremental and partial drawing of the large graphs

I have the following problem: I'm developing a software for data visualization where I get a graph structure and represent it in 3D space. So far, I've been using force-based algorithms to draw graphs ...
4
votes
2answers
91 views

Is it a good idea to have a database row that represents an unknown value within a system

We have two systems, System A imports a list of sports events from system B, it only wants to import the events we will be providing live data for, the filter used to hide events that we are not ...
4
votes
1answer
153 views

Using macros to implement a generic vector (dynamic array) in C. Is this a good idea?

So far I have only done personal projects at home. I hope to get involved in some open source project some time next year. The languages I that have been using the most are C and C++. I have used both ...
4
votes
1answer
224 views

Designing storage service data structure for decoupled models sharing same data

Surely most of you remember the Norton Commander application where similar (sometimes the same) data is displayed in separate decoupled views. I'm building a web application that follows the same ...
4
votes
1answer
690 views

Which data structure to represent a triangular undirected graph

I was wanting to create a graph similar to this in C++: The one I will implement will have more edges and vertices, but will use this triangle style. I was wanting to create something that I could ...
4
votes
0answers
166 views

How do personal assistants typically generate sentences?

This is sort of a follow up to this question about NLG research directions in the linguistics field. How do personal assistant tools such as Siri, Google Now, or Cortana perform Natural Language ...
3
votes
6answers
949 views

Why isn't the line count in Visual Studio zero-based?

This just struck me as an oversight by Microsoft. Since arrays and other data-structures within the .NET framework begin from zero (zero-based) why don't we have a line 0 within the code view in ...
3
votes
9answers
596 views

Standardized Data Structure Interface

I want to work with a variety of data structures (arrays, singly/doubly linked lists, sorted structures, etc.) on a plug-and-play basis. For example, I want to be able to easily swap in and out the ...
3
votes
5answers
1k views

How do you cope with Java coming from javascript?

I learned some javascript as my introduction into programming, and now I'm trying to move on to Java and Python and from there to c++, c#, etc. I've been caught up with schoolwork, so I've had very ...
3
votes
4answers
1k views

Why many programming languages have only 2 data-structures: arrays and hashes?

Many programming languages have only those 2 structures, and even some languages that have more structures still only provide special syntax for those 2; usually, [] and {}. Why is this? Is there ...
3
votes
3answers
507 views

Data structure: sort and search effectively

I need to have a data structure with say 4 keys . I can sort on any of these keys. What data structure can I opt for? Sorting time should be very little. I thought of a tree, but it will be only help ...
3
votes
2answers
3k views

What is the advantage of using map datastructure?

I am a member of some online coding website and today when I submitted a solution in C++, I saw there were much better answers in C++ itself which took considerably less time. The solutions were open ...
3
votes
2answers
822 views

Collection interfaces in C#, coming from Java

In Java, I'm used to declaring collections using the most-abstract interface possible and then constructing them using the concrete implementation that makes sense at the time. It usually looks ...
3
votes
6answers
530 views

Should a database table(s) structure match its intended data structure(s) in the logic?

This question branches out of this question, What are the differences between algorithms using data structures and algorithms using databases?. The General Question Should a database table(s) ...
3
votes
4answers
503 views

Search algorithm with co-ordinate (x,y) hints?

I am about to start writing a UI view where many small text items are laid-out over the view and when the user hovers over a text item a dynamically-generated image is displayed (like a tooltip), ...
3
votes
3answers
400 views

Separating Code into Smaller Files in C

I am in the process of cleaning up my code and making it easier to maintain. I am doing this by turning my 5000+ line file into separate smaller files. I have successfully created separate source and ...
3
votes
2answers
488 views

I have data that sends in “bursts” of 100 records with a significant delay. How do I structure my classes for multithreading?

My datasource sends information in 100 batches of 100 records with a delay of 1 to 3 seconds between batches. I would like to start processing data as soon as it's received, but I'm not sure how to ...