Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

1
vote
0answers
177 views

Single write, multiple read of stateful objects

Scenario: One data input feed (call this Source) Multiple objects (call these Layer1) read this Source, and analyze the feed. This analysis is stateful. Layer1 objects have configuration parameters. ...
7
votes
9answers
2k views

Card deck and sparse matrix interview questions [closed]

I just had a technical phone screen w/ a start-up. Here's the technical questions I was asked ... and my answers. What do think of these answers? Feel free to post better answers :-) Question 1: ...
3
votes
2answers
176 views

Bayes filtering and data storage with expansive data sets

I am looking to write a Bayes filter that will act as an indicator of topic for a number of topics with a variable number of sources. Given a really big number of RSS feeds and here really big might ...
2
votes
4answers
297 views

Plugin Architecture: How to Handle Databases and Logging?

I am in the process of designing an architecture for a plugin based application and I have some questions about how to handle database access and logging. The goal of my application is to allow a ...
3
votes
3answers
266 views

Design pattern to dynamically create patterns found in a list of links

Can anyone help me think of a way to dynamically generate the patterns section of a tool I am creating? I'm not sure how to store and generate these "patterns" dynamically. What the program does is ...
3
votes
1answer
125 views

Algorithm/Strategy or Data structure to capture priorities sub priorities in an app

I am working on a CMS that is starting to evolve a bit. We started off with content that had the following priorities ( columnn in the db on the content table): HIGH MED LOW. Data was fetched by ...
1
vote
2answers
341 views

Shift bytes through a fixed sized buffer

I'm writing an InputStream that processes data streams containing "trailer" data. That is, the final n bytes of the stream is a piece of data that must be handled separately and shouldn't be returned ...
5
votes
2answers
728 views

Efficient Data Structure for holding a graph

Problem Link - http://opc.iarcs.org.in/index.php/problems/NUMTRIPLE In my opinion, the problem can be solved by a data structure, that shows how each number is connected to another, and via recursion ...
0
votes
1answer
289 views

Good Data Structure book to improve programming logic for children [closed]

My cousin (Age 10) who studies in Standard 5. He wants to learn programming and programming logic. Which book or resources should I refer? Is there any Good Data Structure book to improve ...
7
votes
3answers
13k views

Abstract Data Type and Data Structure [duplicate]

It's quite difficult for me to understand these terms. I searched on google and read a little on Wikipedia but I'm still not sure. I've determined so far that: Abstract Data Type is a definition of ...
9
votes
2answers
2k views

Will B-Trees and Other Data Structures Become Obsolete With The Advent of Solid State Drives?

Many (perhaps most?) database applications today use B-Trees and variations to store data, because this data structure optimizes the read, write and seek operations on a hard disk (and these ...
30
votes
7answers
6k views

When to use DAG (Directed Acyclic Graph) in programming?

I recently found a framework named ecto. In this framework, a basic component named "plasm", which is the ecto Directed Acyclic Graph.In ecto, plasm can be operated by ecto scheduler. I am ...
24
votes
12answers
7k views

Do I need to understand algorithms and data structures to be called a programmer? [closed]

It has been six years since I have been coding. Coding into all kinds of things like ActionScript, JavaScript, Java, PHP, Ajax, XML HTML, ASP, etc. I have used arrays, maps, linked lists, sets, etc ...
-1
votes
1answer
162 views

Storing User Form Data

I am trying to come up with a data schema/methodology for storing the data collected from various different user application forms. All forms contain the basic info like name, address, etc but then ...
-2
votes
2answers
255 views

About Artificial Intelligence [closed]

I am interested in starting a career in artificial intelligence. Can anyone suggest how I could prepare for this? What languages should I study that would be best for this career choice?
1
vote
3answers
1k views

The best way to store dictionary from file

I'm working on a translator in C++. Basically I want to parse the file with translations and store it in my program, so I can perform search through the words and simply access the corresponding word. ...
1
vote
1answer
132 views

Format text in a generic and reusable way

I would like to write some long text in some structure to allow a set of operations on that text. The question is which structure or format should I use, which suits best the use that I plan to do of ...
2
votes
1answer
102 views

creating simple states for a stateless input

Given an input device (basically a keyboard) that reports keyup and keydown, how may I most efficiently store and retrieve information about which keys are currently depressed? My first thought was a ...
3
votes
4answers
1k views

Why many programming languages have only 2 data-structures: arrays and hashes?

Many programming languages have only those 2 structures, and even some languages that have more structures still only provide special syntax for those 2; usually, [] and {}. Why is this? Is there ...
6
votes
2answers
685 views

What is the Bible of Hashing?

Is there a Cormen-like reference on Hashes and Hashing? This particular structure has seen little attention in my CS education for some reason but I'd like to learn more as they seem to be everywhere. ...
3
votes
2answers
405 views

How should I structure a C# application that reads & writes binary data?

I have to read and write binary "chunks" of approximately 1Mb each. The data can come in the form of a stream or a in-memory byte[]. Normally I would use a Struct with a fixed layout, but there are ...
4
votes
3answers
220 views

Picking powers of two for sizes/limits

A habit I've observed among programmers, and a habit I sometimes subconsciously exhibit myself, is to pick powers of two (or powers of two minus one) when defining a database schema, a data buffer, ...
4
votes
3answers
436 views

How Immutable Sets are Manipulated

When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead. I know ...
12
votes
5answers
481 views

Storing in-text metadata in a discrete data structure

I am developing an application which will need to store inline, intext metadata. What I mean by that is the following: let's say we have a long text, and we want to store some metadata connected with ...
5
votes
1answer
756 views

How do scalable bloom filters work?

I was reading up on scalable bloom filters and could not understand how each time a constituent bloom filters fills up, a new bloom filter with larger size is added. The elements that contributed to ...
3
votes
3answers
274 views

Is it worth to try write foolproof data structures?

The problem We need to store data in a table-like way, but we have very strict space constraints (~1Mb per table of 10k+ rows). We store data like this: ID | reviews | factor | score | interval | ...
12
votes
2answers
2k views

Pseudocode for Brodal queue

I'm trying to find more resources regarding Brodal heap. All I found is a haskell implementation of Brodal-Okasaki heap, but I think that they are skew heaps, is this correct? Furthermore, I'm ...
2
votes
1answer
129 views

Applications with duplicate data - keep separate or combine into one set?

We currently have 3 similar, but slightly different applications that use the same data. We load the (same) data into each application that uses it. The applications are similar and use the same ...
1
vote
1answer
162 views

Storage and naming of log data

I'm in the process of rewriting a small application from console mode to a GUI mode. The input is a file with log data from a measuring device: ... 00029;00044;00076;00044;00021 ...
2
votes
2answers
2k views

Nested Enum type in C++ or C#?

I've come across a recurring issue in a few of my recent projects in which I find myself using enums to represent state, or type, or something else, and I need to check against a few conditions. Some ...
2
votes
5answers
3k views

An XML file or Database?

I am re-writing a section of my site and am trying to decide how much of a rewrite this will be. At the moment I have a web service feed that generates an xml once per day. I then use this xml file on ...
0
votes
1answer
120 views

overworld vs on-screen 2d tile organization

I am developing a 2d game where the world is made up of tiles. I have an overworld and a "current world": the current world represent the current tiles on screen at any given time, currently this ...
1
vote
1answer
237 views

Database data structures for RESTful api

I am creating a RESTful API. I am struggling to decide on the best way to design my database tables around my resources. Initially, I though a table per resource would be a good way to go, but I'm ...
1
vote
1answer
215 views

Marked nodes in Fibonacci heaps

I don't understand why Fibonacci heaps have marked nodes (picture). A node is marked when its child is deleted. Quoting from Wikipedia: "[Keeping degree of each node low] is achieved by the rule that ...
7
votes
4answers
7k views

What is the point of using lists over vectors, in C++?

I've run 3 different experiments involving C++ lists and vectors. Those with vectors proved more efficient, even when a lot of insertions in the middle were involved. Hence the question: in which ...
7
votes
6answers
3k views

Which self balancing binary tree would you recommend?

I'm learning Haskell and as an exercise I'm making binary trees. Having made a regular binary tree, I want to adapt it to be self balancing. So: Which is most efficient? Which is easiest to ...
0
votes
2answers
193 views

Data structure for fast StartsWith filtering of a dictionary

I have a list of english words. I'm trying to come up with a data structure that allows me to do fast StartsWith filtering, like: var words = dictionary.StartsWith("foo"); I was thinking of ...
16
votes
2answers
1k views

What's the difference in content between Chris Okasaki's 1996 thesis and 1999 book, Purely Functional Data Structures?

I want to read Purely Functional Data Structures. I've easily found the thesis (which is freely available as a PDF), but see that there's a book available also. So I'd like to know what the ...
7
votes
2answers
1k views

Using a stream manipulator (endl) or a newline escape character (\n)?

I don't have a specific context in which I'm asking the question, but while I was reading a beginner book on C++ I noticed the use of both an endl stream manipulator and a newline escape character ...
0
votes
2answers
453 views

Algorithms for data structures in distributed system

The hash table data structure can be easily spread across multiple machines with a simple algorithm to distribute the keys: machine_to_query = item_key % machine_count When you want to read and ...
3
votes
2answers
345 views

I have data that sends in “bursts” of 100 records with a significant delay. How do I structure my classes for multithreading?

My datasource sends information in 100 batches of 100 records with a delay of 1 to 3 seconds between batches. I would like to start processing data as soon as it's received, but I'm not sure how to ...
5
votes
3answers
330 views

Generating Deep Arrays: Shallow to Deep, Deep to Shallow or Bad idea?

I'm working on an array structure that will be used as the data source for a report template in a web app. The data comes from relatively complex SQL queries that return one or many rows as one ...
1
vote
2answers
3k views

How to increase the efficiency of an Immutable Queue in java?

I have been developing an Immutable queue in java, but the version I have developed is supposedly slow and I have been suggested to create a faster version of the same and I have no idea as to how to ...
2
votes
2answers
636 views

Thoughts on a custom data structure for a node/chain implementation

I am designing a simple GUI that will allow a user to create a GPX route by clicking repeatedly on a map panel. I am faced with a design dilemma for how to represent the nodes and route. This ...
4
votes
2answers
726 views

Creating huge decision tree

I'm to write an event correlator. A fundamental part of the system will be a decision tree that recognizes the origin of the fault basing on recorded states and log files. Often many accidents will ...
15
votes
4answers
7k views

Difference between hash and dictionary

What is the difference between Hash and Dictionary. Coming from a scripting background, I feel that they are similar , but wanted to find out the exact differences, Googling did not help me much.
3
votes
6answers
448 views

Should a database table(s) structure match its intended data structure(s) in the logic?

This question branches out of this question, What are the differences between algorithms using data structures and algorithms using databases?. The General Question Should a database table(s) ...
6
votes
4answers
318 views

Representing a rule in a ruleset

How to represent rules for a rule engine as objects? A rule would be if (booleanExpression(input)) then a chain of generic actions" else next rule ...where the generic actions might be e.g. ...
7
votes
4answers
2k views

What are the differences between algorithms using data structures and algorithms using databases?

The General Question What are the differences between algorithms using data structures and algorithms using databases? Some Context This is a question that has been bugging me for some time, and I ...
1
vote
3answers
701 views

What data structure could a word processor use to map the user's caret position/text selection to its internal representation of the document?

Word processors (e.g. Microsoft Word) display documents to the user as styled text. The user can select a part of this text and apply styles to it, and edit the text. The word processor must (I ...