Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

0
votes
1answer
453 views

Priority queue for Kruskal's algorithm with running time O(E lg V)

I'm reviewing my notes on Kruskal's algorithm and have a question about getting the runtime down to O(E lg V). By using a PQ with edges and a boolean array of which vertices we have added to our tree ...
0
votes
1answer
390 views

What data structure is suitable for implementing dynamic huffman encoding and decoding on a piece of text?

Some pseudo code or resources will be appreciated.I was thinking if implementing it in form of a BST stored in an array. However,not all operations can be performed easily using this approach. I am ...
3
votes
3answers
296 views

Design pattern to dynamically create patterns found in a list of links

Can anyone help me think of a way to dynamically generate the patterns section of a tool I am creating? I'm not sure how to store and generate these "patterns" dynamically. What the program does is ...
3
votes
1answer
142 views

Algorithm/Strategy or Data structure to capture priorities sub priorities in an app

I am working on a CMS that is starting to evolve a bit. We started off with content that had the following priorities ( columnn in the db on the content table): HIGH MED LOW. Data was fetched by ...
1
vote
2answers
444 views

Shift bytes through a fixed sized buffer

I'm writing an InputStream that processes data streams containing "trailer" data. That is, the final n bytes of the stream is a piece of data that must be handled separately and shouldn't be returned ...
0
votes
1answer
432 views

Good Data Structure book to improve programming logic for children [closed]

My cousin (Age 10) who studies in Standard 5. He wants to learn programming and programming logic. Which book or resources should I refer? Is there any Good Data Structure book to improve ...
-1
votes
1answer
170 views

Storing User Form Data

I am trying to come up with a data schema/methodology for storing the data collected from various different user application forms. All forms contain the basic info like name, address, etc but then ...
-2
votes
2answers
276 views

About Artificial Intelligence [closed]

I am interested in starting a career in artificial intelligence. Can anyone suggest how I could prepare for this? What languages should I study that would be best for this career choice?
1
vote
1answer
146 views

Format text in a generic and reusable way

I would like to write some long text in some structure to allow a set of operations on that text. The question is which structure or format should I use, which suits best the use that I plan to do of ...
1
vote
3answers
1k views

The best way to store dictionary from file

I'm working on a translator in C++. Basically I want to parse the file with translations and store it in my program, so I can perform search through the words and simply access the corresponding word. ...
3
votes
2answers
220 views

Bayes filtering and data storage with expansive data sets

I am looking to write a Bayes filter that will act as an indicator of topic for a number of topics with a variable number of sources. Given a really big number of RSS feeds and here really big might ...
1
vote
1answer
372 views

Handling timeout in network application

How can I handle timeouts in a network application. I'm implementing a provisioning system on a Linux server, the code is huge so I'm going to put the algorithm, it works as like this Read ...
3
votes
4answers
1k views

Why many programming languages have only 2 data-structures: arrays and hashes?

Many programming languages have only those 2 structures, and even some languages that have more structures still only provide special syntax for those 2; usually, [] and {}. Why is this? Is there ...
3
votes
2answers
478 views

How should I structure a C# application that reads & writes binary data?

I have to read and write binary "chunks" of approximately 1Mb each. The data can come in the form of a stream or a in-memory byte[]. Normally I would use a Struct with a fixed layout, but there are ...
1
vote
3answers
378 views

What can Go chan do that a list cannot?

I want to know in which situation Go chan makes code much simpler than using list or queue or array that is usually available in all languages. As it was stated by Rob Pike in one of his speeches ...
4
votes
3answers
601 views

How Immutable Sets are Manipulated

When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead. I know ...
9
votes
1answer
467 views

Optimal Data Structure for our own API

I'm in the early stages of writing an Emacs major mode for the Stack Exchange network; if you use Emacs regularly, this will benefit you in the end. In order to minimize the number of calls made to ...
2
votes
4answers
353 views

Plugin Architecture: How to Handle Databases and Logging?

I am in the process of designing an architecture for a plugin based application and I have some questions about how to handle database access and logging. The goal of my application is to allow a ...
12
votes
5answers
540 views

Storing in-text metadata in a discrete data structure

I am developing an application which will need to store inline, intext metadata. What I mean by that is the following: let's say we have a long text, and we want to store some metadata connected with ...
3
votes
3answers
293 views

Is it worth to try write foolproof data structures?

The problem We need to store data in a table-like way, but we have very strict space constraints (~1Mb per table of 10k+ rows). We store data like this: ID | reviews | factor | score | interval | ...
5
votes
6answers
4k views

Why is the usage of string keys generally considered to be a bad idea?

This has been bugging me for a while. Most of the time, when it comes to storing data in structures such as hashtables, programmers, books and articles insist that indexing elements in said structures ...
1
vote
1answer
183 views

Storage and naming of log data

I'm in the process of rewriting a small application from console mode to a GUI mode. The input is a file with log data from a measuring device: ... 00029;00044;00076;00044;00021 ...
3
votes
1answer
310 views

How do I transparently cache data in intermediate stages of processing?

I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline. ckt = generate_ckt(ckt_properties); freq = generate_fpoints(fconfig); result = ...
2
votes
1answer
134 views

Applications with duplicate data - keep separate or combine into one set?

We currently have 3 similar, but slightly different applications that use the same data. We load the (same) data into each application that uses it. The applications are similar and use the same ...
6
votes
3answers
4k views

What is the best way to store a table in C++

I'm programming a decision tree in C++ using a slightly modified version of the C4.5 algorithm. Each node represents an attribute or a column of your data set and it has a children per possible value ...
0
votes
1answer
133 views

overworld vs on-screen 2d tile organization

I am developing a 2d game where the world is made up of tiles. I have an overworld and a "current world": the current world represent the current tiles on screen at any given time, currently this ...
1
vote
1answer
330 views

Database data structures for RESTful api

I am creating a RESTful API. I am struggling to decide on the best way to design my database tables around my resources. Initially, I though a table per resource would be a good way to go, but I'm ...
2
votes
2answers
3k views

Nested Enum type in C++ or C#?

I've come across a recurring issue in a few of my recent projects in which I find myself using enums to represent state, or type, or something else, and I need to check against a few conditions. Some ...
2
votes
1answer
298 views

Marked nodes in Fibonacci heaps

I don't understand why Fibonacci heaps have marked nodes (picture). A node is marked when its child is deleted. Quoting from Wikipedia: "[Keeping degree of each node low] is achieved by the rule that ...
2
votes
1answer
106 views

creating simple states for a stateless input

Given an input device (basically a keyboard) that reports keyup and keydown, how may I most efficiently store and retrieve information about which keys are currently depressed? My first thought was a ...
0
votes
2answers
236 views

Data structure for fast StartsWith filtering of a dictionary

I have a list of english words. I'm trying to come up with a data structure that allows me to do fast StartsWith filtering, like: var words = dictionary.StartsWith("foo"); I was thinking of ...
19
votes
2answers
2k views

What's the difference in content between Chris Okasaki's 1996 thesis and 1999 book, Purely Functional Data Structures?

I want to read Purely Functional Data Structures. I've easily found the thesis (which is freely available as a PDF), but see that there's a book available also. So I'd like to know what the ...
11
votes
7answers
16k views

What is the point of using lists over vectors, in C++?

I've run 3 different experiments involving C++ lists and vectors. Those with vectors proved more efficient, even when a lot of insertions in the middle were involved. Hence the question: in which ...
7
votes
2answers
2k views

Using a stream manipulator (endl) or a newline escape character (\n)?

I don't have a specific context in which I'm asking the question, but while I was reading a beginner book on C++ I noticed the use of both an endl stream manipulator and a newline escape character ...
0
votes
2answers
558 views

Algorithms for data structures in distributed system

The hash table data structure can be easily spread across multiple machines with a simple algorithm to distribute the keys: machine_to_query = item_key % machine_count When you want to read and ...
5
votes
2answers
959 views

How do scalable bloom filters work?

I was reading up on scalable bloom filters and could not understand how each time a constituent bloom filters fills up, a new bloom filter with larger size is added. The elements that contributed to ...
2
votes
2answers
6k views

How to increase the efficiency of an Immutable Queue in java?

I have been developing an Immutable queue in java, but the version I have developed is supposedly slow and I have been suggested to create a faster version of the same and I have no idea as to how to ...
2
votes
2answers
946 views

Thoughts on a custom data structure for a node/chain implementation

I am designing a simple GUI that will allow a user to create a GPX route by clicking repeatedly on a map panel. I am faced with a design dilemma for how to represent the nodes and route. This ...
5
votes
2answers
1k views

Creating huge decision tree

I'm to write an event correlator. A fundamental part of the system will be a decision tree that recognizes the origin of the fault basing on recorded states and log files. Often many accidents will ...
3
votes
6answers
567 views

Should a database table(s) structure match its intended data structure(s) in the logic?

This question branches out of this question, What are the differences between algorithms using data structures and algorithms using databases?. The General Question Should a database table(s) ...
6
votes
4answers
509 views

Representing a rule in a ruleset

How to represent rules for a rule engine as objects? A rule would be if (booleanExpression(input)) then a chain of generic actions" else next rule ...where the generic actions might be e.g. ...
7
votes
4answers
2k views

What are the differences between algorithms using data structures and algorithms using databases?

The General Question What are the differences between algorithms using data structures and algorithms using databases? Some Context This is a question that has been bugging me for some time, and I ...
1
vote
3answers
924 views

What data structure could a word processor use to map the user's caret position/text selection to its internal representation of the document?

Word processors (e.g. Microsoft Word) display documents to the user as styled text. The user can select a part of this text and apply styles to it, and edit the text. The word processor must (I ...
3
votes
1answer
120 views

Representing domain objects

This is related to my recent question regarding naming awkward domain objects. A number of answers indicated I was using the wrong representation for the domain objects. To summarize, I chose to use ...
0
votes
1answer
820 views

Simple dependency tree diagram generator [closed]

I have a need to produce a simple dependency tree diagram. The input data would be in the following simple format: ITEM_NAME DEPENDENCY ---------------------------- ITEM_101 ITEM_75 ITEM_102 ...
3
votes
2answers
3k views

How, when, and why do developers use custom data structures in python?

Python's lack of pointers makes data-structure construction intuitively more challenging, and has so much built in functionality that, as an amateur, I can't see when, why, or how you would create ...
20
votes
3answers
777 views

How do I express subtle relationships in my data?

"A" is related to "B" and "C". How do I show that "B" and "C" might, by this context, be related as well? Example: Here are a few headlines about a recent Broadway play: David Mamet's Glengarry ...
4
votes
3answers
845 views

Is it conceivable to have millions of lists of data in memory in Python?

I have over the last 30 days been developing a Python application that utilizes a MySQL database of information (specifically about Norwegian addresses) to perform address validation and correction. ...
17
votes
2answers
12k views

Sets Data Structure in Golang

I really like google golang but could some one explain what the rationale is for the implementors having left out a basic data structure such as sets from the standard library?
5
votes
3answers
361 views

Generating Deep Arrays: Shallow to Deep, Deep to Shallow or Bad idea?

I'm working on an array structure that will be used as the data source for a report template in a web app. The data comes from relatively complex SQL queries that return one or many rows as one ...