Questions regarding efficient storing and representing data within a software application.
3
votes
4answers
168 views
What's the difference between a stream and a queue?
What's the difference between a stream and a queue? They both have the concept of an ordered set of elements, but tend to have different implementations and a different vocabulary of ...
3
votes
1answer
73 views
Building a Graph Editor - How to create a data driven graph
I am developing a graph-editor that uses drag and drop to build hierarchical graphs (containing nodes and links). Each node in the graph should be linked to a table in our database (SQL Server). I did ...
0
votes
6answers
619 views
Heap overflow vs stack overflow
So as a general rule to avoid a stack overflow, big objects should be allocated to the heap (correct me if I am wrong). But, since the heap and the stack expand towards each other, wouldn't this cause ...
0
votes
1answer
114 views
Priority queue for Kruskal's algorithm with running time O(E lg V)
I'm reviewing my notes on Kruskal's algorithm and have a question about getting the runtime down to O(E lg V). By using a PQ with edges and a boolean array of which vertices we have added to our tree ...
0
votes
1answer
73 views
How to model/structure data that contains parent/child relationships for the same entity?
I think it is just my thinking that is off here. Suppose I have two entities: question and choice. A question can be a parent or child, or both (in relation to other questions). It seems that this ...
1
vote
0answers
83 views
Single write, multiple read of stateful objects
Scenario:
One data input feed (call this Source)
Multiple objects (call these Layer1) read this Source, and analyze the feed. This analysis is stateful.
Layer1 objects have configuration parameters. ...
5
votes
6answers
265 views
Data structure for accessing units of measure
TL;DR - I'm looking for the "optimal" data structure to define units within a unit of measure.
A Unit of measure is essentially a value (or quantity) associated with a unit. SI Units have seven ...
10
votes
9answers
1k views
Card deck and sparse matrix interview questions
I just had a technical phone screen w/ a start-up. Here's the technical questions I was asked ... and my answers. What do think of these answers? Feel free to post better answers :-)
Question 1: ...
3
votes
2answers
102 views
Bayes filtering and data storage with expansive data sets
I am looking to write a Bayes filter that will act as an indicator of topic for a number of topics with a variable number of sources. Given a really big number of RSS feeds and here really big might ...
0
votes
0answers
74 views
What data structure is suitable for implementing dynamic huffman encoding and decoding on a piece of text?
Some pseudo code or resources will be appreciated.I was thinking if implementing it in form of a BST stored in an array. However,not all operations can be performed easily using this approach. I am ...
2
votes
4answers
182 views
Plugin Architecture: How to Handle Databases and Logging?
I am in the process of designing an architecture for a plugin based application and I have some questions about how to handle database access and logging. The goal of my application is to allow a ...
7
votes
1answer
255 views
Optimal Data Structure for our own API
I'm in the early stages of writing an Emacs major mode for the Stack Exchange network; if you use Emacs regularly, this will benefit you in the end.
In order to minimize the number of calls made to ...
3
votes
1answer
265 views
Patterns for a tree of persistent data with multiple storage options?
I have a real-world problem which I'll try to abstract into an illustrative example.
So imagine I have data objects in a tree, where parent objects can access children, and children can access ...
0
votes
0answers
85 views
HDF vs NOSQl solutions
Hi I get a look in some scientific data stored in HDF5 format and I was wondering if there is any NoSQl solution that will reach the same performance in read/write as HDF5 .
The data of my example ...
3
votes
3answers
164 views
Design pattern to dynamically create patterns found in a list of links
Can anyone help me think of a way to dynamically generate the patterns section of a tool I am creating? I'm not sure how to store and generate these "patterns" dynamically.
What the program does is ...
0
votes
0answers
11 views
Design pattern to dynamically create patterns found in a list of links [duplicate]
Can anyone help me think of a way to dynamically generate the patterns section of a tool I am creating? I'm not sure how to store and generate these "patters" dynamically.
What the program does is ...
2
votes
1answer
65 views
Algorithm/Strategy or Data structure to capture priorities sub priorities in an app
I am working on a CMS that is starting to evolve a bit.
We started off with content that had the following priorities ( columnn in the db on the content table): HIGH MED LOW. Data was fetched by ...
1
vote
2answers
103 views
Shift bytes through a fixed sized buffer
I'm writing an InputStream that processes data streams containing "trailer" data. That is, the final n bytes of the stream is a piece of data that must be handled separately and shouldn't be returned ...
4
votes
2answers
454 views
Efficient Data Structure for holding a graph
Problem Link - http://opc.iarcs.org.in/index.php/problems/NUMTRIPLE
In my opinion, the problem can be solved by a data structure, that shows how each number is connected to another, and via recursion ...
0
votes
0answers
31 views
Data-structures and memory allocation [closed]
I have a structure:
struct student
{
int id_number;
char first_name[30];
char gender;
};
When I try to enter the gender field it does not work. Skip over this step. Why does that ...
0
votes
1answer
119 views
Good Data Structure book to improve programming logic for children [closed]
My cousin (Age 10) who studies in Standard 5. He wants to learn programming and programming logic.
Which book or resources should I refer? Is there any Good Data Structure book to improve ...
5
votes
3answers
3k views
Abstract Data Type and Data Structure [duplicate]
It's quite difficult for me to understand these terms. I searched on google and read a little on Wikipedia but I'm still not sure. I've determined so far that:
Abstract Data Type is a definition of ...
6
votes
2answers
744 views
Will B-Trees and Other Data Structures Become Obsolete With The Advent of Solid State Drives?
Many (perhaps most?) database applications today use B-Trees and variations to store data, because this data structure optimizes the read, write and seek operations on a hard disk (and these ...
28
votes
7answers
2k views
When to use DAG (Directed Acyclic Graph) in programming?
I recently found a framework named ecto.
In this framework, a basic component named "plasm", which is the ecto Directed Acyclic Graph.In ecto, plasm can be operated by ecto scheduler.
I am ...
6
votes
2answers
236 views
Min cost concatenation alignment
I am doing a project on data mining.
I have a very large string of size, say n and many small strings, l_1,l_2.. each of size at most m. I want to find the sequence of small strings such that their ...
21
votes
12answers
4k views
Do I need to understand algorithms and data structures to be called a programmer? [closed]
It has been six years since I have been coding. Coding into all kinds of things like ActionScript, JavaScript, Java, PHP, Ajax, XML HTML, ASP, etc. I have used arrays, maps, linked lists, sets, etc ...
38
votes
11answers
15k views
How do I learn algorithms and data structures? [closed]
This is in continuation to my previous question where I asked is it necessary to learn algorithms and data structures. I feel yes it is.
Now I work in an environment where I won't ever get the chance ...
-1
votes
1answer
62 views
Storing User Form Data
I am trying to come up with a data schema/methodology for storing the data collected from various different user application forms.
All forms contain the basic info like name, address, etc but then ...
-2
votes
2answers
177 views
About Artificial Intelligence [closed]
I am interested in starting a career in artificial intelligence.
Can anyone suggest how I could prepare for this?
What languages should I study that would be best for this career choice?
1
vote
3answers
398 views
The best way to store dictionary from file
I'm working on a translator in C++. Basically I want to parse the file with translations and store it in my program, so I can perform search through the words and simply access the corresponding word. ...
1
vote
1answer
87 views
Format text in a generic and reusable way
I would like to write some long text in some structure to allow a set of operations on that text. The question is which structure or format should I use, which suits best the use that I plan to do of ...
1
vote
0answers
88 views
Handling timeout in network application
How can I handle timeouts in a network application.
I'm implementing a provisioning system on a Linux server, the code is huge so I'm going to put the algorithm, it works as like this
Read ...
2
votes
1answer
84 views
creating simple states for a stateless input
Given an input device (basically a keyboard) that reports keyup and keydown, how may I most efficiently store and retrieve information about which keys are currently depressed? My first thought was a ...
1
vote
4answers
619 views
Why many programming languages have only 2 data-structures: arrays and hashes?
Many programming languages have only those 2 structures, and even some languages that have more structures still only provide special syntax for those 2; usually, [] and {}. Why is this? Is there ...
5
votes
2answers
486 views
What is the Bible of Hashing?
Is there a Cormen-like reference on Hashes and Hashing? This particular structure has seen little attention in my CS education for some reason but I'd like to learn more as they seem to be everywhere. ...
3
votes
2answers
193 views
How should I structure a C# application that reads & writes binary data?
I have to read and write binary "chunks" of approximately 1Mb each. The data can come in the form of a stream or a in-memory byte[].
Normally I would use a Struct with a fixed layout, but there are ...
4
votes
3answers
217 views
Picking powers of two for sizes/limits
A habit I've observed among programmers, and a habit I sometimes subconsciously exhibit myself, is to pick powers of two (or powers of two minus one) when defining a database schema, a data buffer, ...
0
votes
1answer
157 views
What can Go chan do that a list cannot?
I want to know in which situation Go chan makes code much simpler than using list or queue or array that is usually available in all languages.
As it was stated by Rob Pike in one of his speeches ...
4
votes
3answers
218 views
How Immutable Sets are Manipulated
When working with an immutable set or map, like the ones found in many functional programming languages, operations that would otherwise modify the container generate a new container instead.
I know ...
12
votes
5answers
300 views
Storing in-text metadata in a discrete data structure
I am developing an application which will need to store inline, intext metadata. What I mean by that is the following: let's say we have a long text, and we want to store some metadata connected with ...
6
votes
3answers
2k views
EAV - is it really bad in all scenarios?
I'm thinking to use EAV for some of the stuff in one of the projects, but all questions about it in stackoverflow end up to answers calling EAV an antipattern.
But I'm wondering, if is it that wrong ...
5
votes
1answer
370 views
How do scalable bloom filters work?
I was reading up on scalable bloom filters and could not understand how each time a constituent bloom filters fills up, a new bloom filter with larger size is added.
The elements that contributed to ...
6
votes
5answers
846 views
What is the best way to store a table in C++
I'm programming a decision tree in C++ using a slightly modified version of the C4.5 algorithm. Each node represents an attribute or a column of your data set and it has a children per possible value ...
4
votes
5answers
448 views
Why is the usage of string keys generally considered to be a bad idea?
This has been bugging me for a while. Most of the time, when it comes to storing data in structures such as hashtables, programmers, books and articles insist that indexing elements in said structures ...
3
votes
3answers
212 views
Is it worth to try write foolproof data structures?
The problem
We need to store data in a table-like way, but we have very strict space constraints (~1Mb per table of 10k+ rows). We store data like this:
ID | reviews | factor | score | interval | ...
11
votes
2answers
1k views
Pseudocode for Brodal queue
I'm trying to find more resources regarding Brodal heap. All I found is a haskell implementation of Brodal-Okasaki heap, but I think that they are skew heaps, is this correct? Furthermore, I'm ...
2
votes
1answer
92 views
Applications with duplicate data - keep separate or combine into one set?
We currently have 3 similar, but slightly different applications that use the same data. We load the (same) data into each application that uses it. The applications are similar and use the same ...
1
vote
1answer
102 views
Storage and naming of log data
I'm in the process of rewriting a small application from console mode to a GUI mode.
The input is a file with log data from a measuring device:
...
00029;00044;00076;00044;00021
...
1
vote
2answers
409 views
Nested Enum type in C++ or C#?
I've come across a recurring issue in a few of my recent projects in which I find myself using enums to represent state, or type, or something else, and I need to check against a few conditions. Some ...
2
votes
1answer
77 views
How do I transparently cache data in intermediate stages of processing?
I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline.
ckt = generate_ckt(ckt_properties);
freq = generate_fpoints(fconfig);
result = ...
