Questions regarding efficient storing and representing data within a software application.

learn more… | top users | synonyms

1
vote
2answers
123 views

What design patterns exist for large dataset manipulation within MVC for a rich client data grid?

I want to feed an HTML5 data grid with large data sets that the client will want to manipulate in different ways, e.g. aggregation, grouping, pivoting. The data comes from different web services and ...
2
votes
1answer
281 views

Automatic denormalization for a NoSQL database application

We have a use case where we store table-like data, but we know about the schema of the data only at runtime. In our application, a power user defines a schema and normal user can create records and ...
0
votes
1answer
92 views

How can I avoid duplicating information in this scenario?

I have a Data class that can be placed inside a DataCollection class that can be placed inside a DataCollectionCollection class. Kinda like an egg can placed inside a box that can be placed inside a ...
0
votes
1answer
395 views

Designing online exam

I need to design an online exam server for an exam like GRE in which question difficulty increases if you answer correctly and decreases if you answer wrong. Questions are multiple choice ...
-1
votes
1answer
73 views

Data structure for storing sorted data

What's a good way to store loads of sorted string on a file so that inserts and reads are fast? I could store it up in a tree, and read/modify/store it as needed, but that seems wasteful for ...
4
votes
0answers
76 views

Modelling a complex schedule of work

I've got a real-world problem that I'm trying to represent and automate. I've simplified and abstracted it down to the following: There are n places of work (P1, P2, ..., Pn). Each place, Pn has a ...
2
votes
0answers
41 views

Cache-conscious data structure that does not require hashing

I am looking for a cache conscious data structure that does not require hashing. This is to avoid HashDoS without needing to use cryptographic PRFs such as SipHash, which are slow (~1 cycle/byte) &...
2
votes
0answers
27 views

Use dictionary value property as the dictionary key

I am creating an AST, and I am currently introducing the concept of "scope". By implementing a scope, wherever an identifier (variable name) is used, I am able to determine its original declaration. ...
2
votes
0answers
52 views

IRLBot Paper DRUM Implementation - Why keep key, value and auxiliary buckets seperate?

Repost from here as I think it may be more suited to this exchange. I'm trying to implement DRUM (Disk Repository with Update Management) as per the IRLBot paper (relevant pages start at 4) but as ...
2
votes
0answers
279 views

Most suitable data structure for in-memory string -> int conversion for variable length strings

My project involves validating and normalising email addresses in this format [userpart]@[domainpart].[tld] After syntactic validation of the address, [tld] is checked to exist, otherwise the ...
1
vote
0answers
82 views

How to structure enum data to achieve the desired effect

I'm making a framework for building simple html websites for an embedded system and I want to make it bulletproof in a way that a user can't make mistakes in building the html document. As I've ...
1
vote
0answers
24 views

Pagination with multiple groupings

I have a data model that looks like this: Group 1 --Sub group1 ----item1 ----item2 ----item3 --Sub group2 ----item1 ----item2 ----item3 Group 2 --Sub group1 ----item1 ----item2 ----item3 --Sub ...
1
vote
0answers
99 views

Search algorithm on sequential data model

For sequential data models like LinkedList, search algorithms like linear-search and binary-search are well known. Are their any other search algorithms apart from these two, that work on sequential ...
1
vote
0answers
210 views

Optimal data structure for binary data/Java

I have several input formats mostly restricted to geotiff and some proprietary binary formats. I import them into my program and then need to export them into various other data formats (also binary) ...
1
vote
0answers
51 views

Compound container for storing output from a parser

Consider a hierarchical structure like this directory [ [item_0;item_1;...;item_n] [item_0;item_1;inner_directory[...];item_m] other_directory [...] ] ...
1
vote
0answers
246 views

Single write, multiple read of stateful objects

Scenario: One data input feed (call this Source) Multiple objects (call these Layer1) read this Source, and analyze the feed. This analysis is stateful. Layer1 objects have configuration parameters. ...
0
votes
0answers
16 views

Randomly sampling chunks of a tree

I have a (non-binary) tree from which I want to sample various leaf and non-leaf nodes semi-randomly. The tree is generally less than 50 leaf nodes in all. The criteria for sampling is somewhat fuzzy ...
0
votes
0answers
39 views

Algorithm: Calculate average time in given time frame

I have a complex problem to solve. Problem Statement: I have a list of cities: Lets say New York, LA Las Vegas California The list of cities is limited, not more than 10. Now there are peoples ...
0
votes
0answers
11 views

How to segregate data between user groups in Django?

django.contrib.auth has User and Group models. I'm using the User model for my users, and the Group model to manage their broad permissions (access to admin site, access logs, etc). I also have my ...
0
votes
0answers
19 views

Joins between datasets in data models

I'm struggling to understand how to join data between data domains in application properly. Say, I have three domains, User, Post and Like: User - id - login - password - active - ... Post - id - ...
0
votes
0answers
45 views

Bounded Priority queue with constant time amortized delete min and insert

Given a bounded priority queue (with keys between 1 and k), is there any data structure with amortized constant time for delete min and insert and find min, where the worst case time for these ...
0
votes
0answers
90 views

Best datastructure to use

I am using Amazon's dynamodb nosql database and as a result I need to think of some ways to do certain operations to fit into their limits (e.g 4k item limit, can't combine small items into the same ...
0
votes
0answers
95 views

Storing a many-to-many relationship between aggregate roots in a document store

Intro I'm currently facing a rather difficult problem. An application that we need to expand and partially overhaul, uses RavenDB as its storage backend (a NoSQL document store). The two most ...
0
votes
0answers
82 views

What is the most efficient trie for comple multi-ligual data usage

Background Efficient Trie implementation for unicode strings I'm writing an app that will likely be run on systems that don't have much space or memory like my pi. Ideally my Trie would be efficient ...
0
votes
0answers
54 views

Final array after M updates on Segment Tree

I am stuck in a middle of a problem. I have made segment Tree of N elements and performed M updates through lazy propagation Now I need to store the final array (to some array B of size N) after all ...
0
votes
0answers
125 views

Enterprise Mashups. A good approach for .net

I'm in the middle of trying to talk our management into letting us do a portal that sits across the many, many systems that we use. I believe in using the best tool for the job so we have a good ERP, ...
0
votes
0answers
148 views

scrabble solving with maximum score

I was asked a question You are given a list of characters, a score associated with each character and a dictionary of valid words ( say normal English dictionary ). you have to form a word ...
0
votes
0answers
145 views

How to model dynamic table schema in .Net code

I am preparing to do an upgrade of a fairly large program that is currently written entirely in MS Access, both GUI and data (code and data in separate files). One of the key features of the product ...