The tag has no usage guidance.

learn more… | top users | synonyms

19
votes
1answer
7k views

What are the solutions to the Distributed Queue problem?

I am trying to learn more about the various ways that the problem of a Distributed Queue may be solved. So I would like know what products, services, implementations and research papers that are ...
13
votes
8answers
1k views

What exactly is distributed computing? [closed]

So I'm a senior in college and a couple of weeks ago, I interviewed at Microsoft for an entry-level SDE position. While most of the interviews (out of the total 7) were on the general principles of ...
8
votes
1answer
1k views

Best practice for handling asynchronous inter communication?

Recently completed a project for handling credit card processing. One of the difficulties I faced was handling the delay / possible failure of notification messages. The most complex example was: an ...
7
votes
3answers
1k views

Distributed Issue Tracking [closed]

Distributed issue tracking seems like a belting idea to me, but it has never really taken off in a big way. Is there a good reason for that? I am aware of: bugz everywhere too complex to set up ...
7
votes
4answers
3k views

How do I get into a career in distributed systems? [closed]

I am enamored by reliable distributed systems and want to spend my career improving them rather than merely using them to write the same CRUD applications over and over. What experience will convince ...
7
votes
3answers
299 views

Optimistic work sharing on sparsely distributed systems

What would a system like BOINC look like if it were written today? At the time BOINC was written, databases were the primary choice for maintaining a shared state and concurrency among nodes. Since ...
7
votes
1answer
330 views

What logic is used to control Lock and Update actions in P2P networks? (similar to Bitcoin)

Bitcoin relies on Proof of Work (hashing) to control the writing to a central repository (the blockchain). It also relies on a set of rules for these updates, and defines what happens if a conflict ...
6
votes
4answers
713 views

How to effectively (but minimally) *simulate* a distributed computing environment?

I am doing some personal research on Byzantine Faults. I am trying out algorithms from various research papers which present algorithms for consensus in distributed systems like this one. For this i ...
5
votes
1answer
2k views

Persistent storage for Apache Mesos

Recently I've discovered such a thing as a Apache Mesos. It all looks amazingly in all that demos and examples. I could easily imagine how one would run for stateless jobs - that fits to the whole ...
5
votes
2answers
85 views

How do distributed applications verify that the data was processed correctly?

There are a lot of projects out there that rely on the public sphere to process data for them, but I do wonder how they ensure that every computer participating has processed the data properly. The ...
5
votes
1answer
499 views

Gossip protocol in a Ring topology

I am implementing a gossip based membership detection mechanism. Lets say we have a ring topology, i.e. node1 only knows about the 2 nodes (node4 and node2) around it. node1 ...
4
votes
3answers
146 views

How to store (and find) updated values in a Distributed Hash Table?

Distributed Hash Tables are a great way to store immutable data in a resilient way. However, I am wondering about how it would be possible to store new versions of old data in this system. Because ...
4
votes
3answers
1k views

Making a virtual supercomputer with distributed P2P computing

I was just having this idea once and wondering if there's any possibility to harness the power of p2p distributed computing to make a virtual supercomputer with the following features: Works like a ...
4
votes
1answer
102 views

Patterns for maintaining consistency in a distributed, event sourced system?

I've been reading on event sourcing lately and really like the ideas behind it but am stuck with the following problem. Let's say you have N concurrent processes which receive commands (e.g. web ...
4
votes
1answer
296 views

Distributed long-running jobs

Scenario Various types of files containing customer's data are being uploaded via FTP, and a POST is being made to a REST API to update a database table, which tells the system (a web app) which of ...
4
votes
1answer
306 views

How to implement efficient heterogeneous microservice data queries?

Our team has an idea of implementing a simple declarative DSL that would let users query the enterprise's domain model via a single interface without caring which specific microservices to call to get ...
4
votes
1answer
58 views

Is there any technology for dynamic assignment of execution context for programs that target many platforms with widely varying runtime performance? [closed]

This may very well be nothing new and probably exists, but I've never learned about it and would like to. The simplest way to be more specific is just to give an example. I was on the bus and ...
4
votes
2answers
1k views

How to implement a lightweight clustered architecture for a distributed application

I currently have a distributed application that runs on multiple embedded PCs. The whole application is composed of one master server and several nodes. Each node is an embedded PC that runs Windows 7 ...
4
votes
2answers
1k views

How does Elastic Search approach the problem of distributed membership and consensus?

I already understand that Elastic Search is supposed to be deployed in a distributed topology, in that you can have multiple nodes for a cluster of ES instances. I like the API, and it looks ...
4
votes
1answer
436 views

Use a custom value object or a Guid as an entity identifier in a distributed system?

tl;dr I've been told that in domain-driven design, an identifier for an entity could be a custom value object, i.e. something other than Guid, string, int, etc. Can this really be advisable in a ...
4
votes
3answers
2k views

Building a distributed system on Amazon Web Services

Would simply using AWS to build an application make this application a distributed system? For example if someone uses RDS for the database server, EC2 for the application itself and S3 for hosting ...
4
votes
1answer
2k views

Design patterns for multi-threaded messaging server

I'm designing an instant messaging server as a personal exercise to improve my understanding and application of multi-threading and design patterns in Java. I'm still designing, there's no code yet. ...
4
votes
0answers
69 views

Distributed Calculation of Geometric Distance between vectors

I am looking at a large scale low latency way to calculate the geometric distance between vectors. Let's say I have a vector A. It has size 128 and of type 32 bit float. I want to get it's geometric ...
4
votes
0answers
202 views

Open Grid Engine or Akka/Something more fault tolerant?

My use case is that I have a pipeline of independent, stand alone programs, that I want to execute in a certain order on specific pieces of data that are output from previous pipeline stages. The ...
3
votes
3answers
115 views

What is an accepted method of timestamping in a distributed system?

In a normal application, a simple call to time.Now() in most languages will suffice for getting a timestamp, but what about in a web application spread over multiple servers? As I understand it time....
3
votes
3answers
932 views

have we come full circle with microservices, back to very old school approaches?

In terms of software architecture and design, how do microservices "stack up" (pun intended) against middleware? I'm coming from Java, and it seems like as you move away from straight REST as an API, ...
3
votes
1answer
329 views

Single source of truth within an enterprise distributed system

Within an enterprise distributed system I have many services - an ecommerce service, a CRM, support desk, finance, billing. Many of these services share common data, such as Customer data. These ...
3
votes
1answer
612 views

Is it possible to modify Lamport's mutual exclusion algorithm to work without a FIFO guarantee?

I'm trying to implement a modified version of the Lamport Mutual Exclusion algorithm. The original algorithm assumes FIFO message ordering between connected systems, but I would like to use UDP a ...
3
votes
2answers
225 views

Open Source Cloud Computing

Recently I realized that the most popular social platforms on the net are in the hand of just a few companies, Google, Twitter, Facebook, to name just a few. Taking into account the world changing ...
3
votes
1answer
149 views

Is TCP still considered siginificant overhead compared to UDP? [closed]

I'm working on a distributed application for a graduate class I'm taking. There has been much discussion around implementing a way to maintain peer connection health statuses. Think Gossip. One of the ...
3
votes
1answer
99 views

Distributing work between TCP Clients

I have an application (run by a Windows Service) which connects to a TCP server (let's call the service 'Listener' from here on). [It's important to note that the TCP server is out of my reach, and ...
3
votes
1answer
78 views

Consistent snapshot

Suppose process A has a consitent snapshot of some entities. And it is constantly receiving updates for this enteties. Now client B connecting to A and it should receive the consistent snapshot of ...
3
votes
1answer
75 views

Coordinate access to resource between workers

I have a database accessed by an several servers, let's call them workers. These workers take one item from the database, do some operations on it and then update it back into the database. The ...
3
votes
2answers
194 views

Distributing a set of “Jobs” across multiple computers

I'm rather new to distributed computing and would like some assistance with the overall architecture of my application. My application has Jobs that can be added to a JobQueue. Then one or more ...
3
votes
1answer
249 views

Accepting the UUID collision risk based on number of clients

After reading some questions about the probability of UUID collisions it seems like collisions although unlikely, are still possible and a conflict solution is still needed. Therefore I am wondering ...
2
votes
1answer
335 views

What is eventual consistency?

I've heard people talking about "eventual consistency" (for example in this question and this answer). What is "eventual consistency"? How does it compare to "transactional consistency"? When does it ...
2
votes
2answers
147 views

Starting a Windows task from Linux, what are my options? [closed]

We have server that runs data processing tasks, using Python, Django and Celery, on Linux. Once a batch of data comes in, a number of Celery tasks are scheduled and they run, some of them in parallel, ...
2
votes
3answers
526 views

Can an issue tracking system be distributed?

I was thinking about issue tracking software like Redmine, Trac or even the one that is in Fossil and something hit me: Is there a reason why Redmine and Trac are not possible to be distributed? Or ...
2
votes
2answers
128 views

What does a load balancer return?

When a user hits the load balancer and the load balancer determines which web server to forward to, what happens next? Does the load balancer forward the request and all its data to the webserver, ...
2
votes
3answers
286 views

Promoting scalability in small business applications

I am part of a small software company that specializes in small to medium-sized business applications and my experience with distributed architectures/SOA is limited. Please let me know if I'm going ...
2
votes
6answers
184 views

Distributed/Network application development that is user focused but NOT web application development [closed]

I was curious, what other architectures exist for business or user focused development that aren't written using web applications. Are these architectures used today? If you are or were in the ...
2
votes
1answer
107 views

Message Passing: How to detect and/or handle “dead ends”

I'm playing with a mental model for distributed actor system that communicates using messages. In this, it is possible for an actor to create another actor. I've encountered one specific situation ...
2
votes
1answer
763 views

Client–server models and Client-centric models

For distributed applications, emphasis mine: Where classic software systems of the past century were mostly based on Client–server models and Client-centric application development, both ...
2
votes
1answer
123 views

OAuthv2 for distributed applications

Please note: Months ago I asked this question that received a truly amazing answer and introduction to OAuthv2. I’m now knee deep in the implementation of such a system and have similar (but notably ...
2
votes
1answer
124 views

Does map-reduce transmit the data and executable instructions?

When map-reduce divides a task and sends the data to each worker process, does it also transmit the instructions for how to operate on the data? For example, let's say Google has some huge array of ...
2
votes
1answer
238 views

Scaling websocket client connections (not server) to multiple servers

I wrote a Slack bot which must connect to Slack teams through websocket connections. Since the bot might be used by thousands of team, I will eventually need to distribute the teams across multiple ...
2
votes
2answers
68 views

Parallelising processing of users

Update I updated my question to reflect the fact I'm working with a database. I need to process user actions: The actions for each user change the user's balance which is then persisted to a ...
2
votes
1answer
64 views

Any reference doc /model/ implementation of enforcing events dependency/ordering in distributed computing?

For example, Process P is listening to events E(A) and E(B), but to process either of the events, P needs the other event available too. Naively, when P receives either of the events, it will check if ...
2
votes
0answers
77 views

How do I take data offline for a specific tenant, if their data is spread across multiple databases and multiple microservices?

I'm working on an application with about 50 or so microservices that have separate multi-tenanted databases. The problem arose of taking a tenant's data offline meaning that any requests for a certain ...
2
votes
0answers
75 views

Error handling in distributed system

This is the common sequence of two distributed components in our Java application: 1 A sends request to B 2 B starts some job J in parallel thread 3 B returns response to A 4 A accepts ...