Hot answers tagged

13

Just because source control + distributed was a huge success, issue tracking + distributed isn't necessarily a good idea. What do we gain from distributed source control and would it apply to issue tracking? Easy branching and merging: not really. Actually it is crucial that everybody is on the same page. So branching would be highly undesirable. ...


11

Writing a basic queuing system is fairly simple, but as you noted above with all of the challenges, doing it right is another matter. I've used home grown systems for which I wrote the source code, 3rd party systems, and various JMS providers. JMS (Java Messaging Service) by far is the most complete solution I've encountered thus far. Much of what you ask ...


11

When building distributed systems, the difference between a 'synchronous' system and an 'asynchronous' system is this: A synchronous system has known upper bounds on computation and message delivery times. So: you have an asynchronous system where certain events do not have these known upper bounds. How do you handle it? If these asynchronous processes ...


9

It's simply to completely separate failure of each different tab and plugin. When one fail, whatever the way it fails, only this one will crash. The monitoring process will just report that it failed and will be able to reload it. C++ have shared memory by default and an exception system which makes easy to crash the whole application if one part isn't ...


8

Virtual machines may be the way to go, the problem is that if you setup 5 VM's on one server the response time between them will be effectively 0, which may not be true in your real environment where there could be a delay between servers. If that is an issue (or could be) I would suggest setting up some servers on AWS or the like to test. Chances are that ...


7

You have two distinct but related challenges: learning about "reliable distributed systems" (eventually you will discover that is an incomplete specification within a large and complex field); and getting a job in that field at some point in your education. Two activities are crucial to the education challenge: becoming familiar with the pertinent ...


7

The usual practice is to set the servers to all keep their time updated using NTP. There are limitations to the accuracy when using NTP time syncing which means that you should only rely on the time stamp to give a general idea of when events occurred, which is likely to be good enough for identifying the set of events you are interested in. Timestamps are ...


6

Having gone through this experience, I am led to wonder as to what exactly constitutes distributed computing? Distributed computing is an inherently parallel collection of processing elements that communicate with one another to tackle one or more problems. Those processing elements are sufficiently separated from each other that it is not practical to ...


6

Distributed computing is a computing system that has processing occurring on different computers. The individual programs communicate with each other through a series of communication channels. These channels are usually network connections (TCP sockets, for example), but often use other communication protocols and devices (such as DeviceNET, BACNet, SECS-2, ...


6

Do multiple parallel threads trying to synchronize for access to a resource constitute a problem in the domain of distributed computing? They do if those threads could be running on different machines, or even if they're running on the same machine but in different processes.


6

Both Redmine and TRAC are typical web based CRUD applications, architecturally wise, there isn't a reason why they can't be distributed. The simplest way would be to have them use a distributed database, and since both support MySQL the obvious solution would be MySQL Cluster. Then of course you could save yourself any hassle and run them on a cloud, for ...


6

Short answer: Apache Mesos doesn't provide distributed FS. So, apps have to work with local FS on slaves or you may run any distributed FS alongside Mesos. Mesos is typically deployed together with HDFS, and most of the frameworks that run on top of Mesos can work with HDFS (Hadoop, Spark, Storm, etc.) And in case your app doesn't support any distributed ...


6

What is "eventual consistency"? How does it compare to "transactional consistency"? When does it happen? Consistency models describe how a system (nominally a distributed system) responds to change. In an eventually-consistent system, all nodes will eventually have a consistent view of the overall system state. However, there will be a period of time ...


6

Once you decide that clients cannot be trusted, then it is a given fact that each value is immutable, and each key is immutable. And immutability is forever. This in turn means that this is not a problem of storing and finding values, it is a problem of identifying the key of the most updated version of a value, so it is a versioning problem. When trying ...


5

Order is the least of your problems. Not only does UDP not guarantee delivery order, it doesn't guarantee delivery at all. Lamport's algorithm requires cooperation of all non-requesting processes that would contend for a resource in the form of replies. The loss of a datagram carrying a request would cause the other processes not to send replies. That or ...


5

The most architecturally sound approach I know of is to put that single source of truth behind a microservice. It is perfectly okay for multiple parts of the system to update that data, as long as they do it through something like a microservice that can ensure it's always done correctly and predictably. So for instance, Customer data is probably already in ...


5

Utilizing a webserver for this purpose is actually a standard approach, it is just a simple form of Service Oriented Architecture. Of course, this term might pretend more than there is actually behind it. To keep this lightweight, without the need using a fullblown Webserver, you can use a tool like node.js. It is the most simple solution I can think of for ...


4

I just skipped through the paper that you mention, but here I go. We often use a virtualised environment to create experimental setups that involve multiple computers. Using Hyper-V on Windows Server (or similar software) you can easily create a private, isolated network of machines that you can configure on the fly and reuse as needed. You seem to be ...


4

We use this: http://lxc.sourceforge.net/ Linux Containers are extremely light weight virtual machines (basically zero overhead). We are using only 250 machines per physical machine, but there shouldn't be any issue going for thousands per machine. We successfully used both LVM partitions and overlay filesystems as the system root.


4

There are several such projects. The most publicized system (but certainly not the first) is Diaspora, which is a social network made of many individually-operated servers, called "pods". Pods can be freely set up using the AGPL-licensed source code. An individual user can set up a personal pod, or can join a public pod. Regardless of what pod your account ...


4

Every software development technique we've ever invented has been about managing complexity somehow. A huge portion of them have been and continue to be about abstraction, encapsulation and loose coupling. Microservices are yet another way of doing those things, which is probably why it resembles a lot of older techniques at a high theoretical level, but ...


3

Strictly speaking "distributed computing" is any solution that involves processing a single transaction/request/calculation on more than one computer. You will also come across the term "Distributed Systems" which is a catch all term for windows, unix and other small systems servers which would have originally deployed outside the central data center. ...


3

To answer your general question about what constitutes distributed computing, I would recommend the paper A Note on Distributed Computing by Ann Wollrath, Geoff Wyant, Jim Waldo and Samuel C. Kendall. It covers the recent history of distributed systems and its failures, and it proposes that distributed computing requires thinking differently about the ...


3

Short of virtual machines -- which are probably the way to go -- most client server software lets the same machine behave as client and server. So you might need a distributed pardagim on a single PC.


3

What you are looking for is a niche job. The only way to get one of those is like any other really: apply for a job that fits the description. Of course since you have little experience in the design and building of such systems you'd have to either run for an entry level job, or otherwise leverage your experience if any of it could be of interest to that ...


3

The Wikipedia article doesn't cite sources for that phrase, and I couldn't find a reference to "client-centric application" in my distributed computing book, however I believe I can try to infer something. To me, that sentence is describing two prevalent paradigms in computing - the client-server model, and a non-distributed client-side application. This is ...


3

It depends on your architecture. On UNIX systems, tools like puppet are really usefull : http://www.puppetlabs.com/ . You have also deployement tools on windows systems throw groups management. Then, you'll also need a monitoring tool. I can't be really precise, as long as you don't give many information to build on.


3

Good question! Nice to see someone considering such points. Philosophising over computer application and usage is important. What you describe is a Distributed System. If I were you, I would consider looking at the likes of SETI@Home project and other "screensaver processes" that use redundant CPU cycles to process large amounts of data. Chances are that ...


3

Make sure the Windows Service is loosely coupled to the instance of the DB server. This will allow you to move to a N:1 ratio of Windows Service : DB server. There's a whole host of techniques that can be used to make your DB server more robust, but that's not really what you're getting at in your Q. Isolate the following information: What data is ...


3

I don't think being decentralized is as important as having off-line capabilities. Integration with source control is a big benefit, so you want each user to be able to conveniently handle both tasks. The closer together they do it the more continuity you'll have. Even the most distributed teams should be able to put together a web server and tracking ...



Only top voted, non community-wiki answers of a minimum length are eligible