Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I was just having this idea once and wondering if there's any possibility to harness the power of p2p distributed computing to make a virtual supercomputer with the following features:

  • Works like a normal machine but uses networked RAM
  • Normal applications should be able to work on distributed environment dividing load among peers

This maybe unrealistic, just curious and want to know your views..

EDIT: Some interesting links I found while researching...

share|improve this question

migrated from stackoverflow.com Apr 25 '11 at 16:21

This question came from our site for professional and enthusiast programmers.

    
This is an interesting idea. Distributed computing only works for problems where computation is worth more than bandwidth/latency. There are many projects that do this already, but uniting them could be awesome. –  ProdigySim Apr 25 '11 at 17:04
    
@ProdigySim maybe you can share some links of similar projects? –  Sara Apr 25 '11 at 17:17
1  
GIMPS, SETI@Home, Folding@Home. See Also Volunteer Computing and Distributed Algorithm to find what makes these possible. Edit: Also this List of Distributed Projects. –  ProdigySim Apr 25 '11 at 17:22
add comment

3 Answers

Good question! Nice to see someone considering such points. Philosophising over computer application and usage is important. What you describe is a Distributed System.

If I were you, I would consider looking at the likes of SETI@Home project and other "screensaver processes" that use redundant CPU cycles to process large amounts of data. Chances are that those guys will have considered this kind of situation before.

The main issues are:

This is the question of "how can I ensure my request gets processed when the node I ask to process the request may fail - and how do I handle such inevitable failures?" (see Jeff Atwood's Coding Horror piece on the Chaos Monkey concept (in today for a possible method of investigation

Memory speed (or CAS) would be so variable that this would mean that any application would have to have enough memory available locally, at the place of use, that it could manage its tasks. In the same way that a Windows machine uses a Page File, so a distributed architecture could use specific nodes as memory caches

There are other considerations (such as security, usability, and yes bandwidth to some extent although you can always add more nodes, etc) but these will suffice as a starting point for you. Good luck with your research.

share|improve this answer
add comment

No. And yes.

People both can and have used this type of system as virtual super computers. With special software, lots of companies have used systems like this as well. However every program written for any kind of distributed cluster needs to be painfully aware of latency issues. And in this kind you need to be painfully aware of bandwidth limitations as well.

In short you're dealing with a very different beast than a regular computer, and you won't be able to get any performance unless you are aware of the differences and find ways to accommodate them for your purposes.

Distributed factoring problems, yes. Distributed fluid dynamics models? Not so much.

share|improve this answer
    
thanks for your quick response, in terms of the bandwidth issue, what about a LAN network? Can we make an old machine to work like a super powered one? (Networked RAM architecture: link –  Sara Apr 25 '11 at 16:54
    
@Sara, yes, LAN networks are a better programming platform than random machines on the internet. You'll need to install a cluster manager service (see en.wikipedia.org/wiki/Cluster_manager for more) and then develop for that platform. You'll likely need to use different programming techniques. I'd suggest starting down that path by learning about MapReduce. –  btilly Apr 25 '11 at 17:11
add comment

Well I see some highlighted points and I'd like to add some more major factors to be taken into account, when you are doing a distributed approach the concept of failsafe is the critical one with so many components !!

But looking on the bright side scalability can be an easier transition with this system in place, but If i were to put my money on it the "virtual supercomputer" would definitely be having some serious performance and latency issues but down the lane it might just become the norm of the Super computing !

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.