In a multiple processor computer, an important issue is: How do processors coordinate to solve a problem? Processors must have the ability to communicate with each other in order to cooperatively complete a task. There are two general approaches to address this problem.
One option uses a single address space. Systems based on this concept, otherwise known as shared-memory systems, allow processor communication through variables stored in a shared address space.
The other alternative employs a scheme by which each processor has its own memory module. Such a distributed-memory system (cluster) is constructed by connecting each component with a high-speed communications network. Processors communicate to each other over the network.
The architectural differences between shared-memory systems and distributed-memory systems have implications on how each is programmed. With a shared-memory multiprocessor, different processors can access the same variables. This makes referencing data stored in memory similar to traditional single-processor programs, but adds the complexity of shared data integrity. A distributed-memory system introduces a different problem: how to distribute a computational task to multiple processors with distinct memory spaces and reassemble the results from each processor into one solution.
Distributed Computing is a way of combining the processing power of thousands of small computers (ie: PCs) to solve very complex problems that are too large for traditional supercomputers, which are very expensive to build and run.