Talk:Distributed Shared Memory

Here is the page where we will be discussing the DSM readings.

IVY

Anil: What were the key characteristics of IVY? What exactly did Kai Li build?

Alireza : IVY was a software based DSM system that's been developed to allow users share their local memories in a distributed manner. IVY was designed to be used in loosely coupled environments. It had five main modules including memory allocation, process management, initialization, remote operation and memory mapping. The main advantage of IVY was gaining performance in parallel applications comparing.

Alireza :(Question) Name some of the applications that you would think benefit from using IVY environment? Distributed Database system is the one that is mentioned in the dissertation. Thinks of something different.

Azalia:(Answer) Some of the current sample applications can be (CRM) Customer Relationship Management or (ERP) Enterprise Resource Planning applications that serve multiple users across an organization.Another example, imagine a billing system that has to calculate the telephone bills of thousands of customers can benefit from this environment by calculating the bill of multiple customers at the same time on different machines distributed across the network. Even though, each customer bill calculation can be done separately, using a shared memory space for reading input values like cost per minute, or cost per text message can be very useful. In addition, since each customer bill is a separate object the write operation is done in different pages of the shared memory and even using multiple writer algorithm, in this case, does not introduce any concurrency issue.

Azalia:(Question) What is the potential problem in centralized manager algorithm? What is the alternative algorithm?

Abecevello:(Answer) There are actually several potential problems with using a centralized manager algorithm. Actually, we can even get philosophical about this question (and possible something not covered in the paper). A centralized manager algorithm represents a single point of failure. If the machine running the centralized algorithm fails, then the entire distributed system fails. A centralized component in a distributed system (not even a distributed shared memory system) is probably against the whole point of making a distributed system. You distributed a system, so why not distribute all of it? Especially today, distributed systems are made so that if one computer fails, the entire system doesn't fail. To answer the second question, the alternative is of course a distributed manager algorithm. Of course, in the Kai Li paper the centralized manager problem was discussed in terms of performance, but also, this goes in terms of centralized vs. distributed. In a distributed algorithm there are multiple machines to do the work, so is it not conceivable that it would be faster overall?

Current DSM systems?

Anil: What is a current production system that uses distributed shared memory? What about the underlying problem makes DSM a good technological fit?

Azalia:(Answer) What is a current production system that uses distributed shared memory? Any application with complex independent steps that can be parallelized would be suitable for DSM environment. Some of the current sample applications can be (CRM) Customer Relationship Management or (ERP) Enterprise Resource Planning applications that serve multiple users across an organization. What about the underlying problem makes DSM a good technological fit? Apart from DSM there are alternative methods for using in distributed environment (e.g. RPC and message passing), they have some inadequacies that DSM has been introduced to address them. For instance, message passing and RPC, have difficulties in sending complex data structures and pointers over the network due to different memory address spaces. The distributed shared memory can be a solution of this problem since all the processors share the same memory address space. In addition, if we consider current RPC technologies like Web Services, we'll realize that for each task we have to pack and send a lot of XML data around. With DSM we can share a memory space and prevent overloading network by sending XML messages.

Colin:(Answer) Any systems with a great deal of variability in load on its processors could benefit from DSM. This is because the unified address space makes process migration, and thus load balancing, simpler. (Question) How much more efficient is the movement of data across the networks on a system that implements DSM? Does it not send a comparable amount of data on a page fault as message passing or RPC would to invoke a remote call?

AlexC:(Answer) Cluster computing comes to mind. It depends what type of cluster is implemented, a cluster built for High Availabilty would likely not implement DSM because the primary way to accomplish high availabily is redundency. A cluster build for Load Balancing may implement DSM if it was load balancing large computations with shared data, it would be a good candidate. However if the cluster was built for load balancing many small jobs (say a really big webserver), then it would not be a good DSM candidate. If a cluster was built for large computations then it would certainly be a good candidate for DSM. While looking at clusters I also noticed a Java implementation for cluster computing call JavaSpace that implements distributed shared memory. However instead of sharing memory pages, it shares java objects.

Difference between DSM and NUMA?

Anil What are the differences between DSM and NUMA? Under what circumstances are each appropriate?

Alireza: NUMA follows SMP paradigm where there is common memory bus for accessing shared memories. In addition, one of the most important aspects of the NUMA is that it provides different access time for the processers based on their locations. For instance local processors can have faster access to local memories. In addition, NUMA access to memory is hardware based.

Joshua Tessier: Correct me if I'm wrong but NUMA is basically a type of DSM. In a NUMA system, each processor has access to a common memory, however this common memory is distributed across each of the processors. For example, if there are 8 processors, the total memory is divided into 8 sections. As stated above, the processors have different access times to memory stores. Meanwhile, DSM is just dynamic shared memory; not a specific type like NUMA.

ABecevello: It seems from doing some background reading on NUMA (since I noticed the articles didn't really describe it very well), that NUMA is primarily used for memory access of multiple processors in a single computer. DSM as described sounds like its designed for memory accesses between computers. From my reading, it looks like NUMA runs best when multiple processors do NOT attempt to access the same memory all at the same time.

DSM Implementations?

Azalia:(Question) What are the different types of DSM Implementations?

Yohan:(Answer) There are 3 different types of DSM implementation. The first one is Software-level implementation which can be achieved in user-level, run-time library routine, the OS, or the programming language, for example IVY, Mermaid, Munin, etc. The second one is Hardware-level implementation which ensures that automatic replication of shared data in local memories and processors caches, transparently for software-layer, for example Memnet, Dash, SCI KSR1, etc. Since software is used in hardware support to optimize memory reference, and hardware is used in software solution such as virtual memory management, then the third one is Hybrid Level Implementation which is a combination of both implementation. Several examples of such implementation are Plus, Galactica Net, Alewife, etc.

Joshua Tessier:(Question) Does the hybrid solution hold much relevance today? From what I got in the paper, it came to light due to some limitations of the hardware/OS layers at the time. Today, we have a ton of different tools at our disposal and these limitations are no longer present. How would such a solution be divided today?

William:(Answer) While the general need for DSM systems may be reducing, when they are desired, the hybrid solution does hold some weight today. It is of course a balance of scalability vs. performance, I don't think there will ever be a day when software performance will ever exceed hardware. In order to reduce latencies a hybrid approach would be the most beneficial, but only in the correct circumstances. Software implementations make it much easier to integrate (especially those which do not modify OS functionality), while pure hardware, although costs have come down, would not be all that economical as the system would involve a lot of custom hardware and thus custom software to get the proper benefit from it. This would allow the hardware to perform certain processes, in parallel with the software, much like the internal workings of a single computer. The unfortunate downside would be for programmers, updating hardware and software simultaneously and efficiently is much more difficult than updating software alone and of course the money involved to develop such a system.

Alireza : (Answer) I believe one of the main problems of utilizing any kind of hardware solution is portability and adaptability issue. As soon as we start combining hardware with our solution we introduce a factor that requires special machines and thus reduce the rate of adaptability of our solution. Obviously, customized hardware can surpass software running on general purpose hardware from performance perspective but we should always ask this question during early stages of our design: Do we want a special purpose solution or an adaptable solution?