Haystack

Facebook's Photo Application Storage System.
Previous Fb photo storage based on NFS design. The reason why NFS didn't work is that it took 3 file-system accesses per logical photo read. Haystack only needs one access.
Main goals of Haystack:
- High throughput with low latency. It uses one disk operation to provide these.
- Fault tolerance
- Cost effective
- Simple
Facebook stored all images in haystack with a CDN in front to cache hot data. Haystack still needs to be fast since accessing non-cached data is still common.
Haystack reduces the memory used for filesystem metadata
It has 2 types of metadata:
- Application metadata
- File System metadata
The architecture consists of 3 components:
- Haystack Store
- Haystack Directory
- Haystack Cache
Pitchfork and bulk sync were used to tolerate faults. theTfault tolerance works in a very profound way to make haystack feasible and reliable

Comet

Introduced the concept of distributed shared memory (DSM). In a DSM, RAMs from multiple servers would appear as if they are all belonging to one server, allowing better scalability for caching.
DSM provides advatage over RPC(Remote Procedure Call) including multi threading suuport, thread migration during execution.
client and server model maintain consistency using DSM
Comet model works by offloading the computation intensive process from the mobile to only one server.
The offloading process works by passing the computation intensive process to the server and hold it on the mobile device. Once the process on the server completes, it returns the results and the handle back to the mobile device. In other words, the process does not get physically offloaded to the server but instead it runs on the server and stopped on the mobile device.

Warm Blob Storage System.
- Warm Blob is a store for large quantities of immutable data that isn't frequently accessed, but must still be available.
- Built to reduce the overhead of haystack for old data that doesn't need to be quite as available. Generally data that is a few months old is moved from Haystack to Warm Blob.
- F4 reduce the space usage of Haystack from a replication factor of 3.6 to 2.8 or 2.1 using Reed Solomon coding and XOR coding respectively but still provides consistency.
- Less robust to data center failures as a result.
Reed Solomon coding basically use(10,4) which means 10 data and 4 parity blocks in a stripe, and can thus tolerate losing up to 4 blocks which means it can tolerate 4 rack failure and use 1.4 expansion factor.Two copies of this would be 2* 1.4= 2.8 effective replication factor.
XOR coding use(2,1) across three data center and use 1.5 expansion factor which gives 1.5*1.4= 2.1 effective replication factor.
The caching mechanism provides the reduction in load on storage system and it makes BLOB scaleable.

Represents a building block towards building this global distributed systems. The main critique to it is that it didn’t present a specific use case upon which their design is built upon.
Sapphire does not show their scalability boundaries. There is no such distributed system model that can be “one size fits all”, most probably it will break in some large scale distributed application.
Reaching this global distributed system that address all the distributed OS use cases will be a cumulative work of many big bodies and building it block by block and then this system will evolve by putting all these different building blocks together. In other words, reaching a global distributed system will come from a “bottom up not top down approach” [Somayaji, 2015].
The concept of separate application logic from deployment logic helps programmers in making a flexible system. The other important part that makes it as a scalable system was that it is object based and could be integrated with any object oriented language.