DistOS-2011W FWR

Introduction

The idea behind FWR (First Webocratic Republic) is simple: to create a self-governed community on the web.

The first approach was natural: write a CMS where users can register (become citizens of FWR), communicate freely, elect leaders and be elected. The main goal of such a community is to survive, so some source of income is needed to pay hosting provider. CMS needed to have two parts – public and for-citizens-only. Citizens elect leaders, who decide on the strategy of content-generation, then everybody work on public part (tourist site) to attract visitors and get money by ads, referral links, etc. This approach can't be described as fully democratic, since the owner of root-password on the server is still in complete, god-like power over the FWR.

To fix this issue, another idea was added: distribute the copy of the world (filesystem and DB) to all citizens in some p2p fashion (torrent, probably) every time government changes, so that if new government screws things up, everyone has a "backup world". This also contributes to overall distribution of FWR – every copy is fully-functional and can be set as a separate "country".

But this all wasn't distributed enough.

Here is the structure in mind prior to implementation:

CMS is running on the main server, but in read-only mode. The data (files, databases, etc) is synced by rsync from client machines. Clients are citizens, and only they can write to synced directories. Local http server on client machine runs fully-functional CMS which can be accessed locally. There is also a set of scripts to work with files, databases and rsync. Clients can sync with each other too.

So, even though we still have main server for public access, the system does not depend on it. Main server can be easily changed. System still runs being offline and tries to sync everything as soon as it gets back online.

Amount of governance needed has also changed: in the previous model elected leaders had access to server-files and databases, but now everyone has access to it. The only essential thing left is to moderate content being synced. This can be done by adding personal or global filters, not allowing particular people to sync with main server or with clients. Also, local and server-storage can be set as version control system, so that vandalism can be dealt with as in wikis. Moreover, every citizen can separate at any moment and run their own world.

This scheme can be a base of some collaboration system or just as a safe web-development environment.

Setting up

Using rsync both ways can lead to inconsistencies and errors, that's why another tool was chosen – unison. It allows to sync files on two machines by issuing one command on either machine. Unison can use sockets or ssh to transfer data and can be used together with SVN or any other version control system.

Central server

The following steps are required to set up a central server:

1. Install unison (Debian example):

apt-get install unison

2. Install open-ssh (Debian example):

apt-get install openssh-server openssh-client

Since we use ssh to transfer data, open-ssh should be installed on both server and client. Alternatively, sockets can be used. sshd daemon must be running now. If we want server to invoke synchronization, we need to generate keys and give public key to everyone, who wants to join the commmunity:

ssh-keygen -t dsa

File .ssh/id_dsa.pub is created. It is the public key.

3. Add user

adduser username /home/username

This user account is for FWR server only, it will run servers. Then, create a directory /home/username/fwr. This is where all synced data will be stored.

4. Install Apache (Debian example):

apt-get install apache2

5. Install inotify utility:

inotify-based utility will be monitoring our 'fwr' directory on server side and sync with all the clients on every modification. We could use incron, but it cannot monitor directories recursively. There is a tiny python utility called Watcher which uses Linux kernel's inotify via pyinotify Python module. Watcher supports everything incron does and adds recursive monitoring.

Install Python, pyinotify, python-argparse

sudo apt-get install python python-pyinotify python-argparse

Download and unpack Watcher

wget https://github.com/splitbrain/Watcher/tarball/master --no-check-certificate && tar zxvf master

Client

Client machine runs local FWR server and syncs data with central server. The following steps are required to set up a client machine:

1. Install unison (Debian example):

apt-get install unison

2. Install open-ssh (optional)

apt-get install openssh-client

Since we use ssh to transfer data, open-ssh should be installed on both server and client. Alternatively, sockets can be used.

3. Create a private key for passwordless connections:

ssh-keygen -t dsa

4. Copy the key to central server:

ssh-copy-id -i .ssh/id_dsa.pub username@remote.machine.com

This will allow to avoid entering password while connecting to central server, so that synchronization can be done seamlessly. Of course, it is safer to give the key to central server administrator (elected official) who will then upload it without sharing the password of 'username' account on central server.

5. Install Apache (Debian example):

apt-get install apache2

6. Install inotify-based utility (optional):

Install Python, pyinotify, python-argparse

sudo apt-get install python python-pyinotify python-argparse

Download and unpack Watcher

wget https://github.com/splitbrain/Watcher/tarball/master && tar zxvf master

Configuration

Server

The following daemons should be running at all times on central server:

• sshd (open-ssh daemon allowing remote connections from clients)
• httpd (apache web-server)
• incrond (monitoring system, syncs data on every modification)

The following files should be present in /home/username on central server:

• fwr/www (citizen-site, not public)
• fwr/www_tourist (tourist-site, public)
• .ssh/authorized_keys (clients' keys)
• .ssh/known_hosts (clients' hosts)

Apache

Add the following to httpd.conf:

Alias /fwr /home/username/fwr/www
<Directory /home/username/fwr/www>
Options FollowSymLinks
AllowOverride Limit Options FileInfo
DirectoryIndex index.php
</Directory>

Alias /fwr_tour /home/username/fwr/www_tourist
<Directory /home/username/fwr/www_tourist>
Options FollowSymLinks
AllowOverride Limit Options FileInfo
DirectoryIndex index.php
</Directory>

This can vary depending on CMS you want to use. Restart apache.

/etc/init.d/apache2 restart

Now the contents of /home/username/fwr/www (which is the main citizen site) is available at http://website.com/fwr, and contents of /home/username/fwr/www_tourist (which is public citizen site) available at http://website.com/fwr_tour, where 'website.com' is server's public domain name. The citizen site must be protected on server side, so appropriate settings in /home/username/fwr/www/.htaccess must be added and .htaccess file should be ignored at synchronization. This will be described later.

Much safer approach is to avoid putting fwr-citizen-site on public server altogether (as described in the general scheme in the beginning).

inotify + pyinotify

inotify-based utility will be monitoring our 'fwr' directory on server side and sync with all the clients on every modification. We could use incron, but it cannot monitor directories recursively. There is a tiny python utility called Watcher which uses Linux kernel's inotify via pyinotify Python module. Watcher supports everything incron does and adds recursive monitoring.

Install Python, pyinotify, python-argparse

sudo apt-get install python python-pyinotify python-argparse

Download and unpack Watcher

wget https://github.com/splitbrain/Watcher/tarball/master && tar zxvf master

Make the following changes to watcher.ini:

watch=/home/username/fwr
events=create,delete,attribute_change,write_close,modify
command=/home/username/fwr.sh sync

And run watcher as daemon:

python watcher.py start -c watcher.ini

Now fwr.sh bash script is executed with parameter 'sync' every time any modification occurs in 'fwr' directory.

fwr.sh

to be continued

Conclusion

Summarize the report, point to future work.

References

Give references in proper form (not just URLs if possible, give dates of access).