DistOS-2011W Cassandra and Hamachi - Revision history

Slyons: /* Triple Replication */

2011-03-05T19:47:38Z

Triple Replication

← Older revision		Revision as of 19:47, 5 March 2011
Line 205:		Line 205:
	=== Triple Replication ===		=== Triple Replication ===
	With the third level of replication we can finally see the impact of the [[#Consistency\|ALL]] consistency that my benchmark enforced. With these tests Cassandra had to ensure that every node had the latest update for the test row before the client call could be completed, which quite obviously has impacted both reading and writing for this series of tests.		With the third level of replication we can finally see the impact of the [[#Consistency\|ALL]] consistency that my benchmark enforced. With these tests Cassandra had to ensure that every node had the latest update for the test row before the client call could be completed, which quite obviously has impacted both reading and writing for this series of tests.
	~~[[File:THREE_REP-writes.jpg\|600px\|left\|border\|Benchmark results for WRITING with three replications]]~~
	~~[[File:THREE_REP-reads.jpg\|600px\|right\|border\|Benchmark results for READING with three replications]]~~

			<center>
			<gallery caption="Three Replications" widths="300px">
			Image:THREE_REP-writes.jpg\|Benchmark results for WRITING with three replications
			Image:THREE_REP-reads.jpg\|Benchmark results for READING with three replications
			</gallery>
			</center>
	<br style="clear: both" />		<br style="clear: both" />

Slyons: /* Dobule Replication */

2011-03-05T19:46:59Z

Dobule Replication

← Older revision		Revision as of 19:46, 5 March 2011
Line 194:		Line 194:
	=== Dobule Replication ===		=== Dobule Replication ===
	When we add more replications into the mix we see a noted rise in both read and write times. What's more interesting to see is the statistical aberrations that are much more pronounced as you compound network latency and VPN overhead between the two nodes and ultimately the client.		When we add more replications into the mix we see a noted rise in both read and write times. What's more interesting to see is the statistical aberrations that are much more pronounced as you compound network latency and VPN overhead between the two nodes and ultimately the client.
	~~[[File:TWO_REP-writes.jpg\|600px\|left\|border\|Benchmark results for WRITING with two replications]]~~
	~~[[File:TWO_REP-reads.jpg\|600px\|right\|border\|Benchmark results for READING with two replications]]~~

			<center>
			<gallery caption="Two Replications" widths="300px">
			Image:TWO_REP-writes.jpg\|Benchmark results for WRITING with two replications
			Image:TWO_REP-reads.jpg\|Benchmark results for READING with two replications
			</gallery>
			</center>
	<br style="clear: both" />		<br style="clear: both" />

Slyons: /* Single Replication */

2011-03-05T19:43:02Z

Single Replication

← Older revision		Revision as of 19:43, 5 March 2011
Line 183:		Line 183:
	=== Single Replication ===		=== Single Replication ===
	It's quite apparent that the "append" method of database writing has a significant impact on the responsiveness of the server. While the number of columns in the payload has an affect on the write times, the size of the payload itself has minimal impact on performance. The read times, however small they are, are far more varied in value. Since the benchmarking is very simple in nature I can't pin down the exact cause of the erratic values. Since this test was performed with only single replication it's doubtful Cassandra's to blame, as only one Node need to be contacted and consulted.		It's quite apparent that the "append" method of database writing has a significant impact on the responsiveness of the server. While the number of columns in the payload has an affect on the write times, the size of the payload itself has minimal impact on performance. The read times, however small they are, are far more varied in value. Since the benchmarking is very simple in nature I can't pin down the exact cause of the erratic values. Since this test was performed with only single replication it's doubtful Cassandra's to blame, as only one Node need to be contacted and consulted.
	~~[[File~~:ONE_REP-writes.jpg~~\|300px\|border~~\|Benchmark results for WRITING with one replication]]
	~~[[File~~:ONE_REP-reads.jpg~~\|300px\|border~~\|Benchmark results for READING with one replication]]		<center>
			<gallery caption="One Replication" widths="300px">
			Image:ONE_REP-writes.jpg\|Benchmark results for WRITING with one replication
			Image:ONE_REP-reads.jpg\|Benchmark results for READING with one replication
			</gallery>
			</center>
	<br style="clear: both" />		<br style="clear: both" />

Slyons: /* Single Replication */

2011-03-05T19:32:34Z

Single Replication

← Older revision		Revision as of 19:32, 5 March 2011
Line 183:		Line 183:
	=== Single Replication ===		=== Single Replication ===
	It's quite apparent that the "append" method of database writing has a significant impact on the responsiveness of the server. While the number of columns in the payload has an affect on the write times, the size of the payload itself has minimal impact on performance. The read times, however small they are, are far more varied in value. Since the benchmarking is very simple in nature I can't pin down the exact cause of the erratic values. Since this test was performed with only single replication it's doubtful Cassandra's to blame, as only one Node need to be contacted and consulted.		It's quite apparent that the "append" method of database writing has a significant impact on the responsiveness of the server. While the number of columns in the payload has an affect on the write times, the size of the payload itself has minimal impact on performance. The read times, however small they are, are far more varied in value. Since the benchmarking is very simple in nature I can't pin down the exact cause of the erratic values. Since this test was performed with only single replication it's doubtful Cassandra's to blame, as only one Node need to be contacted and consulted.
	[[File:ONE_REP-writes.jpg\|~~600px\|left~~\|border\|Benchmark results for WRITING with one replication]]		[[File:ONE_REP-writes.jpg\|300px\|border\|Benchmark results for WRITING with one replication]]
	[[File:ONE_REP-reads.jpg\|~~600px\|right~~\|border\|Benchmark results for READING with one replication]]		[[File:ONE_REP-reads.jpg\|300px\|border\|Benchmark results for READING with one replication]]
	<br style="clear: both" />		<br style="clear: both" />

Slyons at 17:56, 1 March 2011

2011-03-01T17:56:19Z

← Older revision		Revision as of 17:56, 1 March 2011
Line 200:		Line 200:

	<br style="clear: both" />		<br style="clear: both" />

			= Conclusions =

			Despite the statistical aberrations (which could have been caused by any number of factors) getting the components to talk together was mostly effortless. As I mentioned previously the implementation would most likely gain a significant performance boost by not relying on a third party for the VPN connection mediation. Not relying on a third party for connection permissions would also greatly increase the security and reliability on external resources.

			As a distributed database Cassandra certainly has potential, but I feel that the project is still too young to be completely reliable for the ideal goal of my project. It was only recently that the project introduced both indexes and paged results, something that I'd expect would be a higher (or more common sense) priority.

			To conclude, I do believe that this idea has merit and has a potential for further development but would require more planning and forethought before a proper implementation would be ready.

	=References=		=References=
	# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011		# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011
	# [http://hamachi.cc\| Hamachi homepage]		# [http://hamachi.cc\| Hamachi homepage]

Slyons: /* References */

2011-03-01T17:32:11Z

References

← Older revision		Revision as of 17:32, 1 March 2011
Line 203:		Line 203:
	=References=		=References=
	# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011		# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011
	# [http://hamachi.cc \| Hamachi homepage]		# [http://hamachi.cc\| Hamachi homepage]

Slyons: /* References */

2011-03-01T17:31:05Z

References

← Older revision		Revision as of 17:31, 1 March 2011
Line 203:		Line 203:
	=References=		=References=
	# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011		# [http://oreilly.com/catalog/0636920010852\| Cassandra: The definitive guide] by Eben Hewitt © 2011
			# [http://hamachi.cc \| Hamachi homepage]

Slyons: /* Caveat */

2011-03-01T17:30:42Z

Caveat

← Older revision		Revision as of 17:30, 1 March 2011
Line 59:		Line 59:
	== Caveat ==		== Caveat ==

	While the original implementation of Hamachi was open source the project is currently part of a commercial vendor called LogMeIn. The reason I chose to use this project was simply to fulfill a need for a quick VPN setup with a minimal amount of fuss. The performance and security of my implementation project would undoubtably be improved by an open source and mediator-free VPN solution such as [[http://openvpn.net/\|OpenVPN]]. Because of the closed source nature of Hamachi I won't be able to comment much about the implementation details, it was simply a means to an end.		While the original implementation of Hamachi[[#References\|<sup>2</sup>]] was open source the project is currently part of a commercial vendor called [http://logmein.com\| LogMeIn]. The reason I chose to use this project was simply to fulfill a need for a quick VPN setup with a minimal amount of fuss. The performance and security of my implementation project would undoubtably be improved by an open source and mediator-free VPN solution such as [http://openvpn.net/\| OpenVPN]. Because of the closed source nature of Hamachi I won't be able to comment much about the implementation details, it was simply a means to an end.

	== The Basics ==		== The Basics ==

Slyons: /* Configuring Cassandra */

2011-03-01T17:28:06Z

Configuring Cassandra

← Older revision		Revision as of 17:28, 1 March 2011
Line 86:		Line 86:
	=== Configuring Cassandra ===		=== Configuring Cassandra ===

	I installed Cassandra 0.7 using [http://wiki.apache.org/cassandra/DebianPackaging\| the official Debian repository] for both ''Cybercontrol'' and ''Manos''. Both of these servers were configured to use ''Intersect'' as the '''''seed node''', the node (or nodes) that would be responsible for bootstrapping new nodes joining the ring. All the configuration that was required to get the nodes talking was to use the Hamachi IP assigned to ''Intersect'' and that was it.		I installed Cassandra 0.7 using [http://wiki.apache.org/cassandra/DebianPackaging\| the official Debian repository] for both ''Cybercontrol'' and ''Manos''. Both of these servers were configured to use ''Intersect'' as the '''''seed node''''', the node (or nodes) that would be responsible for bootstrapping new nodes joining the ring. All the configuration that was required to get the nodes talking was to use the Hamachi IP assigned to ''Intersect'' and that was it.

	=== The Schema ===		=== The Schema ===

Slyons: /* The Data Model */

2011-03-01T17:26:53Z

The Data Model

← Older revision		Revision as of 17:26, 1 March 2011
Line 16:		Line 16:
	== The Data Model ==		== The Data Model ==

	[[File:Column_Family.PNG\|300px\|thumb\|left\|Cassandra Column Family (([[#References\|Source #1]]))]]		[[File:Column_Family.PNG\|300px\|thumb\|left\|Cassandra Column Family ([[#References\|Source #1]])]]
	Cassandra takes a different approach compared to RDBMSs when it comes to how data is conceptualized and managed. In a standard RDBMS, data is ''normalized'' into a series of '''tables''' with a set number of columns. For the most part, the types and number of these columns will only rarely change as the need arises. For a later analogy, we'll refer to this model as a "Narrow Row" model. Cassandra operates on a system of '''''Column Families''''' which are analogous to a '''tables''' in an RDBMS, but is much more flexible. Column Families (hereafter referred to as ''CFs'') are a loose grouping of common data keyed by some value for each row in question and are grouped together into '''Keyspaces'''. However, as opposed to the static feel of a RDBMS table, the number of columns or even the types of columns can and will change under Cassandra. That isn't to say there is no form of schema in Cassandra, because there is a simple type-enforcement schema system in place for each ''CF''. To complete the analogy, the model for Cassandra can be seen as "Wide Row", with the number of columns growing however you please. Different rows in the same CF can even have a different number of columns. Internally, Cassandra uses a "always appending" model when writing data to disk, similar to Google's Bigtable or GFS.		Cassandra takes a different approach compared to RDBMSs when it comes to how data is conceptualized and managed. In a standard RDBMS, data is ''normalized'' into a series of '''tables''' with a set number of columns. For the most part, the types and number of these columns will only rarely change as the need arises. For a later analogy, we'll refer to this model as a "Narrow Row" model. Cassandra operates on a system of '''''Column Families''''' which are analogous to a '''tables''' in an RDBMS, but is much more flexible. Column Families (hereafter referred to as ''CFs'') are a loose grouping of common data keyed by some value for each row in question and are grouped together into '''Keyspaces'''. However, as opposed to the static feel of a RDBMS table, the number of columns or even the types of columns can and will change under Cassandra. That isn't to say there is no form of schema in Cassandra, because there is a simple type-enforcement schema system in place for each ''CF''. To complete the analogy, the model for Cassandra can be seen as "Wide Row", with the number of columns growing however you please. Different rows in the same CF can even have a different number of columns. Internally, Cassandra uses a "always appending" model when writing data to disk, similar to Google's Bigtable or GFS.