Cloudant is a distributed database system, where each logical database is represented by a specific number of shards in a cluster (refereed to as Q). Each shard is furthermore stored N times in the cluster, hence each document will be stored on more than one node in the cluster. The locality of the document is determined by using consistent hashing keys, as is very well described in the Dynamo paper by Amazon.
With Cloudant, there’s a free to start multi-tenant solution, where you can be up and running in 5 minutes, where you pay by request; and there’s also dedicated installations. The point is. You don’t need to worry. The infrastructure is in good hands and instead of worrying about maintenance, you can focus on building your application. But one interesting thing to know about as a developer is the ability of configuring read and write quorum.
Write and Read Quorum, configured for consistency
Whenever a document is written (PUT, POST) to Cloudant, it is by default (as of this writing), written to two nodes. Whenever you read a document (GET), by default, you will not get the document returned unless it can be read from two nodes. By default, the number of nodes in total are three. This means that by default, it is configured for consistency. Why? Well, if you write a document, and it needs to be successfully written to two nodes out of three, a maximum of one node (due to a partition failure) can be without the document. As a result, if you read the document with a read quorum of two nodes out of three, you are guaranteed to get the previously stored document, since the write- and read quorum will overlap each other in at least one node. The equation for this is written:
W + R > N --> Consistency
You keep on saying: “By default”
Yes, by default, Cloudant is configuredwith sensible defaults for consistency. But, you can (can does not mean you should), on an operation basis configure this. So when performing your write (PUT, POST) you can pass an additional query parameter, e.g. w=1. Doing this will increase the throughput of your writes, put you will get a higher risk of getting inconsistencies. The same goes for reads. When performing a read (GET) you can pass an additional query parameter, e.g. r=1, this time indicating how many nodes you need to read the document from, to be accepted as a successful read. This could e.g. be OK if you are reading static data. Data that has non or low frequent updates.
The CAP theorem
The terms Consistency and Partition tolerance used above are two of the three ingredients of the CAP theorem, also known as Brewers theorem. The third ingredient is Availability. Hence CAP stands for:
- C: Consistency
- A: Availability
- P: Partition tolerance
You can read more about the theorem in: “Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services” (Seth Gilbert & Nancy Lynch of MIT). The theorem is in short about, in any distributed system, you can only achieve two out of the three at the same time, and for being a successful distributed system, you need to be partition tolerant, so the choice stands between being either: Consistent; or Available. Being consistent means that all nodes share the same view of the state; while being available means that all nodes might or might not share the same view of the state at a given point, but is configured to eventually catch up and to be in sync, which is expressed as being eventually consistent.
One big misunderstanding is often that a distributed database system is either Consistent & Partition tolerant (CP) or Available and Partition tolerant (AP). But take Cloudant as an example. There you can configure this per operation. If you e.g. would configure write quorum to less than three and read quorum to be one, you would potentially gain a higher chance of being available, since the node serving the request could be partitioned from the other two nodes and still deliver a response. This off course increases the risk with serving data that is more stale than data lying on any of the other two nodes and this is something your domain experts need to take a decision on. The classical example is the banking world and the ATMs. In case an ATM is partitioned, they still allow withdrawals to a certain amount. This means that banks has decided to favor availability before consistency and thereby risking a temporary incorrect view of bank account balances, but instead be operational for customers.
Using MyCouch with Cloudant
MyCouch is the async .Net client for use with CouchDb and Cloudant. It’s currently only in v0.15.0 so the functionality in this post will most likely be improved and simplified soon. MyCouch has the concept of a Client. The Client points to a certain database, identified by an URL, and with it, you get a simplified experience of performing reads, writes, queries etc. The Client makes use of an IConnection implementation. This Connection is the “last stop” before performing the actual HTTP-request against CouchDb or Cloudant, and as of now, there’s only one simple implementation: BasicHttpConnection, but you can easily create your own, and inject that via the constructor to the Client.
Create a custom IConnection to configure read quorum
In this case I will extend the BasicHttpConnection and ensure that all GET requests for documents will have a read quorum of one. The usage of a vanilla Client would look like this (read more about how-to get connected in the documentation):
using (var client = new Client("https://someaccount.cloudant.com/dbname"))
{
var response = await client.Documents.GetAsync("test1");
}
the difference will be the injection of a custom connection:
var cn = new CustomCloudantConnection("https://someaccount.cloudant.com/dbname");
using (var client = new Client(cn))
{
var response = await client.Documents.GetAsync("test1");
}
and the custom connection that intercepts all GetDocumentRequest, looks like this:
public class CustomCloudantConnection : BasicHttpClientConnection
{
public CustomCloudantConnection(string url)
: this(new Uri(url)) {}
public CustomCloudantConnection(Uri uri)
: base(uri) {}
protected override HttpRequest OnBeforeSend(HttpRequest httpRequest)
{
if (httpRequest.Method != HttpMethod.Get)
return httpRequest;
if (httpRequest.Headers.Contains(HttpRequest.CustomHeaders.RequestType))
{
var requestTypeName = httpRequest.Headers
.GetValues(HttpRequest.CustomHeaders.RequestType)
.First();
if (requestTypeName == typeof (GetDocumentRequest).Name)
{
httpRequest.RequestUri = string.IsNullOrEmpty(httpRequest.RequestUri.Query)
? new Uri(httpRequest.RequestUri + "?r=1")
: new Uri(httpRequest.RequestUri + "&r=1");
}
}
return base.OnBeforeSend(httpRequest);
}
}
This code makes use of the meta information associated with the HttpRequest via custom headers. The headers mycouch-type and mycouch-entitytype will be removed before the request is performed. This is done in BasicHttpConnection.OnBeforeSend
That’s all for now. Some info about Cloudant read and write quorum and also a use-case for a custom connection in MyCouch.
Cheers,
//Daniel