Skip to main content

The architecture of a Distributed SQL Database in 2020 | Modern Data Architecture

In this webinar we define the architecture of a Distributed SQL database. Several attempts have been made to deliver truly scalable SQL in a distributed environment. Some have tried to retrofit existing databases to meet their needs but this ultimately does not deliver on the promise of a truly Distributed SQL database. So then, what makes up a Distributed SQL database? The requirements can be summarized into the five core conditions: 1. #Scale A distributed SQL database must seamlessly scale in order to mirror the capabilities of cloud environments without introducing operational complexity. Just as we can scale up compute without heavy lifting, the database should be able to scale as well. This includes an ability to evenly distribute data across multiple distributed participants in the database. 2. #Consistency A distributed SQL database must deliver a high level of isolation in a distributed environment. In a cloud-based world with distributed systems and microservices are the default architectures, transactional consistency becomes difficult as multiple operators may be trying to work on the same data. The database should mediate contention and deliver the same level of isolation of transactions as we expect in a single instance database. 3. #Resiliency A distributed SQL database must naturally deliver the highest level of resiliency without any need of external tooling to accomplish this. The cloud presents an always-on environment for our workloads and the database should have the same properties. With a distributed database we can reduce the time it takes to recover from a failure down to near zero and replicate data naturally without any external configuration. 4. #GeoReplication A distributed SQL database should allow for distribution of data throughout a complex, widely dispersed geographic environment. The cloud presents an ability to reach every corner of the globe with an acceptable quality of service and the database should not restrict your applications from doing so. It should perform to meet your expectations 5. #SQL And while these four technical requirements are paramount, there is one key prerequisite above all. The database must speak SQL. It is the language of data and the default for all application logic. We should not have to retrain developers to use the database. They should be able to use the SQL dialect they are already familiar with. There are a few databases that meet these requirements. The list includes Spanner, of course, but you could also consider Amazon Aurora, Yugabyte, FaunaDB and CockroachDB as members of this new category. All of these members meet the requirements in some form, some better than others. Noticeably missing from this list are Oracle, Postgres, MySQL and all of the NoSQL options. While each may meet some of the requirements, none of them meet all of the requirements and cannot be considered alternatives. The devil is six: #DataLocality Once you live in a distributed world, it becomes apparent that the database itself could actually take care of domiciling data. With participants located in various regions or data centers, it becomes possible to understand the location of each and then tie the data that it stores to a location. Some application architects have implemented this as part of an application but this approach is error-prone and brittle. Using the database to geo-partition data based on some field in a table is a new requirement for Distributed SQL. This allows you to use the database to address data sovereignty concerns. It can also be used to have data follow a user so you can ensure low latency access to their information or to tie data to an explicit cloud so you can minimize egress charges. And god is seven: #MultiCloud A unique trait of a Distributed SQL database is that it has semi-autonomous units that all participate in a larger system. Each unit should be able to be deployed by itself and then join the larger system, the CockroachDB cluster. This is an inherent trait that fuels the first five requirements listed above. However, this can also be used to extend the database to be truly multi-cloud. The database should not rely on a single network to accomplish distribution. It should be divorced from these limits so that a participant can be located anywhere, from any public cloud, a private cloud and even a single on-premise instance. This requirement is important for the future of compute where we live in a distributed hybrid and multi-cloud world. --------------------------------------------------------------------------------------------------------------------------- Careers: https://www.cockroachlabs.com/careers CockroachCloud:https://www.cockroachlabs.com/product/cockroachcloud/ CockroachDB:https://www.cockroachlabs.com/product/ Blog:https://www.cockroachlabs.com/blog/ Docs:https://www.cockroachlabs.com/docs/stable/ Community Slack: https://cockroa.ch/Welcome-to-Slack