我最近一直在阅读有关可伸缩架构的文章。在这种情况下,关于数据库不断出现的两个词是分片和分区。我查了一下描述,但最后还是懵了。

stackoverflow的专家能帮我弄清楚基本的东西吗?

分片和分区之间的区别是什么? “所有的分片数据库本质上都是分区的(在不同的节点上),但所有分区的数据库不一定都是分片的”,这是真的吗?


当前回答

When talking about partitioning please do not use term replicate or replication. Replication is a different concept and out of scope of this page. When we talk about partitioning then better word is divide and when we talk about sharding then better word is distribute. In partition (normally and in common understanding not always) the rows of large data set table are divided into two or more disjoint (not sharing any row) groups. You can call each group a partition. These groups or all the partitions remain under the control of once RDMB instance and this is all logical. The base of each group can be a hash or range or etc. If you have ten years data in a table then you can store each of the year's data in a separate partition and this can be achieved by setting partition boundaries on the basis of a non-null column CREATE_DATE. Once you query the db then if you specify a create date between 01-01-1999 and 31-12-2000 then only two partitions will be hit and it will be sequential. I did similar on DB for billion + records and sql time came to 50 millis from 30 seconds using indices etc all. Sharding is that you host each partition on a different node/machine. Now searching inside the partitions/shards can happen in parallel.

其他回答

假设数据库中的一个表有100万行和100列 在Partitioning中,你可以将表分成2个或更多具有如下属性的表:

40万行(表1),60万行(表2) 100万行60列(表1)和100万行40列(表2) 可能有很多这样的案例

这是一般分区

但Sharding仅指第一种情况,即我们根据行划分数据。如果我们将表划分为多个表,我们需要维护模式的多个类似副本,因为现在我们有多个表。

When talking about partitioning please do not use term replicate or replication. Replication is a different concept and out of scope of this page. When we talk about partitioning then better word is divide and when we talk about sharding then better word is distribute. In partition (normally and in common understanding not always) the rows of large data set table are divided into two or more disjoint (not sharing any row) groups. You can call each group a partition. These groups or all the partitions remain under the control of once RDMB instance and this is all logical. The base of each group can be a hash or range or etc. If you have ten years data in a table then you can store each of the year's data in a separate partition and this can be achieved by setting partition boundaries on the basis of a non-null column CREATE_DATE. Once you query the db then if you specify a create date between 01-01-1999 and 31-12-2000 then only two partitions will be hit and it will be sequential. I did similar on DB for billion + records and sql time came to 50 millis from 30 seconds using indices etc all. Sharding is that you host each partition on a different node/machine. Now searching inside the partitions/shards can happen in parallel.

分区是跨表或数据库划分数据的通用术语。分片是一种特定类型的分区,是所谓水平分区的一部分。

在这里,您可以跨(通常)多个实例或服务器复制模式,使用某种逻辑或标识符来知道要在哪个实例或服务器上查找数据。这种标识符通常被称为“碎片键”。

一种常见的无键逻辑是使用字母表来划分数据。A-D是实例1,E-G是实例2,等等。客户数据非常适合这样做,但是如果分区没有考虑到某些字母比其他字母更常见,那么在实例之间的大小表示就会有些错误。

另一种常用技术是使用键同步系统或逻辑,以确保跨实例的键是唯一的。

你可以研究的一个众所周知的例子是Instagram在早期是如何解决他们的划分问题的(见下面的链接)。他们开始在很少的服务器上进行分区,使用Postgres从一开始就划分数据。我相信是几千个逻辑碎片在那几个物理碎片上。在这里阅读他们2012年的精彩报道:Instagram工程-切分和id

在这里也可以看到:http://www.quora.com/Whats-the-difference-between-sharding-and-partition

看来这回答了你的两个问题

Horizontal partitioning splits one or more tables by row, usually within a single instance of a schema and a database server. It may offer an advantage by reducing index size (and thus search effort) provided that there is some obvious, robust, implicit way to identify in which table a particular row will be found, without first needing to search the index, e.g., the classic example of the 'CustomersEast' and 'CustomersWest' tables, where their zip code already indicates where they will be found. Sharding goes beyond this: it partitions the problematic table(s) in the same way, but it does this across potentially multiple instances of the schema. The obvious advantage would be that search load for the large partitioned table can now be split across multiple servers (logical or physical), not just multiple indexes on the same logical server.

来源:Wiki-Shard。

分片是跨多个存储数据记录的过程 这是MongoDB满足数据需求的方法 增长。随着数据大小的增加,单个机器可能无法实现 足以存储数据,也不能提供可接受的读写 吞吐量。分片解决了水平缩放的问题。与 通过分片,您可以添加更多的机器来支持数据增长和需求 读取和写入操作。

来源:MongoDB。

水平分区移动到另一个数据库实例*时将成为数据库碎片。

数据库实例可以在同一台机器上,也可以在另一台机器上。