前言:
相信维护过有大数据的MySQL的运维人员一定对sharding这个非常了解,MySQL数据库切分自身没有工具需要借助第三方工具进行;MySQL切片是一件非常头疼而又难做的一件事,一旦切分错误,不仅不能优化数据库,反而会加剧数据库负载;mongodb相对于MySQL来说,数据库切分是mongodb与生俱来的功能,mongodb会自动切分数据库;只需要执行切分键,mongodb就会自动切分,这大大减少了运维工作人员的工作量。
mongodb切分结构如图所示:
Config Server节点:保存了所有数据库的Collection,在实际生产中建议使用复制集
Router:路由节点,通过查找Config Server,将数据存储到指定的Shard节点,建议在生产环境中使用keepalived或heartbeat等高可用方案
Shard:存储节点,实际存储数据的节点;在实际生产中建议每个shard做一个复制集
mongodb切分类型:
详细信息请查官网地址:http://docs.mongodb.org/manual/core/sharding-introduction/
(1)基于范围切分:range
(2)基于列表切分:list
(3)基于hash切分:hash
实验环境:
(1)确保所使用主机iptables已经SELinux都以关闭
(2)本次实验中mongodb的各个节点角色如下:
node1: 172.16.2.12 mongodb的shard节点
node2:172.16.2.13 mongodb的shard节点
node3:172.16.2.14 mongodb的config-server节点
node4: 172.16.2.15 mongodb的route节点
(3)确保各节点时间保持一致
二、配置过程:
[mongodb-org-2.6] \\准备yum源 name=MongoDB 2.6 Repository baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/ gpgcheck=0 enabled=1
(1)安装配置config server节点
[root@node3 ~]# yum -y install mongodb-org-server mongodb-org-shell mongodb-org-tools
编辑mongodb配置文件:/etc/mongod.conf;修改信息如下
[root@node3 ~]# mkdir -pv /mongodb/data/;chown -R mongod.mongod /mongodb/ \\创建数据存放目录 [root@node3 ~]# cat /etc/mongod.conf # mongod.conf #where to log logpath=/var/log/mongodb/mongod.log logappend=true # fork and run in background fork=true #port=27017 dbpath=/mongodb/data # location of pidfile pidfilepath=/var/run/mongodb/mongod.pid # Listen to local interface only. Comment out to listen on all interfaces. #bind_ip=127.0.0.1 # Disables write-ahead journaling # nojournal=true # Enables periodic logging of CPU utilization and I/O wait #cpu=true # Turn on/off security. Off is currently the default #noauth=true #auth=true # Verbose logging output. #verbose=true # Inspect all client data for validity on receipt (useful for # developing drivers) #objcheck=true # Enable db quota management #quota=true # Set oplogging level where n is # 0=off (default) # 1=W # 2=R # 3=both # 7=W+some reads #diaglog=0 # Ignore query hints #nohints=true # Enable the HTTP interface (Defaults to port 28017). #httpinterface=true #rest=true # Turns off server-side scripting. This will result in greatly limited # functionality #noscripting=true # Turns off table scans. Any query that would do a table scan fails. #notablescan=true # Disable data file preallocation. #noprealloc=true # Specify .ns file size for new databases. # nssize=<size> # Replication Options # in replicated mongo databases, specify the replica set name here #replSet=one #replIndexPrefetch=_id_only # maximum size in megabytes for replication operation log #oplogSize=1024 # path to a key file storing authentication info for connections # between replica set members #keyFile=/path/to/keyfile configsvr=true \\添加此选项,开启此主机的mongodb中的server config角色
[root@node3 ~]# /etc/init.d/mongod start \\启动mongodb的server config节点 Starting mongod: [ OK ]
[root@node3 ~]# netstat -tpln | grep mongod \\查看监听端口,会发现有原来的TCP/27017更改为TCP/28019 tcp 0 0 0.0.0.0:27019 0.0.0.0:* LISTEN 29526/mongod
(2)配置router节点
[root@node4 ~]# yum -y install mongodb-org-mongos \\只需要安装这一个软件包即可
[root@node4 ~]# mkdir -pv /var/log/mongos/;touch /var/log/mongos/mongos.log \\创建mongos的日志存放位置 [root@node4 ~]# mongos --configdb=172.16.2.14:27019 --logpath=/var/log/mongos/mongos.log --fork 2015-08-22T21:03:08.106+0800 warning: running with 1 config server should be done only for testing purposes and is not recommended for production about to fork child process, waiting until server is ready for connections. forked process: 12939 \\进程pid child process started successfully, parent exiting 参数解释: --configdb:指定mongodb的config server地址 --logpath: 指定日志文件路径 --fork:指定在后端运行 常用参数请查看mongos -h
(3)配置shard节点
[root@node1 ~]# yum -y install mongodb-org-server mongodb-org-shell mongodb-org-tools \\node2节点安装方法相同
编辑mongodb的配置文件:/etc/mongod.conf;修改如下
[root@node1 ~]# mkdir -pv /mongodb/data/;chown -R mongod.mongod /mongodb/ \\创建数据存放目录;node2节点同样需要创建 [root@node1 ~]# cat /etc/mongod.conf # mongod.conf #where to log logpath=/var/log/mongodb/mongod.log logappend=true # fork and run in background fork=true #port=27017 dbpath=/mongodb/data # location of pidfile pidfilepath=/var/run/mongodb/mongod.pid # Listen to local interface only. Comment out to listen on all interfaces. #bind_ip=127.0.0.1 # Disables write-ahead journaling # nojournal=true # Enables periodic logging of CPU utilization and I/O wait #cpu=true # Turn on/off security. Off is currently the default #noauth=true #auth=true # Verbose logging output. #verbose=true # Inspect all client data for validity on receipt (useful for # developing drivers) #objcheck=true # Enable db quota management #quota=true # Set oplogging level where n is # 0=off (default) # 1=W # 2=R # 3=both # 7=W+some reads #diaglog=0 # Ignore query hints #nohints=true # Enable the HTTP interface (Defaults to port 28017). #httpinterface=true #rest=true # Turns off server-side scripting. This will result in greatly limited # functionality #noscripting=true # Turns off table scans. Any query that would do a table scan fails. #notablescan=true # Disable data file preallocation. #noprealloc=true # Specify .ns file size for new databases. # nssize=<size> # Replication Options # in replicated mongo databases, specify the replica set name here #replSet=one #replIndexPrefetch=_id_only # maximum size in megabytes for replication operation log #oplogSize=1024 # path to a key file storing authentication info for connections # between replica set members #keyFile=/path/to/keyfile
[root@node1 ~]# scp /etc/mongod.conf node2:/etc/ \\把配置文件复制给node2节点一份
[root@node1 ~]# /etc/init.d/mongod start \\启动node1节点 Starting mongod: [ OK ] [root@node2 ~]# /etc/init.d/mongod start \\启动node2节点 Starting mongod: [ OK ]
[root@node1 ~]# netstat -tpln | grep mongod ;ssh node2 'netstat -tpln | grep mongod' \\查看监听端口,shard节点的监听端口为tcp/27017 tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 27619/mongod tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 2468/mongod
(4)在shard节点连接router节点,测试切分环境是否搭建成功
[root@node1 ~]# mongo 172.16.2.15 MongoDB shell version: 2.6.11 connecting to: 172.16.2.15/test mongos>
(5)基础环境已经搭建完成,接下来进行sharding具体操作
mongos> sh.addShard("172.16.2.12") \\添加node1节点到mongos(router节点) { "shardAdded" : "shard0000", "ok" : 1 } mongos> sh.addShard("172.16.2.13") \\添加node2节点到mongos(router节点) { "shardAdded" : "shard0001", "ok" : 1 }
mongos> sh.enableSharding("stu") \\指明要切分的数据库,此数据可以不存在,事后创建即可 { "ok" : 1 } mongos> sh.shardCollection("stu.net12", {name: 1}) \\指明要切分的collection,并且指定以那个字段作为切分字段 { "collectionsharded" : "stu.net12", "ok" : 1 } \\这里指明的collection是”net12",指定时要写完整的名称,"dbname.collection_name";以name字段进行切分
mongos> sh.status() \\sharding配置完成,查看各个节点信息 --- Sharding Status --- sharding version: { "_id" : 1, \\表示符 "version" : 4, \\sharding版本 "minCompatibleVersion" : 4, "currentVersion" : 5, 当前版本 "clusterId" : ObjectId("55d872a44a05a590332d1d91")\\集群节点id } shards: \\各个shard节点信息 { "_id" : "shard0000", "host" : "172.16.2.12:27017" } { "_id" : "shard0001", "host" : "172.16.2.13:27017" } databases: \\数据库信息 { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "test", "partitioned" : false, "primary" : "shard0000" } { "_id" : "stu", "partitioned" : true, "primary" : "shard0000" } \\此处显示"stu"数据库可以sharding,主库为shard000即为node1节点 stu.net12 shard key: { "name" : 1 } chunks: shard0000 1 { "name" : { "$minKey" : 1 } } -->> { "name" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)
(6)在mognos中往stu数据库中插入大量数据,查看切分结果
mongos> for(i=1;i<=1000000;i++) db.net12.insert({name: "stu"+i, class: "Network Class", comment: " \\插入数据
mongos> sh.status() \\插入完成后查看各节点信息 --- Sharding Status --- sharding version: { "_id" : 1, "version" : 4, "minCompatibleVersion" : 4, "currentVersion" : 5, "clusterId" : ObjectId("55d872a44a05a590332d1d91") } shards: { "_id" : "shard0000", "host" : "172.16.2.12:27017" } { "_id" : "shard0001", "host" : "172.16.2.13:27017" } databases: { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "test", "partitioned" : false, "primary" : "shard0000" } { "_id" : "stu", "partitioned" : true, "primary" : "shard0000" } stu.net12 shard key: { "name" : 1 } chunks: shard0001 2 \\每个shard节点都各自有两个chunk shard0000 2 { "name" : { "$minKey" : 1 } } -->> { "name" : "stu1" } on : shard0001 Timestamp(2, 0) { "name" : "stu1" } -->> { "name" : "stu25846" } on : shard0000 Timestamp(3, 2) { "name" : "stu25846" } -->> { "name" : "stu999" } on : shard0000 Timestamp(3, 3) { "name" : "stu999" } -->> { "name" : { "$maxKey" : 1 } } on : shard0001 Timestamp(3, 0)
至此实验完成,对mongodb接触不多,还请大家多多给意见。O(∩_∩)O
原创文章,作者:马行空,如若转载,请注明出处:http://www.178linux.com/7812
评论列表(1条)
刚好马内利也有这块的疑问,刚巧共同探讨下 :cool: