MongoDBのReplica SetsをUbuntuで試してみる

ログ溜め込んで解析する基盤としてMongoDBイイんじゃない?ってのは前からよく聞く話でしたが、
こないだのMongoTokyo2012(秀逸なまとめ)みてたら、みんなやってんだなぁと。
以前WindowsでSpringDataからいじった事はあったのですが、(MongoDBをSpring Dataからホゲホゲしてみる | shinodogg.com)
運用考えた構成で試してみようかな的な感じで。
 
■ MongoDBのインストールと動作確認
 
Webサイト(http://www.mongodb.org/downloads)からLinux 64-bitのURLをコピって
wgetでバイナリをダウンロード

# cd /usr/local/src
# export http_proxy=ごにょごにょ
# wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.2.tgz
略

Saving to: `mongodb-linux-x86_64-2.0.2.tgz'

100%[======================================>] 38,350,811  6.81M/s   in 5.2s

2012-01-24 10:18:08 (7.10 MB/s) - `mongodb-linux-x86_64-2.0.2.tgz' saved [38350811/38350811]

 
解凍すると↓こんな感じ。ダンプとかリストアとかtopとか、ほうほうって感じですね。

# tar xvf mongodb-linux-x86_64-2.0.2.tgz
mongodb-linux-x86_64-2.0.2/
mongodb-linux-x86_64-2.0.2/THIRD-PARTY-NOTICES
mongodb-linux-x86_64-2.0.2/GNU-AGPL-3.0
mongodb-linux-x86_64-2.0.2/README
mongodb-linux-x86_64-2.0.2/bin/
mongodb-linux-x86_64-2.0.2/bin/mongo
mongodb-linux-x86_64-2.0.2/bin/mongotop
mongodb-linux-x86_64-2.0.2/bin/mongos
mongodb-linux-x86_64-2.0.2/bin/mongodump
mongodb-linux-x86_64-2.0.2/bin/mongostat
mongodb-linux-x86_64-2.0.2/bin/mongofiles
mongodb-linux-x86_64-2.0.2/bin/mongoimport
mongodb-linux-x86_64-2.0.2/bin/mongosniff
mongodb-linux-x86_64-2.0.2/bin/mongorestore
mongodb-linux-x86_64-2.0.2/bin/mongod
mongodb-linux-x86_64-2.0.2/bin/bsondump
mongodb-linux-x86_64-2.0.2/bin/mongoexport

てか、configureとかmakeとか要らんのねーって。この辺も楽チンなとこでしょうか。
 
/usr/local/mongodbって感じにしておきたいので、ディレクトリ移動して
シンボリックリンク貼っておきやす。

# mv mongodb-linux-x86_64-2.0.2 /usr/local
# cd /usr/local
# ln -s mongodb-linux-x86_64-2.0.2 mongodb
# ls -l | grep mongodb
lrwxrwxrwx 1 root root   26 2012-01-24 10:24 mongodb -> mongodb-linux-x86_64-2.0.2
drwxr-xr-x 3 root root 4096 2011-12-15 06:26 mongodb-linux-x86_64-2.0.2

 
調子に乗ってそのまま起動したらなんか怒られました。
所定のディレクトリがねぇだろうがゴルァって話みたいです。

# /usr/local/mongodb/bin/mongod
/usr/local/mongodb/bin/mongod --help for help and startup options
Tue Jan 24 10:28:42 [initandlisten] MongoDB starting : pid=3275 port=27017 dbpath=/data/db/ 64-bit host=VirtualBoxUbuntu
Tue Jan 24 10:28:42 [initandlisten] db version v2.0.2, pdfile version 4.5
Tue Jan 24 10:28:42 [initandlisten] git version: 514b122d308928517f5841888ceaa4246a7f18e3
Tue Jan 24 10:28:42 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue Jan 24 10:28:42 [initandlisten] options: {}
Tue Jan 24 10:28:42 [initandlisten] exception in initAndListen: 10296 dbpath (/data/db/) does not exist, terminating
Tue Jan 24 10:28:42 dbexit:
Tue Jan 24 10:28:42 [initandlisten] shutdown: going to close listening sockets...
Tue Jan 24 10:28:42 [initandlisten] shutdown: going to flush diaglog...
Tue Jan 24 10:28:42 [initandlisten] shutdown: going to close sockets...
Tue Jan 24 10:28:42 [initandlisten] shutdown: waiting for fs preallocator...
Tue Jan 24 10:28:42 [initandlisten] shutdown: lock for final commit...
Tue Jan 24 10:28:42 [initandlisten] shutdown: final commit...
Tue Jan 24 10:28:42 [initandlisten] shutdown: closing all files...
Tue Jan 24 10:28:42 [initandlisten] closeAllFiles() finished
Tue Jan 24 10:28:42 dbexit: really exiting now
# echo $?
100

 
データを格納しておくようのディレクトリが必要なのねって事で作って叩きます。

# mkdir -p /data/db
# /usr/local/mongodb/bin/mongod
/usr/local/mongodb/bin/mongod --help for help and startup options
Tue Jan 24 10:30:37 [initandlisten] MongoDB starting : pid=3281 port=27017 dbpath=/data/db/ 64-bit host=ほげほげ
Tue Jan 24 10:30:37 [initandlisten] db version v2.0.2, pdfile version 4.5
Tue Jan 24 10:30:37 [initandlisten] git version: 514b122d308928517f5841888ceaa4246a7f18e3
Tue Jan 24 10:30:37 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue Jan 24 10:30:37 [initandlisten] options: {}
Tue Jan 24 10:30:38 [initandlisten] journal dir=/data/db/journal
Tue Jan 24 10:30:38 [initandlisten] recover : no journal files present, no recovery needed
Tue Jan 24 10:30:38 [initandlisten] preallocateIsFaster=true 14.9
Tue Jan 24 10:30:39 [initandlisten] preallocateIsFaster=true 15.24
Tue Jan 24 10:30:41 [initandlisten] preallocateIsFaster=true 15.32
Tue Jan 24 10:30:41 [initandlisten] preallocateIsFaster check took 3.469 secs
Tue Jan 24 10:30:41 [initandlisten] preallocating a journal file /data/db/journal/prealloc.0
		503316480/1073741824	46%
		954204160/1073741824	88%
Tue Jan 24 10:30:50 [initandlisten] preallocating a journal file /data/db/journal/prealloc.1
		377487360/1073741824	35%
		597688320/1073741824	55%
Tue Jan 24 10:30:59 [initandlisten] File I/O errno:2 No such file or directory
preallocating a journal file /data/db/journal/prealloc.2
Tue Jan 24 10:30:59 [initandlisten] File I/O errno:2 No such file or directory
waiting for connections on port 27017
Tue Jan 24 10:30:59 [websvr] admin web console waiting for connections on port 28017

 
それっぽく上がったみたいなのでバックグラウンドにしておきます。
# /etc/init.dがどうのとか、そういうのは置いておいて。。

^Z
zsh: suspended  /usr/local/mongodb/bin/mongod
# bg
[1]  + continued  /usr/local/mongodb/bin/mongod

 
データを入れたり出したりしてみます。
MongoDBのシェルはbin/mongoなので↓のようにシェルを起動します。

# ./mongo
MongoDB shell version: 2.0.2
connecting to: test
Tue Jan 24 11:44:06 [initandlisten] connection accepted from 127.0.0.1:52563 #1
>

 
↓を参考にしてドキュメントをインサートしようとしたら、、
http://www.mongodb.org/pages/viewpage.action?pageId=7209154
diskの残りがねーよ、と。。

Tue Jan 24 12:32:15 [FileAllocator]     will try again in 10 seconds
Tue Jan 24 12:32:25 [FileAllocator] allocating new datafile /data/db/test.ns, filling with zeroes...
Tue Jan 24 12:32:25 [FileAllocator] creating directory /data/db/_tmp
Tue Jan 24 12:32:25 [FileAllocator] error failed to allocate new file: /data/db/test.ns size: 16777216 errno:28 No space left on device

 
またまた、とか思ってみてみたら↓そんなに食うんですかいっていう…

/data/db/journal# ls -l
total 1754916
-rw------- 1 root root 1073741824 2012-01-24 10:30 j._0
-rw------- 1 root root  721301504 2012-01-24 10:30 prealloc.1
-rw------- 1 root root    1888256 2012-01-24 10:30 prealloc.2
/data/db/journal# du -skh
1.7G	.

 
とりあえずjounal配下を全部消して、

/data/db/journal# rm -r *
/data/db/journal# ls -l
total 0

プロセスkillして、、

# kill 3281
VirtualBoxUbuntu# Tue Jan 24 12:35:05 got kill or ctrl c or hup signal 15 (Terminated), will terminate after current cmd ends
Tue Jan 24 12:35:05 [interruptThread] now exiting
Tue Jan 24 12:35:05 dbexit:
Tue Jan 24 12:35:05 [interruptThread] shutdown: going to close listening sockets...
Tue Jan 24 12:35:05 [interruptThread] closing listening socket: 6
Tue Jan 24 12:35:05 [interruptThread] closing listening socket: 7
略
Tue Jan 24 12:35:05 [interruptThread] journalCleanup...
Tue Jan 24 12:35:05 [interruptThread] removeJournalFiles
Tue Jan 24 12:35:05 [interruptThread] shutdown: removing fs lock...
Tue Jan 24 12:35:05 dbexit: really exiting now

[1]  + exit 12    /usr/local/mongodb/bin/mongod

 
ジャーナル無しで上げ直します

# /usr/local/mongodb/bin/mongod --nojournal
Tue Jan 24 12:37:59 [initandlisten] MongoDB starting : pid=3580 port=27017 dbpath=/data/db/ 64-bit host=VirtualBoxUbuntu
Tue Jan 24 12:37:59 [initandlisten] db version v2.0.2, pdfile version 4.5
Tue Jan 24 12:37:59 [initandlisten] git version: 514b122d308928517f5841888ceaa4246a7f18e3
Tue Jan 24 12:37:59 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Tue Jan 24 12:37:59 [initandlisten] options: { nojournal: true }

 
気を取り直して、ドキュメントを突っ込んでみます。

/usr/local/mongodb/bin# ./mongo
MongoDB shell version: 2.0.2
connecting to: test
> doc = { author: 'joe',
...   created : new Date('03/28/2009'),
...   title : 'Yet another blog post',
...   text : 'Here is the text...',
...   tags : [ 'example', 'joe' ],
...   comments : [ { author: 'jim', comment: 'I disagree' },
...                { author: 'nancy', comment: 'Good post' }
...   ]
... }
{
	"author" : "joe",
	"created" : ISODate("2009-03-27T15:00:00Z"),
	"title" : "Yet another blog post",
	"text" : "Here is the text...",
	"tags" : [
		"example",
		"joe"
	],
	"comments" : [
		{
			"author" : "jim",
			"comment" : "I disagree"
		},
		{
			"author" : "nancy",
			"comment" : "Good post"
		}
	]
}
> db.posts.insert(doc);

 
コメントのauthorを指定して検索も出来ました。

> db.posts.find( { "comments.author" : "jim" } )
{ "_id" : ObjectId("4f1e27f47d7053b05fb67e39"), "author" : "joe", "created" : ISODate("2009-03-27T15:00:00Z"), "title" : "Yet another blog post", "text" : "Here is the text...", "tags" : [ "example", "joe" ], "comments" : [ 	{ 	"author" : "jim", 	"comment" : "I disagree" }, 	{ 	"author" : "nancy", 	"comment" : "Good post" } ] }

 
 
■ Replica Setsで冗長性を確保する
 
Shardingとかもあれですが、とりあえず冗長性確保したいのと、
Master/Slaveじゃなくて自動フェールオーバーな仕掛けを持った
Replica Setsってのがあるので、ソレを試してみたいな、と。
 
↓の記事がとてもよくまとまっていて分かりやすかったので参考にさせていただきました。
MongoDBのReplica Setsについての概要 | ryopekoの日記
 
それぞれのデータ用のディレクトリ作ります。

mkdir -p /data/replica1
mkdir -p /data/replica2
mkdir -p /data/replica3

 
それぞれをnojournalで立ち上げます。restオプションはブラウザで確認できるようにする
オプションらしいです。
今回はそれぞれがどんなログ出すのか後からみてみたいので、
コンソールのタブを3つ立ち上げてそれぞれでやってみます。

タブ1:
/usr/local/mongodb/bin/mongod \
--replSet replica --port 27017 --dbpath /data/replica1 --rest --nojournal

タブ2:
/usr/local/mongodb/bin/mongod \
--replSet replica --port 27018 --dbpath /data/replica2 --rest --nojournal

タブ3:
/usr/local/mongodb/bin/mongod \
--replSet replica --port 27019 --dbpath /data/replica3 --rest --nojournal

 
どこのノードでも↓のように設定がちゃんとされてないんじゃね?系のログが出ています。

Tue Jan 24 13:48:16 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)
Tue Jan 24 13:48:16 [rsStart] replSet info you may need to run replSetInitiate -- rs.initiate() in the shell -- if that is not already done
Tue Jan 24 13:48:26 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)

 
↓にアクセスするとHadoop並みのイカした画面が出てきます。
localhost:28017 localhost:28018 localhost:28019

 
続いてレプリカの設定を入れます。

/usr/local/mongodb/bin# ./mongo localhost:27017
MongoDB shell version: 2.0.2
connecting to: localhost:27017/test
> conf = {_id: 'replica', members: [
...    {_id: 0, host: 'localhost:27017'},
...    {_id: 1, host: 'localhost:27018'},
...    {_id: 2, host: 'localhost:27019'}]
... }
{
	"_id" : "replica",
	"members" : [
		{
			"_id" : 0,
			"host" : "localhost:27017"
		},
		{
			"_id" : 1,
			"host" : "localhost:27018"
		},
		{
			"_id" : 2,
			"host" : "localhost:27019"
		}
	]
}
> rs.initiate(conf)
{
	"info" : "Config now saved locally.  Should come online in about a minute.",
	"ok" : 1
}

 
ステータスを見てみるとまだStartup的な。

> rs.status()
{
	"set" : "replica",
	"date" : ISODate("2012-01-24T09:20:22Z"),
	"myState" : 2,
	"members" : [
		{
			"_id" : 0,
			"name" : "localhost:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"optime" : {
				"t" : 1327396814000,
				"i" : 1
			},
			"optimeDate" : ISODate("2012-01-24T09:20:14Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "localhost:27018",
			"health" : 1,
			"state" : 5,
			"stateStr" : "STARTUP2",
			"uptime" : 8,
			"optime" : {
				"t" : 0,
				"i" : 0
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("2012-01-24T09:20:22Z"),
			"pingMs" : 18677460,
			"errmsg" : "initial sync need a member to be primary or secondary to do our initial sync"
		},
		{
			"_id" : 2,
			"name" : "localhost:27019",
			"health" : 1,
			"state" : 5,
			"stateStr" : "STARTUP2",
			"uptime" : 8,
			"optime" : {
				"t" : 0,
				"i" : 0
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("2012-01-24T09:20:22Z"),
			"pingMs" : 18661632,
			"errmsg" : "initial sync need a member to be primary or secondary to do our initial sync"
		}
	],
	"ok" : 1
}
SECONDARY>

 
もう一回叩いてやると、狙った感じになっています。

SECONDARY> rs.status()
{
	"set" : "replica",
	"date" : ISODate("2012-01-24T09:22:16Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "localhost:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"optime" : {
				"t" : 1327396814000,
				"i" : 1
			},
			"optimeDate" : ISODate("2012-01-24T09:20:14Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "localhost:27018",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 122,
			"optime" : {
				"t" : 1327396814000,
				"i" : 1
			},
			"optimeDate" : ISODate("2012-01-24T09:20:14Z"),
			"lastHeartbeat" : ISODate("2012-01-24T09:22:14Z"),
			"pingMs" : 68
		},
		{
			"_id" : 2,
			"name" : "localhost:27019",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 122,
			"optime" : {
				"t" : 1327396814000,
				"i" : 1
			},
			"optimeDate" : ISODate("2012-01-24T09:20:14Z"),
			"lastHeartbeat" : ISODate("2012-01-24T09:22:14Z"),
			"pingMs" : 68
		}
	],
	"ok" : 1
}
PRIMARY>

 
プライマリにデータを入れて、

PRIMARY> doc = { author: 'joe',
...   created : new Date('03/28/2009'),
...   title : 'Yet another blog post',
...   text : 'Here is the text...',
...   tags : [ 'example', 'joe' ],
...   comments : [ { author: 'jim', comment: 'I disagree' },
...                { author: 'nancy', comment: 'Good post' }
...   ]
... }
{
	"author" : "joe",
	"created" : ISODate("2009-03-27T15:00:00Z"),
	"title" : "Yet another blog post",
	"text" : "Here is the text...",
	"tags" : [
		"example",
		"joe"
	],
	"comments" : [
		{
			"author" : "jim",
			"comment" : "I disagree"
		},
		{
			"author" : "nancy",
			"comment" : "Good post"
		}
	]
}
PRIMARY> db.posts.insert(doc);

 
セカンダリからデータを取得してみます。slaveOkってのをやってやらないと
error: { “$err” : “not master and slaveok=false”, “code” : 13435 }
って出て、データが取得できませんでした。

SECONDARY> rs.slaveOk();
not master and slaveok=false
SECONDARY> db.posts.find( { "comments.author" : "jim" } )
{ "_id" : ObjectId("4f1e79261c0db27b00f1954c"), "author" : "joe", "created" : ISODate("2009-03-27T15:00:00Z"), "title" : "Yet another blog post", "text" : "Here is the text...", "tags" : [ "example", "joe" ], "comments" : [ 	{ 	"author" : "jim", 	"comment" : "I disagree" }, 	{ 	"author" : "nancy", 	"comment" : "Good post" } ] }

 
ちなみにrsはhelpのメソッドもあって↓こんな感じに使うんだぜ的な

SECONDARY> rs.help();
	rs.status()                     { replSetGetStatus : 1 } checks repl set status
	rs.initiate()                   { replSetInitiate : null } initiates set with default settings
	rs.initiate(cfg)                { replSetInitiate : cfg } initiates set with configuration cfg
	rs.conf()                       get the current configuration object from local.system.replset
	rs.reconfig(cfg)                updates the configuration of a running replica set with cfg (disconnects)
	rs.add(hostportstr)             add a new member to the set with default attributes (disconnects)
	rs.add(membercfgobj)            add a new member to the set with extra attributes (disconnects)
	rs.addArb(hostportstr)          add a new member which is arbiterOnly:true (disconnects)
	rs.stepDown([secs])             step down as primary (momentarily) (disconnects)
	rs.freeze(secs)                 make a node ineligible to become primary for the time specified
	rs.remove(hostportstr)          remove a host from the replica set (disconnects)
	rs.slaveOk()                    shorthand for db.getMongo().setSlaveOk()

	db.isMaster()                   check who is primary

	reconfiguration helpers disconnect from the database so the shell will display
	an error, even if the command succeeds.
	see also http://<mongod_host>:28017/_replSet for additional diagnostic info

 
↓Web画面から見てもフムフムって感じになっています。

 
これでプライマリ(27017)落としてセカンダリ(27018か27019)がプライマリになればOKです。
さっそくやってみると、、
 
・27017のログ

Tue Jan 24 19:02:36 [interruptThread] closeAllFiles() finished
Tue Jan 24 19:02:36 [interruptThread] shutdown: removing fs lock...
Tue Jan 24 19:02:36 dbexit: really exiting now

 
・27018のログ

Tue Jan 24 19:02:36 [rsSync] replSet syncThread: 10278 dbclient error communicating with server: localhost:27017
Tue Jan 24 19:02:37 [rsHealthPoll] DBClientCursor::init call() failed
Tue Jan 24 19:02:37 [rsHealthPoll] replSet info localhost:27017 is down (or slow to respond): DBClientBase::findN: transport error: localhost:27017 query: { replSetHeartbeat: "replica", v: 1, pv: 1, checkEmpty: false, from: "localhost:27018" }
Tue Jan 24 19:02:37 [rsHealthPoll] replSet member localhost:27017 is now in state DOWN
Tue Jan 24 19:02:37 [rsMgr] replSet info electSelf 1
Tue Jan 24 19:02:37 [rsMgr] replSet PRIMARY

 
・27019のログ

Tue Jan 24 19:02:36 [rsHealthPoll] replSet info localhost:27017 is down (or slow to respond): DBClientBase::findN: transport error: localhost:27017 query: { replSetHeartbeat: "replica", v: 1, pv: 1, checkEmpty: false, from: "localhost:27019" }
Tue Jan 24 19:02:36 [rsHealthPoll] replSet member localhost:27017 is now in state DOWN
Tue Jan 24 19:02:36 [rsMgr] not electing self, localhost:27018 would veto
Tue Jan 24 19:02:37 [conn172] replSet info voting yea for localhost:27018 (1)
Tue Jan 24 19:02:38 [rsHealthPoll] replSet member localhost:27018 is now in state PRIMARY

 
画面からみても↓のように狙った通り。お利口さんです。

 
27018(プライマリ)にデータを入れて、27019(セカンダリ)からデータを取得してみます。

# ./mongo localhost:27018
MongoDB shell version: 2.0.2
connecting to: localhost:27018/test
PRIMARY> db.things.save({"name":"Shinohara","age":"32"});

↓無事取れました。

# ./mongo localhost:27019
MongoDB shell version: 2.0.2
connecting to: localhost:27019/test
SECONDARY> db.things.find();
error: { "$err" : "not master and slaveok=false", "code" : 13435 }
SECONDARY> rs.slaveOk();
not master and slaveok=false
SECONDARY> db.things.find();
{ "_id" : ObjectId("4f1e837e59ff27bd08cf3187"), "name" : "Shinohara", "age" : "32" }

 
最後は落とした27017を戻してやって画面から確認したら
セカンダリとしてちゃんと戻ってくれました↓

  

確かにお手軽な感じなんだなぁってのは実際に手を動かしてみるとよくわかりますね。
 

NoSQLデータベースファーストガイド
佐々木 達也
秀和システム
売り上げランキング: 182936

シェアする

  • このエントリーをはてなブックマークに追加

フォローする

コメント

  1. […] CheckMongoDBのReplica SetsをUbuntuで試してみる | shinodogg.comで立てた MongoDBにApacheのログを突っ込みたいわけなんですが、 […]

  2. […] MongoDBをインストール MongoDBのReplica SetsをUbuntuで試してみる – shinodogg.com – (この記事を書いている時点での最新版2.2.0を使いました) […]