1、实时重建索引
在实际的生产环境中，一个field的设置是不能被修改的，如果要修改一个Field，那么应该重新按照新的mapping，建立一个index，然后将数据批量查询出来，重新用bulk api写入index中。
批量查询的时候，建议采用scroll api，并且采用多线程并发的方式来reindex数据。例如说每次scoll就查询指定日期的一段数据，交给一个线程即可。
PUT /my_index/_doc/1
{
"title": "2019-09-10"
}
PUT /my_index/_doc/2
{
"title": "2019-09-11"
}（2）当后期向索引中加入string类型的title值的时候，就会报错
PUT /my_index/_doc/3
{
"title": "my first article"
}报错
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [title] of type [date] in document with id '3'. Preview of field's value: 'my first article'"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [title] of type [date] in document with id '3'. Preview of field's value: 'my first article'",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "failed to parse date field [my first article] with format [strict_date_optional_time||epoch_millis]",
"caused_by": {
"type": "date_time_parse_exception",
"reason": "Failed to parse with all enclosed parsers"
}
}
},
"status": 400
}（3）如果此时想修改title的类型，是不可能的
PUT /my_index/_mapping
{
"properties": {
"title": {
"type": "text"
}
}
}报错
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "mapper [title] of different type, current_type [date], merged_type [text]"
}
],
"type": "illegal_argument_exception",
"reason": "mapper [title] of different type, current_type [date], merged_type [text]"
},
"status": 400
}（4）此时，唯一的办法，就是进行reindex，也就是说，重新建立一个索引，将旧索引的数据查询出来，再导入新索引。
（5）如果说旧索引的名字，是old_index，新索引的名字是new_index，终端java应用，已经在使用old_index在操作了，难道还要去停止java应用，修改使用的index为new_index，才重新启动java应用吗？这个过程中，就会导致java应用停机，可用性降低。
（6）所以说，给java应用一个别名，这个别名是指向旧索引的，java应用先用着，java应用先用prod_index来操作，此时实际指向的是旧的my_index
PUT /my_index/_alias/prod_index（7）查看别名，会发现my_index已经存在一个别名prod_index了。
GET my_index/_alias（8）新建一个index，调整其title的类型为string
PUT /my_index_new
{
"mappings": {
"properties": {
"title": {
"type": "text"
}
}
}
}（9）使用scroll api将数据批量查询出来
GET /my_index/_search?scroll=1m
{
"query": {
"match_all": {}
},
"size": 1
}返回
{
"_scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAARUMWQWx5bzRmTW9TeUNpNmVvN0E2dF9YQQ==",
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"title" : "2019-09-10"
}
}
]
}
}
（9）采用bulk api将scoll查出来的一批数据，批量写入新索引
POST /_bulk
{"index":{"_index":"my_index_new","_id":"1"}}
{"title":"2019-09-10"}（10）反复循环8~9，查询一批又一批的数据出来，采取bulk api将每一批数据批量写入新索引
（11）将my_index索引的别名prod_index切换到my_index_new上去，java应用会直接通过index别名使用新的索引中的数据，java应用程序不需要停机，零提交，高可用
POST /_aliases
{
"actions": [
{
"remove": {
"index": "my_index",
"alias": "prod_index"
}
},
{
"add": {
"index": "my_index_new",
"alias": "prod_index"
}
}
]
}（12）直接通过prod_index别名来查询，是否ok
GET prod_index/_search可以看到能够查询到新索引my_index_new的数据了
{
"took" : 1117,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "my_index_new",
"_type" : "_doc",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"title" : "2019-09-10"
}
}
]
}
}
2、总结：
基于alias对client透明切换index
PUT /my_index_v1/_alias/my_indexclient对my_index进行操作
reindex操作，完成之后，切换v1到v2
POST /_aliases
{
"actions": [
{ "remove": { "index": "my_index_v1", "alias": "my_index" }},
{ "add": { "index": "my_index_v2", "alias": "my_index" }}
]
}
You will see an “NFS mounts” window as follows:
now that’s a working tutorial! been struggling with my Zyxel NSA220, trying to use NFS instead of SMB – couldn’t mount until found this page 🙂
Wow,
Vivek this is a great tutorial. I as well was struggling and as soon as I found this page I nailed it in one try. Thanks jaysunn
This doesn’t describe how to set up persistent NFS mounts from the command line (hard mounts, not automounts). That should be easy but I can’t find it anywhere.
Do you want to know about soft / hard option that determines the recovery behavior of the NFS client after an NFS request times out? May be you need to give us more info..
you have to fire this command for mout NFS shared folder to MAC OX
sudo mount_nfs -P :
Great job……thanks 🙂
I followed every step(gui and terminal) word for word and I can see files on my nfs server, but I can’t write or modify the contents of the shared directory. It keeps saying “Permission Denied”. Is there anything particular I need to do on the server side?
You need to make sure server is exporting directories in a rw (read write) and not in a ro (read only) mode.
If have tried exporting it as rw as seen below, but still no avail 🙁
/mnt/shared 192.168.0.0/255.255.0.0(rw,sync,all_squash,anonuid=502,anongid=502,insecure)
Same problem here.
ditto Lion 10.7.4 rw enabled in /etc/exports
Have a NFS server running on Debian 5.
I can mounting my shares on Linux Mint 11 like a charm (with r/w permissions).
But under os 10.6.8, i can mount these shares but i haven’t permission to write or modify anything -> “Permission Denied”.
The NFS server is setup with these options in /etc/exports
/mnt/shared 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)
Any idea ?
Thanks in advance
In fact, i can create and modify files via terminal if i use sudo command or logged as root.
All folders/files are uid=1000 and gid=1000 like the screen capture of this article.
You need to add an account on Debian server. For example if you login as fsimerey on OS X. Make fsimerey account on the Debian too. This will map UID and GIDs.
Hi from France,
Great faqs, thank !
I have the same question as fsimerey.
I can access to my nfs server, but not with my single user account under mac Osx. I need to use SUDO into the terminal to access it.
From my others PC, under Ubuntu, single users don’t needs to use root account to access the nfs.
My nfs server exports : rw, sync, no_root_squash, anonuid=1000,anongid=1000
Can you help me please ?
Thank a lot
Worked first time using Fedora 15 and latest Mac OS X. Many thanks!!
got it
Same problem as fsimerey and sx1! Anyone now a fix?
Most modern NFS server need some sort of user authentication and user id mapping. For example, /home/vivek is owned by vivek user id # 501 on the Debian Linux nfs 4 server.
You need to add a user called vivek to Apple OS x with user id # 501 using the following command:
See this page for more info. Use dscl command with care as it can destroy existing users and other system properties.
Thanks for the article.
Now I have a NFS server running on Ubuntu 11.10 exporting a 2TB exFAT external HD drive. Had to install fuse-exfat to be able to mount the HD on Ubuntu.
I mounted a NFS folder on a Mac OSX using: sudo mount -t nfs -o resvport my_nfs_server:/media/HD2TB /private/nfs and from Finder I can navigate through and create folders and even open large files. The problem is that I can’t copy anything to the NFS (neither from Finder nor from Terminal). Here is the error:
cp: /private/nfs/test.txt: fchmod failed: Input/output error
Here is my /etc/exports:
/media/HD2TB my_mac(rw,no_root_squash,no_subtree_check)
Thanks for helping.
For my nfs exports on Debian, i used insecure option.
Try with:
/media/HD2TB my_mac(rw,no_root_squash,no_subtree_check,insecure)
Hope that would solve your prob.
Thanks for the tutorial. It fit the trick for me!
running ox 10.8 and in disk utility. Under file menu mount is grayed out and there is no nfs mount in menu.
Vivek – Brilliant! Was annoyed with my Synology 1512+ shares with AFP going offline and having to remount for my Plex (MMS). This was the solution! I used your tutorial (which was spot on) and a GUI from NFS Manager(now that OSX 10.8 Disk Utility doesn’t have NFS mounting options) and was successful. I’m super happy now with automounts and dismounts for all my shares! Great, great job!!!!
p.s. With OSX 10.8 not including the Disk Utility bit, you may want to update/mention that for future users.
After a solid week of having permission denied messages etc.. out of Mountain Lion, here is the fix…..
Stolen from Apple Discussions:
Hi!
Few days ago i run into same problem. I have Ubuntu server and want to mount some folders from there… Anyway, I was solved this problem this way:
— on Ubuntu —
1. edit /etc/exports (sudo vim /etc/exports) and add this line(s)
—
where “/folder” is a folder you want to mount from your Mac
“0.0.0.0” is an IP address you allow to connect to your share from
“anonuid=xxx,anongid=xxx” is UID and GID of user on Ubuntu you want to be used as owner of files you want to create on mounted share (if you ommit this you will have read-only access to the share folder)
2. export your shares – sudo exportfs -a
———————–
— on your Mac —
Use Connect to Server (Commant+K) menu and write nfs://server-name-or-ip/folder
————
Voila! You have to have rw access to shared folder on your server.
P.S. check man exports to find some additional info to set IP ranges you want to allow to connect from.
Good luck!
Dear Helper,
There are two major flaws with the mount instructions above:
1. The entry “NFS mounts” in the Disk Utility does NOT exist in MAC OS X 10.8!
2. It is easy to NFS-mount disks read only in 10.8 but much harder to mount them
read-write. As you can see above the parameter “-o rw” is missing in the
examples above!
Thanks anyway for the nice tutorial. Have a good day, vonbarth
Thanks for the heads up! The faq has been updated.
Vivek — there is a problem accessing a “normal” nfs server from osx if the mount option “-o resvport” is used on the osx client. Most/normal nfs servers are firewalled; opening port 2049 for nfs connections. Osx is going to pick a random low-numbered port to connect on and this will be blocked on the nfs server.
Is there some sort of solution for this? Cheers
I has no problem at all setting up a NFS client for my macOsx 10.8…. The only thing is that diskutil does not support it but terminal does!
The problem is the dynamic port the mac client try to access. For you who have a Linux nfs server, this is probably is off interests.
I’ve setup a freeipa server on centos mostly following theses guidelines:
http://linsec.ca/Using_FreeIPA_for_User_Authentication#Mac_OS_X_10.7.2F10.8
Everything seems to be working fine, except that the osx client can’t automount the /home share at startup.
I can auth and/or mount the share manually in the command line, but at startup the logs show “server not responding”.
This is the line I’ve added in auto_home (referred to in auto_master):
* -fstype=nfs,rw,resvport,soft,intr,rsize=8192,wsize=8192,nosuid,tcp nfsserver:/home/&
I have tried many options and variations of this command, but have been unsuccessful so far – does anyone have an idea how this can be done ?
It was a firewall issue – its working now
What about reverse situation? I want to share dir from my mac to linux machine.
Mac has ip 192.168.0.26 and linux has 192.168.99.100
In /etc/exports on mac I have this:
/Users -mapall=my_user -alldirs -network 192.168.99.0 -mask 255.255.255.0
Then on linux I do this:
sudo mount 192.168.0.26:/Users /Users -o rw,async,noatime,rsize=32768,wsize=32768,p
roto=tcp
And it says:
mount: 192.168.0.26:/Users failed, reason given by server: Permission denied
mount: mounting 192.168.0.26:/Users on /Users failed: Bad file descriptor
my_user belongs to admin group. Firewall is disabled at all. What the problem can be?
Thanks for your article
how could you configure your export for mounting without option resvport?
thanks
Worked perfectly for me from an Ubuntu host to a macOS Sierra guest, thank you!
those running into ‘permission denied’ for writing into the mount- do not use sudo to mount on the mac side. If you use sudo – you will mount the dir as root and you will have to be ‘sudo’ to make changes. Just do mount, then you will mount as the user and not as root.
On Mac, running 11.1 OS, I got an error nfs can’t mount with remote locks … rpc.statd. On cmd line I ran this:
sudo mount -o nolocks -t nfs …
and it worked