TIPS from Oracle ASM disk owner

issue: Linux disk owner can not be changed simply such as chown oracle:oinstall /dev/sdb1

shortcut:

[root@node1 ~]# cat /etc/udev/rules.d/52-oracle.permissions.rules

# OCR disks

KERNEL=="sda1",OWNER="grid" GROUP="asmdba",MODE="0660"

# UNDO disks

KERNEL=="sdb1",OWNER="grid" GROUP="asmdba",MODE="0660"

# DATA disks

KERNEL=="sdc1",OWNER="grid" GROUP="asmdba",MODE="0660"

from article:

http://download.oracle.com/docs/cd/E11882_01/install.112/e22489/storage.htm#CDEBFDEH

By default, the 2.6 kernel device file naming scheme udev dynamically creates device file names when the server is started, and assigns ownership of them to root. If udev applies default settings, then it changes device file names and owners for voting disks or Oracle Cluster Registry partitions, corrupting them when the server is restarted. For example, a voting disk on a device named /dev/sdd owned by the user grid may be on a device named /dev/sdf owned by root after restarting the server.If you use ASMLIB, then you do not need to ensure permissions and device path persistency in udev.

If you do not use ASMLIB, then you must create a custom rules file. When udev is started, it sequentially carries out rules (configuration directives) defined in rules files. These files are in the path /etc/udev/rules.d/. Rules files are read in lexical order. For example, rules in the file 10-wacom.rules are parsed and carried out before rules in the rules file 90-ib.rules.

When specifying the device information in the UDEV rules file, ensure that the OWNER, GROUP and MODE are specified before any other characteristics in the order shown. For example, if you want to include the characteristic ACTION on the UDEV line, then specify ACTION after OWNER, GROUP, and MODE.

Where rules files describe the same devices, on the supported Linux kernel versions, the last file read is the one that is applied.

To configure a permissions file for disk devices, complete the following tasks:

1. To obtain information about existing block devices, run the command scsi_id (/sbin/scsi_id) on storage devices from one cluster node to obtain their unique device identifiers. When running the scsi_id command with the -s argument, the device path and name passed should be that relative to the sysfs directory /sys (for example, /block/device) when referring to /sys/block/device. For example:

   # /sbin/scsi_id -g -s /block/sdb/sdb1 360a98000686f6959684a453333524174   # /sbin/scsi_id -g -s /block/sde/sde1 360a98000686f6959684a453333524179 

   Record the unique SCSI identifiers of clusterware devices, so you can provide them when required.

   Note:

   The command scsi_id should return the same device identifier value for a given device, regardless of which node the command is run from.

2. Configure SCSI devices as trusted devices (white listed), by editing the /etc/scsi_id.config file and adding options=-g to the file. For example:

   # cat > /etc/scsi_id.config vendor="ATA",options=-p 0x80 options=-g 

3. Using a text editor, create a UDEV rules file for the Oracle ASM devices, setting permissions to 0660 for the installation owner and the group whose members are administrators of the Oracle Grid Infrastructure software. For example, using the installation owner grid and using a role-based group configuration, with the OSASM group asmadmin:

   # vi /etc/udev/rules.d/99-oracle-asmdevices.rules  KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="14f70656e66696c00000000", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?2", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="14f70656e66696c00000001", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?3", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="14f70656e66696c00000002", OWNER="grid", GROUP="asmadmin", MODE="0660" 

4. Copy the rules.d file to all other nodes on the cluster. For example:

   # scp 99-oracle-asmdevices.rules root@node2:/etc/udev/rules.d/99-oracle-asmdevices.rules 

5. Load updated block device partition tables on all member nodes of the cluster, using /sbin/partprobe devicename. You must do this as root.

6. Run the command udevtest (/sbin/udevtest) to test the UDEV rules configuration you have created. The output should indicate that the block devices are available and the rules are applied as expected. For example:

   # udevtest /block/sdb/sdb1 main: looking at device '/block/sdb/sdb1' from subsystem 'block' udev_rules_get_name: add symlink 'disk/by-id/scsi-360a98000686f6959684a453333524174-part1' udev_rules_get_name: add symlink 'disk/by-path/ip-192.168.1.1:3260-iscsi-iqn.1992-08.com.netapp:sn.887085-part1' udev_node_mknod: preserve file '/dev/.tmp-8-17', because it has correct dev_t run_program: '/lib/udev/vol_id --export /dev/.tmp-8-17' run_program: '/lib/udev/vol_id' returned with status 4 run_program: '/sbin/scsi_id' run_program: '/sbin/scsi_id' (stdout) '360a98000686f6959684a453333524174' run_program: '/sbin/scsi_id' returned with status 0 udev_rules_get_name: rule applied, 'sdb1' becomes 'ocr1' udev_device_event: device '/block/sdb/sdb1' validate currently present symlinks udev_node_add: creating device node '/dev/ocr1', major = '8', minor = '17',  mode = '0640', uid = '0', gid = '500' udev_node_add: creating symlink '/dev/disk/by-id/scsi-360a98000686f6959684a453333524174-part1' to '../../ocr1' udev_node_add: creating symlink '/dev/disk/by-path/ip-192.168.1.1:3260-iscsi-iqn.1992-08.com.netapp:sn.84187085 -part1' to '../../ocr1' main: run: 'socket:/org/kernel/udev/monitor' main: run: '/lib/udev/udev_run_devd' main: run: 'socket:/org/freedesktop/hal/udev_event' main: run: '/sbin/pam_console_apply /dev/ocr1 /dev/disk/by-id/scsi-360a98000686f6959684a453333524174-part1 /dev/disk/by-path/ip-192.168.1.1:3260-iscsi-iqn.1992-08.com.netapp:sn.84187085- part1' 

   In the example output, note that applying the rules renames OCR device /dev/sdb1 to /dev/ocr1.

7. Enter the command to restart the UDEV service.

   On Asianux, Oracle Linux 5, and RHEL5, the commands are:

   # /sbin/udevcontrol reload_rules # /sbin/start_udev 

   On SUSE 10 and 11, the command is:

   # /etc/init.d boot.udev restart