Norstore-trd Configuration
by
Einar Næss Jensen
—
last modified
Feb 17, 2011 01:14 PM
List of output from different GPFS-commands
Cluster nodes:
all quorum-managers and quorum nodes are directly attached to the norstore disks via fibrechannel.
the nodes
[root@norstore-trd-io0 ~]# mmlscluster GPFS cluster information ======================== GPFS cluster name: norstore-trd.hpc.ntnu.no GPFS cluster id: 9363288749587742140 GPFS UID domain: norstore-trd.hpc.ntnu.no Remote shell command: /usr/bin/ssh Remote file copy command: /usr/bin/scp GPFS cluster configuration servers: ----------------------------------- Primary server: norstore-trd-io0.hpc.ntnu.no Secondary server: norstore-trd-io1.hpc.ntnu.no Node Daemon node name IP address Admin node name Designation ----------------------------------------------------------------------------------------------- 1 norstore-trd-io0.hpc.ntnu.no 129.241.21.130 norstore-trd-io0.hpc.ntnu.no quorum-manager 2 norstore-trd-io1.hpc.ntnu.no 129.241.21.131 norstore-trd-io1.hpc.ntnu.no quorum-manager 3 norstore-trd-io2.hpc.ntnu.no 129.241.21.132 norstore-trd-io2.hpc.ntnu.no quorum-manager 4 norstore-trd-io3.hpc.ntnu.no 129.241.21.133 norstore-trd-io3.hpc.ntnu.no quorum-manager 5 norstore-trd-app0.hpc.ntnu.no 129.241.21.134 norstore-trd-app0.hpc.ntnu.no quorum 6 norstore-trd-app1.hpc.ntnu.no 129.241.21.135 norstore-trd-app1.hpc.ntnu.no quorum 7 biogw-db.hpc.ntnu.no 129.241.21.136 biogw-db.hpc.ntnu.no 8 norstore-trd-bio1.hpc.ntnu.no 129.241.21.139 norstore-trd-bio1.hpc.ntnu.no 9 genetools.hpc.ntnu.no 129.241.21.137 genetools.hpc.ntnu.no 10 norstore-trd-bio0.hpc.ntnu.no 129.241.21.138 norstore-trd-bio0.hpc.ntnu.no
Configuration of the cluster:
[root@norstore-trd-io0 ~]# mmlsconfig Configuration data for cluster norstore-trd.hpc.ntnu.no: -------------------------------------------------------- clusterName norstore-trd.hpc.ntnu.no clusterId 9363288749587742140 clusterType lc autoload yes minReleaseLevel 3.2.1.5 dmapiFileHandleSize 32 maxMBpS 1000 pagepool 256M cipherList AUTHONLY File systems in cluster norstore-trd.hpc.ntnu.no: ------------------------------------------------- /dev/gpfs_norstore_trd /dev/gpfs_replica_trd
Filesets at norstore: dynamic list
Name Status Path root Linked /gpfs/norstore_trd project_ns9004k Linked /gpfs/norstore_trd/project/noserc work Linked /gpfs/norstore_trd/work project_ns9017k Linked /gpfs/norstore_trd/project/ssb .... .... .... [root@norstore-trd-io0 ~]# mmlsfileset gpfs_replica_trd|grep -v user Filesets in file system 'gpfs_replica_trd': Name Status Path root Linked /gpfs/replica_trd stallo Linked /gpfs/replica_trd/stallo norstore_osl Linked /gpfs/replica_trd/norstore_osl
To filsystem på Norstore-trd: norstore_trd og replica_trd
[root@norstore-trd-io0 ~]# mmlsfs all File system attributes for /dev/gpfs_norstore_trd: ================================================== flag value description ---- ---------------- ----------------------------------------------------- -f 32768 Minimum fragment size in bytes -i 512 Inode size in bytes -I 32768 Indirect block size in bytes -m 2 Default number of metadata replicas -M 2 Maximum number of metadata replicas -r 1 Default number of data replicas -R 2 Maximum number of data replicas -j scatter Block allocation type -D nfs4 File locking semantics in effect -k all ACL semantics in effect -a 1048576 Estimated average file size -n 32 Estimated number of nodes that will mount file system -B 1048576 Block size -Q user;group;fileset Quotas enforced fileset Default quotas enabled -F 134217728 Maximum number of inodes -V 10.00 (3.2.0.0) File system version -u yes Support for large LUNs? -z no Is DMAPI enabled? -L 4194304 Logfile size -E yes Exact mtime mount option -S no Suppress atime mount option -K whenpossible Strict replica allocation option -P system Disk storage pools in file system -d a1l1;a1l2;a1l3;a1l4;a1l5;a1l6;a1l7;a1l8;a1l9;a1l10;a1l11;a1l12;a1l13;a1l14;a1l15;a1l16;a2l1;a2l2;a2l3;a2l4;a2l5;a2l6;a2l7;a2l8;a2l9;a2l10;a2l11;a2l12;a2l13;a2l14; -d a2l15;a2l16;a3l1;a3l2;a3l3;a3l4;a3l5;a3l6;a3l7;a3l8;a3l9;a3l10;a3l11;a3l12;a3l13;a3l14;a3l15;a3l16;a4l1;a4l2;a4l3;a4l4;a4l5;a4l6;a4l7;a4l8;a4l9;a4l10;a4l11;a4l12; -d a4l13;a4l14;a4l15;a4l16;a1l29;a2l29;a3l29;a4l29 Disks in file system -A yes Automatic mount option -o none Additional mount options -T /gpfs/norstore_trd Default mount point File system attributes for /dev/gpfs_replica_trd: ================================================= flag value description ---- ---------------- ----------------------------------------------------- -f 32768 Minimum fragment size in bytes -i 512 Inode size in bytes -I 32768 Indirect block size in bytes -m 1 Default number of metadata replicas -M 2 Maximum number of metadata replicas -r 1 Default number of data replicas -R 2 Maximum number of data replicas -j scatter Block allocation type -D nfs4 File locking semantics in effect -k all ACL semantics in effect -a 1048576 Estimated average file size -n 32 Estimated number of nodes that will mount file system -B 1048576 Block size -Q none Quotas enforced none Default quotas enabled -F 122028038 Maximum number of inodes -V 10.00 (3.2.0.0) File system version -u yes Support for large LUNs? -z no Is DMAPI enabled? -L 4194304 Logfile size -E yes Exact mtime mount option -S no Suppress atime mount option -K whenpossible Strict replica allocation option -P system Disk storage pools in file system -d a1l17;a1l18;a1l19;a1l20;a1l21;a1l22;a1l23;a1l24;a2l17;a2l18;a2l19;a2l20;a2l21;a2l22;a2l23;a2l24;a3l17;a3l18;a3l19;a3l20;a3l21;a3l22;a3l23;a3l24;a4l17;a4l18;a4l19; -d a4l20;a4l21;a4l22;a4l23;a4l24;a1l25;a2l25;a3l25;a4l25;a1l26;a2l26;a3l26;a4l26;a1l27;a2l27;a3l27;a4l27;a1l28;a2l28;a3l28;a4l28 Disks in file system -A yes Automatic mount option -o none Additional mount options -T /gpfs/replica_trd Default mount point
Disks participating in the cluster:
[root@norstore-trd-io0 ~]# mmlsdisk gpfs_norstore_trd
disk driver sector failure holds holds storage
name type size group metadata data status availability pool
------------ -------- ------ ------- -------- ----- ------------- ------------ ------------
a1l1 nsd 512 1 yes yes ready up system
a1l2 nsd 512 1 yes yes ready up system
a1l3 nsd 512 1 yes yes ready up system
a1l4 nsd 512 1 yes yes ready up system
a1l5 nsd 512 1 yes yes ready up system
a1l6 nsd 512 1 yes yes ready up system
a1l7 nsd 512 1 yes yes ready up system
a1l8 nsd 512 1 yes yes ready up system
a1l9 nsd 512 1 yes yes ready up system
a1l10 nsd 512 1 yes yes ready up system
a1l11 nsd 512 1 yes yes ready up system
a1l12 nsd 512 1 yes yes ready up system
a1l13 nsd 512 1 yes yes ready up system
a1l14 nsd 512 1 yes yes ready up system
a1l15 nsd 512 1 yes yes ready up system
a1l16 nsd 512 1 yes yes ready up system
a2l1 nsd 512 2 yes yes ready up system
a2l2 nsd 512 2 yes yes ready up system
a2l3 nsd 512 2 yes yes ready up system
a2l4 nsd 512 2 yes yes ready up system
a2l5 nsd 512 2 yes yes ready up system
a2l6 nsd 512 2 yes yes ready up system
a2l7 nsd 512 2 yes yes ready up system
a2l8 nsd 512 2 yes yes ready up system
a2l9 nsd 512 2 yes yes ready up system
a2l10 nsd 512 2 yes yes ready up system
a2l11 nsd 512 2 yes yes ready up system
a2l12 nsd 512 2 yes yes ready up system
a2l13 nsd 512 2 yes yes ready up system
a2l14 nsd 512 2 yes yes ready up system
a2l15 nsd 512 2 yes yes ready up system
a2l16 nsd 512 2 yes yes ready up system
a3l1 nsd 512 3 yes yes ready up system
a3l2 nsd 512 3 yes yes ready up system
a3l3 nsd 512 3 yes yes ready up system
a3l4 nsd 512 3 yes yes ready up system
a3l5 nsd 512 3 yes yes ready up system
a3l6 nsd 512 3 yes yes ready up system
a3l7 nsd 512 3 yes yes ready up system
a3l8 nsd 512 3 yes yes ready up system
a3l9 nsd 512 3 yes yes ready up system
a3l10 nsd 512 3 yes yes ready up system
a3l11 nsd 512 3 yes yes ready up system
a3l12 nsd 512 3 yes yes ready up system
a3l13 nsd 512 3 yes yes ready up system
a3l14 nsd 512 3 yes yes ready up system
a3l15 nsd 512 3 yes yes ready up system
a3l16 nsd 512 3 yes yes ready up system
a4l1 nsd 512 4 yes yes ready up system
a4l2 nsd 512 4 yes yes ready up system
a4l3 nsd 512 4 yes yes ready up system
a4l4 nsd 512 4 yes yes ready up system
a4l5 nsd 512 4 yes yes ready up system
a4l6 nsd 512 4 yes yes ready up system
a4l7 nsd 512 4 yes yes ready up system
a4l8 nsd 512 4 yes yes ready up system
a4l9 nsd 512 4 yes yes ready up system
a4l10 nsd 512 4 yes yes ready up system
a4l11 nsd 512 4 yes yes ready up system
a4l12 nsd 512 4 yes yes ready up system
a4l13 nsd 512 4 yes yes ready up system
a4l14 nsd 512 4 yes yes ready up system
a4l15 nsd 512 4 yes yes ready up system
a4l16 nsd 512 4 yes yes ready up system
a1l29 nsd 512 1 yes yes ready up system
a2l29 nsd 512 2 yes yes ready up system
a3l29 nsd 512 3 yes yes ready up system
a4l29 nsd 512 4 yes yes ready up system
[root@norstore-trd-io0 ~]# mmlsdisk gpfs_replica_trd
disk driver sector failure holds holds storage
name type size group metadata data status availability pool
------------ -------- ------ ------- -------- ----- ------------- ------------ ------------
a1l17 nsd 512 1 yes yes ready up system
a1l18 nsd 512 1 yes yes ready up system
a1l19 nsd 512 1 yes yes ready up system
a1l20 nsd 512 1 yes yes ready up system
a1l21 nsd 512 1 yes yes ready up system
a1l22 nsd 512 1 yes yes ready up system
a1l23 nsd 512 1 yes yes ready up system
a1l24 nsd 512 1 yes yes ready up system
a2l17 nsd 512 2 yes yes ready up system
a2l18 nsd 512 2 yes yes ready up system
a2l19 nsd 512 2 yes yes ready up system
a2l20 nsd 512 2 yes yes ready up system
a2l21 nsd 512 2 yes yes ready up system
a2l22 nsd 512 2 yes yes ready up system
a2l23 nsd 512 2 yes yes ready up system
a2l24 nsd 512 2 yes yes ready up system
a3l17 nsd 512 3 yes yes ready up system
a3l18 nsd 512 3 yes yes ready up system
a3l19 nsd 512 3 yes yes ready up system
a3l20 nsd 512 3 yes yes ready up system
a3l21 nsd 512 3 yes yes ready up system
a3l22 nsd 512 3 yes yes ready up system
a3l23 nsd 512 3 yes yes ready up system
a3l24 nsd 512 3 yes yes ready up system
a4l17 nsd 512 4 yes yes ready up system
a4l18 nsd 512 4 yes yes ready up system
a4l19 nsd 512 4 yes yes ready up system
a4l20 nsd 512 4 yes yes ready up system
a4l21 nsd 512 4 yes yes ready up system
a4l22 nsd 512 4 yes yes ready up system
a4l23 nsd 512 4 yes yes ready up system
a4l24 nsd 512 4 yes yes ready up system
a1l25 nsd 512 1 yes yes ready up system
a2l25 nsd 512 2 yes yes ready up system
a3l25 nsd 512 3 yes yes ready up system
a4l25 nsd 512 4 yes yes ready up system
a1l26 nsd 512 1 yes yes ready up system
a2l26 nsd 512 2 yes yes ready up system
a3l26 nsd 512 3 yes yes ready up system
a4l26 nsd 512 4 yes yes ready up system
a1l27 nsd 512 1 yes yes ready up system
a2l27 nsd 512 2 yes yes ready up system
a3l27 nsd 512 3 yes yes ready up system
a4l27 nsd 512 4 yes yes ready up system
a1l28 nsd 512 1 yes yes ready up system
a2l28 nsd 512 2 yes yes ready up system
a3l28 nsd 512 3 yes yes ready up system
a4l28 nsd 512 4 yes yes ready up system
