Tutorial on DiskSuite
Solstice DiskSuite 4.2.1
General Info:
- Requires Solaris 8 to run. Also ships with Solaris8. Located on S/W CD2 at:
/cdrom/cdrom0/Solaris_8/EA/products/DiskSuite_4.2.1
- Incompatible with previous versions of DiskSuite. If upgrading => have to reconfigure.
- DiskSuite / Solaris compatibility matrix:
- Solaris 2.5.1 -> DS 4.0, 4.1
- Solaris 2.6 -> DS 4.1, 4.2
- Solaris 7 -> DS 4.2
- Solaris 8 -> DS 4.2.1
Installation:
1. Mount Solaris 8 S/W CD2 (if remote: put "share -F nfs -o ro -d "CDROM" /cdrom/cdrom0" in /etc/dfs/dfstab; shareall; then mount from the remote location)
2. cd /cdrom/cdrom0/Solaris_8/EA/products/DiskSuite_4.2.1/i386/Packages
3. pkgadd -d .
- Administrate DiskSuite either via the DiskSuite tool (metatool) or command line.
Terms:
- metadevice (simple, mirror, RAID5, trans) - a group of physical devices that appears as a single logical device.
- metadevice state database - contains vital information - has multiple replicas - all exist on a separate slice or a slice used in a metadevice.
- hot spare pool - collection of hot spares waiting to be used in case of a failure.
Config files:
- /etc/lvm/mddb.cf - state database replicas
- /etc/lvm/md.tab - input file for "metainit", "metadb", and "metahs"
- /etc/lvm/md.cf - a backup of the current configuration - useful when recovering from a crash.
- /kernel/drv/md.conf - number of metadevices and disksets
- /etc/lvm/mdlogd.cf - SNMP trap generating daemon
- /etc/rcS.d/S35lvm.init - metadevice configuration at boot
- /etc/rc2.d/S95lvm.sync - automatic resyncing of metadevices
/etc/lvm/md.tab examples:
- 3 state database replicas on each slice:
mddb01 -c 3 c0t1d0s0 c0t2d0s0 c0t3d0s0
- single stripe consisting of 2 disks, with 32k interlace value (default is 16k):
d15 1 2 c0t1d0s2 c0t2d0s2 -i 32k
- concatenation of 4 stripes, each consisting of 1 disk:
d7 4 1 c0t1d0s0 1 c0t2d0s0 1 c0t3d0s0 1 c0t4d0s0
- concatenation of 2 stripes, each made of 3 disks:
d75 2 3 c0t1d0s2 c0t2d0s2 c0t3d0s2 -i 16k 3 c1t1d0s2 c1t2d0s2 c1t3d0s2 -i 32k
- mirror (d50) containing 1 submirror (d51). The other 2 mirrors must be attached later with "metattach"
d50 -m d51
d51 1 1 c0t1d0s2
d52 1 1 c0t2d0s2
d53 1 1 c0t3d0s2
- trans metadevice d1, containing a master device d10 and a logging device d20 (both mirrors).
Mirrors d12 and d22 are attached later via "metattach"
d1 -t d10 d20
d10 -m d11
d11 1 1 c0t1d0s2
d12 1 1 c0t2d0s2
d20 -m d21
d21 1 1 c1t1d0s2
d22 1 1 c1t2d0s2
- RAID5 metadevice of 3 slices, using an interlace of 20K:
d80 -r c0t1d0s1 c1t0d0s1 c2t0d0s1 -i 20k
- hot spare pool -- 3 hot spare pools are associated to 3 mirrors:
d10 -m d20
d20 1 1 c1t0d0s2 -h hsp001
d30 1 1 c2t0d0s2 -h hsp002
d40 1 1 c3t0d0s2 -h hsp003
hsp001 c2t2d0s2 c3t2d0s2 c1t2d0s2
hsp002 c3t2d0s2 c1t2d0s2 c2t2d0s2
hsp003 c1t2d0s2 c2t2d0s2 c3t2d0s2
The following general section will also be featured as a separate RAID topic, as it contains valuable advice when selecting a particular RAID level.
General configuration notes:
1. Striping has the best performance, but offers no data protection.
2. For write intensive applications, mirroring has better performance than RAID5.
3. Mirroring and RAID5 both increase data availability, and both decrease writing performance.
4. Mirroring improves random read performance.
5. RAID5 has lower cost than mirroring. Stripes/concatenations have no additional cost.
Concatenation notes:
1. Concatenation uses less CPU time than striping.
2. Concatenation works well for small random I/O.
3. Avoid using physical disks with different geometries.
4. Distribute slices across different controllers and busses to help balance the I/O load.
Striping (RAID0) notes:
1. Set the stripe's interlace value correctly.
2. The more physical disks in a stripe, the greater the I/O performance, and the lower the MTBF (mean time b/n failures).
3. Don't mix differently sized slices, as a stripe's size is limited by its smallest slice.
4. Avoid using physical disks with different geometries.
5. Distribute the stiped metadevice across different controllers and busses.
6. Striping cannot be used to encapsulate existing filesystems.
7. Striping performs well for large sequential I/O and for random I/O distributions.
8. Striping uses more CPU cycles than concatenation, but it is usually worth it.
9. Striping does not provide any redundancy of data.
Mirroring (RAID1) notes:
1. Mirroring may improve read performance; write performance is always degraded.
2. Mirroring improves read performance only in multi-threaded or asynchronous I/O situations.
3. Mirroring degrades write performance by about 15-50 percent, as it has to write everything twice.
4. Using filesystem cache may turn a 80/20 read/write situation to 60/40 or even 40/60.
RAID5 notes:
1. RAID5 can withstand only a single device failure (mirroring MAY withstand several; striping and concatenation leave no room for that).
2. RAID5 provides good read performance under no errors, and poor read performance under error conditions.
3. RAID5 can cause poor write performance -- up to 70 percent degradation (as parity has to be calculated on the fly).
4. RAID5 is much cheaper than mirroring. Amount of disks needed for parity = 1 / total_#_disks.
5. RAID5 can NOT be used for existing filesystems. A backup and restore will be necessary.
Logging device notes:
1. Place them on an unused disk, preferrably around the middle (to minimize the average seek).
2. The log device and the master device of the same trans metadevice should be located on different drives/controllers to balance the I/O load.
3. Trans metadevices can share logs. This is not recommended for heavily used filesystems.
4. Absolute minimum log size is 1 MB. Good average is 1 MB per 100 MB. Recommended minimum is 1 MB per 1 GB.
5. All logs should be mirrored to avoid filesystem problems and/or data loss.
Filesystem notes:
1. Create new filesystems with "newfs -i 8192" -- 1 inode per 8K (default is 1 inode per 2K).
2. For large metadevices (>8G), increase the size of a cylinder group (max is 256) -- "newfs -c 256".
The cluster size should be equial to an integral of the stripe width:
maxcontig = 16 (16*8 Kbyte blocks = 128 Kbyte clusters)
Using a four-way stripe with 32K interlace results in 128K stripe width, which is good in this case.
interlace size = 32K(32K stripe unit size * 4 disks = 128K stripe width)
General section ends here.
State database replica notes:
1. All replicas are written when the configuration changes.
2. Only two replicas (per mirror) are updated for mirror dirty region bitmaps.
3. A good average is two replicas per three mirrors.
4. Use two replicas per one mirror for write intensive applications.
5. Use two replicas per 10 mirrors for read intensive applications.
6. 1 drive => 3 replicas on one slice, as the minimum number of replicas is 3.
7. 2-4 drives => 2 replicas on each drive.
8. 5+ drives => 1 replica on each drive.
9. Each state database replica occupies 517K (1034 disk sectors).
10. Replicas can be stored on a dedicated slice or on one that will be used in a metadevice.
11. The system will run with half of the replicas and will reboot with half+1.
CREATING DISKSUITE OBJECTS:
Create additional state database replicas:
- adding a state database replica:
metadb -a c0t2d0s0 <-- add it
metadb <-- verify it
- adding two state database replicas to the same slice:
metadb -a -c 2 c0t2d0s0
metadb
Create a striped metadevice:
!!! A striped metadevice CANNOT be created on a filesystem containing data.
- two slices, 32K interlace:
metainit d10 1 2 c0t1d0s2 c0t2d0s2 -i 32k
- three slices:
metainit d20 1 3 c0t1d0s2 c0t2d0s2 c0t3d0s2
Create a concatenation:
- two slices:
metainit d25 2 1 c0t1d0s2 1 c0t2d0s2
- four slices:
metainit d40 4 1 c0t1d0s2 1 c0t2d0s2 1 c0t2d0s3 1 c0t2d1s3
Create a mirror from unused slices:
- two-way mirror:
metainit d51 1 1 c0t0d0s2
metainit d52 1 1 c1t0d0s2
metainit d50 -m d51
metattach d50 d52
Create a mirror from a filesystem that can be unmounted:
- two-way mirror:
metainit -f d1 1 1 c1t0d0s0 <-- c1t0d0s0 contains /master
metainit d2 1 1 c2t0d0s0
metainit d0 -m d1
umount /master
(Edit the /etc/vfstab file so that the file system references the mirror)
mount /master
metattach d0 d2
Create a mirror from a filesystem that CANNOT be unmounted:
- mirror from /usr:
metainit -f d12 1 1 c0t3d0s6 <-- d12 contains /usr
metainit d22 1 1 c1t0d0s6metainit d2 -m d12
(Edit the /etc/vfstab file so that /usr references the mirror)
reboot
metattach d2 d22
- mirror from swap:
metainit -f d11 1 1 c0t0d0s1 <-- d11 contains swap
metainit d21 1 1 c1t0d0s1
metainit d1 -m d11
(Edit the /etc/vfstab file so that swap references the mirror)
reboot
metattach d1 d21
Create a mirror from root (/):
- SPARC:
metainit -f d11 1 1 c0t3d0s0 <-- d11 contains root (/)
metainit d12 1 1 c1t3d0s0 <-- d12 >= d11
metainit d10 -m d11
metaroot d10 <-- fix /etc/vfstab and /etc/system
lockfs -fa <-- good idea to do this before reboot
reboot
metattach d10 d12
ls -l /dev/rdsk/c1t3d0s0 <-- look for alternate root device, in case it is needed.
- x86 (SCSI only -- cannot mirror IDE drives):
fdisk /dev/rdsk/c0t1d0s0 <-- create a partition
format <-- create a slice for root (/)
(select disk; partition; 0; root, wm, 4, 400mb; label; quit)
!!! the ID tag for the new partition MUST be "root".
the size should be greater or equal to the size of the original root partition.
It should NOT use cylinders 0-2.
installboot /usr/lib/fs/ufs/pboot /usr/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0
metainit -f d10 1 1 c0t0d0s0
metainit d20 1 1 c1t0d0s0
metainit d0 -m d10
metaroot d0
lockfs -fa
reboot
metattach d0 d20
ls -l /dev/rdsk/c1t0d0s0
Create a RAID5 metadevice:
- 3 slices:
metainit d45 -r c2t3d0s2 c3t0d0s2 c4t0d0s2
Create a trans metadevice for a filesystem that can be unmounted:
- slice for master:
umount /home1
metainit d63 -t c0t2d0s2 c2t2d0s1
(Edit the /etc/vfstab file so that the file system references the trans metadevice)
mount /home1
Note: On subsequent reboots, fsck will show the following: "/dev/md/rdsk/d63: is logging"
- stripe for master:
umount /home2
metainit d40 -t d2 c1t2d0s0
(Edit the /etc/vfstab file so that the file system references the trans metadevice)
mount /home2
Create a trans metadevice for a filesystem that CANNOT be unmounted:
- trans for /usr:
metainit -f d20 -t c0t3d0s6 c1t2d0s1 <-- c0t3d0s6 contains /usr
(Edit the /etc/vfstab file so that the file system references the trans metadevice)
reboot
Create a trans metadevice using mirrors:
- d30 is master device, d12 is logging device:
umount /home1
metainit d64 -t d30 d12
(Edit the /etc/vfstab file so that the file system references the trans metadevice)
mount /home1
Create a hot spare pool:
- 2 hot spares:
metainit hsp001 c2t2d0s2 c3t2d0s2
- add a hot spare to ONE hot spare pool:
metahs -a hsp001 /dev/dsk/c3t0d0s2
- add a hot spare to ALL hot spare pools:
metahs -a -all /dev/dsk/c3t0d0s2
- change the hot spare pool association:
metastat d4 <-- check association
metaparam -h hsp002 d4 <-- change
metastat d4 <-- verify
DISKSETS:
Initial procedure:
1. Drive device names have to be the same on each host in the diskset.
2. Connect drives to be share on both hosts.
3. Configure local metadevice state database replica on each host.
4. Add hosts to a diskset (this creates the diskset).
5. Add drives to the diskset.
- 2 disksets:
metaset -s relo-red -a -h red blue <-- create diskset 1
metaset -s relo-blue -a -h red blue <-- create diskset 2
metaset <-- verify
Note: At this point, neither set has an owner. The hosts that adds disks will become the owner by default.
Add drives to a diskset:
Note: The drive about to be added must NOT be uses in a metadevice, must NOT contain a replica, and must NOT be mounted.
!!! Adding a disk destroys the data on it!
- adding drives:
metaset -s relo-red -a c1t2d0 c1t3d0 c2t2d0 c2t3d0 c2t4d0 c2t5d0 <-- add
metaset <-- verify
Create a mirror in a diskset:
- mirror d50:
metainit -s relo-red d51 1 1 /dev/dsk/c0t0d0s2
metainit -s relo-red d52 1 1 /dev/dsk/c1t0d0s2
metainit -s relo-red d50 -m d51
metattach -s relo-red d50 d52
Create a filesystem on a metadevice:
- on a concatenation:
newfs /dev/md/rdsk/d3
fsck /dev/md/rdsk/d3
- on a concatenation (d33) within a diskset (relo-red):
newfs /dev/md/relo-red/rdsk/d33
MAINTAINING DISKSUITE OBJECTS:
Checking the status of state database replicas:
- (uppercase letters indicate a problem, lowercase letters indicate an OK status)
metadb -i
Checking the status of metadevices and hot spare pools:
- stripe/concatenation => no report, unless it is used as a submirror.
- everything else:
metastat (for a hot spare pool, it is "metastat hsp001")
Checking the status of a diskset:
- specified:
metaset -s relo-red
- all:
metaset
Recreate a stripe/concatenation after slice failure:
- concatenation has a failed slice => error on console: "WARNING: md d35: read error on /dev/dsk/c0t0d0s6":
umount /news
init 0
...
boot -s
...
ufsdump 0ucf /dev/rmt/0 /news
metaclear d35
metainit d35 2 1 c1t0d0s2 1 c1t0d1s2
newfs /dev/md/rdsk/d35
mount /dev/md/dsk/d35 /news
cd /news
ufsrestore rvf /dev/rmt 0
rm restoresymtable
ls /news
Note: If stripe => new slice = failed slice; If concatenation => new slice >= failed slice.
Enable a slice in a submirror:
- mirror d11 contains a slice c1t4d0s7 which had an error but is now ready to be enabled again:
metareplace -e d11 c1t4d0s7
Replace a slice in a submirror:
- mirror d6 has a submirror d26, with a slice c0t2d0s2 in the "Needs maintenance" mode.
metastat d6 <-- check state
metareplace d6 c0t2d0s2 c0t2d2s2 <-- replace failed slice
Replace a submirror in a mirror:
- mirror d20 has a submirror d22 in the "Needs maintenance state". It will be recreated from new slices:
metastat d20
metadetach -f d20 d22
metaclear d22
metainit d22 2 1 c1t0d0s2 1 c1t0d1s2
metattach d20 d22
Enable a slice in RAID5:
- RAID5 metadevice d20 has a slice, c2t0d0s2, which had a soft error. It is now ready to go back in:
metareplace -e d20 c2t0d0s2
Replace a hot spare in a hot spare pool:
- replace in one hot spare pool:
metastat hsp003
metahs -r hsp003 c0t2d0s2 c3t1d0s2
- replace in all associated hot spare pools:
metahs -r all c1t0d0s2 c3t1d0s2
Enable a hot spare:
- do not need to specify a hot spare pool:
metahs -e c0t0d0s2
Recover a trans metadevice with a file system panic:
fsck /dev/md/rdsk/trans
Recover a trans metadevice with hard errors:
- High level steps:
lockfs <-- affected filesystems will be listed with a lock type of "hard"
- unmount affected filesystem(s) -- if a shared logging device, multiple filesystems will be affected.
- back up any accessible data
- fix the device error
- repair the filesystem
metastat d5
fsck /dev/md/rdsk/d5
metastat d5
Expand a slice containing existing data:
- expanding by concatenation:
umount /docs <-- contained in c0t1d0s2
metainit d25 2 1 c0t1d0s2 1 c0t2d0s2
(Edit the /etc/vfstab file so that the file system references the metadevice d25)
mount /docs
Note: The first slice in the "metainit" command must be the slice with data. Otherwise, data will be erased.
Expand an existing stripe:
- concatenate a single slice:
metattach d2 c1t2d0s2
- add the same number of slices in the existing metadevice:
metattach d25 c1t2d0s2 c1t2d1s2 c1t2d3s2 <-- d25 initially had 3 slices
- create a concatenated stripe from scratch:
metainit d1 3 2 c0t0d0s2 c1t0d0s2 -i 16k 2 c1t2d0s2 c1t2d1s0 -i 32k 2 c2t0d0s2 c2t0d1s2
Expand a mirror:
- two-way mirror containing a mounted filesystem -- one slice is added to each submirror:
metastat
metattach d9 c0t2d0s5
metattach d10 c0t3d0s5
Expand a RAID5 metadevice:
- add a slice:
metattach d2 c2t1d0s2
Expand a trans metadevice:
- expand a mirrored master device -- trans d10, with a mirrored master d0, containing submirrors d11 and d12:
metastat d10
metattach d11 c0t2d0s5
metattach d12 c0t3d0s5
Grow a filesystem:
- on a concatenation -- d10 contains /home2. A new slice was added to d10. Then, growfs is run to finalize things.
df -k <-- locate the filesystem
growfs -M /home2 /dev/md/rdsk/d10 <-- tell /home2 to grow to the full d10
df -k <-- check the changes
Note: For a mirror or a trans, ALWAYS run "growfs" in the TOP metadevice.
Rename a metadevice:
- d10, which contains /home, is renamed to d100
umount /home
metarename d10 d100
(Edit the /etc/vfstab file so that the file system references the new metadevice)
mount /home
Unmirror a filesystem:
- high level steps:
- run "metastat" to verify that at least one submirror is ok
- unmount the filesystem
- run "metadetach" on the submirror that will continue to be used
- run "metaclear" on the mirror
- edit /etc/vfstab to use a nonmirror device
- remount the filesystem
- example -- d4 is a mirror containing /var, d2 will continue to be used:
umount /var
metadetach d4 d2
metaclear -r d4
(Edit the /etc/vfstab file so that the entry for /var is changed from d4 to d2)
mount /var
Attach a submirror:
- the concatenation/stripe to be used as a submirror has to be the same size or larger than the existing submirror:
metastat d30
metattach d30 d70
metastat d30
Detach a submirror:
- we are detaching submirror d50 from mirror d5:
metastat
metadetach d5 d50
Place a submirror offline or online:
- offline:
metaoffline d10 d11
- online:
metaonline d10 d11
Reserve a diskset -- safely or forcibly:
- safely -- done from host "red":
metaset
metaset -s relo-red -t
metaset
- forcibly:
metaset -s relo-red -t -f
Release a diskset:
- release relo-red:
metaset -s relo-red -r <-- release it
metaset -s relo-red <-- check the status
Add drives to a diskset:
- data will be lost on drives that are being added:
metaset -s relo-red -a c2t5d0 <-- add a disk
metaset <-- check
Add another host to a diskset:
- maximum of two hosts per diskset are allowed:
metaset -s relo-red -a -h blue <-- add a host
metaset -s relo-red <-- check
CHANGING DISKSUITE OBJECTS
Change a mirror's options:
- read policy:
metaparam -r geometric d30
metaparam d30
- write policy:
metaparam -w serial d40
metaparam d40
- pass number:
metaparam -p 5 d50
metaparam d50
Share a logging device among filesystems:
- share d10 between an existing metadevice and a new metadevice d64. Master is /xyzfs on slice c0t2d0s4:
umount /xyzfs
metainit d64 -t c0t2d0s4 d10
(Edit the /etc/vfstab file so that the entry for /xyzfs references the trans metadevice d64)
mount /xyzfs
REMOVING DISKSUITE OBJECTS
Remove state database replicas:
- delete two replicas:
metadb -d c0t2d0s0 c0t1d0s0
- force delete all replicas:
metadb -f -d c0t1d0s3 c4t1d0s3
Remove stripes / concatenations:
- concatenation:
umount d8
metaclear d8
(Edit the /etc/vfstab file)
Remove a mirror and submirrors:
High level steps:
1. Unmount the filesystem
2. Run "metadetach" on the mirror and submirrors
3. Run "metaclear" on the mirror
4. Optional: edit /etc/vfstab to use a non-mirror device
5. remount the filesystem
- example: remove a mirror, keeping the data on the submirror:
metastat d2 <-- d2 is our mirror
umount /news <-- /news was contained in d2
metadetach d2 d0 <-- detach submirror d0
metaclear d2 <-- delete the mirror
(Edit the /etc/vfstab file so that /news references submirror d0)
mount /news <-- remount the submirror to be used
metaclear d1 <-- delete the unused submirror
Remove a RAID5 metadevice:
- this will delete any data:
umount /nfs
metaclear d80 <-- d80 is our RAID5 metadevice
Remove a trans metadevice:
- remove loggings from trans d64:
umount /abcfs <-- our filesystem for which we have logging
metaclear d64 <-- clear the trans
(Edit the /etc/vfstab file) <-- change back to using raw and block devices
fsck /dev/rdsk/c0t2d0s6 <-- fix the filesystem before mounting
...
FILE SYSTEM STATE IN SUPERBLOCK IS WRONG; FIX? y
...
mount /abcfs
Remove a trans metadevice from a filesystem than cannot be unmounted (system mount):
- removing a trans from /usr:
metadetach -f d20 <-- d20 used to provide UFS logging for /usr
reboot
(Edit the /etc/vfstab file) <-- change back to using ram and block devices
reboot
metaclear d20 <-- delete the unused trans metadevice
Remove a hot spare from a hot spare pool:
- remove a hot spare from one hot spare pool:
metahs -d hsp003 /dev/dsk/c2t1d0s2
- remove a hot spare from all hot spare pools:
metahs -d all /dev/dsk/c2t1d0s2
metahs -i
Remove a hot spare pool:
- hsp001 is shared between submirrors d31 and d32 of mirror d30:
metastat <-- find hot spare pool associations with metadevices
metaparam -h none d30 <-- remove association
metaparam -h none d31 <-- remove association
metahs -d hsp001 <-- delete hot spare pool
Remove a host from a diskset:
- remove host "blue" from the diskset "relo-red":
metaset -s relo-red -d -h blue
metaset
Remove a drive from a diskset:
- "-f" has to be used to delete the last drive in the diskset:
metaset -s relo-red -d c2t5d0
metaset
Remove a diskset:
- When the last host is deleted from a diskset, the diskset is automatically deleted too:
metaclear -s relo-red -a <-- remove all metadevices
metaset -s relo-red -d -h blue <-- delete host "blue"
metaset -s relo-red -f -d c1t2d0 c1t3d0 ... <-- remove all shared drives
metaset -s relo-red -d -h red <-- delete host "red" --> this also deletes the diskset
*** Read "Tips and Tricks" from the Solstice DiskSuite 4.2.1 Collection
-----
DiskSuite stripe maintenance writup -- NOT TESTED!!!
*** Replace a submirror in a mirror (because of a failed slice):
- mirror d20 has a submirror d22 in the "Needs maintenance state". The submirror will be recreated from new slices:
metastat d20 <-- check the mirror
metadetach -f d20 d22 <-- detach the submirror containing the failed slice
metaclear d22 <-- delete the detached submirror
(fix the slice -- "fsck", etc.... replace with another one of the same size if it cannot be fixed)
metainit d22 2 1 c1t0d0s2 1 c1t0d1s2 <-- recreate the submirror with the fresh slices
metattach d20 d22 <-- attach it back to the mirror. It will sync automatically
*** Expand the existing filesystem:
You have 2 stripes (built of 2GB slices), which are mirrored. There are two ways to expand:
1st way:
1. Add an equal number of new slices to each stripe (each slice is the same size as the existing slices)
2. Grow the filesystem:
Steps:
1. metattach d2 c1t2d0s2 <-- add a single slice to a stripe (do this equally for both stripes)
2. growfs -M /home2 /dev/md/rdsk/d10 <-- expands /home2 up to the new full size of d10 (your primary mirror)
Note: While "growfs" is running, the filesystem will be locked for writing. Reading will continue.
2nd way:
1. Create two new stripes, equal to each other, but not necessarily equal to the existing two.
2. Concatenate each new stripe to an existing stripe.
3. Grow the filesystem:
Steps (let's suppose the existing mirror is called d1, and the two stripes are called d11 and d12):
1. metainit d21 1 2 c0t1d0s2 c0t2d0s2 <-- create a new metadevice consisting of 1 stripe made of 2 slices (that's what the numbers are for)
2. metainit d22 1 2 c0t1d1s2 c0t2d1s2 <-- create d22 which is identical to d21
3. metattach d11 d21 <-- attach d21 at the end of d11
4. metattach d12 d22 <-- attach d22 at the end of d12
5. growfs -M /home2 /dev/md/rdsk/d1 <-- grow the filesystem to the new full size of d1 (your primary mirror)