LVM2创建条带化存储卷
安装LVM2(Centos7为例)
yum install lvm2
磁盘情况：
三块500G的云SSD ，使用LVM2
/dev/vdb 500G
/dev/vdc 500G
/dev/vdd 500G
创建条带化LVM卷
创建分区
fdisk /dev/vdb
选择n来创建新分区。
选择p来创建主分区。
选择我们需要创建的分区号。
按Enter两次来使用整个磁盘空间。
我们需要修改新创建的分区类型，输入t。
选择需要修改的分区号，选择我们创建的分区号1。
这里，我们需要修改类型。我们需要创建LVM，因此我们使用LVM的类型代码8e。如果不知道类型代码，按L来列出所有类型代码。
打印我们创建的分区以确认。
这里我们可以看到Linux LVM的ID 8e。
写入修改并退出fdisk，输入w。
依次完成其他 /dev/vdc /dev/vdd
使用fdisk -l查看分区信息：
创建PV
pvcreate /dev/vd[b-d]1 -v
PV创建完成后，你可以使用‘pvs’命令将它们列出来。
pvs
创建VG卷:
现在，我们需要使用这3个物理卷来定义卷组。这里，定义了一个物理扩展大小（PE）为16MB，名为vg_strip的卷组
vgcreate -s 16M vg_strip /dev/vd[b-d]1 -v
保存配置：
vgs vg_strip
查看VG卷情况：
vgdisplay vg_strip -v
创建LV卷:
这里，我创建了一个名为lv_pgdata，大小为1.4T的逻辑卷，它需要放到vg_strip卷组中。我定义了3个条块，就是说数据在写入到我的逻辑卷时，需要条块化分散到3个PV中。
-L –逻辑卷大小
-n –逻辑卷名称
-i –条块化
lvcreate -L 1.4T -i3 -I 64 -n lv_pgdata vg_strip /dev/vdb1 /dev/vdc1 /dev/vdd1
保存
lvs
查看：
lvdisplay -m
挂载分区
mkfs.xfs /dev/vg_strip/lv_pgdata
mkdir /data
mount /dev/vg_strip/lv_pgdata /data
写入fstab
性能对比
顺序读写：
条带化前：
[email protected]]# fio --filename=/data/test -iodepth=64 -ioengine=libaio --direct=1 --rw=read --bs=1m --size=2g --numjobs=4 --runtime=10 --group_reporting --name=test-read
test-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 4 processes
test-read: Laying out IO file (1 file / 2048MiB)
Jobs: 4 (f=4): [R(4)][100.0%][r=307MiB/s,w=0KiB/s][r=307,w=0 IOPS][eta 00m:00s]
test-read: (groupid=0, jobs=4): err= 0: pid=7161: Sat Dec 14 15:13:13 2019
read: IOPS=300, BW=301MiB/s (315MB/s)(3084MiB/10256msec)
slat (usec): min=57, max=105652, avg=12992.51, stdev=25162.23
clat (msec): min=127, max=1114, avg=819.87, stdev=118.38
lat (msec): min=199, max=1115, avg=832.86, stdev=118.82
clat percentiles (msec):
| 1.00th=[ 317], 5.00th=[ 617], 10.00th=[ 709], 20.00th=[ 785],
| 30.00th=[ 802], 40.00th=[ 810], 50.00th=[ 818], 60.00th=[ 860],
| 70.00th=[ 885], 80.00th=[ 902], 90.00th=[ 927], 95.00th=[ 986],
| 99.00th=[ 1011], 99.50th=[ 1028], 99.90th=[ 1083], 99.95th=[ 1083],
| 99.99th=[ 1116]
bw ( KiB/s): min=57344, max=92160, per=24.75%, avg=76205.16, stdev=7643.43, samples=76
iops : min= 56, max= 90, avg=74.41, stdev= 7.47, samples=76
lat (msec) : 250=0.39%, 500=2.43%, 750=13.59%, 1000=81.58%
cpu : usr=0.05%, sys=0.85%, ctx=1504, majf=0, minf=65664
IO depths : 1=0.1%, 2=0.3%, 4=0.5%, 8=1.0%, 16=2.1%, 32=4.2%, >=64=91.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=3084,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=301MiB/s (315MB/s), 301MiB/s-301MiB/s (315MB/s-315MB/s), io=3084MiB (3234MB), run=10256-10256msec
Disk stats (read/write):
dm-0: ios=9957/419, merge=0/0, ticks=2540845/11297, in_queue=2592940, util=99.04%, aggrios=3389/153, aggrmerge=0/0, aggrticks=852856/3581, aggrin_queue=377994, aggrutil=85.76%
vdb: ios=9607/122, merge=0/0, ticks=2557435/10136, in_queue=1132260, util=85.76%
vdc: ios=0/42, merge=0/0, ticks=0/25, in_queue=20, util=0.19%
vdd: ios=562/297, merge=0/0, ticks=1135/582, in_queue=1702, util=1.53%
条带化后：
[[email protected]]# fio --filename=/data/test -iodepth=64 -ioengine=libaio --direct=1 --rw=read --bs=1m --size=2g --numjobs=4 --runtime=10 --group_reporting --name=test-read
test-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 4 processes
Jobs: 4 (f=4): [R(4)][100.0%][r=834MiB/s,w=0KiB/s][r=834,w=0 IOPS][eta 00m:00s]
test-read: (groupid=0, jobs=4): err= 0: pid=11170: Sat Dec 14 15:09:36 2019
read: IOPS=899, BW=899MiB/s (943MB/s)(8192MiB/9108msec)
slat (usec): min=77, max=1991, avg=169.14, stdev=115.92
clat (msec): min=33, max=556, avg=283.71, stdev=43.08
lat (msec): min=33, max=556, avg=283.88, stdev=43.03
clat percentiles (msec):
| 1.00th=[ 73], 5.00th=[ 241], 10.00th=[ 249], 20.00th=[ 271],
| 30.00th=[ 279], 40.00th=[ 284], 50.00th=[ 288], 60.00th=[ 292],
| 70.00th=[ 296], 80.00th=[ 300], 90.00th=[ 313], 95.00th=[ 321],
| 99.00th=[ 451], 99.50th=[ 514], 99.90th=[ 550], 99.95th=[ 550],
| 99.99th=[ 558]
bw ( KiB/s): min=149504, max=237568, per=24.52%, avg=225823.21, stdev=18744.58, samples=72
iops : min= 146, max= 232, avg=220.50, stdev=18.30, samples=72
lat (msec) : 50=0.46%, 100=0.62%, 250=9.42%, 500=88.85%, 750=0.63%
cpu : usr=0.13%, sys=3.93%, ctx=9439, majf=0, minf=2314
IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.4%, 16=0.8%, 32=1.6%, >=64=96.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=8192,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=899MiB/s (943MB/s), 899MiB/s-899MiB/s (943MB/s-943MB/s), io=8192MiB (8590MB), run=9108-9108msec
Disk stats (read/write):
dm-0: ios=131072/0, merge=0/0, ticks=34573317/0, in_queue=35082808, util=98.92%, aggrios=6388/0, aggrmerge=36681/0, aggrticks=1734202/0, aggrin_queue=1132085, aggrutil=96.80%
vdb: ios=6395/0, merge=36671/0, ticks=1739700/0, in_queue=1131684, util=96.79%
vdc: ios=6391/0, merge=36690/0, ticks=1715396/0, in_queue=1132330, util=96.74%
vdd: ios=6378/0, merge=36682/0, ticks=1747511/0, in_queue=1132243, util=96.80%
8K随机读写：
条带化前：
[[email protected]]# fio -filename=/data/test -iodepth=64 -ioengine=libaio -direct=1 -rw=randread -bs=8k -size=2G -numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 64 processes
test-rand-read: Laying out IO file (1 file / 2048MiB)
Jobs: 64 (f=64): [r(64)][100.0%][r=130MiB/s,w=0KiB/s][r=16.7k,w=0 IOPS][eta 00m:00s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=27920: Sat Dec 14 20:45:04 2019
read: IOPS=17.0k, BW=133MiB/s (139MB/s)(2660MiB/20010msec)
slat (usec): min=3, max=90955, avg=3753.46, stdev=12254.09
clat (msec): min=2, max=470, avg=236.08, stdev=50.75
lat (msec): min=5, max=472, avg=239.84, stdev=51.21
clat percentiles (msec):
| 1.00th=[ 116], 5.00th=[ 176], 10.00th=[ 192], 20.00th=[ 201],
| 30.00th=[ 207], 40.00th=[ 211], 50.00th=[ 220], 60.00th=[ 234],
| 70.00th=[ 279], 80.00th=[ 288], 90.00th=[ 296], 95.00th=[ 305],
| 99.00th=[ 372], 99.50th=[ 380], 99.90th=[ 393], 99.95th=[ 397],
| 99.99th=[ 405]
bw ( KiB/s): min= 766, max= 3104, per=1.54%, avg=2102.07, stdev=282.99, samples=2560
iops : min= 95, max= 388, avg=262.73, stdev=35.38, samples=2560
lat (msec) : 4=0.01%, 10=0.04%, 20=0.11%, 50=0.16%, 100=0.09%
lat (msec) : 250=60.23%, 500=39.38%
cpu : usr=0.18%, sys=1.06%, ctx=582073, majf=0, minf=10509
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=0.6%, >=64=98.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=340462,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=133MiB/s (139MB/s), 133MiB/s-133MiB/s (139MB/s-139MB/s), io=2660MiB (2789MB), run=20010-20010msec
Disk stats (read/write):
dm-0: ios=336999/717, merge=0/0, ticks=6298965/5479, in_queue=6311249, util=99.44%, aggrios=113377/328, aggrmerge=116/0, aggrticks=1695616/1421, aggrin_queue=798083, aggrutil=92.58%
vdb: ios=340131/458, merge=350/1, ticks=5086847/3828, in_queue=2393827, util=92.58%
vdc: ios=0/196, merge=0/0, ticks=0/106, in_queue=100, util=0.50%
vdd: ios=2/330, merge=0/0, ticks=3/329, in_queue=322, util=0.28%
条带化后：
[[email protected]]# fio -filename=/data/test -iodepth=64 -ioengine=libaio -direct=1 -rw=randread -bs=8k -size=20G -numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 64 processes
test-rand-read: Laying out IO file (1 file / 20480MiB)
Jobs: 64 (f=64): [r(64)][100.0%][r=383MiB/s,w=0KiB/s][r=48.0k,w=0 IOPS][eta 00m:00s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=31859: Sat Dec 14 20:39:22 2019
read: IOPS=48.9k, BW=382MiB/s (401MB/s)(7649MiB/20010msec)
slat (usec): min=3, max=78356, avg=1301.52, stdev=5248.26
clat (usec): min=622, max=272749, avg=82278.78, stdev=32193.92
lat (usec): min=631, max=273956, avg=83581.15, stdev=32603.39
clat percentiles (msec):
| 1.00th=[ 17], 5.00th=[ 28], 10.00th=[ 35], 20.00th=[ 52],
| 30.00th=[ 71], 40.00th=[ 80], 50.00th=[ 86], 60.00th=[ 92],
| 70.00th=[ 97], 80.00th=[ 105], 90.00th=[ 116], 95.00th=[ 138],
| 99.00th=[ 169], 99.50th=[ 178], 99.90th=[ 192], 99.95th=[ 199],
| 99.99th=[ 211]
bw ( KiB/s): min= 3296, max=10224, per=1.56%, avg=6092.74, stdev=860.56, samples=2560
iops : min= 412, max= 1278, avg=761.57, stdev=107.58, samples=2560
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.03%, 10=0.19%, 20=1.59%, 50=17.60%
lat (msec) : 100=54.84%, 250=25.74%, 500=0.01%
cpu : usr=0.39%, sys=1.49%, ctx=590995, majf=0, minf=10449
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, >=64=99.6%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=979026,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=382MiB/s (401MB/s), 382MiB/s-382MiB/s (401MB/s-401MB/s), io=7649MiB (8020MB), run=20010-20010msec
Disk stats (read/write):
dm-0: ios=1028126/1088, merge=0/0, ticks=11173125/10516, in_queue=11200926, util=99.73%, aggrios=339667/412, aggrmerge=6728/23, aggrticks=3242309/2819, aggrin_queue=1851484, aggrutil=92.27%
vdb: ios=339200/422, merge=6474/30, ticks=1860443/1881, in_queue=1229233, util=67.10%
vdc: ios=340144/404, merge=6606/20, ticks=3606016/3526, in_queue=2070389, util=91.30%
vdd: ios=339658/411, merge=7105/21, ticks=4260470/3050, in_queue=2254831, util=92.27%
参考文章：
https://linux.cn/article-3965-1.html
https://linux.cn/article-4445-1.html
近期评论