LVM2创建条带化存储卷

作者: 阿伟 分类: linux 发布时间: 2020-01-13 20:46 ė 浏览数 6 没有评论

安装LVM2(Centos7为例)

yum install lvm2

磁盘情况:

三块500G的云SSD ,使用LVM2

/dev/vdb 500G
/dev/vdc 500G
/dev/vdd 500G

创建条带化LVM卷

创建分区

fdisk  /dev/vdb 

选择n来创建新分区。
选择p来创建主分区。
选择我们需要创建的分区号。
按Enter两次来使用整个磁盘空间。
我们需要修改新创建的分区类型,输入t。
选择需要修改的分区号,选择我们创建的分区号1。
这里,我们需要修改类型。我们需要创建LVM,因此我们使用LVM的类型代码8e。如果不知道类型代码,按L来列出所有类型代码。
打印我们创建的分区以确认。
这里我们可以看到Linux LVM的ID 8e。
写入修改并退出fdisk,输入w。

依次完成其他 /dev/vdc /dev/vdd

使用fdisk -l查看分区信息:

创建PV

pvcreate /dev/vd[b-d]1 -v 

PV创建完成后,你可以使用‘pvs’命令将它们列出来。

pvs  

创建VG卷:

现在,我们需要使用这3个物理卷来定义卷组。这里,定义了一个物理扩展大小(PE)为16MB,名为vg_strip的卷组

vgcreate -s 16M vg_strip /dev/vd[b-d]1 -v  

保存配置:

vgs vg_strip

查看VG卷情况:

vgdisplay   vg_strip -v

创建LV卷:

这里,我创建了一个名为lv_pgdata,大小为1.4T的逻辑卷,它需要放到vg_strip卷组中。我定义了3个条块,就是说数据在写入到我的逻辑卷时,需要条块化分散到3个PV中。
-L –逻辑卷大小
-n –逻辑卷名称
-i –条块化

lvcreate -L 1.4T -i3 -I 64 -n lv_pgdata vg_strip /dev/vdb1 /dev/vdc1 /dev/vdd1

保存

lvs

查看:

lvdisplay -m

挂载分区

mkfs.xfs /dev/vg_strip/lv_pgdata
mkdir /data
mount /dev/vg_strip/lv_pgdata /data

写入fstab


性能对比

顺序读写:

条带化前:

root@lvm]# fio --filename=/data/test -iodepth=64 -ioengine=libaio --direct=1 --rw=read --bs=1m --size=2g --numjobs=4 --runtime=10 --group_reporting --name=test-read
test-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 4 processes
test-read: Laying out IO file (1 file / 2048MiB)
Jobs: 4 (f=4): [R(4)][100.0%][r=307MiB/s,w=0KiB/s][r=307,w=0 IOPS][eta 00m:00s]
test-read: (groupid=0, jobs=4): err= 0: pid=7161: Sat Dec 14 15:13:13 2019
read: IOPS=300, BW=301MiB/s (315MB/s)(3084MiB/10256msec)
slat (usec): min=57, max=105652, avg=12992.51, stdev=25162.23
clat (msec): min=127, max=1114, avg=819.87, stdev=118.38
lat (msec): min=199, max=1115, avg=832.86, stdev=118.82
clat percentiles (msec):
| 1.00th=[ 317], 5.00th=[ 617], 10.00th=[ 709], 20.00th=[ 785],
| 30.00th=[ 802], 40.00th=[ 810], 50.00th=[ 818], 60.00th=[ 860],
| 70.00th=[ 885], 80.00th=[ 902], 90.00th=[ 927], 95.00th=[ 986],
| 99.00th=[ 1011], 99.50th=[ 1028], 99.90th=[ 1083], 99.95th=[ 1083],
| 99.99th=[ 1116]
bw ( KiB/s): min=57344, max=92160, per=24.75%, avg=76205.16, stdev=7643.43, samples=76
iops : min= 56, max= 90, avg=74.41, stdev= 7.47, samples=76
lat (msec) : 250=0.39%, 500=2.43%, 750=13.59%, 1000=81.58%
cpu : usr=0.05%, sys=0.85%, ctx=1504, majf=0, minf=65664
IO depths : 1=0.1%, 2=0.3%, 4=0.5%, 8=1.0%, 16=2.1%, 32=4.2%, >=64=91.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=3084,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
READ: bw=301MiB/s (315MB/s), 301MiB/s-301MiB/s (315MB/s-315MB/s), io=3084MiB (3234MB), run=10256-10256msec

Disk stats (read/write):
dm-0: ios=9957/419, merge=0/0, ticks=2540845/11297, in_queue=2592940, util=99.04%, aggrios=3389/153, aggrmerge=0/0, aggrticks=852856/3581, aggrin_queue=377994, aggrutil=85.76%
vdb: ios=9607/122, merge=0/0, ticks=2557435/10136, in_queue=1132260, util=85.76%
vdc: ios=0/42, merge=0/0, ticks=0/25, in_queue=20, util=0.19%
vdd: ios=562/297, merge=0/0, ticks=1135/582, in_queue=1702, util=1.53%

条带化后:

[root@lvm]# fio --filename=/data/test -iodepth=64 -ioengine=libaio --direct=1 --rw=read --bs=1m --size=2g --numjobs=4 --runtime=10 --group_reporting --name=test-read
test-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 4 processes
Jobs: 4 (f=4): [R(4)][100.0%][r=834MiB/s,w=0KiB/s][r=834,w=0 IOPS][eta 00m:00s]
test-read: (groupid=0, jobs=4): err= 0: pid=11170: Sat Dec 14 15:09:36 2019
read: IOPS=899, BW=899MiB/s (943MB/s)(8192MiB/9108msec)
slat (usec): min=77, max=1991, avg=169.14, stdev=115.92
clat (msec): min=33, max=556, avg=283.71, stdev=43.08
lat (msec): min=33, max=556, avg=283.88, stdev=43.03
clat percentiles (msec):
| 1.00th=[ 73], 5.00th=[ 241], 10.00th=[ 249], 20.00th=[ 271],
| 30.00th=[ 279], 40.00th=[ 284], 50.00th=[ 288], 60.00th=[ 292],
| 70.00th=[ 296], 80.00th=[ 300], 90.00th=[ 313], 95.00th=[ 321],
| 99.00th=[ 451], 99.50th=[ 514], 99.90th=[ 550], 99.95th=[ 550],
| 99.99th=[ 558]
bw ( KiB/s): min=149504, max=237568, per=24.52%, avg=225823.21, stdev=18744.58, samples=72
iops : min= 146, max= 232, avg=220.50, stdev=18.30, samples=72
lat (msec) : 50=0.46%, 100=0.62%, 250=9.42%, 500=88.85%, 750=0.63%
cpu : usr=0.13%, sys=3.93%, ctx=9439, majf=0, minf=2314
IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.4%, 16=0.8%, 32=1.6%, >=64=96.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=8192,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
READ: bw=899MiB/s (943MB/s), 899MiB/s-899MiB/s (943MB/s-943MB/s), io=8192MiB (8590MB), run=9108-9108msec

Disk stats (read/write):
dm-0: ios=131072/0, merge=0/0, ticks=34573317/0, in_queue=35082808, util=98.92%, aggrios=6388/0, aggrmerge=36681/0, aggrticks=1734202/0, aggrin_queue=1132085, aggrutil=96.80%
vdb: ios=6395/0, merge=36671/0, ticks=1739700/0, in_queue=1131684, util=96.79%
vdc: ios=6391/0, merge=36690/0, ticks=1715396/0, in_queue=1132330, util=96.74%
vdd: ios=6378/0, merge=36682/0, ticks=1747511/0, in_queue=1132243, util=96.80%

8K随机读写:

条带化前:

[root@lvm]# fio -filename=/data/test -iodepth=64 -ioengine=libaio -direct=1 -rw=randread -bs=8k -size=2G -numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 64 processes
test-rand-read: Laying out IO file (1 file / 2048MiB)
Jobs: 64 (f=64): [r(64)][100.0%][r=130MiB/s,w=0KiB/s][r=16.7k,w=0 IOPS][eta 00m:00s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=27920: Sat Dec 14 20:45:04 2019
   read: IOPS=17.0k, BW=133MiB/s (139MB/s)(2660MiB/20010msec)
    slat (usec): min=3, max=90955, avg=3753.46, stdev=12254.09
    clat (msec): min=2, max=470, avg=236.08, stdev=50.75
     lat (msec): min=5, max=472, avg=239.84, stdev=51.21
    clat percentiles (msec):
     |  1.00th=[  116],  5.00th=[  176], 10.00th=[  192], 20.00th=[  201],
     | 30.00th=[  207], 40.00th=[  211], 50.00th=[  220], 60.00th=[  234],
     | 70.00th=[  279], 80.00th=[  288], 90.00th=[  296], 95.00th=[  305],
     | 99.00th=[  372], 99.50th=[  380], 99.90th=[  393], 99.95th=[  397],
     | 99.99th=[  405]
   bw (  KiB/s): min=  766, max= 3104, per=1.54%, avg=2102.07, stdev=282.99, samples=2560
   iops        : min=   95, max=  388, avg=262.73, stdev=35.38, samples=2560
  lat (msec)   : 4=0.01%, 10=0.04%, 20=0.11%, 50=0.16%, 100=0.09%
  lat (msec)   : 250=60.23%, 500=39.38%
  cpu          : usr=0.18%, sys=1.06%, ctx=582073, majf=0, minf=10509
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=0.3%, 32=0.6%, >=64=98.8%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=340462,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=133MiB/s (139MB/s), 133MiB/s-133MiB/s (139MB/s-139MB/s), io=2660MiB (2789MB), run=20010-20010msec

Disk stats (read/write):
    dm-0: ios=336999/717, merge=0/0, ticks=6298965/5479, in_queue=6311249, util=99.44%, aggrios=113377/328, aggrmerge=116/0, aggrticks=1695616/1421, aggrin_queue=798083, aggrutil=92.58%
  vdb: ios=340131/458, merge=350/1, ticks=5086847/3828, in_queue=2393827, util=92.58%
  vdc: ios=0/196, merge=0/0, ticks=0/106, in_queue=100, util=0.50%
  vdd: ios=2/330, merge=0/0, ticks=3/329, in_queue=322, util=0.28%

条带化后:

[root@lvm]# fio -filename=/data/test -iodepth=64 -ioengine=libaio -direct=1 -rw=randread -bs=8k -size=20G -numjobs=64 -runtime=20 -group_reporting -name=test-rand-read
test-rand-read: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=64
...
fio-3.7
Starting 64 processes
test-rand-read: Laying out IO file (1 file / 20480MiB)
Jobs: 64 (f=64): [r(64)][100.0%][r=383MiB/s,w=0KiB/s][r=48.0k,w=0 IOPS][eta 00m:00s]
test-rand-read: (groupid=0, jobs=64): err= 0: pid=31859: Sat Dec 14 20:39:22 2019
   read: IOPS=48.9k, BW=382MiB/s (401MB/s)(7649MiB/20010msec)
    slat (usec): min=3, max=78356, avg=1301.52, stdev=5248.26
    clat (usec): min=622, max=272749, avg=82278.78, stdev=32193.92
     lat (usec): min=631, max=273956, avg=83581.15, stdev=32603.39
    clat percentiles (msec):
     |  1.00th=[   17],  5.00th=[   28], 10.00th=[   35], 20.00th=[   52],
     | 30.00th=[   71], 40.00th=[   80], 50.00th=[   86], 60.00th=[   92],
     | 70.00th=[   97], 80.00th=[  105], 90.00th=[  116], 95.00th=[  138],
     | 99.00th=[  169], 99.50th=[  178], 99.90th=[  192], 99.95th=[  199],
     | 99.99th=[  211]
   bw (  KiB/s): min= 3296, max=10224, per=1.56%, avg=6092.74, stdev=860.56, samples=2560
   iops        : min=  412, max= 1278, avg=761.57, stdev=107.58, samples=2560
  lat (usec)   : 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.03%, 10=0.19%, 20=1.59%, 50=17.60%
  lat (msec)   : 100=54.84%, 250=25.74%, 500=0.01%
  cpu          : usr=0.39%, sys=1.49%, ctx=590995, majf=0, minf=10449
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, >=64=99.6%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=979026,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=382MiB/s (401MB/s), 382MiB/s-382MiB/s (401MB/s-401MB/s), io=7649MiB (8020MB), run=20010-20010msec

Disk stats (read/write):
    dm-0: ios=1028126/1088, merge=0/0, ticks=11173125/10516, in_queue=11200926, util=99.73%, aggrios=339667/412, aggrmerge=6728/23, aggrticks=3242309/2819, aggrin_queue=1851484, aggrutil=92.27%
  vdb: ios=339200/422, merge=6474/30, ticks=1860443/1881, in_queue=1229233, util=67.10%
  vdc: ios=340144/404, merge=6606/20, ticks=3606016/3526, in_queue=2070389, util=91.30%
  vdd: ios=339658/411, merge=7105/21, ticks=4260470/3050, in_queue=2254831, util=92.27%

参考文章:

https://linux.cn/article-3965-1.html
https://linux.cn/article-4445-1.html

逻辑卷管理器管理 RedHat 文档:

https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/stripe_create_ex

知识共享许可协议
本作品采用知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议进行许可。

本文出自阿伟的博客,转载时请注明出处及相应链接。

本文永久链接: https://blog.awei.me/lvm2/

0

发表评论

电子邮件地址不会被公开。 必填项已用*标注

Ɣ回顶部