Author: pepezzzz Original source: https://tidb.net/blog/308842f7
Planning for Raid configuration
The configuration of the physical machine where the TiDB cluster is installed is as follows:
| Component | Configuration Description |
|---|---|
| CPU | 2 * Intel Xeon Gold 5218R(2.1GHz, 20Core) |
| RAM | 384GB |
| System Disk | 2 * 480GB SATA SSD RAID1 |
| Data Disk | 14 * 1.92TB SATA SSD RAID10 |
After using 14 * 1.92TB SATA SSDs to form RAID10, minus the RAID metadata and 1024 conversion loss, the actual total 12819716 MB is available. Because all nodes are configured with SSD disks and the data size is large, it needs to be configured as a hybrid deployment mode of TiDB and TiKV.
At present, we configure a host with 80 VCPUs in total. According to a component of 16 VCPUs, about 5 components can be configured, and each physical host deploys a total of 4 TiKV and 1 TiDB. According to the capacity plan of a TiKV of about 2 TB, configure a space of 3 TB to ensure that the space occupied is below 80%. So the capacity space of 12819716 MB is divided into 4 spaces of 3 TB, mounted on /data{0123}, and 1 space of 500 GB, mounted on /deploy.
For the CPU and memory configuration of hybrid deployment, please refer to the document https://docs.pingcap.com/zh/tidb/dev/hybrid-deployment-topology.
In the operating system, you can use the command line management tool of the RAID card to realize online RAID reconfiguration, eliminating the problems of restarting and console access.
At present, in x86 servers, Microchip’s SmartROC series (products released after the acquisition of adaptec) and Broadcom’s Megaraid series (products released after the acquisition of avago) are mainly used. The configuration of these two RAID cards is taken as an example below.
Microchip Adaptec RAID arcconf
RAID card background information
Use operating system commands to view the RAID card model is ZTE ZTE server R5300 G4 RAID card SDPSA_L-18i
[host bin]$ sudo dmidecode |grep -i "product name"
Product Name: R5300 G4
Product Name: R5300G4
[host bin]$ sudo dmidecode |grep -i RAID
[host bin]$ sudo lspci |grep -i RAID
[host bin]$ sudo dmesg |grep -i RAID
[ 6.360527] smartpqi 0000:3c:00.0: added 14:1:0:0 4000000000000000 Direct-Access ZTE LOGICAL VOLUME SSDSmartPathCap- En- RAID-1(1 + 0)
[ 6.386016] smartpqi 0000:3c:00.0: added 14:1:0:1 4000000100000000 Direct-Access ZTE LOGICAL VOLUME SSDSmartPathCap- En- RAID-1(1 + 0)
[ 6.396947] smartpqi 0000:3c:00.0: added 14:2:0:0 00000000000000000 RAID ZTE SDPSA_L-18i 2G
[ 6.398251] scsi 14:2:0:0: RAID ZTE SDPSA_L-18i 2G 5.00 PQ: 0 ANSI: 5
A search reveals that the SDPSA_L-18i is actually Microchip’s SmartROC3100 RAID card.
https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio &productid=50320&vcl=true
Check the product literature to confirm that RAID can be configured using the arcconf command-line tool.
https://ww1.microchip.com/downloads/en/DeviceDoc/00003022A.pdf
Software download location
https://storage.microsemi.com/en-us/speed/raid/storage_manager/arcconf_B25611_zip.php
Refer to the product manual for configuration
https://download.adaptec.com/pdfs/user_guides/adaptec_cli_smarthba_smartraid_11_22_ug.pdf
View RAID card and data disk information
The host is configured with two vds, and vd1 is a RAID 10 configured with 14 disks, and vd1 needs to be deleted.
[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1 Controllers found: 1 -------------------------------------------------- -------------------- Controller information -------------------------------------------------- -------------------- Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN -------------------------------------------------- -------------------- Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F -------------------------------------------------- -------------------- Array Information -------------------------------------------------- -------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) -------------------------------------------------- -------------------- Array 0 : Ok (SATA SSD, 915725 MB, 0 MB) Array 1 : Ok (SATA SSD, 25639884 MB, 1 MB) -------------------------------------------------- -------------------- Logical device information -------------------------------------------------- -------------------- Logical ID : Status (RAID, Interface, Size MB) Name -------------------------------------------------- -------------------- Logical 0 : Optimal (1, Data, 457830 MB) os Logical 1 : Optimal (10, Data, 12819716 MB) vd -------------------------------------------------- -------------------- maxCache information -------------------------------------------------- -------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) -------------------------------------------------- -------------------- No maxCache device created -------------------------------------------------- -------------------- Physical Device information -------------------------------------------------- -------------------- Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location] -------------------------------------------------- -------------------- Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)] Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)] Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)] Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)] Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)] Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)] Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)] Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)] Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)] Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)] Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)] Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)] Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)] Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)] Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)] Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)] Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable] Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf getconfig 1 ld 1 Controllers found: 1 -------------------------------------------------- ------ Logical device information -------------------------------------------------- ------ Logical Device number 1 Logical Device name: vd Disk Name: /dev/sdb Block Size of member drives : 512 Bytes Array : 1 RAID level: 10 Status of Logical Device : Optimal Size : 12819716MB Stripe-unit size: 256 KB Full Stripe Size : 1792 KB Interface Type : SATA SSD Device Type : Data Boot Type : None Heads: 255 Sectors Per Track : 32 Cylinders : 65535 Caching : Enabled Mount Points : Not Mounted LD Acceleration Method : Controller Cache Logical Device On Boot Connector : No Volume Unique Identifier : 600508B1001CD9EDCFD7F2DC2F11B617 -------------------------------------------------- ------ Logical Device segment information -------------------------------------------------- ------ Group 0, Segment 0 : Present (1831420MB, SATA, SSD, Connector:0, Device:0) 222638F89768 intermediate output Group 6, Segment 1 : Present (1831420MB, SATA, SSD, Connector:3, Device:13) 22253910CA2E Command completed successfully. [root@host /tmp/arcconf/linux_x64]#
Delete VD1
Delete vd1, 14 disks are ready.
[root@host /tmp/arcconf/linux_x64]# ./arcconf delete 1 logicaldrive 1
Controllers found: 1
WARNING: Deleting this logical device will automatically delete array 1 because it is the only logical device present on that array.
All data in logical device 1 will be lost.
Delete the logical device?
Press y, then ENTER to continue or press ENTER to abort: y
Deleting: logical device 1 ("vd")
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1
Controllers found: 1
-------------------------------------------------- --------------------
Controller information
-------------------------------------------------- --------------------
Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN
-------------------------------------------------- --------------------
Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F
-------------------------------------------------- --------------------
Array Information
-------------------------------------------------- --------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
-------------------------------------------------- --------------------
Array 0 : Ok (SATA SSD, 915725 MB, 0 MB)
-------------------------------------------------- --------------------
Logical device information
-------------------------------------------------- --------------------
Logical ID : Status (RAID, Interface, Size MB) Name
-------------------------------------------------- --------------------
Logical 0 : Optimal (1, Data, 457830 MB) os
-------------------------------------------------- --------------------
maxCache information
-------------------------------------------------- --------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
-------------------------------------------------- --------------------
No maxCache device created
-------------------------------------------------- --------------------
Physical Device information
-------------------------------------------------- --------------------
Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
-------------------------------------------------- --------------------
Physical 0,0 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
Physical 0,1 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
Physical 0,2 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
Physical 0,3 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
Physical 0,4 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
Physical 0,5 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
Physical 0,6 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
Physical 0,7 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
Physical 0,8 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
Physical 0,9 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
Physical 0,10 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
Physical 0,11 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
Physical 0,12 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
Physical 0,13 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]
Command completed successfully.
Create first VD
Create a RAID 10 with 14 blocks and carve out the first 3070000 MB LUN.
| command parameter | value |
|---|---|
| arcconf create 1 logicaldrive | Create LD on controller 1 |
| name / VD name | data0 |
| LDcache / cache enable | LON |
| VD capacity | 3070000 |
| RAID level | 10 |
| Disks from 0 to 13 | 0 0,0 1, 0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0 11,0 12,0 13 |
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data0 LDcache LON 3070000 10 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13 Controllers found: 1 Do you want to add a logical device to the configuration? Press y, then ENTER to continue or press ENTER to abort: y Creating logical device: data0 Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf list 1 Controllers found: 1 -------------------------------------------------- -------------------- Controller information -------------------------------------------------- -------------------- Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN -------------------------------------------------- -------------------- Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F -------------------------------------------------- -------------------- Array Information -------------------------------------------------- -------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) -------------------------------------------------- -------------------- Array 0 : Ok (SATA SSD, 915725 MB, 0 MB) Array 1 : Ok (SATA SSD, 25639884 MB, 19499435 MB) -------------------------------------------------- -------------------- Logical device information -------------------------------------------------- -------------------- Logical ID : Status (RAID, Interface, Size MB) Name -------------------------------------------------- -------------------- Logical 0 : Optimal (1, Data, 457830 MB) os Logical 1 : Optimal (10, Data, 3070000 MB) data0 -------------------------------------------------- -------------------- maxCache information -------------------------------------------------- -------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) -------------------------------------------------- -------------------- No maxCache device created -------------------------------------------------- -------------------- Physical Device information -------------------------------------------------- -------------------- Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location] -------------------------------------------------- -------------------- Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)] Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)] Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)] Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)] Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)] Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)] Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)] Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)] Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)] Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)] Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)] Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)] Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)] Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)] Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)] Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)] Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable] Command completed successfully. [root@host /tmp/arcconf/linux_x64]# fdisk -l Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 262144 bytes Disk label type: dos Disk identifier: 0x000a9444 Device Boot Start End Blocks Id System /dev/sda1 * 2048 937506479 468752216 83 Linux /dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA) Partition 2 does not start on physical sector boundary. Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes [root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 1 3200GB Warning: The resulting partition is not properly aligned for best performance.
Attachment: “Warning: The resulting partition is not properly aligned for best performance.” Processing method
http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted/
[root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/queue/optimal_io_size 1835008 [root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/alignment_offset 0 [root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 3584s 100% [root@host /tmp/arcconf/linux_x64]# mkfs.ext4 /dev/sdb1 [root@host /tmp/arcconf/linux_x64]#
Create subsequent VD
Create the ARRAY disk group of RAID 10 in block 14 and continue to divide other LUNs.
| Command | Comment |
|---|---|
| arcconf create 1 logicaldrive | Create LD on controller 1 |
| name /VD name | data1~data4 |
| LDcache / cache enable | LON |
| VD capacity | 3070000 * 3 + 499000 * 1 |
| RAID LEVEL | 10 |
| ARRAY / Existing RAID Group | 1 (1st ARRAY automatically defined when LD is created) |
| noprompt | No interaction, it is recommended not to use it for delete operations. |
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data1 LDcache LON 3070000 10 ARRAY 1 Controllers found: 1 Do you want to add a logical device to the configuration? Press y, then ENTER to continue or press ENTER to abort: y Creating logical device: data1 Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data2 LDcache LON 3070000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data3 LDcache LON 3070000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data4 LDcache LON 499000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# fdisk -l Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 262144 bytes Disk label type: dos Disk identifier: 0x000a9444 Device Boot Start End Blocks Id System /dev/sda1 * 2048 937506479 468752216 83 Linux /dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA) Partition 2 does not start on physical sector boundary. WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk label type: gpt Disk identifier: 08A14123-EF5C-4775-A0C0-2DE80827BD40 # Start End Size Type Name 1 3584 6287357439 3T Microsoft basic primary Disk /dev/sdc: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sdd: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sde: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sdf: 523.2 GB, 523239424000 bytes, 1021952000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Subsequent partition formatting and hooking are omitted
Broadcom avago MegaRAID storcli
RAID card background information
Use the operating system command to view the RAID card model is the OEM Megaraid RAID card of the super fusion (formerly Huawei x86) server 2288H V5
# dmidecode |grep -Ei "manufacturer|product name"
Manufacturer: XFUSION
Product Name: 2288H V5
# dmesg|grep -i megaraid
...
[ 7.846978] scsi host0: Avago SAS based MegaRAID driver
...
Download and install the storcli management tool
-
Log in to the RAID controller card page on the BROADCOM website.
-
On the DOWNLOADS tab, click Management Software and Tools.
-
Open the list of management software and tools.
-
Download the latest version of the “StorCLI” tool.
-
After decompression, you can get toolkits suitable for different operating systems, and install the rpm package of linux.
Use the storcli management tool
There are many tutorials on the use of storcli management tools, a brief list is as follows:
- View RAID card configuration and current disk RAID configuration
Usually v1 is the data disk.
./storcli64 /c0 show ./storcli64 /c0/v1 show all
- remove v1
./storcli64 /c0/v1 del
- Add multiple LUNs such as v1 as needed
./storcli64 /c0 add vd r10 size=2000GB drives=252:0-13