A
A
Andrey Pechenkin2017-03-01 10:34:53
linux
Andrey Pechenkin, 2017-03-01 10:34:53

How to connect HP P2000 storage to HP ProLiant BL460c G7 server in C3000 farm via SAS Switch + Smart Array P712m controller in CentOS 7?

All this scheme is described in this document.
In stock:

  • HP C3000 Server Farm
  • ProLiant BL460c G7 Server with P712m Smart Array Controller
  • Sas Switch HP 6Gb SAS BL Switch
  • HP P2000 storage

Connection scheme in SAS switch:
1c709524000f4eb9ac50f1da4c70ca69.png
A volume is created on the storage system, read-write rights are set in the connection map.
When you install Windows Server 2008 through an HP SmartStart disk, the volume is mounted and visible on the system. 00bfb5ded08443938ed1b5651b780086.png
There is no special disk to install CentOS 7, I installed minimal on the hard disks of the server. The necessary volume through the P712m controller does not see.
My scsi device searches:
#lsscsi
[1:0:0:0]    storage HP       P410i            6.64  -
[1:1:0:0]    disk    HP       LOGICAL VOLUME   6.64  /dev/sda
[1:1:0:1]    disk    HP       LOGICAL VOLUME   6.64  /dev/sdb
# lsscsi -l
[1:0:0:0]    storage HP       P410i            6.64  -
  state=running queue_depth=32 scsi_level=6 type=12 device_blocked=0 timeout=0
[1:1:0:0]    disk    HP       LOGICAL VOLUME   6.64  /dev/sda
  state=running queue_depth=32 scsi_level=6 type=0 device_blocked=0 timeout=30
[1:1:0:1]    disk    HP       LOGICAL VOLUME   6.64  /dev/sdb
  state=running queue_depth=32 scsi_level=6 type=0 device_blocked=0 timeout=30
# ls /sys/class/scsi_device/
1:0:0:0  1:1:0:0  1:1:0:1
# ls /sys/class/scsi_disk/
1:1:0:0  1:1:0:1
# ls /sys/class/scsi_generic/
sg0  sg1  sg2
# ls /sys/class/scsi_host/
host0  host1  host2  host3#

There are only hard drives on the server on another controller and iSCSI initiators. The controller itself is visible in the system. lspci -vvv output:
09:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01)
        Subsystem: Hewlett-Packard Company Smart Array 712m (Mezzanine RAID controller)
        Physical Slot: 2
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 26
        Region 0: Memory at fba00000 (64-bit, non-prefetchable) [size=2M]
        Region 2: Memory at fb9f0000 (64-bit, non-prefetchable) [size=4K]
        Region 4: I/O ports at 3000 [size=256]
        [virtual] Expansion ROM at fb900000 [disabled] [size=512K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [70] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 116.000W
                DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 4096 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM not supported, Exit Latency L0s <512ns, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [ac] MSI-X: Enable+ Count=16 Masked-
                Vector table: BAR=0 offset=001c2000
                PBA: BAR=0 offset=001c4000
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UESvrt: DLP- SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Kernel driver in use: hpsa
        Kernel modules: hpsa

The hpsa driver is used, the documentation for it is here .
In the controller itself, in the selection of the boot disk, it sees the volume as "Shared Storage":
6068ea692e764331973142f2fc3a6987.pngfc5888788ebf44da81a10e1b94810da4.png
In the documentation for the hpsa driver, I did not find support for "Shared Storage", only work with directly connected disks.
Is it possible to run the whole scheme under CentOS 7? What am I doing wrong?
PS With CentOS experience is not great. The driver itself did not collect, most likely from CentOS. I tried to boot from the installation disk with the hpsa.hpsa_allow_any=1 option, I did not find any new disks in anaconda. I dug the Software Delivery Repository from HP and installed the kmod-hpsa package via yum. Maybe he replaced the driver from the distribution.
UPD1. Zoning on the switch without errors, each cable has its own zone. The screenshot shows that Zone1 is assigned to port 1, the server connection status is online, the storage connection status is online, the general status is OK.
d031c925d0724618a46c9382d87a2adb.png
# modinfo hpsa
filename:       /lib/modules/3.10.0-514.6.1.el7.x86_64/weak-updates/hpsa/hpsa.ko
license:        GPL
version:        3.4.16-148
description:    Driver for HP Smart Array Controller version 3.4.16-148 (d162/s598)
author:         Hewlett-Packard Company
rhelversion:    7.3
srcversion:     91FCA999BB76BF152FBD25C
alias: . . .
depends:        scsi_transport_sas
vermagic:       3.10.0-514.el7.x86_64 SMP mod_unload modversions
signer:         Hewlett-Packard Company: HP UEFI Secure Boot 2013 DB key
sig_key:        1D:7C:F2:C2:B9:26:73:F6:9C:8E:E1:EC:70:63:96:7A:B9:B6:2B:EC
sig_hashalgo:   sha256
parm:           hpsa_allow_any:Allow hpsa driver to access unknown HP Smart Array hardware (int)
parm:           hpsa_simple_mode:Use 'simple mode' rather than 'performant mode' (int)
parm:           reply_queues:Specify desired number of reply queues. 1-64, default is number of on-line CPUs, not to exceed 64 (int)

lsblk:
NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                   8:0    0 136,7G  0 disk
├─sda1                8:1    0 136,1G  0 part
│ └─md126             9:126  0   136G  0 raid1
│   ├─cl_node1-root 253:0    0    16G  0 lvm   /
│   └─cl_node1-opt  253:1    0   120G  0 lvm   /opt
└─sda2                8:2    0   586M  0 part
  └─md127             9:127  0   586M  0 raid1 /boot
sdb                   8:16   0 136,7G  0 disk
├─sdb1                8:17   0 136,1G  0 part
│ └─md126             9:126  0   136G  0 raid1
│   ├─cl_node1-root 253:0    0    16G  0 lvm   /
│   └─cl_node1-opt  253:1    0   120G  0 lvm   /opt
└─sdb2                8:18   0   586M  0 part
  └─md127             9:127  0   586M  0 raid1 /boot

UPD2. In /var/log/messages
Mar  1 16:21:24 node1 kernel: hpsa 0000:09:00.0: controller lockup detected: NULL_SDEV_PTR TAG:0x00000000:00000000 LUN:0000000000000000 CDB:c2000000000000002008000000000000
Mar  1 16:21:24 node1 kernel: hpsa 0000:09:00.0: report luns failed, can't track lun changes.
Mar  1 16:21:24 node1 kernel: hpsa 0000:09:00.0: driver discovery polling rescan.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexey, 2017-03-01
@alcyone

How is zoning configured on the switch? There are no errors? How was the driver assembled?
PS Make a conclusion lsblk.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question