InfiniBand technology - rds protocol

Foreword:

　　Has been engaged in oracle work before, the recent one-machine learning encountered ib switch configuration rds protocol and other untouched concepts, it is necessary to summarize some network, storage knowledge.

First, the controller and adapter:

　　Controller: integrated on the motherboard, the cpu logic instructions are converted by the protocol into control signals that the device can recognize.

　　Adapter: Independent of the motherboard, the function is the same as the controller.

　　How does cpu read files on disk?

　　When the user executes a command, such as cat /etc/hosts, the kernel schedules the process to run on the cpu. At this time, the cpu needs to access the file data block in the file system. The file system is built on the disk device. The CPU wants to access the disk device but cannot directly control the disk device. Because the disk device has no block concept, the magnetic device

The disk only has sector tracks. In this case, the controller or the adapter is required to translate the instructions of the CPU into instructions that the disk device can understand. After receiving the command, the disk device takes the data from the specified sector and returns it to the CPU.

Second, hba card and hca card:

　　Early storage systems and computers were connected by fiber optics through switches. Since the server passes the scsi command to the storage system and cannot use the TCP/IP protocol of the ordinary LAN, FC transmission is required, which is called FC-SAN. The latter technology can use the ordinary LAN IP protocol, which is called IP-SCAN. The most typical is ISCSI. This

Both methods require heavy reading and unpacking of data blocks. Therefore, a high-performance SAN system needs to install a network card dedicated to unpacking the server to reduce the burden on the server. This network card is called HBA card, in addition to performing unpacking work, of course, can also provide a fiber interface (if it is an iSCSI HBA card)

It is provided with a normal RJ45 interface ) for connection with the corresponding switch ; in addition, the HBA can physically be inserted into the PCI or PCI-E slot as a network card , so the usage of this device is very similar. Network card, many people also confuse it with ordinary network cards or ordinary fiber-optic network cards. Of course, some iSCSI HBA cards can be used as

Ordinary network cards are used, but considering the price, this is very extravagant.

　　An HBA card can be understood simply as an adapter plugged into a physical motherboard. According to different transmission protocols, network cards can be divided into three types, one is an Ethernet card, the other is an FC network card, and the third is an iSCSI network card.

　　Ethernet card: The Ethernet Adapter, the transmission protocol is IP protocol , and is usually connected to an Ethernet switch through a fiber-optic cable or a twisted pair cable. Interface types are classified into optical ports and electrical ports. Optical interfaces generally use optical fiber cables for data transmission. Interface modules are generally SFP (transmission rate 2Gb/s) and GBIC (1Gb/s). The corresponding interfaces are SC and ST.

And LC. The current interface type of the electrical port is RJ45, which is used to connect to the twisted pair cable, and also has an interface to the coaxial cable, but it has been used less.

　　FC network card: generally called fiber network card, the scientific name Fibre Channel HBA. The transport protocol is a Fibre Channel protocol and is typically connected to Fibre Channel switches through fiber-optic cables. Interface types are classified into optical ports and electrical ports. The optical interface generally transmits data through a fiber-optic cable. The interface module is generally SFP (transmission rate 2Gb/s) and

GBIC (1Gb/s), the corresponding interfaces are SC and LC. The interface type of the electrical port is generally DB9 pin or HSSDC.

　　ISCSI NIC: The scientific name ISCSI HBA, the transmission ISCSI protocol, the interface type is the same as the Ethernet card.

Everyone said that "fiber-optic network card " generally refers to the FC HBA card, which is plugged into the server and externally connected to the fiber-optic switch . The optical port of the optical port is generally called "fiber-optic Ethernet card ", which is also plugged into the server, but it is externally connected. It is an Ethernet switch with an optical port.

　　HCA card:

　　Mellanox introduces the Mellanox ConnectX IB InfiniBand Host Channel Adapter (HCA) card for enterprise data centers, high-performance computing and embedded environments, providing high bandwidth, low latency for server/storage cluster applications solution. Use the infiniBand related protocol.

Third, infiniBand technology and rds protocol:

　　Infiniband technology:

　　Infiniband is widely used for connection between FC/IP SAN, NAS and server. As a storage protocol for iSCSI RDMA, iSER has been standardized by IETF. At present, EMC's full range of products have been switched to Infiniband networking, IBM/TMS FlashSystem series, IBM's storage system XIV Gen3, and DDN's SFA series all using Infiniband.

The internet. Simply put, it is a more efficient connection protocol technology between server and storage system than FC.

　　Rds protocol:

　　Most of the traditional RAC internal interconnections are based on common networks. Currently, the most popular ones are 100M and Gigabit networks, and the fastest is the 10G network that is not yet popular. Due to the speed limitations of ordinary networks, performance cannot be guaranteed in a multi-node RAC database that requires frequent internal communication. It is based on this, Oracle

On February 24, 2006, Qlogic and QLogic jointly released the RDS for Oracle RAC internal interconnection solution based on the Infiniband high-speed Internet.

Compared with the traditional mode, rds does not need to first convert the UDP protocol to the IP protocol through the IPC to convert the IPoIB protocol, and directly interacts with the HCA through the rds to reduce the multi-layer conversion and increase the conversion efficiency.

Not only that, RDS has the following advantages:

1, high bandwidth. Compared to ordinary network switches, the maximum bandwidth is now 10Gbps, and Infiniband can reach 40Gbps at the fastest.

2. Low latency. It can double the network latency compared to the normal UDP internal interconnect.

3. Low CPU utilization. The CPU overhead of the internal interconnect is only half of the original.

 　　Enable rds protocol:

　　 Reference: http://blog.51cto.com/jxplpp/1575681 ------- untested, for reference only

Reference: https://blog.csdn.net/u014753393/article/details/50259207 

　　　Http://yuanjl2006.blog.163.com/blog/static/35266331201471442128525/

　　　Http://www.sohu.com/a/146313500_632967

　　　Https://book.2cto.com/201511/57577.html

　　　Http://www.kuqin.com/database/20081206/29718.html

　　　Http://blog.51cto.com/jxplpp/1575681