ibdm-topo-file(1) [centos man page]
IBDM-TOPO-FILE(1) IB DATA MODEL PACKAGE IBDM-TOPO-FILE(1) NAME
IBDM Topology File The topology file describes the IB connectivity and systems included in the network. It serves two purposes: 1. Support for arbitrary system names to be later used in every report that IBDM generates. 2. Connectivity verification: The specified network topology is verified against the discovered one. Mismatch errors resulting from missing cables and/or wrong connections are reported by IBADM. The topology file is composed "system" sections. Each such section describes the connectivity of one system to other systems in the net- work. The first line of each section is a declaration of the system composed of a system-type, its system-name section, and optional con- figuration details. The lines to follow until the next empty line describe the connections between this system ports to the other systems. The following is a formal definition of a system section syntax. An example is listed afterwards. SYSTEM-TYPE SYSTEM-NAME [CFG: <b1>=<modifier>, [<b2>=<modifier>]...] LOCAL-PORT-PANEL-NAME -> REM-SYS-TYPE REM-SYS-NAME REM-PORT-PANEL-NAME LOCAL-PORT-PANEL-NAME -> REM-SYS-TYPE REM-SYS-NAME REM-PORT-PANEL-NAME LOCAL-PORT-PANEL-NAME -> REM-SYS-TYPE REM-SYS-NAME REM-PORT-PANEL-NAME . . . SYSTEM-TYPE SYSTEM-NAME [CFG: <b1>=<modifier>, [<b2>=<modifier>]...] LOCAL-PORT-PANEL-NAME -> REM-SYS-TYPE REM-SYS-NAME REM-PORT-PANEL-NAME LOCAL-PORT-PANEL-NAME -> REM-SYS-TYPE REM-SYS-NAME REM-PORT-PANEL-NAME . . . . . . Where: SYSTEM-TYPE and REM-SYS-TYPE Any system that has a corresponding IBNL definition. See man ibdm-ibnl-file SYSTEM-NAME The name of the system described in this topology file section. LOCAL-PORT-PANEL-NAME The name of the local system port. The numbers printed on the front panel are used together with Ln for Leaf no. N or Sn for Spine no. N. REM-SYS-NAME The name of the system connected to the local port. REM-PORT-PANEL-NAME A name of the remote system port. We use the numbers as printed on the front panel and Ln for Leaf number N or Sn for Spine number N. EXAMPLE
The following is a topology file for a simple cluster with one 24 port switch and two HCAs. The firts HCA named H-1 and connect to the fab- ric with its two ports. The second HCA named H-2. The switch is of type MTS2400 and is named S-1 MTS2400 S-1 P1 -> MT23108 H-1 P1 P2 -> MT23108 H-1 P2 P24 -> MT23108 H-2 P1 OPTIONAL CFG SECTION
This section in the system declaration line describes the special customization of each board of the system. That is the CFG string is a set of comma-separated sub-fields. Each sub-field describes some special configuration of a corresponding sys- tem board. The actual semantics of the specific board modifiers is defined by the IBNL of the specific system. EXAMPLE
The following is an example of a definition-line in a topology file of the MTS9600 switch system. This switch system can have up to eight leafs and four spines. This example of the MTS9600 lacks (R) leafs no.6,7 and 8, and lacks spines no. 3 and 4. MTS9600 PartialGz1 CFG: leaf3=R,leaf5=R,leaf7=R,spine1=R AUTHOR
Eitan Zahavi, Mellanox Technologies LTD, eitan@mellanox.co.il IBDM 1.0 2008-06-16 IBDM-TOPO-FILE(1)
Check Out this Related Man Page
IBDM-IBNL-FILE(1) IB DATA MODEL PACKAGE IBDM-IBNL-FILE(1) NAME
ibnl - a generic IB netlist format DESCRIPTION
IBDM topology file ibdm-topo-file provide means to describe the IB fabric using a set of predefined systems. A system definition is pro- vided in a single file in IBNL format that describes the internal InfiniBand connectivity of the system in terms of boards and devices. When IBDM starts it parses all the available system definition files before it handles the topology file. The files are located in the fol- lowing directory relative to the installation prefix: <prefix>/lib/ibdm1.0/ibnl. This man page describes the IBNL file format used to define an arbitrary IB system internal IB connectivity. It outlines the main concepts used by the file, provides details for how to write such a file and provides a formal definition of the file syntax in BNF like format (YACC/Bison readable). 1. Main Concepts The described file format for defining generic system IB connectivity uses the following terminology: NODE The instantiation of an IB device may it be a switch or a channel adapter SUB-SYSTEM The instantiation of a board or module in a system SYSTEM A definition of a board or a module TOPSYSTEM The top most system defined in the given file SUB-SYSTEM MODIFIER Many systems allow multiple variations of their subsystem such as a 12X version of a leaf board in a MTS9600. The modifier is a suffix to the board name. The IBNL format supports assigning multiple names to the same board. 2. File Format The exact file format is provided in section 4. This section only provides extra information regarding the specific semantics of various sections. The IBNL file is line sensitive as it requires different sections to start on a new line. The file is broken into several SYSTEM (optional) and one TOPSYSTEM sections. Each has one or more names. Connections are defined inside the SYSTEM/TOPSYSTEM section only and thus might be of two types: 1. Between any node or sub-system to other node or sub-system 2. From any node or sub-system to a connector of the system. NOTE-1: The top system can define ports that might be redundant and not connected to any lower level board connector. In these cases the ports are simply omitted from the resulting system. This feature enables defining the front panel ports of a 12X/4X such that if a 12X leaf is selected only the 12X front panel ports are used. NOTE-2: Port width and speed provided at the lowest level have precedence over definitions provided at upper levels of the hierarchy. 3. Guidelines for writing a System IBNL File The following procedure should be followed in order to provide a new system IBNL: a) Name the file after the system name: <any sys name (no spaces)>.ibnl b) Define a SYSTEM section for each board included in the system c) The port names of the boards are just strings, we use the simple format of Pn where N is just a serial number but you can pick any name you want. Just make sure it is unique. d) When different flavors of the boards exists like when you have a 4x and 12x option for a board name the optional boards with a modifier postfix. An example for such system that supports a LEAF board of 12x and 4x types would be to define two SYSTEMS: SYSTEM LEAF,LEAF:4x,LEAF:4X ... SYSTEM LEAF:12x,LEAF:12X ... NOTE-3: The instantiations of the LEAF boards in another board or the top system need not specify the postfix and are being decided upon only when the topology file is being parsed. The "SYSTEM LEAF" with no postfix will be used by default. To continue the exam- ple here is how the LEAFs are instantiated in the top system: TOPSYSTEM MyIbSystem LEAF leaf1 ... LEAF leaf2 ... The actual 4x or 12x version of the LEAF board can then be specified in the topology file CFG section to select the right combina- tion of optional boards in the system. An example will be: MyIbSystem N001 CFG: leaf2=12x In this case leaf1 will be 4x as no special modifier is defined for it (and LEAF is by default a 4x leaf). Leaf2 will be 12x as defined in the CFG section. e) Special considerations should be made to design the top system section when several optional board types exist. Top system sec- tion might include multiple definitions for front panel plugs like P1-4x and P1-12x (yes these are just names that should fol- low the writing on the front or back panels). As the different flavors of the boards are not required to define the same names for their ports including a 12x port might leave some of the top level connections dangling (not connected) and thus the front panel ports of the other flavor will be removed from the final system definition. As an example let us consider a simple board LEAF with 3 4x port flavor and one 12x port flavors. We recommend connecting it in the top level using the following scheme: SYSTEM LEAF:4x NODE U1 1 -4x-> 4XP1 2 -4x-> 4XP2 3 -4x-> 4XP3 ... SYSTEM LEAF:12x NODE U1 1 -12x-> 12XP1 ... TOPSYSTEM myIbSystem SUBSYSTEM LEAF leaf1 4XP1 -> L1/P1 4XP2 -> L1/P2 4XP3 -> L1/P3 12XP1 -> L1/P1-12x f) Place the file in the <prefix>/lib/IBDM/ibdm1.0/ibnl directory g) Check the new file syntax by placing it in the ibnl directory as described above, creating a simple topology (ibdm-topo- file file with one such system and running ibdmtr utility to parse it tracing a simple path through it. 4. Formal definition in YACC Syntax: INT ::= ([1-9][0-9]*|0) ; WIDTH ::= (4x|8x|12x) ; SPEED ::= (2.5G|5G|10G) ; NODETYPE ::= (SW|CA|HCA) ; NAME ::= ([[]\*/A-Za-z0-9_.:%@~]+) ; NL: LINE | NL LINE; ONL: | NL; ibnl: ONL systems topsystem; systems: | systems system ; sub_inst_attributes: | sub_inst_attributes sub_inst_attribute NL; sub_inst_attribute: NAME '=' NAME '=' NAME | NAME '=' NAME '=' INT | NAME '=' NAME ; topsystem: TOPSYSTEM system_names NL sub_inst_attributes insts ; system: SYSTEM system_names NL insts ; system_names: system_name | system_names ',' system_name ; system_name: NAME ; insts: | insts node | insts subsystem ; node: node_header NL node_connections ; node_header: NODE NODETYPE INT NAME NAME ; node_connections: | node_connections node_connection NL ; node_connection: node_to_node_link | node_to_port_link ; node_to_node_link: INT '-' WIDTH '-' SPEED '-' '>' NAME | INT '-' WIDTH '-' '>' NAME INT | INT '-' SPEED '-' '>' NAME INT | INT '-' '>' NAME INT ; node_to_port_link: INT '-' WIDTH '-' SPEED '-' '>' NAME | INT '-' WIDTH '-' '>' NAME | INT '-' SPEED '-' '>' NAME | INT '-' '>' NAME ; subsystem: subsystem_header NL subsystem_connections ; subsystem_header: SUBSYSTEM NAME NAME ; subsystem_connections: | subsystem_connections subsystem_connection NL ; subsystem_connection: subsystem_to_subsystem_link | subsystem_to_port_link ; subsystem_to_subsystem_link: NAME '-' WIDTH '-' SPEED '-' '>' NAME NAME | NAME '-' WIDTH '-' '>' NAME NAME | NAME '-' SPEED '-' '>' NAME NAME | NAME '-' '>' NAME NAME ; subsystem_to_port_link: NAME '-' WIDTH '-' SPEED '-' '>' NAME | NAME '-' WIDTH '-' '>' NAME | NAME '-' SPEED '-' '>' NAME | NAME '-' '>' NAME ; AUTHOR
Eitan Zahavi, Mellanox Technologies LTD, eitan@mellanox.co.il IBDM 1.0 2008-06-16 IBDM-IBNL-FILE(1)