Back to Contents Page

Linux Driver Software: Broadcom NetXtreme® User Guide

Linux Driver Software

Broadcom Advanced Server Program Driver Software


Linux Driver Software

Limitations

Packaging

Installing Linux Driver Software

Patching PCI Files

Patching the Driver onto the Kernel

Network Installations

Unloading/Removing the Linux Driver

Setting Values for Optional Properties

Driver Messages

Statistics


Limitations

The current version of the Linux driver has been tested on selected Linux distributions for ix86–64, 2.4x, and 2.6x kernels. Refer to the Distrib.txt file on the installation CD for a list of the specific Linux distributions on which the driver has been tested .

Packaging

The Linux driver is released in the following packaging formats (file names):

Identical source files to build the driver are included in both RPM and TAR source packages. The tar file contains additional utilities such as patches and driver diskette images for network installation.

Installing Linux Driver Software

Installing the Source RPM Package

Building the Driver from a TAR file

NOTE: If a BCM5700 driver is loaded and the Linux kernel is updated, the BCM5700 driver module must be recompiled if the driver module was installed using the source RPM or the TAR package.

Installing the Source RPM Package

  1. Install the source RPM package:

rpm -ivh bcm5700-version.src.rpm

  1. Change the directory to the RPM path and build the binary driver for your kernel (the RPM path is different for different Linux distributions):

cd /usr/src/redhat,OpenLinux,turbo,packages,rpm …

rpm -bb SPECS/bcm5700.spec or rpmbuild -bb SPECS/bcm5700.spec

rpmbuild -bb SPECS/bcm5700.spec (for RPM version 4.x.x)

    NOTE: During your attempt to install a source RPM package, the following message may be displayed:

    error: cannot create %sourcedir /usr/src/redhat/SOURCES

    The most likely cause of the error is that the rpm-build package has not been installed. Locate the rpm-build package on the Red Hat installation media and install it using the following command:

    rpm -ivh rpm-build-version.i386.rpm

    Complete the installation of the source RPM.


  1. Install the newly built package (driver and man page):

rpm -ivh RPMS/i386/bcm5700-version.i386.rpm

The --force option is needed if installing over an existing distribution that may already contain an older version of the driver.

Depending on the kernel, the driver is installed to one of the following paths:

2.2.x kernels:

/lib/modules/kernel_version/net/bcm5700.o

2.4.x kernels:

    /lib/modules/kernel_version/kernel/drivers/net/bcm5700.o

    2.4.x kernels with the bcm5700 driver patched in:

    /lib/modules/kernel_version/kernel/drivers/net/bcm/bcm5700.o

    or

    /lib/modules/kernel_version/kernel/drivers/addon/bcm5700/bcm5700.o

2.6.0 kernels:

/lib/modules/kernel_version/kernel/drivers/net/bcm5700.ko

2.6.0 kernels with bcm5700 driver patched in:

/lib/modules/kernel_version/kernel/drivers/net/bcm/bcm5700.ko

  1. Load the driver:

insmod bcm5700

To configure the network protocol and address, refer to the Linux version-specific documentation.

Building the Driver from the TAR File

  1. Create a directory and extract the TAR files to the directory:

tar xvzf bcm5700-version.tar.gz

  1. Build the driver bcm5700.o as a loadable module for the running kernel:

CD src
make

  1. Test the driver by loading it:
    NOTE: If you are loading the driver on Red Hat 7.3, 2.1 AS, or other newer kernels that have the tg3 driver, refer to "Remove tg3 Driver" in the Distrib.txt file before loading the driver.

insmod bcm5700.o

or, for Linux 2.6 kernels:

insmod bcm5700.ko

No message should be returned if this command runs properly

  1. Install the driver and man page:

    make install

  2. NOTE: See the RPM instructions above for the location of the installed driver.
  3. To configure network protocol and address, refer to the manuals supplied with your operating system.

Patching PCI Files (Optional)

To use the Red Hat kudzu hardware detection utility, a number of files containing PCI vendor and device information need to be patched with information on the BCM57XX series NICs. The most recent Red Hat distributions are included. Apply the appropriate patch by running the patch command. For example, on Red Hat Enterprise Linux 2.1 and 3.0 for i386, apply the patch by doing the following:

patch -N -p1 -d /usr < pci-rh80-i386.patch

Run kudzu:

kudzu

Patching the Driver Into the Kernel (Optional)

Patch files are included for patching the driver into some of the latest 2.4.x kernel source trees. This step is optional and should only be done by users familiar with configuring and building the kernel. The patch modifies the original kernel's source code.

To patch the driver into the kernel

1. Select the patch file that matches your kernel and apply the patch:

patch -p1 -d kernel_src_root < bcm5700-version-2.4.x.patch

where version is the version of the BCM57XX driver and 2.4.x is the version of the kernel to patch (for example, 2.4.10).

NOTE: kernel_src_root is usually /usr/src/linux or /usr/src/linux-2.4.x.
  1. Configure the kernel to include the bcm570x driver. It can be found under Network Device Support > Ethernet (1000 Mbps) > Broadcom BCM5700 support when make menuconfig is run. Select built-in or module for the driver:

cd kernel_src_root
make menuconfig

  1. Compile the kernel:

make dep
make clean
....
....

Network Installations

For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver disk that contains the BCM57XX driver may be needed. The driver disk images for the most recent Red Hat versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from the Red Hat website, http://www.redhat.com.

To create the driver disk, select the appropriate image file and type the following:

dd if=dd.img of=/dev/fd0H1440

Unloading/Removing the Linux Driver

Unloading/Removing the Driver from an RPM Installation

Removing the Driver from a TAR Installation

Unloading/Removing the Driver from an RPM Installation

To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, and then type the following:

rmmod bcm5700

If the driver was installed using rpm, do the following to remove it:

rpm -e bcm5700

Removing the Driver from a TAR Installation

If the driver was installed using make install from the tar file, the bcm5700.o driver file has to be manually deleted from the operating system. See Installing the Source RPM Package for the location of the installed driver.

Setting Values for Optional Properties

Values for optional properties can be set for each installed adapter using command line arguments in the insmod command. Typically, the values for the properties are set in the /etc/modules.conf file (see the man page for the modules.conf file). These properties take the following form:

property=value[,value,...]

Multiple values for the same property are for multiple adapters installed in the system.

NOTES:

  • The default value or other appropriate values are used if you specify an invalid value.
  • Some combinations of property values may conflict and result in failures. The driver cannot detect all such conflicting combinations.

line_speed

The line_speed property selects the line speed of the link. This property is used together with the full_duplex and auto_speed properties to select the speed and duplex operation of the link and the setting of auto-negotiation.

0 Auto-negotiates for the highest speed supported by link partner (default).
10 Sets the speed at 10 Mbit/s.
100 Sets the speed at 100 Mbit/s.
1000 Sets the speed at 1000 Mbit/s.

If line_speed is set to 10, 100, or 1000 Mbit/s, the adapter auto-negotiates for the selected speed (and selected duplex mode) if auto_speed is set to 1. If auto_speed is set to 0, the selected speed and duplex mode are set without auto-negotiation. The 1000 Mbit/s speed must be negotiated for copper twisted-pair links.

auto_speed

The auto_speed property enables or disables auto-negotiation.

0 Disables auto-negotiation.
1 Enables auto-negotiation (default).

This property is ignored and is assumed to be 1 if line_speed is set to 0.

full_duplex

The full_duplex property selects the duplex mode of the link. This property is used together with line_speed to select the speed and duplex mode of the link. This property is ignored if line_speed is 0.

0 Sets the mode to Half-Duplex.
1

Sets the mode to Full-Duplex (default).

rx_flow_control

The rx_flow_control property enables or disables receiving flow control (PAUSE) frames. This property is used together with auto_flow_control.

0 Disables receiving PAUSE frames.
1 Enables receiving PAUSE frames auto_flow_control is set to 0, or advertises PAUSE frame receive if auto_flow_control is set to 1 (default).

tx_flow_control

The tx_flow_control property enables or disables transmitting flow control (PAUSE) frames. This property is used together with auto_flow_control.

0 Disables the transmission of PAUSE frames.
1 Enables the transmission of PAUSE frames if auto_flow_control is set to 0, or advertises PAUSE frame transmit if auto_flow_control is set to 1 (default).

auto_flow_control

The auto_flow_control property enables or disables the auto-negotiation of flow control. This property is used together with rx_flow_control and tx_flow_control to determine the advertised flow control capability.

0 Disables flow control auto-negotiation.
1 Enables flow control auto-negotiation with the capability specified in rx_flow_control and tx_flow_control (only valid if line_speed is set to 0 or auto_speed is set to 1) (default).

mtu

The mtu property enables jumbo frames up to the specified MTU size. The valid range for this property is 1500 to 9000. The default value is 1500, which is standard Ethernet (non-jumbo) MTU size. Note that the MTU size excludes the Ethernet header size of 14 bytes. The actual frame size is MTU size + 14 bytes. Jumbo MTU sizes are not supported on BCM5705/BCM5721/BCM5751 chips.

The MTU size can also be changed using ifconfig after the driver is loaded. Refer to the ifconfig man page for details.

tx_checksum

The tx_checksum property enables or disables hardware transmit TCP/UDP checksum.

0 Disables hardware transmit TCP/UDP checksum.
1 Enables hardware transmit TCP/UDP checksum (default).

rx_checksum

The rx_checksum property enables or disables hardware receive TCP/UDP checksum.

0 Disables hardware receive TCP/UDP checksum.
1 Enables hardware receive TCP/UDP checksum (default).

scatter_gather

The scatter_gather property enables or disables scatter/gather and 64-bit DMA on IA32. This option is only useful when running on TUX-enabled kernels or kernels with zero-copy TCP.

0 Disables scatter/gather and 64-bit DMA on IA32.
1 Enables scatter/gather and 64-bit DMA on IA32 (default).

tx_pkt_desc_cnt

The tx_pkt_desc_cnt property configures the number of transmit descriptors. The default value is 100. The valid range of values is from 1 to 600. Depending on kernel and system architecture, the driver may require up to 268 bytes per descriptor. The driver may not be able to allocate the required amount of memory if this property is set too high. This property should not be set to a value of less than 80 if adaptive_coalesce is enabled.

rx_std_desc_cnt

The rx_std_desc_cnt property configures the number of receive descriptors for frames up to 1528 bytes. The default value is 200. The valid range of values is from 1 to 511. This property should not be set with a value less than 80 on systems with high network traffic. Setting the value of this property higher allows the adapter to buffer larger bursts of network traffic without dropping frames, especially on slower systems. Note that the driver may not be able to allocate the required amount of memory if the value of this property is set too high. This property should not be set to a value of less than 50 if adaptive_coalesce is enabled.

rx_jumbo_desc_cnt

The rx_jumbo_desc_cnt property configures the number of receive descriptors for jumbo frames larger than 1528 bytes. The default value is 128 and the valid range of values is from 1 to 255. When jumbo frames larger than 1528 bytes are used, the value of this property should not be set lower than 60 on systems with high network traffic. Setting the value of this property higher allows the adapter to buffer larger bursts of jumbo traffic without dropping frames, especially on slower systems. Depending on kernel and system architecture, the driver may require up to 268 bytes per descriptor. Each descriptor also requires a buffer the size of a maximum jumbo frame. On systems with insufficient memory, it may be necessary to set a lower value for this property. When the maximum frame size is less than 1528 (MTU size less than 1514), this property is not used and always has a value of 0.

adaptive_coalesce

The adaptive_coalesce property enables or disables adaptive adjustments to the various interrupt coalescing properties. Enabling this property allows the driver to dynamically adjust the interrupt coalescing properties to achieve high throughput during heavy traffic and low latency during light traffic. The value of rx_std_desc_cnt (and rx_jumbo_desc_cnt if using Jumbo frames) should not be set less than 50, and the value of tx_pkt_desc_cnt should not be set less than 80 when this property is enabled.

0 Disables adaptive adjustments to the various interrupt coalescing properties.
1 Enables adaptive adjustments to the various interrupt coalescing properties (default).

rx_coalesce_ticks

The rx_coalesce_ticks property configures the number of 1-microsecond ticks before the NIC generates a receive interrupt after receiving a frame. This property works in conjunction with the rx_max_coalesce_frames property. An interrupt is generated when either of these thresholds is exceeded. A 0 means this property is ignored and an interrupt is generated when the rx_max_coalesce_frames threshold is reached. The valid range of values is from 0 to 500, and the default value is 80. This property is not used and is adjusted automatically if adaptive_coalesce is set to 1.

rx_max_coalesce_frames

The rx_max_coalesce_frames property configures the number of received frames before the NIC generates receive interrupt. The valid range of values is from 0 to 100, and the default value is 15. The values of both this property and rx_coalesce_ticks cannot be set to 0; otherwise, no receive interrupts are generated. The value should also be set significantly lower than the value of rx_std_desc_cnt (and rx_jumbo_desc_cnt if using jumbo frames). This property is not used and is adjusted automatically if adaptive_coalesce is set to 1.

tx_coalesce_ticks

The tx_coalesce_ticks property configures the number of 1-microsecond ticks before the NIC generates a transmit interrupt after transmitting a frame. This property works in conjunction with tx_max_coalesce_frames. An interrupt is generated when either of these thresholds is exceeded. A 0 means this property is ignored and an interrupt is generated when the tx_max_coalesce_frames threshold is reached. The valid range of values is from 0 to 500, and the default value is 200. This property is not used and is adjusted automatically if adaptive_coalesce is set to 1.

tx_max_coalesce_frames

The tx_max_coalesce_frames property configures the number of transmitted frames before the NIC generates a transmit interrupt. The valid range of values is from 0 to 100, and the default value is 35. The values of both this property and tx_coalesce_ticks cannot be set to 0; otherwise, no transmit completion interrupt can be generated. This property should always be set to a value lower than the value of tx_pkt_desc_cnt. This property is not used and is adjusted automatically if adaptive_coalesce is set to 1.

stats_coalesce_ticks

The stats_coalesce_ticks property configures the number of 1-microsecond ticks between periodic statistic block DMAs. The valid range of values is from 0 to 3 600 000 000, and the default value is 1 000 000 microseconds (1 second). Setting this property to 0 disables statistics updates. This property is not used and is set to the default value if adaptive_coalesce is set to 1.

enable_wol

The enable_wol property enables or disables Magic Packet™ Wake on LAN when the system is shut down. Note that not all systems support Wake on LAN.

0 Disables Magic Packet Wake on LAN (default).
1 Enables Magic Packet Wake on LAN.

enable_tso

The enable_tso property enables or disables the TCP Segmentation Option (TSO) when you are using kernels that support it.

0 Disables TSO (default).
1 Disables TSO.

vlan_tag_mode

The vlan_tag_mode property controls the stripping of VLAN tags on incoming packets, and is used to allow VLAN tagged ASF or IPMI packets to be received properly.

0 Auto mode (default).
1 Normal strip mode.
2 Forced strip mode.

In normal mode, VLAN tags are stripped only if VLANs are registered by the IEEE 802.1q VLAN module or BASP. In forced strip mode, VLAN tags are always stripped. Auto mode selects normal strip mode if ASF/IPMI is disabled, or selects forced strip mode if ASF/IPMI is enabled.

delay_link

If the delay_link property is set to 1, the driver returns -EOPNOTSUPP when the SIOCGMIIREG or ETHTOOL_GLINK I/O controls are called during the first 6 seconds after driver reset. When the driver resets the NIC during ifconfig, the link is dropped and it may take several seconds for the link to come up after auto-negotiation completes. Some applications, such as ifup, may not wait long enough for the link before giving up. Setting this property to 1 may get around such problems. The default value is 0, which means that the driver always return true link states to all I/O control calls, when applicable.

disable_d3hot

If the disable_d3hot property is set to 1, the driver never puts the device in the D3Hot power state when the NIC is shut down or is suspended. If set, this property also disables the Wake on Lan setting. A rare D3Hot related problem was seen during repeated shutdown of PCI Express devices on systems running 2.6 kernels.

Driver Messages

The following are the most common sample messages that may be logged in the /var/log/messages file. Use dmesg -nlevel to control the level at which messages appear on the console. Most systems are set to level 6 by default.

Broadcom Gigabit Ethernet Driver bcm5700 with Broadcom NIC Extension (NICE) version (date)

Driver Signon

eth#: Broadcom BCM5701 1000Base-T found at mem faff0000, IRQ 16, node addr 0010180402d8
eth#: Broadcom BCM5701 Integrated Copper transceiver found
eth#: Scatter-gather ON, 64-bit DMA ON, Tx Checksum ON, Rx Checksum ON

NIC Detected

bcm5700: eth# NIC Link is Up, 1000 Mbps full duplex

Link Up and Speed Indication

bcm5700: eth# NIC Link is Down

Link down indication.

Statistics

Detailed statistics and configuration information can be viewed in the /proc/net/nicinfo/eth#.info file.


Broadcom Advanced Server Program (BASP) Driver Software

NOTES:

BASP Overview

Packaging

Installing BASP

BASP Files

Configuring Teaming

Installing Broadcom NICE Patches

Uninstalling the BASP RPM Package

Removing a Physical Interface in Generic Trunking Mode and 802.3ad Mode

Installing BASP SNMP Agent

Known Problems


BASP Overview

Broadcom Advanced Server Program is a kernel module designed for Linux 2.4.x kernels. See Broadcom Advanced Server Program Overview for a detailed description.

Broadcom Advanced Server Program also provides remote management through the SNMP protocol, and this package is installed separately (see Installing BASP SNMP Agent).

Back to BASP Contents

Packaging

Broadcom Advanced Server Program is released in 2 packaging formats: source RPM and compressed tar formats. The file names for the 2 packages are basplnx-version.src.arch.rpm and basplnxversion.arch.tgz, respectively. Identical source files to build the driver are included in both RPM and TAR source packages.

Back to BASP Contents

Installing Broadcom Advanced Server Program

Broadcom Advanced Server Program for Linux is shipped in mixed forms; the platform and kernel-specific files are in source code, and the core file is in object form. Four packages are shipped in this release: three RPM packages and one tar archive. The tar archive for i386 platform is basplnx-version.i386.tgz.

Back to BASP Contents

Installing the BASP Source RPM Package

  1. Run % rpm -i basplnx-version.src.archrpm.
  2. Change to the directory containing the RPM package and build the binary driver for the kernel (use rpmbuild for Red Hat Linux 2.1 and 3.0 or later). The path to the RPM package is different for different Linux distributions.

% CD /usr/src/redhat

% rpm -bb SPECS/basplnx.spec

or

rpmbuild -bb SPECS/basplnx.spec

NOTE: During the installation process, the following message may be displayed:

error: cannot create %sourcedir /usr/src/redhat/SOURCES

The most likely cause of the error is that the rpm-build package has not been installed. To install the rpm-build package, see Installing the Source RPM Package.

  1. Install the driver and other required files:

% rpm -i RPMS/i386/basplnx-version.archrpm

  1. Load the driver:

% insmod basp

See Configuring Teaming for Red Hat Distributions to set up the teams.

Back to BASP Contents

Installing the BASP TAR Archive

Uncompress and expand the tar archive:

% tar xvfz basplnx-version.arch.tgz

To install the BASP TAR archive

  1. Change to the directory where the BASP source files are located:

% CD basplnx-version

  1. Build the kernel module, basp.o:

% make

NOTE: The Make process automatically builds the correct module for different kernel options, for example, symbol version and SMP support. There is no need to define -DMODVERSIONS in the Makefile.
  1. Create a device file and copy files:

% make install

  1. Update the module reference:

% depmod -a

  1. Load the driver:

% insmod basp

See Configuring Teaming for Other Linux Distributions to set up the teams.

Back to BASP Contents

BASP Files

File Name Description
makefile makefile
baspcfg Precompiled configuration utility
bcmtype.h Commonly used type header file
blf.c BASP module entry points
blf.h IOCTL interface
blfcore.h Core interface
blfcore.o Precompiled core object
blfopt.h Automatically generated header file from Make
blfver.h Version header file
nicext.h NICE header file
pal.c Platform abstraction implementation
pal.h Header for platform abstraction
release.txt Release notes for BASP driver for Linux 2.4x and 2.6x kernels
nice-2.2.16 NICE enabled driver for Linux 2.2 kernel
nice-2.4.16 NICE enabled driver for Linux 2.4 kernel
scripts Contains sample scripts
scripts/basp Initialization script, goes to /etc/rc.d/init.d
scripts/baspteam Start/stop script, goes to /etc/basp
scripts/baspif Start/stop network, interface, goes to /etc/basp
scripts/team-sample Sample script of an SLB team with three adapters
scripts/team-gec Sample script of GEC team with three adapters
scripts/team-vlan Sample script of an SLB team with 2 VLANs
basp.4
man page
baspcfg.8 man page for baspcfg utility

Back to BASP Contents

Configuring Teaming

Configuring Teaming for Linux Red Hat Distributions

NOTES:

  • If you do not enable LiveLink™ when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is recommended. This minimizes the downtime due to spanning tree loop determination when failing over. LiveLink mitigates such issues.
  • When adding 64 VLANs, the 64th VLAN must have a VLAN ID of 0 (63 VLANs are tagged and 1 VLAN is untagged).
  • Although teaming increases the load on the CPU, teaming does not change the CPU utilization rate.
  • VLANs are not supported on non-Broadcom adapters. It is supported on the Alteon® adapters if the ALT.LAN provided by Broadcom is used. If a non-Broadcom adapter is a member of a failover team, VLANs are not supported for that team.
  • If remote management (IPMI) is enabled on a device, the IPMI service could be disrupted when the device is added to a team other than an SLB team.

Because Red Hat distribution installations do not automatically load drivers for network devices unless the device is configured with an IP address, you must manually configure a network script file for all of the physical (installed) adapters that are potential team members. Network script files are located in the /etc/sysconfig/network-scripts file. The file name of the script file must be prefixed with ifcfg- followed by the physical adapter alias. For interface eth0, you would create a file with the name ifcfg-eth0, then add the content, as shown in the following example.

Example

DEVICE = eth0
BOOTPROTO = static
ONBOOT = yes

Broadcom Advanced Server Program includes several scripts for configuring teams for Linux Red Hat distributions.

Enabling Dynamic Host Configuration Protocol (DHCP) is not recommended for members of an SLB team.

To configure a team

  1. Copy a configuration script from the /etc/basp/samples directory to the /etc/basp directory. The configuration script name must be prefixed with team-.
  2. Modify the configuration script:
    1. Change the team type.
    2. Add/delete a physical network interface.
    3. Add/delete a virtual network interface.
    4. Assign an IP address to each virtual network interface.
    See BASP Scripts for Configuring Teams for the syntax of the configuration script or in the /etc/basp/sample/team-sample script file. When configuring teaming, you must designate at least one adapter as a primary team member.
  1. Manually start the team for the first time:

% /etc/init.d/basp start

NOTES:

  • This step is only required for the first time installation. The team configuration is automatically started on subsequent reboots.
  • If not all the virtual network interfaces are configured with an IP address, an error in starting the BASP team results. When this happens, modify the configuration script to assign an IP address for all of the virtual network interfaces.
  • Forming multiple teams is possible by copying the sample files to the /etc/basp/team-name file and modifying this file as described in the sample file.
  • For instructions on how to create more that one virtual interface (VLAN) for each team, refer to the appropriate section in the sample files.
Using BASP Scripts for Configuring Teams

The team-sample configuration script creates an SLB team named Team1. The team consists of 3 network interfaces, eth0, eth1, and eth2. All 3 interfaces are primary adapters. One virtual interface named sw0 is added to the team and the VLAN is not enabled. This script is part of the Broadcom Advanced Server Program driver for Linux distributions.

The syntax used in the sample scripts, team-sample and team-gec (see BASP Files) is the same as the syntax shown in the following table:

Configurable Property Description
TEAM_ID This number uniquely identifies a team.
TEAM_TYPE

0 = SLB

1 = Generic Trunking/GEC/FEC

2 = 802.3ad

3 = SLB (Auto-Fallback Disable)

LIVE_LINK_ENABLE

1 = SLB (with LiveLink)

0 = SLB (without LiveLink)

LiveLink is supported for SLB teams only. A value of 1 is ignored when TEAM_TYPE is not 0 (SLB).

TEAM_NAME ASCII name of the team.
TEAM_PAx_NAME ASCII name of the physical interface x, where x can be 0 to 7.
TEAM_PAx_ROLE

Role of the physical interface x:

0 = Primary

1 = Hot standby.

This field must be 0 for Generic Trunking/GEC/FEC teams and 802.3ad teams.

TEAM_PAx_IP

Unique IP address to be used by each adapter when LiveLink is enabled. The format should be x.x.x.x.

If an IP address unique to the subnet is not assigned, the adapter will not be used when LiveLink is enabled.

TEAM_VAx_NAME ASCII name of the virtual interface x, where x can be 0 to 63.
TEAM_VAx_VLAN

802.1p VLAN ID of the virtual interface x.

For an untagged virtual interfaces (without VLAN enabled) , set the VLAN ID to 0. The valid VLAN ID can be 0 to 4094.

TEAM_VAx_IP IP address of the virtual interface x. The format should be aa.bb.cc.dd.
TEAM_VAx_NETMASK Subnet mask of the virtual interface x. The format should be mm.nn.oo.pp.
TEAM_VAx_BROADCAST Optional broadcast address of the virtual interface x. The format should be qq.rr.ss.tt.
TEAM_VAx_GW Optional default gateway. The format should be ww.xx.yy.zz. Usually one default gateway is specified for the system, and it should be reachable from one network interface.
PROBE_TARGET_IPx Target IP addresses for the LiveLink option. The format shoud be x.x.x.x. The first LiveLink probe target is required when LiveLink is enabled. Up to 3 additional probe targets may be defined.
PROBE_INTERVAL The interval in seconds between sending LiveLink packets to probe targets. Allowable values are 1, 2, 5, 10, 20. 30 and 60.
MAX_RETRY The number of consecutively missed responses from a LiveLink probe target before a failover is triggered. The value is n * 5 (for example, a setting of 5 = Value of 25 (5 * 5)). Allowable values of n are 1 to 10.
NOTE: Teaming scripts are intended for Red Hat distributions only. Do not use teaming scripts with other Linux distributions.

Configuring Teaming for Other Linux Distributions

Broadcom Advanced Server Program Configuration (baspcfg) is a command-line tool to configure the BASP teams, add/remove NICs, and add/remove virtual devices. This tool can be used in custom initialization scripts. Read your distribution-specific documentation for more information on startup procedures.

Example

baspcfg v6.2.7 — Broadcom Advanced Server Program Configuration Utility Copyright (c) 2000–2004 Broadcom Corporation. All rights reserved.

Usage: baspcfg command

BASP Configuration Commands

Command Action
addteam tid type tname Create a team.
addteam tid type tname target ip 0–3 max interval max retries Create a LiveLink team.
delteam tid Delete a team.
txoffld tid y|n Enable or disable tx offload features
addva tid vlan_id vname [macaddr] Add a virtual adapter to a team.
delva tid vlan_id Delete a virtual adapter from a team.
bind tid role device Bind a physical adapter to a team.
bind tid role device nic ip Bind a LiveLink physical adapter to a team.
unbind tid device Unbind a physical adapter from a team.
show tid Display team configurations.

BASP Configuration Command Placeholders

Placeholder Description
tid A unique ID for each team, starting from 0.
type

Team type:

0 = SLB

1 = FEC/GEC

2 = 802.3ad

3 = SLB (Auto-Fallback Disable)

tname ASCII string of the team.
vlan_id VLAN ID: from 1 to 4094, 0 = untagged or no VLAN.
vname ASCII string of the virtual device.
macaddr MAC address (optional), for example, 00:10:18:00:11:44.
role

Role of the physical device:

0 = primary

1 = hot-standby

device ASCII string of the physical device, for example. eth0.
target/nic ip Probe target IP address (for example, 192.168.1.1)
probe interval The interval in seconds between sending LiveLink packets to probe targets. Allowable values are 1, 2, 5, 10, 20. 30 and 60.
max retry The number of consecutively missed responses from a LiveLink probe target before a failover is triggered. The value is n * 5 (for example, a setting of 5 = Value of 25 (5 * 5)). Allowable values of n are 1 to 10.

NOTE: The baspcfg command can be run only in Super User mode. Attempting to run baspcfg as a standard user returns the following error message:

Error in communicating to BASP Module. Is it loaded?

When configuring teaming, you must designate at least one adapter as a primary team member.

Configuring LiveLink™

Read the following notes before you attempt to configure LiveLink.

NOTES:

  • Before you begin, review the description of LiveLink.
  • A probe target must be on the same subnet as the team, have a valid (not a broadcast, multicast, or unicast), statically-assigned IP address, and be highly available (always on).
  • You can specify up to 4 probe targets.

To configure LiveLink

  1. Open the BASP directory.

    /etc/basp

  2. Change to the SAMPLES directory.

    cd samples

  3. Copy the team-sample file to the BASP directory.

    cp /etc/basp/samples/team-sample /etc/basp/

  4. Edit the team-sample script to enable LiveLink, assign IP addresses to probe targets, assign IP addresses to team members, specify the time interval for sending link packets to probe targets, and to specify the maximum number of probe retries. You must delete the # sign at the beginning of each respective line item.
    1. Go to #LIVE_LINK_ENABLE=0 and change the value from 0 to 1.
    2. Go to #PROBE_TARGET_IP0= and type the target IP address.

      NOTE: Only the first probe target is required. You can specify up to 3 additional probe targets to serve as backups by assigning IP addresses to the other #PROBE_TARGET_IPx= line items.

    3. Go to #TEAM_PA0_IP= (the first physical Interface in the team) and type the team member IP address.

      NOTE: You must assign an IP address to each team member for that member interface to be used by the team.

    4. Go to #PROBE_INTERVAL=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
    5. Go to #MAX_RETRY=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
  5. Save the file.

NOTES:

  • Before you start the team, it is recommended that you use the ifconfig command to temporarily bring up one of the member interfaces using the IP address that you specified for that member in step 4.C. Then ping the probe target that you specified in step 4.B to verify connectivity. After you confirm that there is connectivity, type the ifconfig down command to bring down the interface. Now you are ready to start the team.
  • To view the status of LiveLink activity (the number of probes sent and the number of probe retries) for the team, use the baspcfg show command.

CAUTION:

  • If the MAC address of a probe target is changed for any reason, even though the IP address remains the same, the team must be restarted before that target is again a valid target.
  • If the IP address of the probe target is changed, the value of PROBE_TARGET_IPx must be updated, and the team must be restarted.

To Configure LiveLink in VLAN-tagged environments

CAUTION: For the teams with VLANs (on which LiveLink is enabled): to be able to communicate with the probe target, both the probe target and the team must be on VLAN ID 0 (untagged). Otherwise, the team loses connectivity.
  1. Ensure that the team has an untagged VLAN (VLAN ID 0).
  2. Ensure there is network connectivity between the team and the probe target on the untagged VLAN.
  3. Configure LiveLink as specified in To configure LiveLink.

Back to BASP Contents

Installing Broadcom NICE Patches

Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.4.16 kernel distribution. To install patched drivers:

  1. Copy the Broadcom NICE header file (nicext.h) to the appropriate Linux kernel include directory:

% cp /usr/src/nice-2.4.16/nicext.h /usr/src/linux/include/linux

  1. Rename the original network device driver under the Linux kernel source tree (/usr/src/linux/drivers/net).
  2. Copy the patched drivers to the Linux kernel network driver source directory (for example, /usr/src/linux/drivers/net).
  3. Follow the kernel rebuild instructions to configure kernel support for these drivers:

% CD /usr/src/linux
% make config

  1. If the patched drivers are configured into the kernel, go to step 7. If the patched drivers are configured as modules, go to step 6.
  2. When supporting only the module version of these drivers, it is possible to run the following to compile patched drivers and to install them into the proper module directory:

% make modules
% make modules_install

There is no need to compile the complete kernel. Go to step 8.

  1. Rebuild the kernel to compile these patched drivers:

% make clean
% make dep
% make

  1. Either reboot the system or unload/load the patched modules. Then run the configuration scripts to test the patch.

Back to Broadcom Advanced Server Program Contents

Uninstalling the BASP RPM Package

  1. Uninstall the BASP RPM package:

% rpm -e basplnx

  1. Reboot the system:

% reboot

Back to BASP Contents

Removing a Physical Interface in Generic Trunking and IEEE 802.3ad Mode

In Generic Trunking and IEEE 802.3ad mode, all of the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface that is bound to the team. Removing the first physical interface from the team using baspcfg and binding it directly to the protocol could lead to having a duplicate MAC address problem on the network. If the removed physical interface does not participate in any traffic, no problem occurs.

To properly remove a physical interface

  1. Backup the original team configuration script:

% cp /etc/basp/team-gec /etc/basp/backup-gec

NOTES:

  • team-gec is the name of the configuration script.
  • backup-gec is the name of the backup script. The name of the backup script must not be prefixed with team-.
  1. Modify the team configuration script to remove the physical interface.
  2. Stop the team that is running:

% /etc/basp/baspif /etc/basp/backup-gec stop
% /etc/basp/baspteam /etc/basp/backup-gec del

  1. Restart the team.

% /etc/basp/baspteam /etc/basp/team-gec add
% /etc/basp/baspif /etc/basp/team-gec start

Back to BASP Contents

Installing BASP SNMP Agent

This BASP SNMP agent is designed to support the configuration and statistics information pertaining to the BASP driver. The BASP SNMP agent is available in two packaging formats: TAR archive and RPM. Both packages include the same script and MIB files.

Installing BASP SNMP Agent from the TAR Archive

  1. Uncompress and expand the tar archive:

% tar xvfz baspsnmp-version.tar

  1. Install the TAR archive
  2. Copy the getBaspInfo and genBaspTraps script files to the /usr/bin directory.
  3. Copy the BASP-Config-MIB.txt, BASP-Statistics-MIB.txt and Brcm-BSAPTrap-MIB.txt files to the /usr/share/snmp/mibs directory.
  4. Locate the snmpd.conf file (it is normally located at: /etc/snmp or /usr/lib/snmp or $HOME/.snmp) and add the following lines to the snmpd.conf file:

pass .1.3.6.1.4.1.4413.1.2.1 /usr/bin/getBaspInfo
pass .1.3.6.1.4.1.4413.1.2.2.1 /usr/bin/getBaspInfo
pass .1.3.6.1.4.1.4413.1.2.2.2 /usr/bin/getBaspInfo
pass .1.3.6.1.4.1.4413.1.2.2.3 /usr/bin/getBaspInfo

  1. Stop the snmpd daemon and restart it:

% /etc/init.d/snmpd stop
% /etc/init.d/snmpd start

  1. Run the genBaspTraps script to allow monitoring of the BASP trap events:

% genBaspTraps

If BASP trap event monitoring is no longer needed, terminate this script by pressing CTRL+C.

The snmpget and snmpgetnext commands can be used to receive the BASP SNMP objects such as:

% snmpget localhost public BASP-Config-MIB::btTeamNumber
% snmpgetnext localhost public BASP-Config-MIB::btTeamNumber

BASP SNMP objects are provided in the following text files:

BASP-Config-MIB.txt

BASP-Statistics-MIB.txt

Brcm-BSAPTrap-MIB.txt

Installing BASP SNMP Agent from the RPM Package

  1. Install BASP SNMP Agent from the RPM package:

% rpm -i baspsnmp-version.i386.rpm

This modifies the snmpd.conf configuration file to add support for the BASP SNMP agent.

  1. Follow steps 4–6 in Installing BASP SNMP Agent from the TAR Archive.
NOTE: The current RPM installation fails to append the additional directives to the snmpd.conf file that are needed to support BASP objects. To modify the snmpd.conf file, see step 3 in Installing BASP SNMP Agent from the TAR Archive.

BASP SNMP Files

File Name Description
genBaspTrap Script monitoring the BASP trap events
getBaspInfo Script to process SNMP get/getnext inquiries
BASP-Config-MIB.txt SNMP MIB file for BASP configuration objects
BASP-Statistics-MIB.txt SNMP MIB file for BASP statistics objects
Brcm-BSAPTrap-MIB.txt SNMP MIB file for BASP trap objects
release.txt Release notes about the BASP SNMP agent

Uninstalling BASP SNMP Agent from the RPM Package

  1. Uninstall BASP SNMP.

% rpm -e baspsnmp-version.i386.rpm

  1. Reboot the system.

% reboot

Back to BASP Contents

Known Problems

IEEE 802.3ad team-member links disconnect and reconnect continuously when they are connected to the HP2524 switch. This is a third-party issue. It is seen only when configuring an IEEE 802.3ad team with greater than 2 members on the server and connecting an HP2524 switch with LACP enabled as passive or active. The HP switch shows an LACP channel being brought up successfully with only 2 members. All other member links disconnect and reconnect. This does not occur with a Cisco Catalyst 6500.

Back to BASP Contents


Back to Contents Page