These are the installation notes for U-Net version 2.1, 13-January-1998. Please see the U-Net documentation tree and release notes first.
Installation of U-Net is not automated, but it's meant to be quite easy. There are two steps: Compiling the code, and installing the various pieces. Compilation is covered in the next section, and installation for each implementation is covered later in this document.
The first step is to unpack the U-Net release distribution; create a directory unet-2 and unpack the gzipped tarfile there. The subdirectories are described in the Release Notes.
Before you compile the U-Net release, please check the Makefiles
in each directory relevant for your release. All of the paths and
CFLAGS should be correctly set; however, you may need to make small
changes (e.g., if you are using a compiler other than gcc).
Note that the SBA-200 device driver for Solaris is compiled using
the SparcWorks cc compiler, and for SunOS is compiled using
cc as shipped with the operating system. All other code is
compiled using gcc.
Note that all Makefiles assume use of GNU make. If you don't
have (or want to use) GNU make, you'll need to modify the
Makefiles appropriately.
If you are using a FORE 200-series ATM board, please edit the
file utils/uload-fore200/Makefile and modify the TOOLIBPATH
variable for your system. You need to have toolib.a as
compiled from the GNU i960 tools distribution (see the
release notes for details).
The subdirectory utils/uload-fore200/i960-binaries contains a binary
of this library which may work for you; however, you need to
modify the TOOLIBPATH in this Makefile anyway.
Other than this we hope that no other Makefile changes are necessary.
Please let us know if this isn't so.
To compile the release, issue the command
For example, to compile for the PCA-200 under Linux, issue
To remove all binary files for a given architecture/device, use
Once compiled, the various binary files will be left in the
subdirectories
Notes on installing each U-Net driver are included below.
First, patch your kernel with the bigphysarea patch in
install/linux. This adds several files to /usr/src/linux/init
to allocate physical memory at boot time (the amount is controlled in
init/unet.h) which can be used for U-Net endpoint mappings.
Patches are available for 1.3.71 and 2.0.30 kernels.
If you are using Linux 2.0.30 or later, you also need to patch the
kernel with install/linux/linux-2.0.30.tulip.patch which
adds two entries to the list of symbols exported by the kernel for
kernel modules. You can apply this patch before or after the
bigphysarea patch.
Install the new kernel and reboot. Be sure to use the LILO boot option
The Tulip device driver is located in dev-tulip-linux/tulip.o.
It is a stand-alone kernel module. Simply load it with
In order to use the device you need to create device nodes in /dev.
The major number of the U-Net device is defined in
devtulip.h. If this major number conflicts with your system,
change it and recompile the driver.
The device numbering scheme works as follows: The minor number of the
device corresponds to a single endpoint. devtulip.h contains a
definition for TULIP_MAX_ENDPTS, the maximum number of endpoints
supported by each physical Tulip device. If TULIP_MAX_ENDPTS
is 10, minor numbers 0 through 9 correspond to the first Tulip device in
the system, 10 through 19 to the second, and so forth.
You should create at least one minor device node in order to use the
system; you may wish to create one for each potential endpoint. For
each, execute the command
You should also chmod 777 the device entries, so that non-root
users can use them.
Now you're ready to test it out; see the section Testing
below.
First, patch your kernel with the bigphysarea patch in
install/linux. This adds several files to /usr/src/linux/init
to allocate physical memory at boot time (the amount is controlled in
init/unet.h) which can be used for U-Net endpoint mappings.
Patches are available for 1.3.71 and 2.0.30 kernels. Install the new kernel
and reboot. Be sure to use the LILO boot option
The PCA-200 device driver is located in dev-pca200-linux/pca200.o.
It is a stand-alone kernel module. Simply load it with
In order to use the device you need to create device nodes in /dev.
The major number of the U-Net device is defined in
devpca200.h. If this major number conflicts with your system,
change it and recompile the driver.
You should create at least one minor device node in order to use the
system; you may wish to create one for each potential endpoint. For
each, execute the command
You should also chmod 777 the device entries, so that non-root
users can use them.
The next step is to load the i960 firmware onto the board. This can
be done at any time after the PCA-200 device driver is loaded.
See Loading the Firmware, below.
First, patch your kernel with the bigphysarea patch in
install/linux. This adds several files to /usr/src/linux/init
to allocate physical memory at boot time (the amount is controlled in
init/unet.h) which can be used for U-Net endpoint mappings.
Patches are available for 1.3.71 and 2.0.30 kernels. Install the new kernel
and reboot. Be sure to use the LILO boot option
The Myrinet device driver is located in dev-myrinet-linux/unet-myrinet.o.
It is a stand-alone kernel module. Simply load it with
In order to use the device you need to create device nodes in /dev.
The major number of the U-Net device is defined in
devmyrinet.h. If this major number conflicts with your system,
change it and recompile the driver.
You should create at least one minor device node in order to use the
system; you may wish to create one for each potential endpoint. For
each, execute the command
You should also chmod 777 the device entries, so that non-root
users can use them.
The next step is to load the LanAI firmware ("LCP") onto the board.
This can be done at any time after the Myrinet device driver is loaded.
See Loading the Firmware, below.
The SBA-200 device driver is a standalone kernel module, located in
dev-sba200-sunos/sba200.o.
Simply load it with a command such as
In order to use the device you need to create device nodes in /dev.
The major number of the U-Net device is defined in
devsba200-sunos.h. If this major number conflicts with your system,
change it and recompile the driver.
You should create at least one minor device node in order to use the
system; you may wish to create one for each potential endpoint. For
each, execute the command
You should also chmod 777 the device entries, so that non-root
users can use them.
The next step is to load the i960 firmware onto the board. This can
be done at any time after the SBA-200 device driver is loaded.
See Loading the Firmware, below.
Note that use of the U-Net driver for SunOS is probably incompatible
with use of the FORE Systems SBA-200 kernel driver. For best results
use a SunOS kernel without the FORE Systems driver installed to prevent
conflicts.
The SBA_200 device driver is a standalone kernel module, located in
dev-sba200-solaris/sba200. Use of the U-Net SBA-200
driver is probably incompatible with use of the original FORE SBA-200
driver (as they attach to the same device properties). You should
uninstall your original FORE driver first, saving copies of the various
files modified below.
To install the driver,
This will load the device driver into the system, and automatically
create /dev/sbaNNN entries for each endpoint, with
the numbering system described above.
The next step is to load the i960 firmware onto the board. This can
be done at any time after the SBA-200 device driver is loaded.
See Loading the Firmware, below.
Note that use of the U-Net driver for SunOS is probably incompatible
with use of the FORE Systems SBA-200 kernel driver. For best results
unload the FORE Systems driver before loading the U-Net driver, and
vice versa.
The Myrinet device driver is a standalone kernel module, located in
dev-myrinet-solaris/unet-myrinet. Use of the U-Net Myrinet
driver is incompatible with any other Myrinet driver or code, such
as Myricom's original driver, Fast Messages, and so on. You should
uninstall any other Myrinet driver before attempting to install
U-Net. In most cases it is sufficient to simply unload the driver
with a command such as:
To install the driver,
This will load the device driver into the system, and automatically
create /dev/myriNNN entries for each endpoint, with
the numbering system described above.
The next is to load the LanAI firmware ("LCP") onto the board. This can be
done at any time after the Myrinet device driver is loaded.
See Loading the Firmware, below.
For U-Net implementations on FORE Systems ATM boards, the i960 firmware
must be loaded after installing the appropriate kernel driver. The
uload program included in the U-Net distribution is used.
There are different firmware binaries for each FORE architecture. They
are:
The uload binary is located in
utils/uload-fore200/bin-arch where arch is one of
pca200, sba200-sunos, or sba200-solaris.
To download the i960 firmware using uload, do the following:
You should see messages such as the following:
At this point the i960 firmware should be running and you're ready
to test the system out.
For U-Net implementations on Myrinet LanAI3.x and LanAI4.x boards,
the LanAI firmware (LCP, or LanAI Control Program)
must be loaded after installing the appropriate kernel driver. The
uload program included in the U-Net distribution is used.
There are different firmware binaries for each Myrinet board architecture.
They are:
The uload binary is located in
utils/uload-lanai/bin-arch where arch is
one of myrinet-linux, myrinet-solaris, or myrinet-bsdi.
Use of uload depends on a configuration file which informs the LCP
of the Myrinet network topology. This is necessary so that the LCP knows
the route from the local node to all other nodes on the network.
The format of the file is
For example, a simple four-port switch with four hosts connected would
use the configuration file:
Needless to say the format of the configuration file must correspond to
the topology of your network. If a port is left unconnected you can use
a 'dummy' placeholder hostname for that port, but all hostnames must be
unique.
The format of the configuration file is identical to that used by
Myrinet Fast Messages from the University of Illinois.
(Thanks to that group for allowing me to steal their route-generation code!)
Before running uload you must define the environment variable
UNET_HOSTS which assigns a numeric ordering to hostnames in the
network. For example,
To download the LanAI firmware using uload, do the following:
The test directory contains three programs which test the basic
U-Net functionality and the libunet library:
The syntax for pingpong is
The addr argument is a string corresponding to the address of
some other machine communicating via U-Net. It is converted by routines
in libunet to the appropriate format for U-Net channel addresses
(see U-Net Device Reference). This argument
should be a string in one of three formats:
The size argument is limited by the maximum size of a PDU
supported by the U-Net device; see Device Reference
for details.
For example, to pingpong 40-byte messages between two systems (one
at VCI 100, the other at VCI 200), you would execute the commands:
On the sending end you should see something like:
The syntax for loopback is
You should see something like:
The syntax for window is
Note that for large values of window-size messages may be silently
dropped, either in your switch or by the U-Net implementation (if not enough
free buffers or receive FIFO entries are available, for example). There is
some practical limit to the arguments to window which should be
determined empirically; no intelligent flow-control is being used by this
benchmark application!
On the sending end you should see something like:
U-Net should now be working for you; you can modify your system
boot scripts to automatically load the drivers and firmware, if you wish.
Please see the Device Reference for information
on programming the U-Net device.
Compiling the Software
make ARCH=architecture DEV=device
from the top-level unet-2 directory, where
architecture is one of
and device is one of
This does not imply that all combinations of the above are supported.
make ARCH=linux DEV=pca200
This should compile all of the components for a particular operating
system. You can also issue this command in the various subdirectories by
hand, although no checks are made that ARCH and
DEV are set in that case. Be careful!
make ARCH=architecture DEV=device clean
bin-device-architecture
of each subdirectory. You will need to follow the
installation instructions below to make use of the binaries.
Installing the Kernel Driver
Tulip device, Linux
bigphysarea=
to specify the number of pages which should be pinned down for U-Net buffers
systemwide. For example,
bigphysarea=256
should suffice for most systems.
insmod tulip.o
Once installed the device should print messages such as the following:
tulip: Found device index 0, bus 0, function 160.
tulip: Revision 0x12, using IRQ 9.
tulip: CBIO: 0xfc00, CBMA: 0xf0cffc00
tulip: Setting latency timer from 64 to 100 clocks.
tulip: MAC address 00:00:c0:17:bc:c7
tulip0: Rx ring at 0x00a34034, Tx ring at 0x00a34434
tulip: 1 devices found.
If you see error messages which you don't understand, please
contact us.
mknod /dev/tulipN c major N
Where major is the major number (say, 33) and N is the
minor number as described above.
PCA-200 device, Linux
bigphysarea=
to specify the number of pages which should be pinned down for U-Net buffers
systemwide. For example,
bigphysarea=256
should suffice for most systems.
insmod pca200.o
Once installed the device should print messages such as the following:
pca200: Found device index 0, bus 0, function 144.
pca200: Revision 0x0, using IRQ 11.
pca200: Board memory at 0xf0d00000 mapped at 0x0181c000.
pca200: Setting latency timer from 0 to 100 clocks.
pca200: MAC address prom of device 0 reads 00:20:48:17:03:27
mknod /dev/pcaN c major N
Where major is the major number (say, 33) and N is the
minor number as described above.
Myrinet device, Linux
bigphysarea=
to specify the number of pages which should be pinned down for U-Net buffers
systemwide. For example,
bigphysarea=256
should suffice for most systems.
insmod unet-myrinet.o
Once installed the device should print messages such as the following:
unet-myrinet: Loading devmyrinet-linux.c (Oct 30 1997 09:17:29)
unet-myrinet: EEPROM SRAM size reads 262144.
unet-myrinet: Found device index 0, bus 0, function 96.
unet-myrinet: Revision 0x1, using IRQ 9.
unet-myrinet: regbase 0xfdc00000, kernel_regbase 0x4e37000
unet-myrinet: membase 0xfdc00000, kernel_membase 0x4e37000
unet-myrinet: 1 devices found.
mknod /dev/myriN c major N
Where major is the major number (say, 46) and N is the
minor number as described above.
SBA-200 device, SunOS 4.1.x
modload -o /tmp/sba200.o sba200.o
Once installed the device should print messages such as the following:
sba200 U-Net driver loaded.
FORE,sba-2000 at SBus slot 0 0x800000 and SBus
slot 0 0x1000000 pri 3 (sbus level 2)
mknod /dev/sbaN c major N
Where major is the major number (say, 33) and N is the
minor number as described above.
SBA-200 device, Solaris 2.x
Be sure to delete any lines associated with the FORE SBA-200 driver.
# U-Net/SBA-200 driver
type=ddi_pseudo;name=FORE,sba-200 sba\N0
exclude: drv/sba200
If installation or use of the U-Net driver causes problems, you
can always boot using the etc/system.nounet file to
prevent the driver from being loaded at boot time.
add_drv -i 'FORE,sba-200' -m '* 0666 root bin' sba200
Myrinet device, Solaris 2.x
rather than doing a complete uninstall of the software.
rem_drv myrinet
# U-Net/Myrinet driver
type=ddi_pseudo;name=MYRICOM,mlanai myri\N0
exclude: drv/unet-myrinet
If installation or use of the U-Net driver causes problems, you can
always boot using the etc/system.nounet file to prevent the
driver from being loaded at boot time.
add_drv -i 'MYRICOM,mlanai' -m '* 0666 root bin' unet-myrinet
Loading the Firmware - PCA200 and SBA-200
uload -r -a -s device firmware
Where device is an entry in /dev corresponding
to the appropriate physical board, and firmware is the
filename of the appropriate firmware binary (see above).
Downloading /tmp/comma00531
Sending /tmp/comma00531, 311 blocks:
Sector 311 38k
ureset device
This will reset the U-Net device driver and the firmware code.
You should see messages from the kernel such as:
If the firmware magic does not match, then the firmware was not
loaded correctly. If the version number does not match, the kernel
driver and firmware version do not correspond.
pca0: U-Net reset
pca0: U-Net firmware magic is 0xf00d.
pca0: U-Net firmware version is 0x5.
In this way you can verify that uload is at least
talking to the i960 monitor correctly.
Mon960
Mon960 monitor for the Intel i960 CA (step 04)
Version 1.0 HPA-200 V1.1 (12), Fore Systems, Inc. Apr 4 1994
Copyright 1992, Intel Corporation
=>
Loading the Firmware - Myrinet boards
where switchnum is a decimal number identifying a switch, and
nodename is an alphanumeric hostname. The first column of each
line specifies the switch number for that line; the remaining columns
each correspond to one port on that switch. The first port entry in a line
corresponds to port 0 of that switch, the second port entry to port 1, and so
on.
0 host1 host2 host3 host4
while two four-port switches connected together might use:
0 host1 1 host2 host3
1 host4 host5 0 host6
Here, we see that port 1 of switch 0 connects to port 2 of switch 1 (where
ports are numbered 0-3).
export UNET_HOSTS="host1 host2 host3 host4"
will make host1 host number 0, host2 host number 1, and
so forth. U-Net applications use host numbers rather than host names to
identify machines on the network. The hostname-to-hostnumber mapping is
defined systemwide by the setting of UNET_HOSTS when uload
is run. Note that different hosts may have different name-to-number mappings
(that is different values of UNET_HOSTS may be used). For
simplicity it's probably best to use the same value of UNET_HOSTS
on all network nodes, and to advertise the value of the variable so that
user programs can determine the name-to-number mapping.
uload device lcpfile configfile
Where device is an entry in /dev corresponding
to the appropriate physical board, lcpfile is the filename
of the appropriate firmware binary (see above), and configfile
is the name of the configuration file.
Testing U-Net
Note that loopback will not work for the Tulip device unless the
driver is recompiled to use full-duplex mode. Full-duplex operation is not
recommended for normal use (as this disables collision detection).
pingpong
pingpong [send | recv] device addr num size
Where device is the U-Net device for the desired endpoint
(e.g., /dev/sba0), addr is the address of the machine
to send to or receive from, num is the number of pingpongs, and
size is the size of each pingpong.
xx:xx:xx:xx:xx:xx.yy
Where xx are hexadecimal digits corresponding to the
Ethernet MAC address of the destination controller,
and yy is the
hex value of a single-byte ``port'' address on the receiving end.
Port addresses are used to allow multiple endpoints/channels to use
one physical device; the sending and receiving ends of a U-Net
connection over Fast Ethernet should agree on the port address(es)
used. Note that only one endpoint on a single physical network
device can listen to a particular port.
nnn/ppp
Where nnn is the decimal hostnumber of the destination
machine (defined by the setting of UNET_HOSTS when
uload was run), and ppp is a decimal ``port''
address (in the range 0-255). Port addresses are used to allow
multiple endpoints/channels to use
one physical device; the sending and receiving ends of a U-Net
connection over Myrinet should agree on the port address(es)
used. Note that only one endpoint on a single physical network
device can listen to a particular port.
Receiver: pingpong recv /dev/sba0 200 1000 40
The ``receiver'' should be started before the ``sender''. In pingpong,
both stations send and receive packets, but the receiver receives first,
while the sender sends first.
Sender: pingpong send /dev/sba0 100 1000 40
Activated channel 0.
Sending 1000 40-byte messages.
Sent 1000 40-byte messages in 0.107453 seconds.
107.452989 usec/message or 2.840086 Mbit/sec
Deactivating channel...
Destroying endpoint...
Closing and exiting...
This time corresponds to the round-trip ping-pong time, so divide by two
for the one-way latency per message.
loopback
loopback device addr num size
With arguments are as described above. For loopback on ATM, you should have
the VCI addr configured at the switch for send/receive, or use
a loopback cable (connect send/receive ends of the fibre to both ports on
the card), in which case the VCI is arbitrary. For loopback on Fast Ethernet,
the Tulip driver must be compiled for full duplex operation and a loopback
cable used (hack up a UTP5 and twist the Rx and Tx lines together).
For loopback on Myrinet, everything will work fine if the Myrinet board
is connected to a switch; if you wish to use a loopback cable you must
recompile the LanAI firmware with MYRINET_LOOPBACK_CABLE defined
in lcp.c.
Got channel 0.
Sending 10000 40-byte messages.
Sent 10000 40-byte messages in 0.441592 seconds.
44.159210 usec/message or 6.910807 Mbit/sec
Deactivating channel...
Destroying endpoint...
Closing and exiting...
window
window [send | recv] device addr num size window-size
With arguments as described above. The sender sends window-size
messages of size bytes each to the receiver, which sends a single
(small) acknowdledgement after receiving window-size messages.
This is repeated num times.
Sending 3000 1000-byte messages, window size 30.
Sent 3000 1000-byte messages (window size 30) in 0.346486 seconds.
115.495324 usec/message or 69.266873 Mbit/sec
Deactivating channel...
Destroying endpoint...
Closing and exiting...
Return to U-Net Documentation.
M. Welsh, mdw at cs dot berkeley dot edu