Introduction
Members of our team completed a real-time network emulation development effort to support the addition new technologies and capabilities for a networked radio for test and evaluation. The networking waveform emulated is a military radio protocol based on an OFDM Physical Layer as used with mobile ad hoc networking (MANETs) for software defined radios (SDR). This radio uses the Software Communications Architecture (SCA), an open architecture framework that defines a standard way for radios to instantiate, configure, and manage waveform applications running on their platform. Real-time network emulation of a 30 node radio network was implemented using a software defined radio protocol stack executed within the Extended Mobile Ad-hoc Network Emulator (EMANE) real-time network emulation framework.
EMANE Network Emulation
EMANE is an open source infrastructure used for Mobile Ad-hoc Network Emulation. EMANE provides a set of APIs to allow independent development of network emulation modules (NEMs), emulation/application boundary interfaces (transports) and emulation environmental data distribution mechanisms (events). The key to the flexibility of EMANE is the use of XML based factories to determine which emulation components to instantiate and where they reside once deployed.
Unlike OPNET which is a discrete event network simulation, EMANE is a real-time network emulation framework. EMANE provides statistics of real time behavior; supports open source traffic generating applications like MGEN, Iperf or video; and allows visualization of nodes and connectivity. The differences between network emulation and simulation are further described in the post Network Emulation vs. Simulation.
EMANE Benefits
EMANE real-time network emulation provides the following benefits:
- Repeatable: once established, emulations can be quickly created and rerun for verification and validation.
- Dynamic: supports model specific configuration and a variety of scenarios.
- Scalable: supports emulation of simple as well as large, complex network architectures.
- Transferable: large scale test beds can be replicated at multiple facilities.
- Costs: Very cost effective relative to hardware demonstrations.
EMANE Virtualization and Linux Containers (LXC)
EMANE depends upon virtualization to provide separate network IP stacks to implement independent NEMs. This virtualization is most often implemented using LinuX Containers (LXCs). LXC is an operating system-level virtualization method for running multiple isolated Linux systems (containers) on a single control host (LXC host). Linux containers provide independent network IP stacks for executing networking applications under a single version of the Linux kernel using virtual network interfaces.
EMANE when based on light-weight Linux containers facilitates separate network stack emulation, is more scalable than a simulation system built on heavier weight virtual machines (VM). EMANE containerized network stacks encapsulate the functionality necessary to transmit, receive and operate on data routed through the emulation space. This allows for heterogeneous network emulation using a pluggable Media Access Control (MAC) and Physical (PHY) Layer architecture. This inherently provides a method for changing waveforms.
Network stack applications running in a Linux container to support this emulation effort include:
- EMANE binaries with XML based cascading configuration files.
- Open source network routing and management applications (such as OLSR or OSPF).
- Custom SDR waveform executable.
- Custom EMANE models and shims.
Emulation Custom Scripting
The EMANE binary distribution itself does not include any inherit virtualization. Therefore, external scripting is required to create a multi-node real-time network emulation framework based on virtualized NEMs.
Network emulation using EMANE required the development of thousands of lines of Linux bash shell scripting, which was responsible for:
- Creation of thousand of configuration files for EMANE and application execution necessary for the complex 30 node emulation.
- Creation and configuration of the Linux containers (LXC) for each EMANE NEM.
- Creating and configuring Linux interface bridges to allow containers to communicate with other containers with network packets.
- Populating the created Linux containers with container specific applications being emulated, handling application dependency sequences, startup timing, and interface dependencies.
- Configuration of the LXC IP stack including routing and ARP table population.
EMANE NEM Stack Customization and Shims
The EMANE NEM protocol stack supports customization through the development of custom “shim” layers that exist as part of the executing NEM software stack. A custom shim layer is inserted between component layers in an NEM layer stack without requiring those components to have knowledge of its presence. This is shown in Figure 1, which illustrates an EMANE NEM Layer stack as it appears without shims, and with three shims inserted; two shims reside between the MAC and PHY layers and one shim resides between the PHY Layer and the OTA message interface.
Figure 1: EMANE NEM Layer Stack without and with Shims
This development required the development of two custom, technology specific shims in the NEM stack for a radio node: one that sits above the EMANE MAC layer and the other sits below the EMANE PHY layer. This is illustrated by the following XML configuration file nem.xml as Figure 2, whose contents represents the NEM configuration for a radio black side node.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nem SYSTEM "file:///usr/share/emane/dtd/nem.dtd">
<nem type="unstructured">
<shim definition="macshim.xml"/>
<mac definition="mac.xml"/>
<phy>
<param name="subid" value="4" />
<param name="noisemode" value="all"/>
<param name="noisemaxsegmentduration" value="2000000"/>
</phy>
<shim definition="physhim.xml"/>
</nem>Figure 2: EMANE NEM Configuration File with custom MAC and PHY Layer Shims
EMANE MAC Layer Custom Shim
The custom MAC Layers shim provide control messaging eavesdropping between the emulation application boundary and a custom EMANE OFDM model. This shim decodes message content using Google Protobuf serialization technology, and forwards the decoded content on to a frequency sensing detector application.
EMANE Physical Layer Custom Shim
The custom PHY layer shim is responsible for adding and extracting custom metadata to the EMANE Common PHY Layer network packet including a complex 1024 data point FFT representing the Power Spectral Density (PSD) signature for the emulated networking waveform. Creation of the transmit PSD included the addition of noise, and path-loss information used by the receiver.
This shim extracts received PSDs from the EMANE over the air (OTA) network packets, scales the PSD content based on antenna gains and path-loss information, and then forwards the modified PSD content to a detector application similar to the custom MAC shim.
This shim monitors the system path-loss events for populating the transmitted metadata. EMANE Events are messages sent to EMANE components containing control information relevant to the functional operation of the component.
EMANE Jammer Custom Shim
The benefits of frequency agile communications, such as the emulated networking waveform, is the system detects and responds to attempted jamming of communications. To introduce jamming into the network emulation environment for evaluating system responses, a custom jammer “shim” layer was developed which executes within a separate LXC, and is started and stopping dynamically using the EMANE event system. Detailed information on the custom jammer is provided in the post EMANE Jammer.
The jammer “shim” is configured for various operating modes as follows:
- Fixed frequency at a configured center frequency.
- Sweep frequency between a start and stop frequency with sweep frequency step interval and sweep rate.
- Hop-set frequencies with sweep rate and optional random hop-set frequency ordering.
EMANE OTA Spectrum and Waterfall Viewer
A very important tool used by communications engineers in the analysis and evaluation of network emulation, is real-time spectrum and waterfall plots. For this development effort, it was necessary to develop a custom real-time spectrum/waterfall viewer application in C++ and Qt that was capable of reading and decoding OTA network frames on the EMANE OTA control bus. This development is described in the post EMANE Real-Time Spectrum and Waterfall Viewer.
Emulation Server Platform
This emulation effort was performed using a centralized EMANE deployment running on an enterprise class Dell PowerEdge server. The EMANE emulation server provided 64 CPU cores that were assigned by way of CPU pinning to the various LXC guest containers to provide the required performance for 30 node emulation. Each radio node required 4 independent LXC containers communicating with each other by way of virtual Ethernet interfaces:
- a red side radio container,
- a black side radio container,
- a red side host container, and
- a black side host container.
The platform host operating system was a standard Ubuntu 64-bit standard desktop Linux distribution. No operating system tuning was required.
Demonstration
The demonstration system for the radio network consisted of a 30 radio and 5 jammer node network emulation. Each radio node consisted of a red and black side LXC, as well as red and black side host devices as separate LXCs acting as the source and destination of network message traffic. Host traffic was simulated using the commonly used industry standard IPerf network performance test application to observe and measure network latencies and through-puts. IPerf traffic was generated using both TCP and UDP protocols, and was executed within the content of Linux containers guest sessions controlled by way of SSH terminal sessions with each container.
The jammer nodes were activated using the EMANE event system targeting specific jammers by NEM ID, to view the response of the system to jamming waveform presence. The EMANE OTA viewer application was used to view the real-time spectrum and waterfall displays of the EMANE OTA network during test scenario execution including the presence of jamming.
Conclusion
We would like to thank Adjacent Link, the designers, developers and maintainers of the open source Extendable Mobile Ad–hoc Network Emulator (EMANE) real-time network emulation framework, for answering our questions about EMANE setup, configuration, customization, and operation.