Testing and tools

## 9 Testing and tools

### 9.1 Tracing

We have instrumented our code in order to enable tracing. Running the application with tracing deactivated, causes a neglectible performance overhead (an external call to a function which returns an atom). Activation of tracing does not require any recompilation of the code, since we rely on Erlang/OTP's built in support for dynamic trace activation. In our case tracing of calls to a given external function.

Event traces can be viewed in a generic message sequence chart tool, that we have written. It can either be used in batch by loading event traces from file or interactively, as we are doing at demos and when we debug our own code. The event trace stuff can for the moment be found under megaco/utils but, will later be documented and released as an own application.

### 9.2 Measurement and transformation

We have included a simple tool for codec measurement and message transformation.

The tool is located in the example directory.

#### 9.2.1 Requirement

- Erlang/OTP, version R12B or later.
- Version 3.7 or later of this application.
- Version 1.4.7 or later of the asn1 application.
- The flex libraries. Without it, the flex powered codecs cannot be used.

#### 9.2.2 Results

The results from the measurement run is four excel-compatible textfiles:

- decode_time.xls -> Decoding result
- encode_time.xls -> Encoding result
- total_time.xls -> Total (Decoding+encoding) result
- message_size.xls -> Message size

#### 9.2.3 Instruction

The tool contain three things:

- The transformation module
- The measurement module(s)
- The basic message file archive

##### 9.2.3.1 Message Transformation

The message transformation is done by the transformation module. It is used to transform a set of messages encoded with one codec into the other base codec's.

The transformation can either be done by updating and running the skeleton script provided with the example (trans.sh.skel) or by manually running the code in an erlang shell:

Start an erlang node, and make sure it has the path to both the latest megaco ebin-dir as well as the dir containing the transformation module:

            % erl -pa <path-megaco-ebin-dir> -pa <path-to-tranformation-module-dir>
            Erlang (BEAM) emulator version 5.6 [source]

            Eshell V5.6  (abort with ^G)
            1> megaco_codec_transform:t(pretty, [compact, per, ber, erlang]).
            ...
            2> halt().

Now the messages in the 'pretty' directory has been transformed and stored into the other codec dir's.

It is possible to transform from any codec to any other.

##### 9.2.3.2 Measurement(s)

There are two different measurement tools:

- meas:
  Used to perform codec measurements. That is, to see what kind of performance can be expected by the different codecs provided by the megaco application.
  The measure is done by iterating over the decode/encode function for approx 2 seconds per message and counting the number of decodes/encodes.
  Is best run by modifying the meas.sh.skel skeleton script provided by the tool.
  To run it manually do the following:

              % erl -pa <path-megaco-ebin-dir> -pa <path-to-meas-module-dir>
              Erlang (BEAM) emulator version 5.6 [source]

              Eshell V5.6  (abort with ^G)
              1> megaco_codec_meas:t([pretty, compact, per, ber, erlang]).
              ...
              2> halt().

  or to make it even easier, assuming a measure shall be done on all the codecs (as above):

              % erl -noshell -pa <path-megaco-ebin-dir> \
                    -pa <path-to-meas-module-dir> \
                    -s megaco_codec_meas t -s init stop

  When run as above (this will take some time), the measurement process is done as followes:

                For each codec:
                    For each message:
                        Read the message from the file
                             Detect message version
                             Measure decode
                             Measure encode
                Write results, encode, decode and total, to file

- mstone1:
  This is an SMP performance monitoring tool.
  It creates a process for each codec config supported by the megaco application and let them run for a specific time (all at the same time), encoding and decoding megaco messages. The number of messages processed in total is the mstone1(1) value.
  Is best run by modifying the mstone1.sh.skel skeleton script provided by the tool.

Both these tools use the megaco messages stored in the codec dirs (after the transformation has been performed).

##### 9.2.3.3 Message file archive

This is basically a gzipped tar file of a directory tree with the following structure:

              time_test/pretty/<message-files>
                        compact/
                        per/
                        ber/<message-files>
                        erlang/

The only directories containing any files are the pretty-dir and the ber-dir. It's the same messages encoded with different codec's. This means it is possible to choose the message basis for the (transformation and) measurement.

These files include both version 1 and version 2 messages.

It is of course possible to add and remove messages at will. The messages included are the ones used in our own measurements.

#### 9.2.4 Notes

##### 9.2.4.1 Binary codecs

There are two basic ways to use the binary encodings: With package related name and termination id transformation (the 'native' encoding config) or without. This transformation converts package related names and termination id's to a more convenient internal form (equivalent with the decoded text message).

The transformation is done _after_ the actual decode has been done.

Furthermore, it is possible to make use of a linked in driver that performs some of the decode/encode, decode for ber and encode for per (the 'driver' encoding config).

Therefor in the tests, binary codecs are tested with four different encoding configs to determine exacly how the different options effect the performance: with transformation and without driver ([]), without transformation and without driver ([native]), with transformation and with driver ([driver]) and finally without transformation and with driver ([driver,native]).

##### 9.2.4.2 Included test messages

Some of these messages are ripped from the call flow examples in an old version of the RFC and others are created to test a specific feature of megaco.

##### 9.2.4.3 Measurement tool directory name

Be sure not no name the directory containing the measurement binaries starting with 'megaco-', e.g. megaco-meas. This will confuse the erlang application loader (erlang applications are named, e.g. megaco-1.0.2).

megaco 3.9.1.1
Copyright © 1991-2008 Ericsson AB