The testing shell for the Anjay client

LwM2M is a client-server protocol, which means that to utilize it, you need an implementation of both a client and a server. While this statement is obvious, its consequences might not be immediately apparent – here at AVSystem, the Anjay (LwM2M client) and Coiote DM (LwM2M server) teams did not initially have each other’s counterpart product to test against during development. Similar problems occur whenever we implement new features, such as support for the EST security mode recently. It is impossible to synchronize both teams perfectly, some features are always developed first in one project or the other. How to test a new feature when the other side is not yet ready?

In AVSystem’s Embedded development team, we decided to just… roll our own implementation of the server as well. However, our little “server” exists for the sole purpose of testing Anjay and client applications based on it. It does not have a GUI, does not care about performance or scalability, and does not even allow connecting more than one client at once. It’s written in Python and mostly used as a library in our test suites, For example, the create.pyfile contains a couple of simple tests for the LwM2M Create operation. Whenever a new feature is added to Anjay, we also create a simplified server-side implementation so that the client can be tested against something. This is further used for regression testing, to ensure that each feature still works properly in every version that follows.

There is some overlap between the Python testing framework bundled with Anjay and the automated integration testing scenarios available in Coiote DM. However, the tests in Coiote DM are geared more towards verifying compliance of complete client applications. In the earlier stages of development, the LwM2M testing shell (called “NSH” internally) is often more useful. It’s a simple command-line interface to the aforementioned Python testing framework, that allows precise creation and inspection of individual LwM2M packets, while automatically answering to the basic requests from the client.

Below is an example showing NSH communicating with Anjay. Commands entered on the command line are highlighted in bold.

By launching NSH with the -l switch, it will wait for the incoming connection and handle the Register message:

$ ./tests/integration/framework/nsh-lwm2m/nsh_lwm2m.py -l
ignoring Lwm2mMsg: not command-compatible
ignoring Lwm2mResponse: not command-compatible
loaded 30 message types
waiting for a client on port 5683 ...
<- Register /rd?lwm2m=1.1&ep=demo_client&lt=86400: </1/1>,</2>,</3/0>,</4/0>,<...
-> Created /rd/demo

Using the details command, any message that has been sent or received so far can be examined. The argument is the message number, counting from the most recently sent or received – the incoming Register message is the second newest known message (the newest one is the response sent to it), so the command we want is details 2:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ details 2
* exact: details
*** Recv ***
Register /rd?lwm2m=1.1&ep=demo_client&lt=86400: </1/1>,</2>,</3/0>,</4/0>,<...
version: 1
type: CONFIRMABLE
code: 0.02 (REQ_POST)
msg_id: 43763
token: \xc4\x13\x9c\x11\xc1\xfd=d (length: 8)
options:
    option 11 (URI_PATH), content (2 bytes): rd
    option 12 (CONTENT_FORMAT), content (1 bytes): 40 (APPLICATION_LINK)
    option 15 (URI_QUERY), content (9 bytes): lwm2m=1.1
    option 15 (URI_QUERY), content (14 bytes): ep=demo_client
    option 15 (URI_QUERY), content (8 bytes): lt=86400
content: 147 bytes
ascii-ish:
</1/1>,</2>,</3/0>,</4/0>,</5/0>,</6/0>,</7/0>,</10>;ver="1.1",</10/0>,</11>,</16>,</19>,</20/0>,</33605>,</33606/0>,</33607/0>,</33608>,</33609/0>

There are also tlv and cbor sub-shells that allow building messages in TLV and SenML CBOR formats, respectively.

For example, the commands shown below can be used to serialize a Multiple-instance Resource /*/1/0 with two instances 1 and 2:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ tlv
* exact: tlv
[Lwm2mCmd/TLV] port: 5683, client: 127.0.0.1:55132 $ add_instance 1
* exact: add_instance
Selected top-level: instance
[Lwm2mCmd/TLV] port: 5683, client: 127.0.0.1:55132 $ add_multiple_resource 0
* exact: add_multiple_resource
[Lwm2mCmd/TLV] port: 5683, client: 127.0.0.1:55132 $ add_resource_instance 1 hello
* exact: add_resource_instance
[Lwm2mCmd/TLV] port: 5683, client: 127.0.0.1:55132 $ add_resource_instance 2 world
* exact: add_resource_instance

Upon exiting from the TLV sub-shell, the built payload is printed on the screen:

[Lwm2mCmd/TLV] port: 5683, client: 127.0.0.1:55132 $ exit
* exact: exit
exiting
\x08\x01\x11\x88\x00\x0e\x45\x01\x68\x65\x6c\x6c\x6f\x45\x02\x77\x6f\x72\x6c\x64

We can now paste it as payload to e.g. the Create operation:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ create /16 format=APPLICATION_LWM2M_TLV \x08\x01\x11\x88\x00\x0e\x45\x01\x68\x65\x6c\x6c\x6f\x45\x02\x77\x6f\x72\x6c\x64
* exact: create
-> Create /16: \x08\x01\x11\x88\x00\x0eE\x...
<- Created /16/1

Let’s now Read the created object:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ read /16 accept=APPLICATION_LWM2M_TLV
* exact: read
-> Read /16: accept APPLICATION_LWM2M_TLV
<- Content (11542 (APPLICATION_LWM2M_TLV); 20 bytes)

The details command will also decode the TLV payloads:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ details
* exact: details
*** Recv ***
Content (11542 (APPLICATION_LWM2M_TLV); 20 bytes)
version: 1
type: ACKNOWLEDGEMENT
code: 2.05 (RES_CONTENT)
msg_id: 4925
token: \xae\xbe\x9c\xe3\x85X\xed\xa6 (length: 8)
options:
    option 12 (CONTENT_FORMAT), content (2 bytes): 11542 (APPLICATION_LWM2M_TLV)
content: 20 bytes
TLV (1 elements):
  instance 1 (1 resources)
    multiple resource 0 (2 instances)
      resource instance 1 = b'hello' (int: 448378203247)
      resource instance 2 = b'world' (int: 512970878052)

We can also do the same for the SenML CBOR format:

[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ read /16 accept=APPLICATION_LWM2M_SENML_CBOR
* exact: read
-> Read /16: accept APPLICATION_LWM2M_SENML_CBOR
<- Content (112 (APPLICATION_LWM2M_SENML_CBOR); 38 bytes)
[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $ details
* exact: details
*** Recv ***
Content (112 (APPLICATION_LWM2M_SENML_CBOR); 38 bytes)
version: 1
type: ACKNOWLEDGEMENT
code: 2.05 (RES_CONTENT)
msg_id: 4926
token: \x97\x9f\xfb\x9b\x9d\x17\xb5\x16 (length: 8)
options:
    option 12 (CONTENT_FORMAT), content (1 bytes): 112 (APPLICATION_LWM2M_SENML_CBOR)
content: 38 bytes
CBOR (2 elements):
  {<SenmlLabel.BASE_NAME: -2>: '/16', <SenmlLabel.NAME: 0>: '/1/0/1', <SenmlLabel.STRING: 3>: 'hello'}
  {<SenmlLabel.NAME: 0>: '/1/0/2', <SenmlLabel.STRING: 3>: 'world'}
[Lwm2mCmd] port: 5683, client: 127.0.0.1:55132 $

NSH commands can also be stored in a file, creating a form of basic scripting. For example, we can save the commands from the above example that actually interact with the client application. Note that the listen command is equivalent to the -l switch:

listen
create /16 format=APPLICATION_LWM2M_TLV \x08\x01\x11\x88\x00\x0e\x45\x01\x68\x65\x6c\x6c\x6f\x45\x02\x77\x6f\x72\x6c\x64
read /16 accept=APPLICATION_LWM2M_TLV

When executed, this yields:

$ ./tests/integration/framework/nsh-lwm2m/nsh_lwm2m.py < simple-test.nsh
ignoring Lwm2mMsg: not command-compatible
ignoring Lwm2mResponse: not command-compatible
loaded 30 message types
Warning: Input is not to a terminal (fd=0).
[Lwm2mCmd] $ * exact: listen
waiting for a client on port 5683 ...
<- Register /rd?lwm2m=1.1&ep=demo_client&lt=86400: </1/1>,</2>,</3/0>,</4/0>,<...
-> Created /rd/demo
[Lwm2mCmd] port: 5683, client: 127.0.0.1:45581 $ * exact: create
-> Create /16: \x08\x01\x11\x88\x00\x0eE\x...
<- Created /16/1
[Lwm2mCmd] port: 5683, client: 127.0.0.1:45581 $ * exact: read
-> Read /16: accept APPLICATION_LWM2M_TLV
<- Content (11542 (APPLICATION_LWM2M_TLV); 20 bytes)
[Lwm2mCmd] port: 5683, client: 127.0.0.1:45581 $

For manual testing, this can be combined with, for example, collecting a PCAP file of the communication and examining it afterwards.

You can learn about more features of NSH in its documentation that is available publicly.

While it is arguably most useful while developing Anjay itself, the testing shell may also prove invaluable during LwM2M client application development, especially in the early stages. Some of the advantages it offers over using a fully fledged LwM2M server such as Coiote DM, include:

• The testing shell keeps very little state, does not require any database or complicated configuration – allowing to test the client in different configurations (e.g. different endpoint names, security modes, etc.) much quicker.
• It allows more fine-grained control over each and every packet sent to the client, including the possibility to send invalid messages – allowing for more specific testing scenarios.
• Can be run locally or easily installed on any server with shell access, providing for faster communication with the tested device and allowing the user to perform tests without access to the public Internet.