Monday, January 30, 2017

Partial/Selective Decoding

These days, many critical applications require real time processing of large volumes of data/messages. Often only the data contained in a few selected fields is of value.

Some such application areas include Big Data, Self Optimizing Networks (SON), Lawful Interception, Routers, Protocol Analyzers, V2X, etc. Big Data is characterized by three main elements - the “3Vs”, volume, variety, and velocity. Extremely large and complex data sets (volume) of different types and natures (variety) need to be processed in real time at very fast rates (velocity). SONs need to process in real time, huge numbers of messages received from different nodes. V2X applications need to work with huge amounts of data received from both vehicles and the roadside infrastructure.

When designing such applications, you will be challenged to process a few selected fields contained within a large set of ASN.1 messages at very fast rate without traversing the complex message structures. In addition, you may need to modify some of those fields without decoding and re-encoding the entire message. OSS’ new partial/selective decoding feature is specifically targeted to meet these requirements.

The partial/selective decoding feature of the OSS ASN.1 Tools is very easy to use. Simply mark the fields in your ASN.1 specification that  you are interested in decoding, and for each field specify a callback function which you want the decoder to invoke after decoding the field. That’s it! You are done. No need to decode the whole message, no need to traverse a complex decoded structure value to find the fields of interest. The decoder will start decoding the message, reach the first field as fast as possible, and call the callback function with the decoded value, along with its offset and length within the encoded message. The decoder will repeat this process for each of the marked fields. You can also instruct the decoder to skip the decoding of the remaining fields after decoding a particular field. This functionality is useful in implementing conditional decoding logic.

Here is a little insight into the implementation of the partial/selective decoding feature. The OSS ASN.1 Tools offer two directives OSS.DataCallback and OSS.InfoCallback to support this feature. The former should be applied to all fields you are interested in decoding. However, if a field is part of many types, the OSS.InfoCallback directive can be used to find which type contains the field being decoded. For example, if a ZipCode field is part of both the HomeAddress and CompanyAddress message and you are interested in decoding only the ZipCode of HomeAddress, you can mark ZipCode with the OSS.DataCallback directive, and HomeAddress and CompanyAddress with OSS.InfoCallback. The callback functions of the OSS.InfoCallback directives can be used to identify which ZipCode is being decoded.

In the callback function, it is also possible to change the encoding of a field within the encoded message. When the decoder invokes the callback function, along with the decoded value, it also returns the offset and length of the encoding of the field. If the encoding length of the modified value is same as the encoding length of its original value, the new encoding of the field can be updated in the message. This feature is useful for the router applications which need to update only an address field within the message. There is no need to decode the whole message, modify it, and re-encode it.

The partial/selective decoding feature can be used along with traditional (full) decoding, or the Tools can be instructed to generate code only for the partial/selective decoding feature. The latter significantly reduces the footprint, for it generates only code sufficient to decode the marked fields.

Please refer to this link for detailed technical information about this feature. If you are interested in a trial of this feature, you can download the trial tools, or contact OSS at info@oss.com. This feature is currently supported in the ASN.1/C and ASN.1/C++ Tools.

Thursday, September 1, 2016

IoT Ready OSS ASN.1 Tools

Minimizing the ASN.1 encoder/decoder footprint is an increasingly common requirement of customers in various industries. It is a particularly critical requirement of customers developing solutions for IoT (Internet-of-Things) devices. A great numbers of the devices expected to be connected to the network will have very limited memory capacity, typically a few hundred kilobytes, available for the application.


The OSS ASN.1 Tools have several features customers can use to reduce the size of the ASN.1 encoder/decoder and meet this challenging memory requirement. In several cases, simply using these features we were able to achieve a footprint size of 200 kilobytes for the OSS PER (Packed Encoding Rule) encoder/decoder supporting a large ASN.1 protocol such as RRC (Radio Resource Control).  We were even able to achieve a footprint size of 61 kilobytes for one customer!


The OSS ASN.1 tools for C have two main components, an ASN.1 compiler and a set of runtime libraries. The ASN.1 compiler takes an ASN.1 specification as input and generates C structures for use by your application. The runtime library is linked with your application to support and complement the code generated by the ASN.1 compiler. These generic components have been designed to be easily configured by you to meet your specific needs.


The OSS ASN.1 compiler treats all unreferenced types in the ASN.1 specification as Protocol Data Unit (PDU) types, i.e. the compiler expects that your application will encode/decode messages of these types, and generates encoder/decoder code for all of these types. However, in your particular application, you can optimize space utilization if you don’t need to encode/decode all of these types. By applying the OSS.NOPDU directive, you can instruct the OSS ASN.1 compiler not to generate the code for those types you don’t need.


In cases where there are many message types which your application is not interested in encoding/decoding, applying the OSS.PDU directives to all of those types may not be very convenient. In such cases, simply apply the OSS.NOPDU directive globally and then apply the OSS.PDU directive only to the types you need to encode/decode. This approach, which reduces the number of directives applied is quite useful in applications which handle only a few selected message types.


After you have tailored the generated code to encode and decode only the types you need, it’s possible that you need to only encode some of these types, and only need to decode some other types. Such customization of the generated code is possible by using another set of directives, OSS.ENCODEONLY and OSS.DECODEONLY. If the OSS.ENCODEONLY directive is applied to a message type, only the encoder code will be generated for that type. The OSS.DECODEONLY directive can be used to generate only decoder code for a particular type.


In several cases message types contain many optional fields. If the values corresponding to those fields will not be present in the messages you are receiving, you can instruct the ASN.1 compiler not to generate encoder/decoder code for those optional fields. The ASN1.Remove directive can be used for this. Note, however, that this directive should be used with utmost care, since an incorrect change may invalidate the ASN.1 specification, resulting in unwanted decoding errors. You may refer to the ASN.1 Remove documentation for details.


There is no need to modify the ASN.1 specification to add these directives. You can write them in a separate configuration file and provide it as an input to the OSS ASN.1 compiler. You may even be able to use the same configuration file when you upgrade to a newer version of the specification.  Note that if there are new message types in the new version that your application needs to encode/decode, you may need to make minor adjustments to the configuration file.


The documentation of the OSS ASN.1 Tools for C can be found here.