DCCL uses the Google Protocol Buffers (Protobuf) language to define messages. The DCCL IDL is defined as extensions to the Protobuf language message and field options to allow more compact encoding than is possible with the default Protobuf meta-data. You should familiarize yourself with basic Protobuf usage before reading the rest of this document: see https://protobuf.dev/overview/.
An example DCCL message is as follows:
In the above message, the snippet option (dccl.msg) = { codec_version: 4 id: 124 max_bytes: 32 }; represents the message options extensions since they affect the design of the entire DCCL message (in this case "NavigationReport"). The field options affect a given field, e.g. [(dccl.field) = { min: -10000 max: 10000 precision: 1 }];
The full Protobuf definition of the DCCL extensions is given in option_extensions.proto (as messages DCCLFieldOptions and DCCLMessageOptions).
DCCL options
The available DCCL options is given in the following Table 1:
Table 1: Definition of the DCCL Interface Description Language
| Extension Name | Extension Type | Explanation | Applicable Fields | Symbol | Default |
|---|---|---|---|---|---|
| Message Extensions [a] | |||||
(dccl.msg).id | int32 | Unique identifying integer for this message | - | ||
(dccl.msg).omit_id | bool | If true, omit (dccl.msg).id from the message definition and encoded message. This requires some other means of determining the message type at the receiver (useful when wrapping DCCL messages in another protocol or when using only a single DCCL type in a particular link). | False | ||
(dccl.msg).max_bytes | uint32 | Enforced upper bound for the encoded message length | - | ||
(dccl.msg).codec_version | int32 | Default codec set to use (corresponds to DCCL major version) | - | ||
(dccl.msg).codec | string | Name of the codec to use for encoding the base message. | dccl.defaultN, where N is codec_version | ||
(dccl.msg).codec_group | string | Group of codecs to be used for encoding the fields. | dccl.defaultN | ||
(dccl.msg).unit_system | string | Unit system to use for all fields in this message (unless explicitly specified in the field definition.) | "si" | ||
| Field Extensions (core) [b] | |||||
(dccl.field).precision | int32 | Decimal digits to preserve; can be negative. | double, float, (u)intN [c] (negative values of precision only) | \(p\) | 0 |
(dccl.field).resolution | double | Defines the spacing between encoded values (generalized alternative to precision) | double, float, (u)intN | \(dx = 10^{-p}\) | 1 |
(dccl.field).min | double | Minimum value that this field can contain (inclusive). Should be an exact multiple of \(dx = 10^{-p}\) | (u)intN , double, float | \(x_m\) | - |
(dccl.field).max | double | Maximum value that this field can contain (inclusive). Should be an exact multiple of \(dx = 10^{-p}\) | (u)intN, double, float | \(x_M\) | - |
(dccl.field).max_length | uint32 | Maximum length (in bytes) that can be encoded | string, bytes | \(L_M\) | - |
(dccl.field).min_repeat | uint32 | Minimum number of repeated values. | all repeated | \(r_m\) | - |
(dccl.field).max_repeat | uint32 | Maximum number of repeated values. | all repeated | \(r_M\) | - |
(dccl.field).codec | string | Codec to use for this field (if omitted, the default is used). | all | - | - |
(dccl.field).units | Units | Physical dimensions and units system information | (u)intN, double, float | - | - |
(dccl.field).dynamic_conditions | Conditions | Runtime defined field inclusion or bounds using Lua scripts | all | - | - |
| Field Extensions (special purpose) [b] | |||||
(dccl.field).omit | bool | Do not include field in encoded message | all | - | False |
(dccl.field).in_head | bool | If true, the field is included in the DCCL header (encoded after the DCCL ID but before the rest of the fields. Not encrypted when using encryption) | all | - | False |
(dccl.field).packed_enum | bool | If false, encode use enum assigned values, not enum index values (0-N in order of definition) | enum | - | True |
(dccl.field).static_value | string | Statically defined value for StaticCodec (codec = "dccl.static") | all | - | - |
(dccl.field).num_days | uint32 | Number of days to include in TimeCodec (codec = "dccl.time") encoding (+/- 12 hours of validity for each day added.) | double, (u)int64 | - | 1 |
- [a]: Extensions of
google.protobuf.MessageOptions - [b]: Extensions of
google.protobuf.FieldOptions - [c]: (u)intN refers to any of the integer types: int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64
DCCL ID: (dccl.msg).id
The DCCL ID is used to uniquely identify a given message name without having to encode the name in the message (encoding a number is much cheaper than a string). To interoperate with other groups, please see http://gobysoft.org/wiki/DcclIdTable. For private work, please use IDs 124-127 (one-byte) and 128-255 (two-byte).
DCCL Maximum bytes: (dccl.msg).max_bytes
This value is the maximum message size before you get an error from DCCL. This is a design tool to help ensure messages do not exceed a desired value, typically the path maximum transmission unit (MTU). Messages that do not take the actual max_bytes size are encoded only as the size they take up (i.e. they are not padded to max_bytes).
DCCL Codec Version: (dccl.msg).codec_version
This option sets the default codec version (which is not wire-compatibility between Goby/DCCL 2, DCCL 3, and DCCL 4). This should always be set to "4" when you are able to use DCCL v4 for all nodes that deploy this message, or an earlier version if required (e.g., if one of the nodes has access only to DCCL3, use "3").
Precision versus Resolution
DCCL numeric fields originally were only defined using precision which is a negative power of 10 to which the message should be rounded. For example, a precision of 2 will round the message the hundredths place ( \(10^{-2}\)), and a precision of -1 will round to the tens place ( \(10^1\)).
Now DCCL also provides a more generalized and intuitive option resolution that can be used instead of precision. Resolution doesn't have to be a power of 10, so for example, a resolution of 0.25 would round to the closest quarter of a value, and a resolution of 30 would round to the nearest multiple of 30 (-30, 0, 30, 60, etc.). When using resolution, the min and max values must be a multiple of the resolution. For example, if resolution is 0.25, min could be -0.5, 0.75, etc. but not 0.33 or -0.12 (and similarly for max).
Use whichever definition you prefer, and keep in mind that when using precision this is exactly equivalent to setting resolution as \(10^{-precision}\) (e.g., precision of 2 is a resolution of 0.01).
DCCL Static Units
Since the DCCL field bounds (min, max, and precision) are often based off the physical origins of the data, it is important to define the units of measure of those fields. The DCCL IDL has support for defining the units of a numeric field's quantity. When using the DCCL C++ library, this support is directly connected to the Boost Units C++ library: Boost Units. The units of a given field are given by two parameters: the physical dimension (e.g., length, force, mass, etc.), and the unit system which defaults to the International System of Units (SI). The units of the field can also be specified directly, outside of a canonical system (e.g., nautical mile, fathom, yard, knot, etc.).
The fields defined with units generate additional C++ methods using the DCCL plugin (protoc-gen-dccl) to the GPB compiler (protoc). The Debian package for the plugin is
These additional methods provide accessors and mutators for the dimensioned Boost Units quantities, with full static "unit safety", and correct conversions between different units of the same dimensions (e.g., feet to meters). Unit safety is defined as static (compiler-checked) dimensional analysis. The term is a blending of the (computer science) notion of type safety with (physical) dimensional analysis. For example, in a unit-safe system, the compiler will not allow the user to set a field with dimensions of length to a quantity of hours.
The Units field extension has the following options:
base_dimensions(string): Specifies the dimensions of the field as a combination of powers of the base dimensions given in Table 3. For example, acceleration would be defined as"LT^-2".derived_dimensions(string): As a convenience alternative to thebase_dimensionsspecification, any of the Boost Units "derived dimensions" can be used. For example, instead ofbase_dimensions: "L^-1 M T^-2"for pressure, one can usederived_dimensions: "pressure". Multiplication and division of derived dimensions are also supported using the "*" and "/" operators. The available derived dimensions are those listed here: Dimensions Reference. The string to use here is the name of the dimension as given in Boost Units, minus the "_dimension". For example, to use boost::units::acceleration_dimension, specify "acceleration" in this field.system(string, defaults to "si"): A boost::units or user-defined system of units to use for this field. Defaults to the SI system with base units of kelvin (temperature), second (time), meter (length), kilogram (mass), candela (luminous intensity), mole (amount of substance) and ampere (electric current). The available Boost Units systems include:- "si": SI System Reference
- "cgs": CGS System Reference
- "temperature::celsius": Temperature System Reference
- "temperature::fahrenheit": Temperature System Reference
- "angle::degree": Trigonometry and Angle System Reference
- "angle::gradian": Trigonometry and Angle System Reference
- "angle::revolution": Trigonometry and Angle System Reference
You can also create your own system and use it in DCCL. To do this, make the namespace the same as the include file path, where "::" is replaced by "/". For example, if you define a system "foo::barsystem", you need to provide the definition of this system to the C++ compiler in "foo/barsystem.hpp".
relative_temperature(bool, defaults to false): A special extension only used for temperature fields. Setting this to true means that the temperature is relative (i.e., a difference of absolute temperatures) instead of an absolute temperature. This matters to support correct unit conversions between different temperature systems. For example, relative degrees Kelvin are the same as relative degrees Celsius, but the absolute scales differ by 273.15 degrees.unit(string): As an alternative to thedimensionsandsystemspecification, the field can be set to use particular (typically non-SI) units. A few examples of such units that are still often encountered in the marine domain areunit: "metric::nautical_mile",unit: "metric::bar", andunit: "us::yard". Here you can use any of the Boost Units Base Unit types, given here: Base Units by Category. The string to specify here is the Base Unit to use, minus the "boost::units::" prefix and the "_base_unit" suffix. For example,boost::units::metric::nautical_mile_base_unitshould be specified as "metric::nautical_mile" in this field.
DCCL Units Generated Code
The DCCL Units C++ generated accessors and mutators mirror those provided by the standard Google Protocol Buffers compiler for numeric fields, with the method name appended by the string "_with_units". For the standard Protobuf generated code see Google Protocol Buffers Reference. DCCL Units are only valid on numeric fields (either singular or repeated). Two accessors are provided for convenience: a non-template accessor that returns the value as the Quantity (i.e., boost::units::quantity) defined in the DCCL field, and a template accessor that can take any valid Boost Units Quantity (i.e., a type with the same dimensions as the DCCL field) and return the value in that type, accounting for all conversion factors.
Singular Numeric Fields
For a singular (optional or required) field "foo" with the following parameters:
- Type: "[foo type]", the original unit-less field type (e.g.
google::protobuf::int32,double) - Unit dimension: "[foo dimension]" (e.g.
boost::units::length_dimension,boost::units::derived_dimension<boost::units::length_base_dimension,1, boost::units::time_base_dimension,-1>::type) - Unit system: "[foo system]" (e.g.
boost::units::si::system,boost::units::degree::system)
the following additional methods are defined for unit safe access to the DCCL message:
Repeated Numeric Fields
For a repeated field "foo" with the following parameters:
- Type: "[foo type]"
- Unit dimension: "[foo dimension]"
- Unit system: "[foo system]"
the following additional methods are defined for unit safe access to the DCCL message:
DCCL Units examples
Here are a few example DCCL messages which include unit specification:
AUVStatus
See test/dccl_units/auv_status.proto
For example, to set an AUVStatus message's x and y fields to meters (the default for the base dimension of length, since the default system is SI), and then later access them as nautical miles, one can use this C++ example:
The value of x_nm is 0.54 nautical miles and y_nm is 0.27 nautical miles.
CTDMessage
See test/dccl_units/ctd_message_units.proto
CommandMessage
See test/dccl_units/command_message.proto
DCCL Dynamic Conditions (DCCL4+)
Dynamic Conditions is a new feature in DCCL4 that allows for conditional encoding of the fields based on runtime conditions (values of other parts of the message). This feature allows you to omit a field, mark a field "required", or change the values of the min/max bounds based on the value of one or more fields in the message.
Each dynamic condition is a string that contains a script written in Lua (https://www.lua.org/) that is evaluated each time the message is encoded or decoded.
The available dynamic_conditions are:
- required_if: The Lua script must return a boolean value: true means the field is now required (overriding the optional or required value in the .proto file).
- omit_if: The Lua script must return a boolean value: true means the field is omitted from the encoded message.
- only_if: A commonly used combination of required_if and omit_if. If true, this is the same as required_if returning true. If false, this is the same as omit_if returning true.
- min: The Lua script must return a numeric value setting the new minimum value. Note that this cannot be less than the hardcoded (dccl.field).min value (if it is, (dccl.field).min is used instead).
- max: The Lua script must return a numeric value setting the new maximum value. Note that this cannot be greater than the hardcoded (dccl.field).max value (if it is, (dccl.field).max is used instead).
Special variables
Within the Lua script you are given access to some special variables set by DCCL:
- this (a Lua table acting as a struct) is the defined as the current contents of the innermost Message
- root (a Lua table) is the outermost message.
- this_index (a numeric, aka integer) is the index to the current repeated field element if this (sub)message is contained within a repeated field.
For example, given the following message:
"this" and "root" refer to the contents of TestMsg.
However, given a different message:
Now within the context of field "i", "this" refers to Child, and "root" refers to TestMsg.
Other notes
Since this is a common idiom and to reduce extra code, you can omit "return" if it is the first part of the Lua script. That is, these are identical:
If your script doesn't begin with "return" you must put it in explicitly:
The Lua Protobuf functionality uses this wonderful Github project: https://github.com/starwing/lua-protobuf. Please reference the documentation in the event you need more details about the "this" or "root" tables, which are built using this library.
For more details, and an example usage, see the dccl_dynamic_conditions unit test.
