2022 November release

This release is for version 1.0.0-beta of the KOS platform. This is intended as a pre-release before the first major release, 1.0.0, is delivered in the first quarter of 2023.

The goal for this release is to make it easy for a brand new developer to understand KOS and begin creating and running simple but realistic programs.

Highlights

KOS Mix plugins

The Kry10 Secure platform required understanding and configuring many low-level aspects of a system, and required modifying and writing C code, FDT (for manifest), cmake files, nix files, elixir, mix. It also involved invoking various different tools including cmake, ninja, nix, various home brewed scripts, picocom, iex, docker, etc.

A Mix extension for developing Kry10 systems in order to hide that complexity. It aims to achieve the following:

• Provide a development environment that is comfortable for developers.
• A development environment and workflow that does not require complicated tasks to prepare the environment, to develop a KOS-based system or application, to build the system, to deploy the system, to test the system.
• A development environment and workflow that hides the “systems” details and technologies (e.g. C programming, cmake, nix, partitioning SD cards, etc.) for the common case, only requiring access to those technologies when absolutely necessary to make low-level changes to the target system.
• A mechanism for downloading and installing Kry10 platform releases.

You can do:

• mix kos This provides a mix task that outputs the available KOS mix tasks.
• mix kos.orbit.start This will pull the Docker images from ‘ghcr.io’ and use Docker and Docker Compose to start a copy of Orbit.
• mix kos.env This will look for the toolchains in a pre-installed KOS installation and spawn a new terminal running the KOS development shell. This shell environment contains all of the KOS mix actions available for a particular Kry10 platform release.
• mix kos.docs This will build a copy of the KOS documentation using a pre-installed KOS installation. After the build is complete, the documentation will be opened using the default web browser of the current user. If this fails, the task will output a URL for the documentation to be manually opened. The documentation will be placed in the docs folder in the pre-installed KOS installation.
• mix kos.install This will install a copy of the KOS sources into a given destination directory. The code will be placed into the directory pointed to environment variable KOS_HOME which defaults to the .kos folder of the current user’s home directory.

Within a kos.env shell:

[nix-shell:~]$ mix kos
mix kos.build                      # Builds the KOS system
mix kos.deps.check                 # Verify system prerequisites are sufficient
mix kos.docs                       # Load the Kry10 documentation
mix kos.env                        # Launch a KOS development environment shell
mix kos.install                    # Installs a copy of the KOS code
mix kos.local.scenic_driver_remote # Create a local copy of the scenic_driver_remote sources
mix kos.local.testbed              # Create a local copy of the kos_testbed sources
mix kos.new                        # Creates a new KOS system project
mix kos.new_app                    # create a new KOS Elixir app
mix kos.orbit.start                # Launch Orbit, the Kry10 Management Service
mix kos.toolbox.build_bootfs       # Build a bootfs image from a KOS build
mix kos.toolbox.kos_trace_to_json  # Convert KOS trace binary data to Google's JSON Trace Format
mix kos.toolbox.kos_tunnel_ifdown  # Bring down the KOS tunnel network interface
mix kos.toolbox.kos_tunnel_ifup    # Bring up the KOS tunnel network interface
mix kos.toolbox.reload_board       # Fastboot a KOS build to a device.
mix kos.toolbox.remote_iex         # Creates an iex shell on the device
mix kos.toolbox.sign_manifest      # Sign a KOS manifest image for upgrades
mix kos.toolbox.simulate_arm       # Run a KOS build on the simulated ARM platform
mix kos.toolbox.simulate_x86       # Run a KOS build on the simulated x86 platform
mix kos.toolbox.upload_overlay     # Uploads an overlay to Orbit and sets it as the device's current overlay.
mix kos.toolbox.upload_release     # Uploads a release package to Orbit.

Tunnel IP management

Prior to this release, the private IP that devices should use to communicate with Orbit had to be manually set in the manifest, with the developer ensuring that the private IP did not conflict with any existing device IPs.

Now, all devices believe they’re communicating from a private IP in the 192.168.2.0/24 range; the developer does not need to configure anything manually. To determine which device a message is coming from and where to send replies, Orbit assigns a unique private /24 IP range to any device with a tunnel public key, so that developers only need to manage the keys.

Elixir Manifests

A manifest is a contract of a KOS system. The contract is a formalized description of the applications allowed to run on a KOS system and how they can interact and rely on each other.

The Elixir library kos_manifest allows a user to construct a KOS manifest inside an Elixir-based environment, rather than needing to understand a DTS format. A system manifest can be declared inside a project mix.exs project and supports the following manifest features:

• Defining applications
• Creating message servers
• Publishing and connecting to protocols
• Including existing KOS applications
• Restart dependencies
• Clock and pin multiplexer setups

A new KOS system image can be created with executing the kos.manifest mix task:

arm-kos_qemu-linux-musleabihf-beam-env mix kos.manifest --module AdminExampleManifest --output-dts-path manifest.dts --output-zip-path latest.zip

From here it is easy to create a new Orbit release with the kos.toolbox.upload_release mix task: mix kos.toolbox.upload_release --orbit http://localhost:4000/ --credential-path creds --name new latest.zip

More information can be found in the Develop a KOS system manifest section of the developers guide.

KOS GCC toolchain

The KOS developer environment now provides an actual cross-compiler that can compile C source code to KOS target application environments.

The KOS gcc toolchain currently supports the targets: arm-kos_<plat>-linux-musleabihf for Arm and x86_64-kos-linux-musl for x86_64.

For the qemu-arm-virt platform, these tools are provided by the kos_runtime package:

arm-kos_qemu-linux-musleabihf-addr2line
arm-kos_qemu-linux-musleabihf-ar
arm-kos_qemu-linux-musleabihf-as
arm-kos_qemu-linux-musleabihf-c++
arm-kos_qemu-linux-musleabihf-c++filt
arm-kos_qemu-linux-musleabihf-cc
arm-kos_qemu-linux-musleabihf-cpp
arm-kos_qemu-linux-musleabihf-elfedit
arm-kos_qemu-linux-musleabihf-g++
arm-kos_qemu-linux-musleabihf-gcc
arm-kos_qemu-linux-musleabihf-gcc-9.4.0
arm-kos_qemu-linux-musleabihf-gcc-ar
arm-kos_qemu-linux-musleabihf-gcc-nm
arm-kos_qemu-linux-musleabihf-gcc-ranlib
arm-kos_qemu-linux-musleabihf-gcov
arm-kos_qemu-linux-musleabihf-gcov-dump
arm-kos_qemu-linux-musleabihf-gcov-tool
arm-kos_qemu-linux-musleabihf-gprof
arm-kos_qemu-linux-musleabihf-ld
arm-kos_qemu-linux-musleabihf-ld.bfd
arm-kos_qemu-linux-musleabihf-nm
arm-kos_qemu-linux-musleabihf-objcopy
arm-kos_qemu-linux-musleabihf-objdump
arm-kos_qemu-linux-musleabihf-ranlib
arm-kos_qemu-linux-musleabihf-readelf
arm-kos_qemu-linux-musleabihf-size
arm-kos_qemu-linux-musleabihf-strings
arm-kos_qemu-linux-musleabihf-strip

KOS provides a different OS environment for KOS applications than what common OSes such as Linux provide. This means that pre-built C compiler toolchains for KOS applications aren’t available without providing a lot of overriding compilation options. Until now, CMake has been used to encode all of the build configuration for building KOS systems as configuration options and wrapper scripts around a regular Linux target toolchain.

Manifest Authorization Signatures

This release adds support for signing manifest images. Signed manifests can be used to ensure that all software and configurations that run on a device are authorized. This is achieved by employing a signing mechanism that allows a device to independently authenticate that an upgrade manifest is valid and authorized.

This change introduces three main features:

• A new toolbox utility to generate signed manifest images, kos.toolbox.sign_manifest / kosmanifest-sign
  • This utility takes in a private .pem key (for signing) and a manifest zip archive (to be used as an upgrade) and produces a signed variant of the archive.
  • The utility currently only supports the ed25519 key scheme when signing the manifest.
  • A signed variant of the manifest includes a signature binary of the manifest blob, stored as an additional file within the archive i.e apps/manifest.dtb.sig.
• Extended the kos_options node to accept new signing-related properties:
  • public-key : A base64 encoded public key that is used to authenticate manifest signatures from incoming upgrades. When a device receives an upgrade, it will access this field from its currently loaded manifest to authenticate the upgrades signature.
  • disable-signing: A boolean property that indicates whether the next subsequent upgrade image needs to be signed to be successfully loaded. If the disable-signing property exists, validation of signatures does not need to successfully pass in order for the upgrade to be loaded (mostly useful for development workflows).
  • e.g.
  • kos_options {
     public-key = <base-64 encoded string>;
     disable-signing;
    };
    
  • The kos.toolbox.sign_manifest / kosmanifest-sign utility is able to automatically insert the above properties into the resulting signed manifest image through the --apply-props command line parameter.
• Updated the KOS admin component to support validating manifest authorization signatures. On an upgrade, this fetches the devices current manifest to authenticate the incoming upgrade. If signing is enforced, this will either authorize or reject an upgrade, based on the signature of the incoming upgrade.

Data diode mechanism

A data diode is a unidirectional communication mechanism that enables the one-way transfer of data between segmented applications and networks.

KOS now provides abstractions that facilitate the building of software-defined data diode mechanisms. This in turn can be used to develop high-assurance communication bridges that can exist between KOS message server domains.

Data diode mechanisms are enabled by the introduction of two new features:

• Introduction of a new manifest primitive - KOS channels :
  • These being global communication channels that can be shared between multiple applications, which the root server can instantiate at initialization time, outside the context of the message server.
  • A channel currently consists of a single frame (for data transfer) and a notification object (for asynchronous signalling).
  • A channel is expected to be defined in a manifest under the /channels device tree node. An application is able to define their access to a set of channels, along with their READ/WRITE/SIGNAL/RECV rights to each, by providing a channelsfield in their resources sub-node.
  • At runtime, channels are made available to each application through their dev_resource book-keeping structures. A dev_resource slot for a given channel contains a cap to a CNode. The CNode encapsulates the caps for that channel (currently being a single frame and notification cap).
  • e.g
    

• A new utility library, kos_diode : To better facilitate the building of software diodes over KOS channels, introduced a new utility library, kos_diode. The library provides functions to instantiate a diode protocol over a KOS channel and utilities for sending and receiving payloads over a diode.

Device Overlays

Devices within a single deployment should all be largely the same, although we now support device overlays for managing minor differences between devices. These are intended to be used for minor changes - things like how the device identifies itself in logs. We use the DTBO format for specifying overlays.

An overlay that overrides a specific environment key for the app weatherstation could be something like:

/dts-v1/;
/plugin/;

&{/apps} {
 weatherstation {
   environ {
     DEVICE_ID = "1a7aa52a-3543-4cde-8117-f176edb91daf";
   }
 };
};

The overlay can be applied as long as the device manifest has the following kos_options to allow such override:

kos_options {
 variables = "/apps/weatherstation/environ/DEVICE_ID";
};

Improved documentation

We’ve reorganised our documentation to try to make it easier for developers to find the information they’re looking for or to quickly get started with KOS if they’re brand new. The structure is inspired by the Diátaxis framework which has helped us come up with a roadmap for our docs. There are many additional pages we hope to complete over the coming releases to make KOS even more accessible.

Examples

Weather Station

• MQTT Support: Extended the weather station with a simple example of using an MQTT client (EMQTT in this instance) to publish and subscribe to a MQTT broker. This example demonstrates publishing the weather station data to an MQTT broker at a scheduled interval. The example also uses the MQTT subscribe functionality to implement a simple control plane for the weather station.

MQTT + Data Diode Example

Introduces a new KOS testbed example system, mqtt_example. The system demonstrates the example construction of an MQTT gateway, utilizing KOS’s data diode infrastructure. The example creates 2 main apps:

• mqtt_broker: An app that launches a NanoMQ broker. Network access to the NanoMQ broker is made available through the KOS tunnel via the default IP address 192.168.2.6. The broker also instantiates a communication bridge with the app mqtt_diode_bridge to forward all PUB/SUB/CONNECT messages through.
• mqtt_diode_bridge: A MQTT bridge client that receives all MQTT traffic from the mqtt_broker and forwards it onto an ‘external’ broker. By default, the bridge client connects to the external broker via the KOS tunnel gateway IP address 192.168.2.100:1883. The app expects to receive all forwarded messages from the mqtt_broker via a KOS data diode (backed by a KOS channel instance).

Removed features

Clang Compiler support

Clang compiler support has been temporarily removed from the KOS developer environment. The existing support was still fairly unstable and would sometimes fall back to using compilation options for the host platform instead of the target platform. We plan to provide updated support for Clang on top of the KOS GCC + binutils toolchain in an upcoming release.

Third-party software dependency updates

• OTP 23.3.4.4 → 25.1.2
• Elixir 1.12.1 → 1.14.2
• nixpkgs 16105403bdd → 22.11

‍