1 Stream Abstraction
- 1.1 Abstraction Intent
- 1.2 Benefits
- 1.3 Background
  - 1.3.1 Device I/O vs. File I/O
  - 1.3.2 Simulate Software on Linux
2 Device Stream Contract and Device Stream Glue
- 2.1 Note about copying sockets:
3 Use Cases
- 3.1 UART
- 3.2 GPIO
- 3.3 SPI
  - 3.3.1 SPI Chip-Select
  - 3.3.2 SPI with Mutex
  - 3.3.3 Thoughts on bi-directional SPI
- 3.4 Flash Memory
- 3.5 I2C
  - 3.5.1 I2C Read
- 3.6 CAN I/O
  - 3.6.1 Read All Messages
  - 3.6.2 Read Specific Messages
  - 3.6.3 Write Messages
- 3.7 Network I/O
  - 3.7.1 TCP Client
    - 3.7.1.1 TCP Client with Send & Receive Timeouts
  - 3.7.2 TCP Server
  - 3.7.3 UDP Receive
  - 3.7.4 UDP Transmission
4 Device Stream Dispatcher
- 4.1 Platform Dispatchers
- 4.2 End-to-End Example

Stream Abstraction

Abstraction Intent

The dev_stream abstraction targets stream based drivers, such as UART, CAN etc. where information can flow in and out of the peripheral. This abstraction is more than just a hardware abstraction (HAL) residing just above the peripheral. For example, a UART Stream I/O abstracts it away from specific configuration of the end-user peripheral, such as the baud rate, and the parity bit configuration. This also abstracts the use of the RTOS; for instance, if the intent is to write 10 characters to UART, the underlying Device Stream Glue could send the data to a queue, and the high level Application code is agnostic to the specific driver level details. In this sense, the Stream I/O is a Platform Abstraction, and not necessarily just a HAL.

The objective is that the high level Sibros code should be agnostic to specific workings of a peripheral of the end-user.

Device Stream abstraction does not address specialized register based peripherals, such as a PWM configuration or the manipulation of a Watchdog timer; that abstraction is addressed using a different design. Device stream can abstract anything you believe is feasible, and factoring in the benefits, you can apply it where it fits. For instance, the GPIO is not necessarily a stream based peripheral device, but we can easily apply it to reap the benefits; see the GPIO example below.

Benefits

The Stream Abstraction provides:

Consistent API to read and write diverse set of peripheral devices (i.e.: CAN, I2C, Flash memory), while avoiding API fragmentation.
Portable and re-usable higher level Application code which increases interoperability between product developers.
A single tunnel for many communication peripherals. This helps:
- Facilitate Unit-Testing; it becomes easier for the developers to deal with only one kind of "Mock".
- Make your software "ready" for Software in-the-loop (SIL) simulation. You can re-compile the code for Linux and re-direct the I/O to behave differently

Background

A consistent and well understood I/O abstraction is necessary to encapsulate vendor specific peripheral drivers and reduce API fragmentation. An embedded platform can vary wildly between different micro-controllers, and in order to maximize productivity and code re-use, there is benefit to be able to read and write diverse I/O through the same code interface.

The abstraction mimics a Linux device such that any interaction with the underlining hardware can occur using read() or write() methods. To minimize namespace and application integration conflicts we prepend each API with dev_stream__, such asdev_stream__read() and dev_stream__write().

Device stream abstraction is ideally suited for stream based I/O, however, the aim is to also solve for sophisticated I/O to minimize abstraction fragmentation. Network I/O is a more complex case, and the goal includes to abstract this case also because it leaves the end-user with a single code module (Device Stream Glue) to hook Sibros applications to your platform.

Our solution resembles FreeRTOS+IO, but we do not use the same function names as FreeRTOS+IO because the RTOS itself may be abstracted away. Another distinction from FreeRTOS+IO is that we do not implement very domain specific control, such as polled, or zero copy ioctl types, hence making trivial Device stream glue simpler.

Sometimes, developers create an abstraction but the use cases are not documented or tested. When this occurs, it creates two problems to deal with. First, the user of the API may not understand how to use different pieces of API to solve a particular problem. Secondly, and worse, the API may not even have solved very specific problems at all because maybe a awkward use case was missing. With this in mind, we created the Sibros Device Stream not only with abundant documentation, but also with examples of how it would work for different use cases.

Device I/O vs. File I/O

To perform I/O, the two variants are File I/O and Device I/O. Here is the comparison of the read() function between the two:

// File
size_t fread ( void * ptr, size_t size, size_t count, FILE * stream );

// Device
ssize_t read(int fd, void *buf, size_t count);

Since we want to be able to get error codes back while reading (or writing) a device, it is preferred to use the Device I/O since the return type is a signed type instead of size_t. The one thing that the Device I/O lacks is that the size_t count is missing, which we added to our dev_stream__read() and dev_stream__write() API.

Simulate Software on Linux

The Device stream easily allows to re-direct the API to linux devices.

#ifdef LINUX
#define dev_stream__read(device, input_buffer, bytes_per_element, num_elements) \
  read(device, input_buffer, (bytes_per_element * num_elements))
#endif

Device Stream Contract and Device Stream Glue

The specifications and the contact of the Device Stream Abstraction is contained in the code below. The implementation of this file is what we call Device Stream Glue which the end-user (you) implements to glue our modular code blocks to your software. Don't worry, there is not much to implement if you already have your platform drivers, and hence it should be just a re-direction from an API intent to your existing code.

dev_stream.h This header file provides the contract or the "tunnel" to read and write stream based peripheral drivers. There is sample code at the end of this guide which provides helpful reference for the implementation.

#pragma once

#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>

/*******************************************************************************
 *
 *                               D E F I N E S
 *
 *******************************************************************************/

/*******************************************************************************
 *
 *                                 E N U M S
 *
 *******************************************************************************/

/**
 * This provides different control types for dev_stream__control()
 * The use cases are documented for the Device Stream control types
 */
typedef enum {
  DEV_STREAM__CONTROL_NONE = 0,

  /**
   * @{
   * Set a read or a write timeout on a device.  Initial timeouts on a device should
   * default to infinity.  After a read or write timeout is applied, then the device
   * should start to respect the read timeout during read() and write timeout during
   * write()
   * Argument: 'uint32_t timeout_ms'
   *
   * Examples:
   * - Set a transmit() timeout for a UART putchar
   * - Set a recv() timeout on a socket
   */
  DEV_STREAM__CONTROL_SET_READ_TIMEOUT,
  DEV_STREAM__CONTROL_SET_WRITE_TIMEOUT,
  /** @} */

  /**
   * Flush the data stream completely before returning
   * Argument: None
   *
   * Examples
   * - Flush transmit FIFO of the UART
   */
  DEV_STREAM__CONTROL_FLUSH_TX,

  /**
   * @{
   * A device may require a mutex to be able to use it from multiple contexts
   *
   * For a device like an SPI, DEV_STREAM__CONTROL_ACCESS_REQUEST can perform multiple
   * things upon this request such as:
   *  - Take mutex
   *  - Device chip-select (CS)
   */
  DEV_STREAM__CONTROL_ACCESS_REQUEST,  ///< Argument: 'uint32_t timeout_ms'
  DEV_STREAM__CONTROL_ACCESS_RELEASE,  ///< Argument: None
  /** @} */

  /**
   * I2C bus requires the START and STOP conditions for a data transfer much
   * like the SPI chip-select and de-select.  In particular, the I2C bus
   * read transaction involves, START, WRITE ... REPEAT-START, READ ... STOP
   */
  DEV_STREAM__CONTROL_I2C_START,         ///< Argument: 'uint32_t timeout_ms' that can be used to take an I2C mutex
  DEV_STREAM__CONTROL_I2C_REPEAT_START,  ///< Argument: None
  DEV_STREAM__CONTROL_I2C_STOP,          ///< Argument: None
  /** @} */

  /**
   * A TCP listening socket uses this control type to perform the accept()
   * Argument: 'dev_stream_T *accepted_client'
   */
  DEV_STREAM__CONTROL_TCP_ACCEPT,

  /**
   * @{
   * A UDP socket is just a resource and it can be used to send the data packet
   * to any IP address.  These control types "set" the destination IP and port.
   */
  DEV_STREAM__CONTROL_UDP_SEND_TO_HOST,  ///< Argument: 'uint16_t port'
  DEV_STREAM__CONTROL_UDP_SEND_TO_PORT,  ///< Argument: 'const char * host'
                                        /** @} */
} dev_stream__control_E;

/**
 * Flag types while opening a device
 * This provides distinctions such as:
 *   - An input or output GPIO
 *   - A transmit or a receive CAN mailbox
 */
typedef enum {
  DEV_STREAM__FLAGS_NONE = 0,
  DEV_STREAM__FLAGS_INPUT,
  DEV_STREAM__FLAGS_OUTPUT,
  DEV_STREAM__FLAGS_INPUT_AND_OUTPUT,
} dev_stream__flags_E;

/// Seek types used at dev_stream__lseek()
typedef enum {
  DEV_STREAM__SEEK_NONE = 0,
  DEV_STREAM__SEEK_SET,      ///< Set the seek to the absolute location
  DEV_STREAM__SEEK_CURRENT,  ///< Set the seek to current location plus the offset
  DEV_STREAM__SEEK_END,      ///< Set the seek to size of the file plus the offset
} dev_stream__seek_E;

/*******************************************************************************
 *
 *                               T Y P E D E F S
 *
 *******************************************************************************/

/**
 * The Device Stream is simply a unique descriptor
 * It is used as an identifier to figure out how the read() and write() should
 * re-direct the request for a particular descriptor (i.e.: UART vs. SPI)
 */
typedef int dev_stream_T;

/*******************************************************************************
 *
 *                        P U B L I C    F U N C T I O N S
 *
 *******************************************************************************/

/**
 * Open a device for read and/or write
 * @param descriptive_path   The path of the device, such as "/dev/uart0"
 * @param flags              @see dev_stream__flags_E; this is an int to conform with linux style open()
 *
 * @returns the descriptor (0 is a valid value), or -1 if there was an error acquiring the device
 */
dev_stream_T dev_stream__open(const char *descriptive_path, dev_stream__flags_E flags);

/**
 * Closes a device
 * Typically, a persistent device such as UART doesn't need a closure
 * The use case for this is to close network sockets
 *
 * @returns 0 on success, or non-zero which is an error code
 */
int dev_stream__close(dev_stream_T device);

/**
 * Reads num_elements with each element of size bytes_per_element
 * to the input_buffer from the device referenced by dev_stream_T
 *
 * @returns -1 upon an error
 * @returns the number of bytes read into the input_buffer
 *
 * @note the number of bytes read (and returned) may be fewer than requested (num_elements * bytes_per_element)
 */
int dev_stream__read(dev_stream_T device, void *input_buffer, size_t bytes_per_element, size_t num_elements);

/**
 * Writes num_elements with each element of size bytes_per_element
 * from the output_buffer to the device referenced by dev_stream_T
 *
 * @returns -1 upon an error
 * @returns 0 if the device was not ready to be written
 * @returns the number of bytes written to the device
 */
int dev_stream__write(dev_stream_T device, const void *output_buffer, size_t bytes_per_element, size_t num_elements);

/**
 * Perform a special operation on the descriptor
 * @param request_type @see dev_stream__control_E
 * @param argument Depends on the value of request_type; @see dev_stream__control_E
 *
 * @returns zero upon success, or non-zero error code upon failure
 */
int dev_stream__control(dev_stream_T device, dev_stream__control_E request_type, void *argument);

/**
 * Perform a seek operation on the device (file seek) for the next read() or write()
 * The meaning of this API depends on the device, but is ideally suited for
 * an address based device, such as a file, or flash memory device
 *
 * @returns -1 upon an error
 * @returns the resulting offset location as measured in bytes from the beginning of the file.
 */
int dev_stream__lseek(dev_stream_T device, int offset, dev_stream__seek_E whence);

/**
 * @returns true if the device descriptor is valid
 *
 * Typically, this should return true if the device descriptor is >= 0
 * Sometimes, value of zero may be considered invalid
 */
bool dev_stream__is_valid(dev_stream_T device);

Note about copying sockets:

The dev_stream_t device descriptor essentially acts as the socket number. It should NEVER be copied. Copying this is similar to copying a pointer, leaving it at the mercy of future developers. The core issue is that if one copy is destructed (ie: connection closed) then the second copy should also be destructed. This may be somewhat possible in C++ and other higher-level languages but this is especially hard to enforce in C.

Use Cases

The stream abstraction is designed to solve various different use cases of common micro-controller peripherals.

UART
- Write output
- Wait for input (with timeout)
GPIO
- Read and Write
- Configure a complex GPIO
SPI
- Transmit only
- Receive only
- Data transfer with a mutex
Flash Memory
- Read
- Write (to a specific location)
I2C
- Write a byte stream to an addressed I2C device
- Read a byte stream
CAN
- Write a dedicated mailbox
- Poll for a message from a receive mailbox
- Read a message from a mailbox with a timeout
Sockets
- Client socket to connect to a server
- Server (listening) socket
- UDP receive
- UDP transmit

UART

We do not implement polled and queue configuration on the high level code. Instead, we leave it up to the Device Stream Glue to provide the desired behavior. For example, in the code snippet below, we can assume that the UART0 is opened with the RTOS backed queue to send and receive data. Initially when the device is opened, the timeout value should be infinity, but after the timeouts are set using dev_stream__control_E enumeration then the dev_stream__write() and dev_stream__read() should use the configured timeouts.

GPIO

The GPIO can be configured as input, output, or input/output through the dev_stream__flags_E, and from thereafter dev_stream__read() or dev_stream__write() can be used to access or control it. This abstraction has a very small overhead, in general if an application needs faster timing requirements than this supports, a different module should be used.

SPI

The SPI peripheral must send data out in order to receive the data. The transmit case is simple because the write() would simply discard the input data. The read() case assumes that the Device Stream Glue sends 0xFF for each byte that shall be received.

SPI Chip-Select

Often times, SPI devices may implement a transaction such that there are bytes written to the bus, and then bytes can be read back, and this "transaction" is a single chip-select data transfer.

SPI with Mutex

End-user code that utilizes SPI may implement mutex to guard the SPI resource from being used from multiple contexts. To implement portability with this caveat in mind, there is a control type with the intent of taking and giving back a mutex. The Device Stream Glue shall be responsible to create the mutex if necessary.

Thoughts on bi-directional SPI

Typically this peripheral writes data and reads back the data onto the same source. Fundamentally, this peripheral is designed such that a shift register latches an output bit, and as it shifts the one bit that was output, it reads the new bit from the other side. When 8-bits have been shifted-out, 8 new bits would have been shifted-in.

The write() function sends data out, although we have not yet implemented storing the received data that was input while sending the data out of the SPI peripheral. The plan of record is to add a control type to perform something like DEV_STREAM__CONTROL_SET_READ_BACK_PTR that can be used during the next dev_stream__write() but since there has not been a use-case for this (so far), it has not taken inception yet. We are following the Agile or YAGNI principle here.

Flash Memory

For a generic flash memory driver, such as on-board flash driver, the usual intent is to write blocks of data at a specific location. Remember that flash memory can only be written if it was erased. With this in mind, we can design with the following key points:

dev_stream__open() can not only "open" but erase the memory block
- If the type was DEV_STREAM__MODE_INPUT then it can be read-only and erase is not necessary
Once the flash memory block is opened, it can be written by seeking to an appropriate address
- Address bounds can be checked within dev_stream__write()

I2C

Below is an example where we write 0x02 and 0x03 to an I2C device with address 0xF0. Typically, this means that you want to write the register 0x02 with the value of 0x03 (a typical I2C transaction).

I2C Read

The read transaction is a little tricky with the I2C bus because you have to send bytes, and then receive bytes in a single transaction. The sample code below assumes the the I2C bus has an underlining driver that can send and receive bytes independently.

CAN I/O

For CAN I/O, we can implement various different descriptors that map to the CAN mailboxes. The flags argument for dev_stream__open() is used to identify if the descriptor is going to be used for either reading or writing a CAN HW mailbox.

Read All Messages

Read Specific Messages

CAN drivers typically implement a mask and message id that can be used to setup filters to receive particular messages. The Stream Device does not target very specific behavior for such a case, and it can be implied that there is an intent to receive filtered data only.

Write Messages

The descriptor name can identify the purpose for a CAN mailbox transmission. For example, if Sibros performs a dev_stream__open() with /dev/can0/ecu_info then the Device Stream Glue can trigger on this name and map the descriptor to a dedicated transmission mailbox. Therefore, the code sitting above the Sibros Device abstraction is agnostic to a particular CAN mailbox and it is up to the Device Stream Glue to link the descriptor to their choice of actual hardware CAN mailbox.

It is also up to the Device Stream Glue to ensure that the write() API blocks forever until the underlining driver is ready to write the Hardware CAN mailbox. If the user sets the timeout on the CAN_transmit descriptor, then the Device Stream Glue can deposit the data into an RTOS queue with the configured timeout.

Network I/O

The network I/O is a more complex type of stream. There needs to be a consistent method of encoding and decoding network paths and we designed a code module to handle this. Basically, the following type of encoded paths allow you to open a stream while identifying what type of stream we really desire:

/net/udp/hostname/1234
/net/tcp/server/localhost/1234
/net/tcp/client/localhost/1234

TCP Client

The intention of the TCP client is to connect to another peer. The Device Stream Glue can default the connection timeout to a sane value, such as 3 seconds before giving up and returning NULL as a response from dev_stream__open().

TCP Client with Send & Receive Timeouts

TCP Server

A listening socket doesn't perform read() or write()and thus we need a special function to perform the equivalent of an accept() function which we implement by using a control() function.

UDP Receive

The one limitation to the code below is that a UDP socket can receive data from anyone on the network (it is connection-less), and the dev_stream__read() does not identify where the data came from. For the time being, there is no proposal to address this limitation, and can be designed upon request.

UDP Transmission

The UDP transmission is a little tricky because a UDP socket may be used to send data to anyone because the UDP socket is not associated with just one peer, and this socket is just a resource that can be used to send data to anyone.

Device Stream Dispatcher

You do not need to implement a lot of boiler plate code to glue your hardware platform to the Device Stream. There is a dev_stream_dispatcher.h that can perform the heavy lifting and allow you to focus on defining I/O for specific devices. This section provides starter code to glue Sibros Device with your hardware. There may be control() types that are not relevant for certain devices, and thus the implementation can be omitted.

The Device Stream Dispatcher provides the implementation of the dev_stream.h for you, and you only should care about the registered callbacks. In addition, it features the following:

Match the dev_stream__flags_E during dev_stream__open()
Protection against multiple dev_stream__open()
Not invoking open_fptr(), write_fptr() etc. if they are NULL
close() will invoke your close_fptr() and free the descriptor for re-use
dev_stream__open() takes care of matching the descriptor name and supports
- Partial matching to allow for socket API behavior using IP lib (LWIP)

Note that each dispatcher you register, such as CAN or LWIP can have their own descriptor IDs and they may even overlap! The dispatcher will take care of mapping the right IDs to each of the registered dispatchers. This is why the d0 and d1 can both exist at different dispatchers.

Platform Dispatchers

Here is the sample code that registers your CAN and SPI Device stream handlers. Contact us to get the implementation of the dev_stream_dispatcher.h

End-to-End Example

Refer to this link.