Platform Abstraction

Owned by Former user (Deleted)
Last updated: Apr 03, 2021 by Former user (Deleted)
18 min read

Stream Abstraction

Abstraction Intent

TheĀ dev_streamĀ abstraction targets stream based drivers, such as UART, CAN etc. where information can flow in and out of the peripheral. This abstraction is more than just a hardware abstraction (HAL) residing just above the peripheral. For example, a UART Stream I/O abstracts it away from specific configuration of the end-user peripheral, such as the baud rate, and the parity bit configuration. This also abstracts the use of the RTOS; for instance, if the intent is to write 10 characters to UART, the underlying Device Stream Glue could send the data to a queue, and the high level Application code is agnostic to the specific driver level details. In this sense, the Stream I/O is aĀ Platform Abstraction, and not necessarily just a HAL.

The objective is that the high level Sibros code should be agnostic to specific workings of a peripheral of the end-user.

Device Stream abstraction does not address specialized register based peripherals, such as a PWM configuration or the manipulation of a Watchdog timer; that abstraction is addressed using a different design. Device stream can abstract anything you believe is feasible, and factoring in the benefits, you can apply it where it fits. For instance, the GPIO is not necessarily a stream based peripheral device, but we can easily apply it to reap the benefits; see theĀ GPIO example below.

Benefits

The Stream Abstraction provides:

  • Consistent API to read and write diverse set of peripheral devices (i.e.: CAN, I2C, Flash memory), while avoiding API fragmentation.

  • Portable and re-usable higher level Application code which increases interoperability between product developers.

  • A single tunnel for many communication peripherals. This helps:

    • Facilitate Unit-Testing; it becomes easier for the developers to deal with only one kind of "Mock".

    • Make your software "ready" forĀ Software in-the-loop (SIL)Ā simulation. You can re-compile the code for Linux and re-direct the I/O to behave differently

Background

A consistent and well understood I/O abstraction is necessary to encapsulate vendor specific peripheral drivers and reduce API fragmentation. An embedded platform can vary wildly between different micro-controllers, and in order to maximize productivity and code re-use, there is benefit to be able to read and write diverse I/O through theĀ same code interface.

The abstraction mimics a Linux device such that any interaction with the underlining hardware can occur usingĀ read()Ā orĀ write()Ā methods. To minimize namespace and application integration conflicts we prepend each API withĀ dev_stream__, such asdev_stream__read()Ā andĀ dev_stream__write().

Device stream abstraction is ideally suited for stream based I/O, however, the aim is to also solve for sophisticated I/O to minimize abstraction fragmentation. Network I/O is a more complex case, and the goal includes to abstract this case also because it leaves the end-user with a single code module (Device Stream Glue) to hook Sibros applications to your platform.

Our solution resemblesĀ FreeRTOS+IO, but we do not use the same function names as FreeRTOS+IO because the RTOS itself may be abstracted away. Another distinction from FreeRTOS+IO is that we do not implement very domain specific control, such as polled, or zero copyĀ ioctlĀ types, hence making trivial Device stream glue simpler.

Sometimes, developers create an abstraction but the use cases are not documented or tested. When this occurs, it creates two problems to deal with. First, the user of the API may not understand how to use different pieces of API to solve a particular problem. Secondly, and worse, the API may not even have solved very specific problems at all because maybe a awkward use case was missing. With this in mind, we created the Sibros Device Stream not only with abundant documentation, but also with examples of how it would work for different use cases.

Device I/O vs. File I/O

To perform I/O, the two variants are File I/O and Device I/O. Here is the comparison of theĀ read()Ā function between the two:

// File size_t fread ( void * ptr, size_t size, size_t count, FILE * stream ); // Device ssize_t read(int fd, void *buf, size_t count);

Since we want to be able to get error codes back while reading (or writing) a device, it is preferred to use the Device I/O since the return type is a signed type instead ofĀ size_t. The one thing that the Device I/O lacks is that theĀ size_t countĀ is missing, which we added to ourĀ dev_stream__read() Ā andĀ dev_stream__write()Ā API.

Simulate Software on Linux

The Device stream easily allows to re-direct the API to linux devices.

#ifdef LINUX #define dev_stream__read(device, input_buffer, bytes_per_element, num_elements) \ read(device, input_buffer, (bytes_per_element * num_elements)) #endif

Device Stream Contract and Device Stream Glue

The specifications and the contact of the Device Stream Abstraction is contained in the code below. The implementation of this file is what we callĀ Device Stream GlueĀ which the end-user (you) implements to glue our modular code blocks to your software. Don't worry, there is not much toĀ implementĀ if you already have your platform drivers, and hence it should be just aĀ re-directionĀ from an API intent to your existing code.

dev_stream.hĀ This header file provides theĀ contractĀ or the "tunnel" to read and write stream based peripheral drivers. There isĀ sample code at the end of this guideĀ which provides helpful reference for the implementation.

#pragma once #include <stdbool.h> #include <stdint.h> #include <stdlib.h> /******************************************************************************* * * D E F I N E S * *******************************************************************************/ /******************************************************************************* * * E N U M S * *******************************************************************************/ /** * This provides different control types for dev_stream__control() * The use cases are documented for the Device Stream control types */ typedef enum { DEV_STREAM__CONTROL_NONE = 0, /** * @{ * Set a read or a write timeout on a device. Initial timeouts on a device should * default to infinity. After a read or write timeout is applied, then the device * should start to respect the read timeout during read() and write timeout during * write() * Argument: 'uint32_t timeout_ms' * * Examples: * - Set a transmit() timeout for a UART putchar * - Set a recv() timeout on a socket */ DEV_STREAM__CONTROL_SET_READ_TIMEOUT, DEV_STREAM__CONTROL_SET_WRITE_TIMEOUT, /** @} */ /** * Flush the data stream completely before returning * Argument: None * * Examples * - Flush transmit FIFO of the UART */ DEV_STREAM__CONTROL_FLUSH_TX, /** * @{ * A device may require a mutex to be able to use it from multiple contexts * * For a device like an SPI, DEV_STREAM__CONTROL_ACCESS_REQUEST can perform multiple * things upon this request such as: * - Take mutex * - Device chip-select (CS) */ DEV_STREAM__CONTROL_ACCESS_REQUEST, ///< Argument: 'uint32_t timeout_ms' DEV_STREAM__CONTROL_ACCESS_RELEASE, ///< Argument: None /** @} */ /** * I2C bus requires the START and STOP conditions for a data transfer much * like the SPI chip-select and de-select. In particular, the I2C bus * read transaction involves, START, WRITE ... REPEAT-START, READ ... STOP */ DEV_STREAM__CONTROL_I2C_START, ///< Argument: 'uint32_t timeout_ms' that can be used to take an I2C mutex DEV_STREAM__CONTROL_I2C_REPEAT_START, ///< Argument: None DEV_STREAM__CONTROL_I2C_STOP, ///< Argument: None /** @} */ /** * A TCP listening socket uses this control type to perform the accept() * Argument: 'dev_stream_T *accepted_client' */ DEV_STREAM__CONTROL_TCP_ACCEPT, /** * @{ * A UDP socket is just a resource and it can be used to send the data packet * to any IP address. These control types "set" the destination IP and port. */ DEV_STREAM__CONTROL_UDP_SEND_TO_HOST, ///< Argument: 'uint16_t port' DEV_STREAM__CONTROL_UDP_SEND_TO_PORT, ///< Argument: 'const char * host' /** @} */ } dev_stream__control_E; /** * Flag types while opening a device * This provides distinctions such as: * - An input or output GPIO * - A transmit or a receive CAN mailbox */ typedef enum { DEV_STREAM__FLAGS_NONE = 0, DEV_STREAM__FLAGS_INPUT, DEV_STREAM__FLAGS_OUTPUT, DEV_STREAM__FLAGS_INPUT_AND_OUTPUT, } dev_stream__flags_E; /// Seek types used at dev_stream__lseek() typedef enum { DEV_STREAM__SEEK_NONE = 0, DEV_STREAM__SEEK_SET, ///< Set the seek to the absolute location DEV_STREAM__SEEK_CURRENT, ///< Set the seek to current location plus the offset DEV_STREAM__SEEK_END, ///< Set the seek to size of the file plus the offset } dev_stream__seek_E; /******************************************************************************* * * T Y P E D E F S * *******************************************************************************/ /** * The Device Stream is simply a unique descriptor * It is used as an identifier to figure out how the read() and write() should * re-direct the request for a particular descriptor (i.e.: UART vs. SPI) */ typedef int dev_stream_T; /******************************************************************************* * * P U B L I C F U N C T I O N S * *******************************************************************************/ /** * Open a device for read and/or write * @param descriptive_path The path of the device, such as "/dev/uart0" * @param flags @see dev_stream__flags_E; this is an int to conform with linux style open() * * @returns the descriptor (0 is a valid value), or -1 if there was an error acquiring the device */ dev_stream_T dev_stream__open(const char *descriptive_path, dev_stream__flags_E flags); /** * Closes a device * Typically, a persistent device such as UART doesn't need a closure * The use case for this is to close network sockets * * @returns 0 on success, or non-zero which is an error code */ int dev_stream__close(dev_stream_T device); /** * Reads num_elements with each element of size bytes_per_element * to the input_buffer from the device referenced by dev_stream_T * * @returns -1 upon an error * @returns the number of bytes read into the input_buffer * * @note the number of bytes read (and returned) may be fewer than requested (num_elements * bytes_per_element) */ int dev_stream__read(dev_stream_T device, void *input_buffer, size_t bytes_per_element, size_t num_elements); /** * Writes num_elements with each element of size bytes_per_element * from the output_buffer to the device referenced by dev_stream_T * * @returns -1 upon an error * @returns 0 if the device was not ready to be written * @returns the number of bytes written to the device */ int dev_stream__write(dev_stream_T device, const void *output_buffer, size_t bytes_per_element, size_t num_elements); /** * Perform a special operation on the descriptor * @param request_type @see dev_stream__control_E * @param argument Depends on the value of request_type; @see dev_stream__control_E * * @returns zero upon success, or non-zero error code upon failure */ int dev_stream__control(dev_stream_T device, dev_stream__control_E request_type, void *argument); /** * Perform a seek operation on the device (file seek) for the next read() or write() * The meaning of this API depends on the device, but is ideally suited for * an address based device, such as a file, or flash memory device * * @returns -1 upon an error * @returns the resulting offset location as measured in bytes from the beginning of the file. */ int dev_stream__lseek(dev_stream_T device, int offset, dev_stream__seek_E whence); /** * @returns true if the device descriptor is valid * * Typically, this should return true if the device descriptor is >= 0 * Sometimes, value of zero may be considered invalid */ bool dev_stream__is_valid(dev_stream_T device);

Note about copying sockets:

The dev_stream_t device descriptor essentially acts as the socket number. It should NEVER be copied. Copying this is similar to copying a pointer, leaving it at the mercy of future developers. The core issue is that if one copy is destructed (ie: connection closed) then the second copy should also be destructed. This may be somewhat possible in C++ and other higher-level languages but this is especially hard to enforce in C.

Use Cases

The stream abstraction is designed to solve various different use cases of common micro-controller peripherals.

  • UART

    • Write output

    • Wait for input (with timeout)

  • GPIO

    • Read and Write

    • Configure a complex GPIO

  • SPI

    • Transmit only

    • Receive only

    • Data transfer with a mutex

  • Flash Memory

    • Read

    • Write (to a specific location)

  • I2C

    • Write a byte stream to an addressed I2C device

    • Read a byte stream

  • CAN

    • Write a dedicated mailbox

    • Poll for a message from a receive mailbox

    • Read a message from a mailbox with a timeout

  • Sockets

    • Client socket to connect to a server

    • Server (listening) socket

    • UDP receive

    • UDP transmit

UART

We do not implement polled and queue configuration on the high level code. Instead, we leave it up to the Device Stream Glue to provide the desired behavior. For example, in the code snippet below, we can assume that the UART0 is opened with the RTOS backed queue to send and receive data. Initially when the device is opened, the timeout value should be infinity, but after the timeouts are set usingĀ dev_stream__control_EĀ enumeration then theĀ dev_stream__write()Ā andĀ dev_stream__read()Ā should use the configured timeouts.

GPIO

The GPIO can be configured as input, output, or input/output through theĀ dev_stream__flags_E, and from thereafterĀ dev_stream__read()Ā orĀ dev_stream__write()Ā can be used to access or control it. This abstraction has a very small overhead, in general if an application needs faster timing requirements than this supports, a different module should be used.

SPI

The SPI peripheral must send data out in order to receive the data. The transmit case is simple because theĀ write()Ā would simply discard the input data. TheĀ read()Ā case assumes that the Device Stream Glue sendsĀ 0xFFĀ for each byte that shall be received.

SPI Chip-Select

Often times, SPI devices may implement a transaction such that there are bytes written to the bus, and then bytes can be read back, and this "transaction" is a single chip-select data transfer.

SPI with Mutex

End-user code that utilizes SPI may implement mutex to guard the SPI resource from being used from multiple contexts. To implement portability with this caveat in mind, there is aĀ controlĀ type with the intent of taking and giving back a mutex. The Device Stream Glue shall be responsible to create the mutex if necessary.

Thoughts on bi-directional SPI

Typically this peripheral writes data and reads back the data onto the same source. Fundamentally, this peripheral is designed such that a shift register latches an output bit, and as it shifts the one bit that was output, it reads the new bit from the other side. When 8-bits have been shifted-out, 8 new bits would have been shifted-in.

TheĀ write()Ā function sends data out, although we have not yet implemented storing the received data that was input while sending the data out of the SPI peripheral. The plan of record is to add aĀ controlĀ type to perform something likeĀ DEV_STREAM__CONTROL_SET_READ_BACK_PTRĀ that can be used during the nextĀ dev_stream__write()Ā but since there has not been a use-case for this (so far), it has not taken inception yet. We are following the Agile orĀ YAGNIĀ principle here.

Flash Memory

For a generic flash memory driver, such as on-board flash driver, the usual intent is to write blocks of data at a specific location. Remember that flash memory can only be written if it was erased. With this in mind, we can design with the following key points:

  • dev_stream__open()Ā can not only "open" but erase the memory block

    • If the type wasĀ DEV_STREAM__MODE_INPUTĀ then it can be read-only and erase is not necessary

  • Once the flash memory block is opened, it can be written by seeking to an appropriate address

    • Address bounds can be checked withinĀ dev_stream__write()

I2C

Below is an example where we writeĀ 0x02Ā andĀ 0x03Ā to an I2C device with addressĀ 0xF0. Typically, this means that you want to write the registerĀ 0x02Ā with the value ofĀ 0x03Ā (a typical I2C transaction).

I2C Read

The read transaction is a little tricky with the I2C bus because you have to send bytes, and then receive bytes in a single transaction. The sample code below assumes the the I2C bus has an underlining driver that can send and receive bytes independently.

CAN I/O

For CAN I/O, we can implement various different descriptors that map to the CAN mailboxes. TheĀ flagsĀ argument forĀ dev_stream__open()Ā is used to identify if the descriptor is going to be used for eitherĀ readingĀ orĀ writingĀ a CAN HW mailbox.

Read All Messages

Read Specific Messages

CAN drivers typically implement aĀ maskĀ andĀ message idĀ that can be used to setup filters to receive particular messages. The Stream Device does not target very specific behavior for such a case, and it can be implied that there is an intent to receive filtered data only.

Write Messages

The descriptor name can identify the purpose for a CAN mailbox transmission. For example, if Sibros performs aĀ dev_stream__open()Ā withĀ /dev/can0/ecu_infoĀ then the Device Stream Glue can trigger on this name and map the descriptor to a dedicated transmission mailbox. Therefore, the code sitting above the Sibros Device abstraction is agnostic to a particular CAN mailbox and it is up to the Device Stream Glue to link the descriptor to their choice of actual hardware CAN mailbox.

It is also up to the Device Stream Glue to ensure that theĀ write()Ā API blocks forever until the underlining driver is ready to write the Hardware CAN mailbox. If the user sets the timeout on theĀ CAN_transmitĀ descriptor, then the Device Stream Glue can deposit the data into an RTOS queue with the configured timeout.

Network I/O

The network I/O is a more complex type of stream. There needs to be a consistent method of encoding and decoding network paths and we designed a code module to handle this. Basically, the following type of encoded paths allow you toĀ openĀ a stream while identifying what type of stream we really desire:

  • /net/udp/hostname/1234

  • /net/tcp/server/localhost/1234

  • /net/tcp/client/localhost/1234

TCP Client

The intention of the TCP client is to connect to another peer. The Device Stream Glue can default the connection timeout to a sane value, such as 3 seconds before giving up and returningĀ NULLĀ as a response fromĀ dev_stream__open().

TCP Client with Send & Receive Timeouts

TCP Server

A listening socket doesn't performĀ read()Ā orĀ write()and thus we need a special function to perform the equivalent of anĀ accept()Ā function which we implement by using aĀ control()Ā function.

UDP Receive

The one limitation to the code below is that a UDP socket can receive data from anyone on the network (it is connection-less), and theĀ dev_stream__read()Ā does not identify where the data came from. For the time being, there is no proposal to address this limitation, and can be designed upon request.

UDP Transmission

The UDP transmission is a little tricky because a UDP socket may be used to send data to anyone because the UDP socket is not associated with just one peer, and this socket is just a resource that can be used to send data to anyone.

Device Stream Dispatcher

You do not need to implement a lot of boiler plate code to glue your hardware platform to theĀ Device Stream. There is aĀ dev_stream_dispatcher.hĀ that can perform the heavy lifting and allow you to focus on defining I/O for specific devices. This section provides starter code to glue Sibros Device with your hardware. There may beĀ control()Ā types that are not relevant for certain devices, and thus the implementation can be omitted.

TheĀ Device Stream DispatcherĀ provides the implementation of theĀ dev_stream.hĀ for you, and you only should care about the registered callbacks. In addition, it features the following:

  • Match theĀ dev_stream__flags_EĀ duringĀ dev_stream__open()

  • Protection against multipleĀ dev_stream__open()

  • Not invokingĀ open_fptr(),Ā write_fptr()Ā etc. if they are NULL

  • close()Ā will invoke yourĀ close_fptr()Ā and free the descriptor for re-use

  • dev_stream__open()Ā takes care of matching the descriptor name and supports

    • Partial matching to allow for socket API behavior using IP lib (LWIP)

Note that each dispatcher you register, such asĀ CANĀ orĀ LWIPĀ can have their own descriptor IDs and they may even overlap! The dispatcher will take care of mapping the right IDs to each of the registered dispatchers. This is why theĀ d0Ā andĀ d1Ā can both exist at different dispatchers.

Platform Dispatchers

Here is the sample code that registers yourĀ CANĀ andĀ SPIĀ Device stream handlers. Contact us to get the implementation of theĀ dev_stream_dispatcher.h

End-to-End Example

Refer to thisĀ link.

SIBROS TECHNOLOGIES, INC. CONFIDENTIAL