Using libevl
What is an “EVL application process”?
An EVL application process is composed of one or more EVL threads, running along with any number of regular POSIX threads.
-
an EVL thread is initially a plain regular POSIX thread spawned by a call to pthread_create(3) or the
main()context which has issued the evl_attach_self() system call. This service binds the caller to the EVL core, which enables it to invoke the real-time, ultra-low latency services the latter delivers. -
once attached, such thread may call EVL core services, until it detaches from the core by a call to evl_detach_self(), or exits, whichever comes first.
-
whenever an EVL thread has to meet real-time requirements, it must rely on the services provided by
libevlexclusively. If such a thread invokes a common C library service while in the middle of a time-critical code, the EVL core does keep the system safe by transparently demoting the caller to in-band context. However, the calling thread would loose any real-time guarantee in the process, meaning that unwanted jitter would happen.
To sum up, the lifetime of an EVL application usually looks like this:
-
When a new process initializes, a regular thread - often the
main()one - invokes routines from the C library in order to get common resources, like opening files, allocating memory buffers and so on. At some point later on, this thread calls evl_attach_self() in order to bind itself to the EVL core, which in turn allows it to create other EVL objects (e.g. evl_create_mutex(), evl_create_event(), evl_create_xbuf()). If the application needs more EVL threads, it simply spawns additional POSIX threads using pthread_create(3), then ensures those threads bind themselves to the core with a call to evl_attach_self(). -
Each EVL thread runs its time-critical work loop, only calling EVL services which operate from the out-of-band context, therefore guaranteeing bounded, ultra-low latency. The pivotal EVL service from such loop has to be a blocking call, waiting for the next real-time event to process. For instance, such call could be evl_wait_flags(), evl_get_sem(), evl_poll(), oob_read() and so on.
-
Eventually, EVL threads may call common C library services in order to cleanup/unwind the application context when their time-critical loop is over and time has come to exit.
This page is an index of all EVL system calls available to applications, which should help you finding out which call is legit from which context. In order to use the ultra-low latency EVL services, you need to link your application code against the libevl library which provides the EVL system call wrappers.
Is libevl a replacement for my favourite C library? (glibc, musl, uClibc etc.)
NO, not even remotely. This is a drop-in complement to the common C library and NPTL support you may be using, which enables your thread(s) of choice to be scheduled with ultra-low latency guarantee by the EVL core. As it should be clear now from the above section, you may - and actually have to - use a combination of these libraries into a single application, but you must do this in a way that ensures your time-critical code only relies on either:
-
libevl’s well-defined set of low-latency services which operate from the out-of-band context. -
a (very) small subset of the common C library which is known not to depend on regular in-band kernel services. In other words, only routines which do not issue common Linux system calls directly or indirectly are fine in a time-critical execution context. For instance, routines from the string(3) section may be fine in a time-critical code, like strcpy(3), memcpy(3) and friends. At the opposite, any routine which may directly or indirectly invoke malloc(3) must be banned from your time-critical code, which includes stdio(3) routines, or C++ default constructors which rely on the standard memory allocator.
Outside of those time-critical sections which require the EVL core to guarantee ultra-low latency scheduling for your application threads, your code may happily call whatever service from whatever C library.
What about multi-process applications?
libevl does not impose any policy regarding how you might want to
organize your application over multiple processes. However, the design
and implementation of the interface to the EVL core makes sharing EVL
resources between processes a fairly simple task. EVL elements can be
made visible as common devices, such as threads, mutexes, events, semaphores. Therefore, every element
you may want to share can be exported to the device file system, based
on a visibility attribute
mentioned at creation time.
In addition, EVL provides a couple of additional features which come in handy for sharing data between processes:
-
a general memory-sharing mechanism based on the file proxy, which is used as an anchor point for memory-mappable devices.
-
the Observable element which gives your application a built-in support for implementing the observer design pattern among one or more processes transparently.
How to build an EVL-based application?
This is plain simple: in short, you just need to point your compiler
at the installation root of the EVL header files (e.g
-I$prefix/include), then link the executable against libevl.so
(e.g. -L$prefix/lib -levl). There is no other dependency beyond the
common one on the POSIX native threading library. In other words, an
EVL-based application is not more than a common application which may
talk to the EVL core via the libevl API.
As a result, integration of this build process with any build system
is a no-brainer. Speaking of which, since libevl r29, Xenomai 4
uses the meson build system to generate its
own artefacts. You may want to consider meson if you are looking for
an elegant, well-documented and lightweight build system which makes
things easy for you. Typically, a basic meson.build file which would
define the rules for building a simple application foo composed of a
single file foo.c could look like this:
project('a_foo_system', [ 'c' ], version : '0.0.0')
pthread_dep = dependency('threads')
libevl_dep = dependency('evl', method : 'pkg-config')
executable('foo',
'foo.c',
install: true,
dependencies : [ libevl_dep, pthread_dep ],
)
Once installed, libevl comes with pkgconfig meta-data, so
determining which include directories and libraries should be used to
build an EVL-based application amounts to asking the pkgconfig
utility to retrieve them. meson has built-in support for this.
Visibility: public and private elements
As hinted earlier, EVL elements created by the user API can be either
publically visible to other processes, or private to the process which
creates them. This is a choice you make at creation time, by passing
the proper visibility attribute to any of the evl_create_*() system
calls, either EVL_CLONE_PUBLIC or EVL_CLONE_PRIVATE.
A public element is represented in the /dev/evl hierarchy by a device file, which is visible to any process. Once a file descriptor is available from opening such file, it can be used to send requests to the element it refers to.
Conversely, a private element has no presence in the /dev/evl
hierarchy. Only the process which created such element receives a file
descriptor referring to it, directly from the creation call.
The /dev/evl device file hierarchy
Because of its everything-is-a-file mantra, EVL exports a
number of device files in the /dev/evl hierarchy, which lives in the
DEVTMPFS file system. Each device file either represents an active
public element, or a special
command device used internally by libevl.
In opening a public element device, an application receives a file descriptor which can be used to submit requests to the underlying element. For instance, the scheduling parameters of a thread running in process A could be changed by a thread running in process B by a call to evl_set_schedattr() using the proper file descriptor.
Element device files are organized in directories, one for each element class: clock, monitor, proxy, thread, cross-buffer and observable; general command devices appear at the top level, such as control, poll and trace:
~ # cd /dev/evl
/dev/evl # ls -l
total 0
drwxr-xr-x 2 root root 80 Jan 1 1970 clock
crw-rw---- 1 root root 246, 0 Jan 1 1970 control
drwxr-xr-x 2 root root 60 Apr 18 17:38 monitor
drwxr-xr-x 2 root root 60 Apr 18 17:38 observable
crw-rw---- 1 root root 246, 3 Jan 1 1970 poll
drwxr-xr-x 2 root root 60 Apr 18 17:38 proxy
drwxr-xr-x 2 root root 60 Apr 18 17:38 thread
crw-rw---- 1 root root 246, 6 Jan 1 1970 trace
drwxr-xr-x 2 root root 60 Apr 18 17:38 xbuf
Inside each class directory, the live public elements of that class are visible, in addition to the special clone command device. For the curious, the role of this special device is documented in the under-the-hood section.
/dev/evl/thread # ls -l
total 0
crw-rw---- 1 root root 246, 1 Jan 1 1970 clone
crw-rw---- 1 root root 244, 0 Apr 19 10:45 timer-responder:2562
Managing access permissions to EVL device files
In some situations, you may want to restrict access to EVL devices
files present in the /dev/evl
file hierarchy to a particular user or group of users. Because a
kernel device object is associated to each live EVL element in the
system, you can attach rules to UDEV events generated for public EVL
elements or special command devices appearing in the /dev/evl file hierarchy, in order to set up
their ownership and access permissions at creation time.
First of all, you need to set proper ownership to the
/dev/evl/control device, which is accessed by libevl for
requesting basic services to the core, such as attaching the calling
process to it.
For public elements which come
and go dynamically, EVL enforces a simple rule internally to set the
initial user and group ownership of any element device file, which is
to inherit it from the clone device file of the class it belongs
to. For instance, if you set the ownership of the
/dev/evl/thread/clone device file via some UDEV rule to
square.wheel, all public EVL threads will belong at creation time to
user square, group wheel.
Element names
Every evl_create_*() call which creates a new element, along with
evl_attach_thread() accepts a
printf-like
format string to generate the element name. A common way of generating
unique names is to include the calling process’s pid somewhere into
the format string, so that you may start multiple instances of the
same application without running into naming conflicts. The
requirement for a unique name does not depend on the visibility
attribute: distinct elements must have distinct names, regardless of
their visibility. For instance:
#include <unistd.h>
#include <evl/thread.h>
ret = evl_attach_self("a-private-thread:%d", getpid());
~# ls -l /dev/evl/thread
total 0
crw-rw---- 1 root root 248, 1 Apr 17 11:59 clone
The generated name is used to create a /sysfs attribute directory
exporting the state information about the element. For public
elements, a device file is
created with the same name in the /dev/evl hierarchy, for accessing the element via the
open(2) system
call. Therefore, a name must contain only valid characters in the
context of a file name.
As a shorthand, libevl forces in the EVL_CLONE_PUBLIC visibility
attribute whenever the element
name passed to the system call starts with a slash ‘/’ character, in
which case this leading character will be skipped to form the actual
element name:
#include <unistd.h>
#include <evl/thread.h>
ret = evl_attach_self("/a-publically-visible-thread:%d", getpid());
~# ls -l /dev/evl/thread
total 0
crw-rw---- 1 root root 248, 1 Apr 17 11:59 clone
crw-rw---- 1 root root 246, 0 Apr 17 11:59 a-publically-visible-thread
Note that when found, such shorthand overrides the EVL_CLONE_PRIVATE
visibility attribute which might have been mentioned in the creation
flags for the same call. The slash character is invalid in any other
part of the element name, although it would be silently remapped to a
placeholder for private elements without leading to an API error.
