Quick introduction to the base library

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This project comes with a custom bundled C library and does not rely on

an external one like most userspace C projects do. It builds more like

the Linux kernel than like say GNU coreutils or even busybox, using

a freestanding compiler. All library functions used in the project are

provided by the project.

A major feature of the base library is the way it deals with syscalls:

if((fd = sys_open(name, flags)) < 0)

/* fd is negative error code, e.g. -ENOENT */

else

/* fd is a valid descriptor */

Syscall names are prefixed with sys_, unlike in POSIX, so there is a clear

distinction between syscalls and library functions. Syscall code is inlined,

that sys_open above is not a function call. There is no global errno, the

calls return error code on failure.

The library does (almost) no wrapping for syscalls. Whatever the kernel

interface and semantics for the call is, it's the same in the userspace.

For instance, sys_ppoll does update its timestamp argument.

What is struct top* ctx?

~~~~~~~~~~~~~~~~~~~~~~~~

Global variables declared the usual way force at least a page to be allocated

for .bss segment, even though the total size of the variables may be way less

than a whole page. To avoid wasting a whole page, the variables get moved to

the stack. There's always at least one dirty page of stack so no loss happens.

Busybox has `struct G` which is exactly the same thing.

POSIX libc applications cannot drop .bss completely because of errno (and other

accidentally linked global stuff). In this project, it is possible, and quite

a lot of smaller tools have no .bss/.data segments.

Unlike .bss/.data, the stuff in the stack cannot be addressed statically.

The workaround is to pass the pointer to every single function that uses

them:

struct top {

int foo;

};

void do_blah(struct top* ctx, ...)

{

ctx->foo++;

}

int main(...) {

struct top context = { ... }, *ctx = &context;

do_blah(ctx, ...);

...

}

This way, first-argument register holds the base address and all globals

are accesses via offsets to that address. The base address is not known

at link time but once set early in main(), it does not change.

Extended syscalls

~~~~~~~~~~~~~~~~~

There are several groups of "syscall extensions" in Linux when a newly

introduced syscall supersedes an older one, like so:

sys_open(fd, name, mode) = sys_openat(AT_FDCWD, fd, name, mode)

Pretty much al at-file syscalls operate like that.

In a lot of cases, the code using them looks better with the original calls.

The preferred approach taken in this project is define older syscalls

as macros over the newer ones. Even though there is NR_open, sys_open()

calls NR_openat with at=AT_FDCWD to emulate the behavior of the original

call.

Rationale: on the kernel side of things, using newer NR-s seems to be the

preferred approach, to the point they are willing to make older NR-s defines

conditional in the headers. There are no real downsides to doing this either,

except maybe for strace not spelling the exact syscalls used.

Prototype for main()

~~~~~~~~~~~~~~~~~~~~

The entry trampoline `_start.s` only supplies argc and argv to the C code:

int main(int argc, char** argv);

There is no envp there. If needed, it can be derived like this:

char** envp = argv + argc + 1;

It's rarely used and easy to do in C. No point in doing it in assembly.

Handling signals

~~~~~~~~~~~~~~~~

The way signal handlers work in Linux is weird. It seems to be designed around

some compatibility issues (POSIX sigaction perhaps?) and makes little sense out

of that context. Normally it is all heavily wrapped by a POSIX libc, and most

users aren't even aware of the issue. In this project, wrappers are much

thinner, so more of this stuff is visible.

Signal handlers are not really functions. They cannot return, instead they are

supposed to call sys_sigreturn() to restore process context and continue

with the regular program flow. Linux sys_sigaction however is designed to use

regular functions as signal handlers.

struct sigaction sa = {

.handler = sighandler,

.flags = SA_RESTORER | ...,

.restorer = sigrestorer

}

With this setup, sighandler is allowed to return, and the return address is

sigrestorer, which has to be a chunk of assembly invoking NR_sigreturn.

Pretty much none of this is ever relevant in the code that uses signals.

The library defines a macro, SIGHANDLER, to create this structure and fill

the .restorer field so that the .handler would work as expected.

# Note this is another case of extended syscalls, the actual syscalls used

# are NR_rt_sigaction and NR_rt_sigreturn respectively.

Memory allocation

~~~~~~~~~~~~~~~~~

There is no universal malloc/free in the base library. None of the library

functions rely on being able to allocate arbitrary amounts of memory either.

Memory management is always left to the application.

Whenever necessary, library functions expect to be provided with pre-allocated

memory space to work with.

The way applications from this project manage memory varies. Some have

constant memory footprint. Some use heap (via sys_brk) in data-stack mode,

without conventional free(). Some use sys_mmap for large buffers.

Formatted output

~~~~~~~~~~~~~~~~

There is no general-purpose printf in the library. The problem with printf

is that it's a runtime interpreter, which means the compiler does not know

which sequences have been used and has to compile support for all of them.

Even if that wasn't an issue, removing stuff from printf is not an easy task,

not something a compiler or linker can do. This all really doesn't bode well

with the whole small and static idea.

So instead, the library has a set of explicit formatting functions that work

like this:

char buf[N];

char* p = buf; /* current pointer, initially at the start of buf */

char* e = buf + sizeof(buf) - 1; /* end of buffer, 1 char to spare */

p = fmtstr(p, e, "Some number: ");

p = fmtint(p, e, 123);

*p = '\0'; /* terminate the string using the spare char */

All fmt* functions advance p, so that it always points to remaining space in

buf, while also ensuring p <= e.

The linker can then easily pick only the functions used, making sure there's

no dead formatting code in the executable, like it would be with printf. And

this scheme also make it very easy to write application-specific formatters,

something that's very difficult with printf.

# Why (p, e) tuple instead of a struct? It actually makes code simpler.

# The first argument and the return value share registers on all supported

# targets, so fmt* effectively mutate the first arg register in place.

#

# The second pointer, e, could be left in place as well across several fmt*

# calls, but the language is not expressive enough to describe it.

Formatting is always done into a buffer of fixed size, similar to snprintf.

There are no corresponding stream-printing functions similar to printf proper

or fprintf. Buffered output, whenever necessary, is always explicit.

Note there is a simple printf implementation defined in printf.h, but it is

only meant to be used for printf debugging. Check the comments there on its

limitations.

Error reporting

~~~~~~~~~~~~~~~

Almost all cases where errors are reported to the user in this project are

handled by the following two functions:

warn(msg, arg, ret);

fail(msg, arg, ret);

The first one just prints the message, the second one also does _exit(-1).

The message printed is always "tag: msg arg: err", but if msg or arg are NULL

or ret is 0 relevant parts are skipped, resulting in "tag: msg: err" etc.

This may sound extremely limiting compared to say BSDish err() family, but

actually turns out fine.

The last argument is the negative error from syscall:

if((ret = sys_foo(blah)) < 0)

fail("foo", blah, ret);

Error codes are reported as literally "ENOENT", "EINVAL" in cases where

convential libc would use "No such file or directory", "Invalid argument"

respectively. Any error code that has no corresponding string in the list

gets reported as a raw integer (e.g. -111), the user may look it up later

using `errno` tool.

This setup allows executables to link only the strings for the set of codes

that syscalls the application uses can actually return. Linux defines a lot

of error codes, but the vast majority of syscalls only return a handful.

Linking the rest in would result in dead data.

ERRTAG and ERRLIST macros define the static data needed for warn and fail,

namely the leading tag and the list of error strings. These should always

be in the same file with main().