Introduction

Welcome to ukoOS, a small operating system for RISC-V. The code can be found on GitHub.

UMN Kernel Object

The operating system is meant to be educational for the UMN Kernel Object club, so there are tutorials describing how to develop on it.

Kernel documentation

In addition to tutorials, the kernel itself is documented.

Linux Setup Guide

Install Visual Studio Code if you do not have it already. Note that you will need to follow the instructions for your Linux distribution

Install these packages:

git
podman-docker

If you are using Debian/Ubuntu:

sudo apt install git podman-docker

If you are using Fedora:

sudo dnf install git podman-docker

If you are using Arch:

sudo pacman -S git podman-docker

If you are using a distribution not listed here, install with your distribution's package manager.

Open Visual Studio Code, and navigate to the Extensions menu located at the bottom of the left hand side bar. Install the Dev Containers extension.

vscode extension

git clone ukoOS (git clone https://github.com/UMN-Kernel-Object/ukoos), open the folder in Visual Studio Code (File -> Open Folder). It should prompt you to reopen in Dev Container. If not, press Ctrl + Shift + P and type Reopen in Dev Container.

You are now in the ukoOS Dev Container. To verify this, run the below command and verify the line NAME="Alpine Linux" is present. cat /etc/os-release

When you have a change ready to be committed, you must sign off your commits.

How to sign off and commit changes (in VSCode):

Go to the "Source Control" tab in VSCode, and in the message box, write a description of what you've done. Press the 3 dots icon shown below, go down to the commit menu, and select "Commit (Signed Off)."

vscode extension

When the pop-up "Would you like to stage all your changes and commit them directly" pops up, click yes. To push the changes, click "Sync Changes."

How to sign off and commit changes (in the CLI):

Your commits should look something like this: git commit -s -m 'description of what you've done'

macOS Setup Guide

Install Docker Desktop. Download it, run the installer, and follow any installation prompts.

Install Visual Studio Code.

Open Visual Studio Code, and navigate to the Extensions menu located at the bottom of the left hand side bar. Install the Dev Containers extension.

vscode extension

Install git if you do not have it already. You can install it using Homebrew (brew install git), or download it from their website.

git clone ukoOS (git clone https://github.com/UMN-Kernel-Object/ukoos), open the folder in Visual Studio Code (File -> Open Folder). It should prompt you to reopen in Dev Container. If not, press Cmd + Shift + P and type Reopen in Dev Container.

You are now in the ukoOS Dev Container. To verify this, run the below command and verify the line NAME="Alpine Linux" is present. cat /etc/os-release

How to sign off and commit changes (in VSCode):

vscode extension

When the pop-up "Would you like to stage all your changes and commit them directly" pops up, click yes. To push the changes, click "Sync Changes."

How to sign off and commit changes (in the CLI):

Your commits should look something like this: git commit -s -m 'description of what you've done'

Windows 11 Setup Guide

We need to first install everything needed for running our environment.

First, install WSL. You need to verify Hardware Virtualization is enabled (note all pre-installed Windows 11 computers should come with Virtualization enabled). Go to this link to learn how to enable Hardware Virtualization. Once that is enabled, open a PowerShell window and run the following command:

wsl --install

Install git if you do not have it already. You can install it using Winget (winget install --id=Git.Git -e), or downloading from their website.

Install Docker Desktop. Download it, run the installer and follow any installation prompts.

Open Docker Desktop and verify these settings (you can skip the account page if you'd like):

Settings -> verify "Use the WSL 2 based engine" is checked.

docker

Settings -> Resources -> WSL integration -> verify "Enable integration with my default WSL distro" is enabled.

docker engine

Download and install Visual Studio Code.

Open Visual Studio Code, and navigate to the Extensions menu located at the bottom of the left hand side bar. Install these extensions:

vscode extension

git clone ukoOS (git clone https://github.com/UMN-Kernel-Object/ukoos), open the folder in Visual Studio Code, and follow any installation prompts that pop up. It should prompt you to reopen in Dev Container, If not, press Ctrl + Shift + P and type 'Reopen in Dev Container`.

You are now in the ukoOS Dev Container. To verify this, run the below command and verify the line NAME="Alpine Linux" is present. cat /etc/os-release Before you make any local changes, you must run git reset --hard. NOTE: IF YOU HAVE ANY LOCAL CHANGES, THIS COMMAND WILL DISCARD THEM. This allows you to run ./configure without issues.

How to sign off and commit changes (in VSCode):

vscode extension

When the pop-up "Would you like to stage all your changes and commit them directly" pops up, click yes. To push the changes, click "Sync Changes."

How to sign off and commit changes (in the CLI):

Your commits should look something like this: git commit -s -m 'description of what you've done'

Troubleshooting

Linux

If you are not using the dev container, and your distribution's tftp times out (error shown below), install busybox, and try again. Then, running busybox tftp should work. This is an issue we are having with Fedora's tftp package, and possibly more.

> tftp 192.168.2.91 -c put kernel.elf
Transfer timed out.

The error on the board/tftp server side:

uko-uboot# tftpsrv
Speed: 100, full duplex
Using ethernet@4070000 device
Listening for TFTP transfer on 169.254.96.84
Load address: 0x81800000
Loading: T T T T T T T T T T
Retry count exceeded; starting again

macOS

Currently no known issues.

Windows 11

If you open ukoOS in the dev container, and you get this error:

/workspaces/ukoos # ./configure
env: ‘bash\r’: No such file or directory
env: use -[v]S to pass options in shebang lines

run git reset --hard. NOTE THIS WILL ERASE ALL YOUR LOCAL CHANGES.

If you get the error shown below, you will need to launch Docker Desktop, then try connecting to the dev container again.

Docker daemon error

If Docker Desktop does not open, you need to open Task Manager and end the "Docker Desktop Backend" task, shown below. (To end a task, click on it, then click "End task" on the top right.)

Docker backend process

Dev Container

Currently no known issues.

GDB

GDB is a debugger that will allow us to debug the kernel under a virtual machine.

From the first terminal run qemu, but we need to add options to enable debugging

make qemu-debug

Which will be controlled by GDB once we start it here (from a new shell)

make gdb

Commands

Here are some useful commands, note that commands can be abbreviated so next can become n. There are many other commands so make sure to read the docs and help menus.

General Commands

Command	Meaning
q	quit gdb
kill	kill debugged process (in our case ukoOS)
info r	list register values
file file	load symbols from file
tui enable	enable tui (to split screen source and gdb)
help cmd	help for cmd

Running Commands

Command	Meaning
n	next line
s	step (like next line, but enters function calls)
c	continue until next breakpoint (or end)
si	step instruction
ni	next instruction

Breakpoints

Command	Meaning
b	add breakpoint at current point
b symbol	add breakpoint to symbol (symbol is a function name, etc.)
en n	enable breakpoint n
dis n	disable breakpoint n
info b	list breakpoints

Printing

The p command can do quite a lot:

Command	Meaning
p *addr	print value at address addr
p expr	print out some c-like expression, e.g. p struct->name
p/s *addr	print c-string at addr

Demo

These are from the first day docs.

The first terminal, running make qemu-debug will look like this

And the second, running make gdb

The `print()` and `format()` functions

The ukoOS kernel provides different printing functions than one may be used to from userspace C. They more closely resemble what's provided by Python or Rust, although they have some differences because they can't rely on runtime type information. (If we can get runtime type information in the future, it would absolutely make sense to use it here.)

These functions are:

char *format(const char *fmt, ...);
void print(const char *fmt, ...);

Both functions take the same arguments, the only difference is whether they print the string immediately or return it in a heap-allocated buffer.

TODO: print shouldn't print "immediately" either; it should print to a ringbuffer, so that it can be used in e.g. an interrupt handler.

The first argument is a format string. This string is composed of chunks of literal characters, format directives, and curly brace escapes. We can see all three in a print call like:

void foo(bool b, u32 n) {
    print("f({bool}) = {{ 1, 2, {u32:#010x} }}", b, n);
}

In this call, the string breaks down as follows:

                            f({bool}) = {{1, 2, {u32:#010x}}}
               literal "f(" ┴┘└────┤└──┤└┤└────┤└─────────┤└┤
format directive without arguments ┘   │ │     │          │ │
                        literal ") = " ┘ │     │          │ │
                             escaped '{' ┘     │          │ │
                             literal "1, 2, "  ┘          │ │
                          format directive with arguments ┘ │
                                                escaped '}' ┘

Literal chunks are printed as-is, and curly brace escapes print as the characters they're escaping. Format directives print content computed at print-time, typically content computed from the arguments to print or format.

Format directives are split into two parts; inside the curly braces, there's a type name and optional arguments, separated by a colon (:).

List of format directives

`i8`, `i16`, `i32`, `i64`, `isize`, `u8`, `u16`, `u32`, `u64`, `usize`

These directives expect the appropriate C type in the arguments to print or format.

These directives print numbers. They take a variety of arguments:

#: Prints the a marker for the base before the sign (0b, 0o, or 0x).
0: Left-pads the number with 0 (after the sign) instead of with (before the sign).
a number: Left-pads the number until it is at least this length.
b, o, x: Prints the number in binary, octal, or hex.

`paddr`, `uaddr`, `uptr`

These directives expect the appropriate C type in the arguments to print or format.

These directives print the address or the address part of the pointer. They act like usize, except they default to having arguments of #018x.

`bool`

This directive expects a C bool in the arguments to print or format.

It prints either true or false, corresponding to the value. This directive does not take any arguments.

`cstr`

This directive expects a C const char * in the arguments to print or format.

It prints it as a null-terminated string. This directive does not take any arguments.

`indent`

This directive expects a C usize in the arguments to print or format.

It prints that number of space characters ( ). This directive does not take any arguments.

`va`

This directive expects a C const char* and a C va_list in the arguments to print or format.

It prints the content that format would print if that format string and those arguments were passed to it. This directive does not take any arguments.

Threads and Harts

In ukoOS, there are three related concepts that are important to keep separate.

harts, or hardware threads. Colloquially, we might call these "cores" or "CPUs." This terminology comes from RISC-V, but the concept applies to any architecture.
kthreads, or kernel threads. The kernel manages these automatically, and will create and destroy them at various times.
uthreads, or user threads. These are the threads that userspace programmers talk about. They are created only in response to userspace syscalls.

Each of these notions of threads also has its own notion of "thread-locals." These are stored in various places.

Hart-locals are pointed to by the sscratch CSR, so we can get to them in trap handlers, regardless of whether the trap handler interrupted kernel-space or user-space execution.
Kernel and user thread-locals are pointed to by the tp register (x4).

Tutorials

Here there are a series of tutorials that will guide the reader through building and modifying ukoOS.

First Day

The First Day tutorial will start from nothing and end with fixing a bug in operating system. It will walk through installing the tools (using docker), downloading code, setting up and building the operating system, and finally running and testing.

First Day

Setting Up

Start by setting up a dev container for Linux, MacOS, Windows. You will need VSCode and Docker to use the provided devcontainer (in the git repo).

Troubleshooting

If there are any issues, check out the troubleshooting guide and ask us questions.

Checkout code

The kernel is hosted on GitHub so if you haven't already downloaded the code, here is how:

git clone https://github.com/UMN-Kernel-Object/ukoos.git
cd ukoos

Compiling

Once we have the code and tools, we can compile the code from the dev container

mkdir build
cd build
../configure
make

Running

Once the kernel is built, we can run the code inside a virtual machine (qemu)

make qemu

Debugging

From the first terminal run qemu, but we need to add options to enable debugging

make qemu-debug

Which will be controlled by GDB once we start it here (from a new shell)

make gdb

The first terminal, running make qemu-debug will look similar to the video below. However, it won't do anything until you run make gdb in the second window.

And the second, running make gdb

The gdb window will be how you will control the OS (with the controls below) but the output will show up in the first. There is a more detailed guide on using gdb, but here are the basics:

Command	Meaning
n	next line
s	step (like next line, but enters function calls)
c	continue until next breakpoint (or end)
b	add breakpoint at current point
b symbol	add breakpoint to symbol (symbol is a function name, etc.)
p variable	print the value of the C variable

Fixing a bug

Checking out the first day code

Switch to the tutorial code

git checkout tutorials/first-day

Please note, do not run the ./configure script from the tutorials/first-day branch, it will make the following commands not work.

and rebuild

cd build
make

Once you have the code and are able to run and debug, you'll notice that "Hello, World!" is misspelled! Try using the debugger to step through and find the line that prints this out incorrectly, then go to the file, edit it, and re-run.

ukoOS Docs