StuBS
StuBS Development Environment

Table of Contents

tl;dr:

  • SSH and X2Go are available on our Linux computersauf die Pool-Rechner sind verfügbar. Zu einer Remote-Abgabe treffen wir uns auf l00p34. 
      $ ssh ccd5802@l00p34.rz.tu-harburg.de # Computer 00, Pool 3c
  $ source /opt/OO-MPStuBS/env.sh       # Setup PATH for qemu/gcc
  • Shared tmux session 
      $ tmux -S /tmp/my-shared-tmux new                       # tmux session erstellen
  $ setfacl -m "u:xxd1234:rw" /tmp/my-shared-tmux         # xxd1234 hat zugriff
  $ tmux -S /tmp/my-shared-tmux attach                    # auf die Session verbinden (von beiden auszuführen)
  • Setup development environment on your on Linux computer (or WSL2) 
      $ apt install nasm qemu qemu-system-x86 qemu-kvm build-essential binutils gcc-multilib g++ g++-multilib
    • QEMU (qemu, qemu-system-x86, qemu-kvm)
    • GCC (build-essentials, binutils, gcc-multilibs, g++)
    • nasm

Introduction

For working on the practical exercises (= development of OOStuBS/MPStuBS/StuBSmI), all components are installed on the Linux pools of TUHH. The machines are reachable via SSH and X2Go and further setup instructions can be found in this document. Of course you can also work on the exercises on your own computer, but you will need a modern Linux installation or setup your Windows with Windows Subsystem for Linux. If we talk about your solution remotely, you have to setup a shared TMUX session on the l00p35 machine.

Attention
Those who wish to install the software at home are, of course, fully responsible for any problems that may occur. Failed installations are also not accepted an excuse for late solution deliveries.

As OS development is quite error prone, it is essential that you test your solution thouroughly, before you ask us to accept the solution. Our tool of choice for testing is the full-system emulator qemu (with and without kvm acceleration). The exercise acceptance will always be tested with qemu and on the physical machine in the pool room. Therfore, you have to test you solution with both. The pool is the reference plattform.

Using the Linux pool remotely

You can remotely log into the Linux pool with SSH. Alternatively, you can use X2Go to get a whole graphical desktop enviroment.

SSH

Important: The SSH login only forms from the TUHH network. Therefore, you have to use the TUHH VPN or the SSH Jumphost to log in from home.

As we have to do some load balancing when working remotely, you have to do some suffling for selecting a computer. For this, you take your group number modulo 12 and log into the machine: 

$ ssh <your_login_name>@l<computer number>p34.rz.tu-harburg.de

So, for the user xxd1234 and group 17, the login would be on computer 5: 

$ ssh xxd1234@l05p34.rz.tu-harburg.de

At home, you can define a shortcut for this in your SSH configuration (~/.ssh/config), which allows you to do ssh pool: 

Host pool
     User xxd1234
     Hostname l05p34.rz.tu-harburg.de
     GSSAPIAuthentication yes
     GSSAPIDelegateCredentials yes
     ForwardAgent yes

As the computer pools have installed an antique Redhat Linux, we had to compile a few tools (GCC 10, Nasm 2.15, etc.) by our own. Therefore, you have to include those tools in your PATH by sourcing our enviroment file: 

$ source /opt/OO-MPStuBS/env.sh

You can, and you should include this in your ~/.bashrc. Thereby, this happens automatically when you login.

Using SSH-Keys and Gitlab

For collaborating, we use the TUHH Gitlab, where we create group repositories for you. To access this repository, you have to create an SSH keypair and register the public key with your login. With the following command you can create a private-public keypair with ssh-keygen 

$ ssh-keygen -t rsa -b 4096

By default the key is stored as ~/.ssh/id_rsa in your home in the hidden directory .ssh and the key is automatically offered to the remote server when you try to login into the remote machine. If you give it another name, you have to configure it in your ssh configuration (see ssh_config(1)).

If you have created the SSH key on you local machine, you can use an ssh-agent to forward the key over SSH so that you can use git commands on the pool machine. For this you have to start an ssh-agent if your desktop enviroment does not already start it and add the key: 

$ eval $(ssh-agent)   # Start ssh-agent
$ ssh-add             # Add .ssh/id_rsa to the ssh-agent
$ ssh-add .ssh/gitlab # Add .ssh/gitlab to the ssh-agent
$ ssh-add -l          # List all keys the agent knows about

The created key has to be registered with our Gitlab: User Settings > SSH Keys. Afterwards, you can create all git commands without entering a passworts (with ssh-agent also on the remote machine).

Shared tmux session

For working together you can use tmux to share a TTY session with another user on the same local machine. The two users can then first see and edit the same terminal. Furthermore, tmux works session-based, i.e. one shares not only one terminal, but a whole session, specifically any number of terminals and their state and layout.

A new tmux session is created and attached to a Unix doman socket with (-S / -L): 

$ tmux -S /tmp/my-shared-tmux new

If no other tmux session listens on that file, a new session is created. Please note, that you should change the socket name my-shared-tmux to something else, like group8.

Other uses can now attach on this socket with the attach command: 

$ tmux -S /tmp/my-shared-tmux attach

When tmux is connected, you can detach from the session with Ctrl-b d.

In order to give access to another user, he must have the access to the socket file, which can be done in a fine-grained manner with ACLs (abc9876 is the username of your partner): 

$ setfacl -m "u:abc9876:rw" /tmp/my-shared-tmux

Please be aware, that giving someone access to your tmux session allows him to execute commands on your behalf. So do not use the usual Linux permissions and 777 or something like that.

For more details on tmux, please consult the official tmux documentation or tmux' excellent manpage.

Required Software

For compiling, our Makefiles use g++, for assembling the startup code and the hardware-related subroutines we use the Netwide Assembler (nasm). The x86 emulator QEMU is suitable for preliminary testing and, by a built-in GDB stub, it is also very useful in combination with the GNU Debugger. In the Linux pools have already set up the appropriate environment; if you want to do the whole thing at home, you have to set up the mentioned software accordingly. If you have any problems, feel free to ask us.

*StuBS-Template

  • For BST, we will provide you with a single template that you have to extend over the course of the semester. We will push the template in your group repository.
  • The compilation of *StuBS is done by calling make in the solution directory. All .cc and .asm files in the solution directory are then compiled with the appropriate tools (compiler and assembler respectively) and linked together as a bootable system image. Afterwards the commands make {kvm,qemu}{,-gdb,-ddd}{,-noopt} are available for testing and debugging (more about this in the next section).
  • You can get an overview of the possible make targets with make help.

Testen und Debuggen von OOStuBS (Make-Targets)

  • The simplest and fastest way to test your implmenentation can be done with make qemu-curses, which builds your system image and starts it in QEMU: 
    ccd5802@l00p34:~/oostubs$ make qemu-curses
    By default, we start QEMU to emulate a system with four processors. For the development of OOStuBS/StuBSmI this should not bother you much, because the additional CPUs are simply "left aside" without any further action. For the MPStuBS tinkerers: through the KVM mode your system is really run in parallel on several cores. Therefore, we are relatively close to the test on the real hardware in terms of race conditions and faulty synchronization.
Attention
To quit a QEMU that was started in Curses mode, you have to switch to the monitor view (Alt-2) and type quit. To return from the monitor view to the CGA view, press Alt-1. By the way, on another virtual QEMU window (Alt-3) the serial port is displayed.
  • To ease debugging, you can qemu in the full emulation mode (qemu-* targets) where all cores are emulated pseudo-paralell. With the kvm-* targets you get full parallel execution.

  • If you are stuck with simple printf debugging, you can use QEMU's built-in GDB stub to connect to the emulation with a debugger (gdb). This way you can easily execute your operating system code step by step to find out the reason of any crashes or unwanted behavior. To make it easier for you we provide several targets to start qemu and the gdb. With this command, you start a QEMU (in windowed) mode and gdb in the same console. 

      dietrich@obelix:~/oostubs$ make gdb

    If you work remotely, or if you want to see the output of QEMU, you have to open two terminals (one for QEMU, one for GDB) and use the -gdb targets: 

    dietrich@obelix:~/oostubs$ make kvm-gdb     # Terminal 1: Start QEMU in KVM mode and connect via gdb
dietrich@obelix:~/oostubs$ make connect-gdb # Terminal 2: Start gdb and connect to QEMU.

    A short GDB reference can be found here here. For detailed information, how to a specific GDB command works, you can use the built-in help function 

    (gdb) help <command>

    Hint: As QEMU pauses the operating system when the gdb connects, you have to resume executeion with continue instead of starting the "program" with run.

    For a faster overview about registers and stacks you can use the excellent gdbinit configuration by storing it in your home under the name .gdbinit

  • The easiest way to work from home is via an SSH connection. However, you cannot start a window on this connection by itself. But QEMU is able to map the CGA screen in text mode completely with NCurses to a terminal. We have therefore set up targets for you, which start QEMU/KVM with curses and work from SSH. 
      dietrich@obelix:~/oostubs$ make qemu-curses
  dietrich@obelix:~/oostubs$ make kvm-curses
  • By default, the build system compiles everything with -O3. If you suspect that you have some problem with undefined behavior, you can append an -noopt to the make target to build everything without optimizations. For example, the following command starts an unoptimized StBS in QEMU in the curses mode: 
    dietrich@obelix:~/oostubs$ make qemu-curses-noopt
  • An overview over all make targets is shown with: 
     dietrich@obelix:~/oostubs$ make help
    Here is a collection of very useful gdb commands for OS development:
command Help
bt Show a function call back trace
info threads Show an overview about all four CPUs (CPUs are modeled as "threads"
thread 3 Switch to CPU 3
disassemble Disassemble the current function
info registers Show all CPU registers (gdb style)
display /i *$pc Display the next executed instruction on each prompt
x/10a $esp Display the 10 top-most words on the stack an interpret them as **a**dresses
monitor info registers Instruct QEMU to prin all registers (more detailed version)
monitor info mem Show the current memory mapping!
monitor system_reset Warm-Reboot of the emulated system