Escape

What is Escape?

Escape is a UNIX-like microkernel operating system on which I'm working since october 2008. It's implemented in ANSI C, C++ and a bit assembler. Except that I'm using the bootloader GRUB and the libraries that GCC provides (libgcc, libsupc++), I've developed the whole OS by myself.
Escape runs on x86, ECO32 and MMIX. ECO32 is a 32-bit big-endian RISC architecture, created by Hellwig Geisse at the University of Applied Sciences in Gießen for research and teaching purposes. MMIX is a 64-bit big-endian RISC architecture, developed by Donald Knuth as the successor of MIX, which is the abstract machine that is used in the famous bookseries The Art of Computer Programming. More precisely, Escape runs only on GIMMIX, a simulator for MMIX developed by myself in my master thesis (the differences are minimal, but currently required for Escape).

My goal is not to write an OS that can be used productivly some time (although I don't want to exclude that ;)), but the project is rather an experiment to see what works well and what doesn't and to learn much about operating systems, hardware and in particular the x86-architecture. Of course the OS is not finished yet and therefore not all concepts and implementations are stable.

If you have any questions, problems or feedback don't hesitate to contact me: Nils Asmussen

License

I publish the operating system as open-source under the GPL2.

Changelog

Version 0.4 (09/12/2011)

• Escape uses a gcc-cross-compiler now and has a complete tool-chain to build C- and C++-applications, statically or dynamically linked. I.e. the C++ apps can also use RTTI and exceptions. I'm using the libsupc++ of gcc for that. Additionally libgcc is used (static or shared version).
• I've implemented a dynamic linker that, although not complete, seems to support all relocation-types that are necessary to run "normal" C- and C++-applications.
• The virtual-memory-manager has been rewritten from scratch because the old concept was too simple and unflexible. Now, we have a region-based concept that works really nice. It supports copy-on-write, text-sharing, shared-libraries, shared-memory, mapping of physical mem (e.g. VESA-mem), demand-loading, swapping and thread-local-storage.
• Improved the shell-scripting-language. Syntax is nicer now and it has support for functions, arrays, array-to-string conversion and vice versa and some other things.
• The services/driver concept has been changed. Now, all are called drivers because in principle they're the same. They can specify which "standardized" messages (open, read,write,close) they support and add arbitrary other messages. The other messages are basically an alternative approach to the ioctl-stuff.
• Two additional architectures are supported: ECO32 and MMIX. Since both don't provide many devices yet to make use of, the main goal wasn't to support these architectures because of their cool features or something, but primarily to see what interface between the architecture-dependent and the -independent part of the kernel is required and works well. And in general, what difficulties occur when multiple architectures should be supported.
• SMP support has been added with a lot of subsequently changes.
• The GUI-library has been improved a lot. It does now support mouse-wheels, scrollpanes and layout-manager.
• Resizing and moving of windows does now work by showing a preview-rectangle during that operation instead of applying the operation immediatly. This way, its much more usable on slow hardware.
• New boot- and shutdown procedure
• CPU-speed is determined and CPU-time is measured instead of just cycles.
• Added users and groups (support in kernel and applications to manage them in userspace).
• The kernel uses a cache in front of the heap now. That means, it maintains a storage of fixed-size-objects for common object-sizes, so that one can allocate and free objects very quickly. The heap is only used as a fallback (if requested size and object-size differ too much). But of course, we're buying better performance with more memory-usage here.
• Addea a kernel-debugging-console. That means, a console that can be entered at any time (by pressing F12) and in which all important data-structures of the kernel can be displayed. Additionally, a small ls is supported and the opportunity to read a file.
• Added a new event-system with which one can wait for one or multiple events and optionally for a specific object. This is usable in kernel and (limited) in userspace as well. For example one can until data is readable from a driver by specifying EV_DATA_READABLE as event and the file-descriptor for that driver as the object.
• Parallelized the fs-driver, i.e. a threadpool is used and one thread waits for a request and passes that request to a thread in the pool that is idling. Well, its cool that it works now (it has definitely improved the code of fs), but it turned out that the performance is worse than before :D Even when using many threads that read stuff from the filesystem concurrently and even with multiple CPUs. It seems that the overhead for locking and context-switching is so high that the amount of work every thread does is simply to small to make the whole system better than a single-threaded version. Although there is of course still room for improvements; so we'll see.
• Improved the ATA- and ATAPI-driver a lot. Now it supports DMA and works on all emulators and most of my real machines. Unfortunatly, there are still problems with DMA and other things on real hardware :/

Version 0.3 (03/04/2010)

• The kernel logs now to /system/log
• The GUI supports 16, 24 and 32 bit colordepth (and arbitrary resolutions)
• GUI-Windows are movable out of the screen (left, right, bottom) and can be resized
• Added a nicer mouse-cursor and font
• The VM86-task is used to query the BIOS for the available VESA-modes and setting a mode => works on all emulators and real machines :)
• Added a VM86-task
• Added a stupid game (raptor) ^^
• German umlaute etc. are usable now
• Added a keymap-driver that acts on top of the keyboard-driver
• The ATAPI-drive used for booting is automatically detected
• Built a simple CLI-editor
• Added a small shell-scripting-language (the syntax isn't perfect yet...I'll try to improve it)
• Added a VESA-textmode (hardcoded atm, no mode-detection)
• Changed pipe-implementation so that pipes are "unlimited" now
• Added a PCI-detection service
• Threads can get arguments now
• Added the ability to retrieve the exit-state of a process
• Added a power-manager for reset and shutdown (shutdown isn't implemented yet)
• Added a ISO9660-fs. Means we can boot from cd and hdd now :)
• Its possible to read from ATAPI now

Version 0.2 (09/16/2009)

• Added the user-apps link, unlink, mkdir, mount and umount
• Mounts are supported now
• Added write-support in the ext2-driver
• Rewrote the escape-code-handling
• Added an idle-thread that uses hlt
• Changed the filesystem-layout: "/foo/bar" instead of "foo:/bar". The URL-like scheme introduced to many special cases
• Added /drivers/zero, null and random
• Rewrote service-system: No "single-pipe"-services anymore, special system-calls for sending and receiving messages, a fixed driver-interface, ...
• Added CPU-detection
• Added multi-threading-support
• Started to build a GUI, based on a C++-GUI-library. Some controls are available and a "GUI-shell" that has the same functionality as the text-shell.
• Added Text-sharing

Version 0.1 (04/10/2009)

• Initial release

General structure

Escape consists of a kernel, drivers, libraries and user-applications. The kernel is responsible for processes, threads, memory-management, a virtual file system (VFS) and some other things. Drivers run in userspace and work via message-passing. They do not necessarily access the hardware and should therefore more seen as an instance that provides a service.
The VFS is one of the central points of Escape. Besides the opportunity to create files and folders in it, it is used by the kernel to provide information about the system for the userspace (memory-usage, CPU, running processes, ...). Most important, it is used to communicate with devices, which are registered and handled by drivers. That means, each device gets a node in the VFS over which it can be accessed. So, for example an open("/dev/zero",...) would open the device "zero". Afterwards one can use the received file-descriptor to communicate with it.
Basically, the communication works over messages. Escape provides the system-calls send and receive for that purpose. Additionally, Escape defines standard-messages for open,read,write and close, which devices may support. For example, when using the read system-call with a file-descriptor for a device, Escape will send the read-standard-message to the corresponding driver and handle the communication for the user-program. As soon as the answer is available, the result is passed back to the user-program that called read. The same mechanism is used for the "real" file-system, which is realized by the driver "fs". Escape defines messages for that purpose and handles the communication with fs when a user-program accesses a "real" file.
Because of this mechanism the user-space doesn't have to distinguish between virtual files, real files and devices. So, for example the tool cat can simply do an open and use read to read from that file and write to write to stdout.

Features

Escape has currently the following features:

• Kernel

  • Memory-management supporting copy-on-write, text-sharing, shared-libraries, shared-memory, mapping of physical mem (e.g. VESA-mem), demand-loading, swapping (not finished yet) and thread-local-storage.
  • Process- and thread-management, event-system, ELF-loader, environment- variables, scheduler, signal-management (signals are also used to pass interrupts to drivers) and timer-management (for sleeping and preemption).
  • SMP-support, CPU-detection and CPU-speed-calculation, FPU-support
  • VM86-task to perform BIOS-calls
  • Virtual file system for driver-access, accessing the real-filesystem, creating virtual files and directories, using pipes and providing information about the system (memory-usage, cpu, running processes, ...)
  • Debugging-console

• libc

  Except of some IMO not very important things like locale, math and so on, most of a C standard library exists by now.

• libcpp

  The C++ standard library isn't that complete as the libc. But the most basic things like string, iostreams, vector, list, map, algorithm and a few other things are available (some of them are a bit simpler than the standard specifies it). That means, one can develop applications with the library when one doesn't need the more exotic stuff ;)
  

• libgui

  The GUI-library isn't finished yet, but does already provide things like basic controls, layout-manager, BMP-support, event-system (keypresses, ...) and so on. It is written in C++, which was one of the main reason to provide C++-support in Escape :)

• drivers

  Escape has some architecture-specific drivers and some common drivers that are architecture-independent. Currently, there are:
  • common
    • fs: The filesystem. Has a nearly complete ext2-rev0 fs and a iso9660 fs. Additionally it manages mount-points.
    • kmmanager: The keymap-manager knows the current keymap (can be changed), reads from keyboard, translates the keycodes to characters and provides them for other processes.
    • ui: Registers keyboard-shortcuts at kmmanager and allows this way to switch the virtual terminal via CTRL+1-6. CTRL+7 switches to the GUI.
    • vterm: An abstraction-layer for user-processes. It reads from kmmanager, writes to the video-driver and registers multiple virtual terminals as devices. They can be used to read characters or write characters (terminal-like).
    • random/null/zero: Like /dev/{random,null,zero} in e.g. Linux
    • winmanager: Manages all existing windows. Reads from kmmanager and mouse and passes it to the corresponding application. Additionally it can move and resize windows and is responsible for setting the corresponding mouse-cursor (default, resize, ...)
  • x86
    • ata: Reading and writing from/to ATA/ATAPI-devices. It supports PIO-mode and DMA, LBA28 and LBA48
    • cmos: Provides the current date
    • keyboard: Gets notified about keyboard-interrupts, converts the scancodes to keycodes and stores them in a ringbuffer. Other processes can read them.
    • mouse: Gets notified about mouse-interrupts, stores the information in a ring-buffer and gives other processes read-access to them.
    • pci: Collects the available PCI-devices and gives others access to them. This is e.g. used by ata to find the IDE-controller
    • speaker: Produces beeps with the PC-speaker with a specified frequency and duration.
    • vesa: Reads the supported VESA-modes from BIOS (via VM86), determines the best matching mode and sets it. It creates a shared-memory-region which all GUI-processes use directly. On demand it copies a part of it to the VESA-memory. Additionally one can let VESA draw the mouse-cursor at a specified position and it is also responsible for drawing the window-move/resize-preview-rectangle.
    • vesatext: VESA driver for text-mode. Can be used as an alternative to the video-driver (VGA-text)
    • video: Can be used to write to the VGA-textmode-memory.
  • ECO32/MMIX
    Both architectures have the same devices atm (the only difference is that ECO32-devices are accessed with 32-bit words and MMIX-devices with 64-bit words).
    • disk: the disk-driver and therefore the equivalent to ata on x86.
    • keyboard: the keyboard-driver
    • rtc: the real-time-clock. This is a dummy atm, because both machines don't provide a real-time-clock. Therefore, a timestamp is simply increased once a second, starting with 0 :)
    • video: the video-driver (text-based)

• user

  There are quite a few user-programs so that you can do something with Escape ;) The most important ones are:
  • initloader: Is included in the kernel and is used as the first process. Loads the boot-modules (disk-driver, pci (on x86 only), cmos/rtc and fs because we need them to load other drivers and user-apps from the disk). After that it replaces itself with init.
  • init: Is responsible for loading the drivers and login-shells. Which drivers should be loaded is determined by /etc/drivers. It starts one login-shell for each vterm.
  • login: Accepts username+password, sets uid, gid and further group-ids using /etc/users and /etc/groups, creates stdin, stdout and stderr, sets the env-vars CWD and USER and exchanges itself with the shell.
  • shell: The shell implements a small shell-scripting-language-interpreter with pipes, io-redirection, path-expansion, background-jobs, arithmetic, loops, if-statements, functions, variables and arrays. The interpreter is realized with flex and bison.
  • cat, chmod, chown, cut, date, dd, dump, grep, groups, head, kill, less, ln, ls, mkdir, more, mount, ps, readelf, rm, rmdir, sort, stat, sync, tail, time, umount, users, wc: The well-known UNIX-tools. They don't do exactly the same and are, of course, much simpler, but in principle they are intended for the same things.
  • ts: analogously to ps, ts lists all threads
  • power: for reboot / shutdown using init.
  • keymap: A tool to change the current keymap (us and german are available)
  • ...

  Although a GUI exists, there are not really interesting GUI-programs yet. But a desktop-app exists, responsible for the desktop-background, the desktop-icons and the taskbar and a GUI-shell with which you can do the same things as with the command-line-shell :)

Try Escape

• x86

  All you need is a x86-virtualisation/-emulation-software (or a real machine, of course). I have tested Escape in Bochs, Qemu, VMWare, VirtualBox and several real machines.
  For example:
  qemu -hda escape-hd.img
  qemu -cdrom escape-cd.img
  When using VirtualBox or VMWare you have to create a virtual machine first and use the VDMK-image or CD-image.

• ECO32

  You need the ECO32-simulator to run Escape. A slightly modified version of the official one is provided with Escape. You can run Escape via (assuming that you are in the directory of ECO32):
  
  ./build/bin/sim -r build/monitor/monitor.bin -t 1 -d escape-eco32-hd.img -o log.txt -c -i
  Note: If you use the official version, you have to increase the amount of main memory to something like 10 MB in sim/common.h.

• MMIX

  Running Escape on GIMMIX is similar to running it on ECO32. You can find a slightly modified version of the official one in Escape. You can run Escape via (assuming that you are in the directory of GIMMIX):
  ./build/gimmix -r build/tests/manual/hexmon.rom -t 1 -d escape-mmix-hd.img -o log.txt -c -i

Build/change Escape

The build-process should work with every standard linux system (I'm using Ubuntu 11.04 2.6.38-10-generic x86_64). At first, you need to build the cross-compiler for the architecture you want to run Escape on: Note that the source of the cross-compiler will be put into cross/<arch>/src, the build-files into crossbuild/<arch> and the produced binaries into toolchain/<arch>.
Now you can build Escape: Finally, you can run it: Take a look into Makefile to see further options.

Note: The ECO32-cross-compiler isn't available yet. It will follow soon.

Downloads

Version 0.4

• i586 CD-Image (1826 KiB)
• i586 HD-Image (2012 KiB)
• i586 VMDK-Image (2036 KiB)
• ECO32 HD-Image (3973 KiB)
• MMIX HD-Image (4142 KiB)
• Source-Code (947 KiB)
• Source-Code incl. ECO32 and GIMMIX (6564 KiB)

Version 0.3

• i586 CD-Image (1462 KiB)
• i586 HD-Image (1445 KiB)
• i586 VMDK-Image (1422 KiB)
• Source-Code (1279 KiB)

Version 0.2

• i586 HD-Image (1714 KiB)
• i586 VMDK-Image (1657 KiB)
• Source-Code (824 KiB)

Version 0.1

• i586 HD-Image (520 KiB)
• i586 VMDK-Image (520 KiB)
• Source-Code (449 KiB)

Screenshots

Version 0.4

Boot progress:	Dynamic linking:	Users and groups:	GUI scrollpanes:
GUI window moving:	Kernel debugging console 1:	Kernel debugging console 2:	Kernel debugging console 3:
Maps of a process:	The shell-language:	SMP support:	Listing the running threads:
Running on bochs:	Running on VirtualBox:	Running on VMWare:	Running on real-machine 1:
Running on real machine 2:	Running on real machine 3:	Running on ECO32 1:	Running on ECO32 2:
Running on MMIX 1:	Running on MMIX 2:

Version 0.3

ATAPI & ISO9660:	PCI-info:	VBE-info:	Processes:
dump /dev/random | less:	Raptor:	The shell-scripting-language:	GUI-shell:
GUI-controls:	Resizing of windows:

Version 0.2

Colortable piped through cat:	Read from driver:	The root-directory:	Running processes:
Recursive mounting ^^:	The GUI:	The GUI - combobox:

Version 0.1

After the start:	Help - part1:	Help - part2:	Running processes:
The program ls:	Playing with the shell ;):	Read information from VFS: