Courses/Computer Science/CPSC 457.W2012/Links and Other Resources

= x86 Architecture =

http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html

= How To Configure, Compile, and Install a Linux Kernel from Source =


 * http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html
 * http://cromwell-intl.com/unix/linux-kernel.html
 * http://www.linuxforums.org/forum/kernel/55612-mini-howto-compile-linux-kernel-2-6-a.html
 * Kernel configuration options language: http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt
 * http://linux.about.com/library/cmd/blcmdl8_depmod.htm
 * http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/3/html/System_Administration_Guide/s1-custom-kernel-modularized.html
 * http://fedoraproject.org/wiki/Building_a_custom_kernel#Build_the_New_Kernel
 * http://stackoverflow.com/questions/6307206/linux-kernel-make-modules-install-problem

= Kernel Modules =

Extending the functionality of the kernel can be accomplished by modifying kernel code directly, but through the loadable kernel module (LKM) feature of the Linux kernel, one can write a fairly self-contained module that adds or modifies existing kernel functionality without requiring a recompile, re-install, or reboot of the OS.

Example
This example is based primarily on the documentation from the Linux source tree on how to build modules and on the Linux Kernel Module Programming Guide.


 * for the 2.6 version we've been using in class examples: modules.txt
 * for the latest 3.2.9 version: modules.txt
 * Hello World Example

The full example run can be found here.

Links / References

 * The "HOWTO" on Writing a Linux LKM
 * Compiling modules as part of the Linux compilation process.
 * http://tldp.org/HOWTO/Module-HOWTO/x839.html
 * http://tldp.yolinux.com/HOWTO/Kernel-HOWTO.html#steps
 * The difference between 2.4 and 2.6 related to writing and installing LKMs
 * individual LKMs
 * Passing parameters to modules.
 * http://lxr.linux.no/#linux+v3.2.11/Documentation/kbuild/modules.txt
 * http://www.thc.org/papers/LKM_HACKING.html (nearly) Complete Linux Loadable Kernel Modules "the definitive guide for hackers, virus coders and system administrators"

= KProbes =


 * Phrack: Kernel instrumentation using kprobes
 * http://sourceware.org/systemtap/kprobes/
 * An Introduction to Kprobes
 * Linux Kernel Documentation on Kprobes
 * From IBM: Kernel Debugging With Kprobes
 * From RedHat: Gaining Insight With Kprobes
 * SystemTap (scripting framework for kprobes)
 * SystemTap Example Scripts

= Other Approaches to Teaching OS =


 * Nachos
 * Minix 3 (cool research OS)
 * Minix On VMWare
 * Minix on VirtualBox

= Virtualization Platforms =


 * VMWare Player
 * Bochs
 * QEMU
 * Microsoft VirtualPC
 * VirtualBox Tech Docs
 * VirtualBox User Docs

= Useful links =
 * Hyperlinked Linux Source Code via the Linux Cross Reference project: LXR
 * Advanced Bash Scripting
 * printk: http://lxr.linux.no/#linux+v2.6.35.14/kernel/printk.c#L601
 * A blog post about rdtsc: http://people.summit-servers.com/monk/2010/07/26/hpet-rdtsc-based-userland-gettimeofday/
 * The Linux From Scratch Book Linux From Scratch
 * Linux Kernel Newbies Forum
 * Setting up SVN accounts: http://svnbook.red-bean.com/en/1.7/svn.serverconfig.svnserve.html#idp40102672
 * SVN Basic Work Cycle
 * Useful Unix Commands
 * Some Unix history
 * Programming Kernel Modules
 * Ars: What is a CPU Thread?
 * GCC inline assembly HOWTO Write Inline ASM
 * Netiquette Advice: http://freenode.net/channel_guidelines.shtml
 * Manipulating the Intel Debug registers: http://www.phrack.org/issues.html?issue=65&id=8&mode=txt
 * mm structure: http://lxr.linux.no/#linux+v2.6.38/include/linux/mm_types.h#L222
 * http://stackoverflow.com/questions/6134984/access-permissions-of-dev-mem
 * http://lxr.linux.no/#linux+v2.6.35.14/drivers/char/mem.c#L63 (useful for HW2 and writing to /dev/mem)
 * IBM article on initrd: Linux initial RAM disk (initrd) overview
 * Examining load average: http://www.linuxjournal.com/article/9001?page=0,1
 * http://www.linuxquestions.org/questions/programming-9/writing-a-trivial-kernel-module-help-810822-print/
 * "Understanding the /bin/ /sbin/ and /usr/sbin split": http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

Misc

 * syntax highlighting in emacs: https://www.gnu.org/software/emacs/manual/html_node/efaq/Turning-on-syntax-highlighting.html
 * What bugs can teach us: http://lwn.net/Articles/482345/
 * taskstats: http://lwn.net/Articles/406975/
 * Who needs /dev/kmem? http://lwn.net/Articles/147901/
 * http://fedoraproject.org/wiki/Building_a_custom_kernel#Build_the_New_Kernel
 * The C10K issue: http://www.kegel.com/c10k.html
 * accept scalability on Linux: http://www.citi.umich.edu/projects/linux-scalability/reports/accept.html
 * LKM programming: talking to devices: http://tldp.org/LDP/lkmpg/2.6/html/x892.html
 * memory management vulnerabilities: http://jon.oberheide.org/blog/2010/09/10/linux-kernel-can-slub-overflow/
 * http://en.wikipedia.org/wiki/Mach_%28kernel%29
 * http://www.gnu.org/software/hurd/community/weblogs/ArneBab/technical-advantages-of-the-hurd.html
 * http://forums.androidcentral.com/thunderbolt-rooting-roms-hacks/123060-guide-how-root-unroot-thunderbolt-revolutionary.html#post1649296
 * http://androidforums.com/thunderbolt-all-things-root/459410-htc-thunderbolt-all-things-root-guide.html
 * wiki.cyanogenmod.com/wiki/Howto:_Install_the_Android_SDK#Mac_OS_X
 * http://www.androidauthority.com/how-to-root-unlock-htc-thunderbolt-20651/
 * http://lxr.linux.no/#linux+v3.3/kernel/pid.c#L429
 * http://lxr.linux.no/#linux+v2.6.27.41/include/linux/sched.h#L1173
 * http://lxr.linux.no/#linux+v2.6.27.41/include/linux/sched.h#L1173
 * http://lxr.linux.no/#linux+v2.6.27.41/kernel/exit.c
 * http://lxr.linux.no/#linux+v2.6.27.41/include/linux/rwsem-spinlock.h#L69

Kernel LXR Links We Have Looked At
Here is a collection of links (in no particular order) to every piece of kernel source code we have looked at.


 * task_struct: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/sched.h#L1215
 * the mm_struct http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/mm_types.h#L202
 * the struct_file structure: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/fs.h#L908
 * the file operations structure: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/fs.h#L1482


 * the do_fork system call: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L1369
 * the copy_process helper: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L972
 * actually loading an ELF: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/binfmt_elf.c#L563


 * show_map_vma (i.e., /proc/PID/maps implementation): http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/proc/task_mmu.c#L202


 * the brk(2) system call entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L246
 * the do_brk helper function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L1993
 * the helper routine for finding a vma intersection http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/mm.h#L1211
 * the find_vma routine: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L1480


 * the sys_read entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L372
 * the VFS read function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L277
 * do_sync_read function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L252


 * the poll(2) entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/select.c#L898
 * the do_pollfd helper: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/select.c#L709


 * schedule http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched.c#L5412
 * pick_next_task http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched.c#L5380
 * CFS's pick_next_task_fair http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched_fair.c#L1676
 * context_switch: http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L2831
 * the x86 switch_to macro: http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/arch/x86/include/asm/system.h#L43
 * yield http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L6740
 * sched_yield http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L6649


 * example of creating a kernel thread: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/apm_32.c#L2369
 * operator table for devices: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/netdevice.h#L489
 * the poll controller ops field in netdevice ops: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/netdevice.h#L627
 * allocating a network device: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/net/core/dev.c#L5126
 * implementation of 'ifconfig up' for 3c501 network card: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/3c501.c#L338
 * initializing a network device kobject: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/net/core/net-sysfs.c#L530
 * servicing an interrupt in the tulip driver: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/tulip/interrupt.c#L508
 * the 3COM 3c501 driver: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/3c501.c#L1


 * copy_process: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L972
 * task struct's 'stack_start' (cf. mm_struct's start_stack): http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L1126
 * entry_32.S, syscall dispatch...riveting! http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/entry_32.S#L449
 * sys_clone declaration: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/asm-generic/syscalls.h#L12
 * sys_clone implementation, uses do_fork, no surprise: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/process_32.c#L433
 * getpid: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/timer.c#L1274
 * langsec. the kernel caught in the act of parsing an executable: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/proc/vmcore.c#L497

= BSD =


 * NetBSD kernel documentation: http://www.netbsd.org/docs/kernel/index.html
 * the OpenBSD operating system
 * OpenBSD Journal
 * Papers and presentation of the OpenBSD project
 * "Memory Allocators in Modern Operating Systems"
 * Building OpenBSD from source