Courses/Computer Science/CPSC 457.W2012/Links and Other Resources
From wiki.ucalgary.ca
< Courses | Computer Science | CPSC 457.W2012
Contents
x86 Architecture
http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html
How To Configure, Compile, and Install a Linux Kernel from Source
- http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html
- http://cromwell-intl.com/unix/linux-kernel.html
- http://www.linuxforums.org/forum/kernel/55612-mini-howto-compile-linux-kernel-2-6-a.html
- Kernel configuration options language: http://kernel.org/doc/Documentation/kbuild/kconfig-language.txt
- http://linux.about.com/library/cmd/blcmdl8_depmod.htm
- http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/3/html/System_Administration_Guide/s1-custom-kernel-modularized.html
- http://fedoraproject.org/wiki/Building_a_custom_kernel#Build_the_New_Kernel
- http://stackoverflow.com/questions/6307206/linux-kernel-make-modules-install-problem
Kernel Modules
Extending the functionality of the kernel can be accomplished by modifying kernel code directly, but through the loadable kernel module (LKM) feature of the Linux kernel, one can write a fairly self-contained module that adds or modifies existing kernel functionality without requiring a recompile, re-install, or reboot of the OS.
Example
This example is based primarily on the documentation from the Linux source tree on how to build modules and on the Linux Kernel Module Programming Guide.
- for the 2.6 version we've been using in class examples: modules.txt
- for the latest 3.2.9 version: modules.txt
- Hello World Example
The full example run can be found here.
Links / References
- Modifying kernel boot params (see especially the "S" flag) https://www.kernel.org/doc/Documentation/kernel-parameters.txt
- The "HOWTO" on Writing a Linux LKM
- Compiling modules as part of the Linux compilation process.
- http://tldp.org/HOWTO/Module-HOWTO/x839.html
- http://tldp.yolinux.com/HOWTO/Kernel-HOWTO.html#steps
- The difference between 2.4 and 2.6 related to writing and installing LKMs
- individual LKMs
- Passing parameters to modules.
- http://lxr.linux.no/#linux+v3.2.11/Documentation/kbuild/modules.txt
- http://www.thc.org/papers/LKM_HACKING.html (nearly) Complete Linux Loadable Kernel Modules "the definitive guide for hackers, virus coders and system administrators"
KProbes
- Phrack: Kernel instrumentation using kprobes
- http://sourceware.org/systemtap/kprobes/
- An Introduction to Kprobes
- Linux Kernel Documentation on Kprobes
- From IBM: Kernel Debugging With Kprobes
- From RedHat: Gaining Insight With Kprobes
- SystemTap (scripting framework for kprobes)
- SystemTap Example Scripts
Other Approaches to Teaching OS
Virtualization Platforms
Useful links
- http://eudyptula-challenge.org/
- http://lwn.net/Articles/599231/
- Hyperlinked Linux Source Code via the Linux Cross Reference project: LXR
- Advanced Bash Scripting
- printk: http://lxr.linux.no/#linux+v2.6.35.14/kernel/printk.c#L601
- A blog post about rdtsc: http://people.summit-servers.com/monk/2010/07/26/hpet-rdtsc-based-userland-gettimeofday/
- The Linux From Scratch Book Linux From Scratch
- Linux Kernel Newbies Forum
- Setting up SVN accounts: http://svnbook.red-bean.com/en/1.7/svn.serverconfig.svnserve.html#idp40102672
- SVN Basic Work Cycle
- Useful Unix Commands
- Some Unix history
- Programming Kernel Modules
- Ars: What is a CPU Thread?
- GCC inline assembly HOWTO Write Inline ASM
- Netiquette Advice: http://freenode.net/channel_guidelines.shtml
- Manipulating the Intel Debug registers: http://www.phrack.org/issues.html?issue=65&id=8&mode=txt
- mm structure: http://lxr.linux.no/#linux+v2.6.38/include/linux/mm_types.h#L222
- http://stackoverflow.com/questions/6134984/access-permissions-of-dev-mem
- http://lxr.linux.no/#linux+v2.6.35.14/drivers/char/mem.c#L63 (useful for HW2 and writing to /dev/mem)
- IBM article on initrd: Linux initial RAM disk (initrd) overview
- Examining load average: http://www.linuxjournal.com/article/9001?page=0,1
- http://www.linuxquestions.org/questions/programming-9/writing-a-trivial-kernel-module-help-810822-print/
- "Understanding the /bin/ /sbin/ and /usr/sbin split": http://lists.busybox.net/pipermail/busybox/2010-December/074114.html
Misc
- syntax highlighting in emacs: https://www.gnu.org/software/emacs/manual/html_node/efaq/Turning-on-syntax-highlighting.html
- What bugs can teach us: http://lwn.net/Articles/482345/
- taskstats: http://lwn.net/Articles/406975/
- Who needs /dev/kmem? http://lwn.net/Articles/147901/
- http://fedoraproject.org/wiki/Building_a_custom_kernel#Build_the_New_Kernel
- The C10K issue: http://www.kegel.com/c10k.html
- accept() scalability on Linux: http://www.citi.umich.edu/projects/linux-scalability/reports/accept.html
- LKM programming: talking to devices: http://tldp.org/LDP/lkmpg/2.6/html/x892.html
- memory management vulnerabilities: http://jon.oberheide.org/blog/2010/09/10/linux-kernel-can-slub-overflow/
- http://en.wikipedia.org/wiki/Mach_%28kernel%29
- http://www.gnu.org/software/hurd/community/weblogs/ArneBab/technical-advantages-of-the-hurd.html
- http://forums.androidcentral.com/thunderbolt-rooting-roms-hacks/123060-guide-how-root-unroot-thunderbolt-revolutionary.html#post1649296
- http://androidforums.com/thunderbolt-all-things-root/459410-htc-thunderbolt-all-things-root-guide.html
- wiki.cyanogenmod.com/wiki/Howto:_Install_the_Android_SDK#Mac_OS_X
- http://www.androidauthority.com/how-to-root-unlock-htc-thunderbolt-20651/
- http://lxr.linux.no/#linux+v3.3/kernel/pid.c#L429
- http://lxr.linux.no/#linux+v2.6.27.41/include/linux/sched.h#L1173
- http://lxr.linux.no/#linux+v2.6.27.41/include/linux/sched.h#L1173
- http://lxr.linux.no/#linux+v2.6.27.41/kernel/exit.c
- http://lxr.linux.no/#linux+v2.6.27.41/include/linux/rwsem-spinlock.h#L69
Kernel LXR Links We Have Looked At
Here is a collection of links (in no particular order) to every piece of kernel source code we have looked at.
- task_struct: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/sched.h#L1215
- the mm_struct http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/mm_types.h#L202
- the struct_file structure: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/fs.h#L908
- the file operations structure: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/fs.h#L1482
- the do_fork() system call: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L1369
- the copy_process() helper: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L972
- actually loading an ELF: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/binfmt_elf.c#L563
- show_map_vma (i.e., /proc/PID/maps implementation): http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/proc/task_mmu.c#L202
- the brk(2) system call entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L246
- the do_brk() helper function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L1993
- the helper routine for finding a vma intersection http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/mm.h#L1211
- the find_vma routine: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L1480
- mm_take_all_locks: http://lxr.cpsc.ucalgary.ca/lxr/#linux/mm/mmap.c#L2389 (example of good commenting)
- 64-bit x86 mmap(2) entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/sys_x86_64.c#L21
- IA32 mmap2 entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/ia32/sys_ia32.c#L542
- note both call do_mmap_pgoff: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/mm/mmap.c#L905
- the sys_read entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L372
- the VFS read function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L277
- do_sync_read function: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/read_write.c#L252
- the poll(2) entry point: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/select.c#L898
- the do_pollfd helper: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/select.c#L709
- schedule() http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched.c#L5412
- pick_next_task() http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched.c#L5380
- CFS's pick_next_task_fair() http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/sched_fair.c#L1676
- context_switch(): http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L2831
- the x86 switch_to() macro: http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/arch/x86/include/asm/system.h#L43
- yield() http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L6740
- sched_yield() http://lxr.cpsc.ucalgary.ca/#linux+v2.6.32/kernel/sched.c#L6649
- example of creating a kernel thread: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/apm_32.c#L2369
- operator table for devices: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/netdevice.h#L489
- the poll controller ops field in netdevice ops: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/linux/netdevice.h#L627
- allocating a network device: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/net/core/dev.c#L5126
- implementation of 'ifconfig up' for 3c501 network card: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/3c501.c#L338
- initializing a network device kobject: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/net/core/net-sysfs.c#L530
- servicing an interrupt in the tulip driver: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/tulip/interrupt.c#L508
- the 3COM 3c501 driver: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/drivers/net/3c501.c#L1
- copy_process: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L972
- task struct's 'stack_start' (cf. mm_struct's start_stack): http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/fork.c#L1126
- entry_32.S, syscall dispatch...riveting! http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/entry_32.S#L449
- ia32entry.S emulated syscalls via INT 0x80, has calling convention laid out: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/ia32/ia32entry.S#L380
- sys_clone declaration: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/include/asm-generic/syscalls.h#L12
- sys_clone implementation, uses do_fork(), no surprise: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/arch/x86/kernel/process_32.c#L433
- getpid: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/kernel/timer.c#L1274
- langsec. the kernel caught in the act of parsing an executable: http://lxr.cpsc.ucalgary.ca/lxr/#linux+v2.6.32/fs/proc/vmcore.c#L497
BSD
- NetBSD kernel documentation: http://www.netbsd.org/docs/kernel/index.html
- the OpenBSD operating system
- OpenBSD Journal
- Papers and presentation of the OpenBSD project
- "Memory Allocators in Modern Operating Systems"
- Building OpenBSD from source