DDB(4) Device Drivers Manual DDB(4)

ddb
in-kernel debugger

options DDB
To enable history editing:
options DDB_HISTORY_SIZE=integer
To disable entering ddb upon kernel panic:
options DDB_ONPANIC=0
To enable teeing all ddb output to the kernel msgbuf:
options DDB_TEE_MSGBUF=1
To specify commands which will be executed on each entry to ddb:
options DDB_COMMANDONENTER="trace;show registers" In this case, "trace" and then "show registers" will be executed automatically.
To enable extended online help:
options DDB_VERBOSE_HELP.

ddb is the in-kernel debugger. It may be entered at any time via a special key sequence, and optionally may be invoked when the kernel panics.

Unless DDB_ONPANIC is set to 0, ddb will be activated whenever the kernel would otherwise panic.
ddb may also be activated from the console. In general, sending a break on a serial console will activate ddb. There are also key sequences for each port that will activate ddb from the keyboard:
alpha
<Ctrl>-<Alt>-<Esc> on PC style keyboards.
amd64
<Ctrl>-<Alt>-<Esc>
<Break> on serial console.
amiga
<LAlt>-<LAmiga>-<F10>
atari
<Alt>-<LeftShift>-<F9>
hp300
<Shift>-<Reset>
hpcarm
<Ctrl>-<Alt>-<Esc>
hpcmips
<Ctrl>-<Alt>-<Esc>
hpcsh
<Ctrl>-<Alt>-<Esc>
hppa
<Ctrl>-<Alt>-<Esc> on PC style keyboards.
+++++ (five plus signs) on PDC console
<Break> on serial console.
i386
<Ctrl>-<Alt>-<Esc>
<Break> on serial console.
mac68k
<Command>-<Power>, or the Interrupt switch.
macppc
Some models: <Command>-<Option>-<Power>
mvme68k
Abort switch on CPU card.
pmax
<Do> on LK-201 rcons console.
<Break> on serial console.
sandpoint
<Break> on serial console.
sparc
<L1>-A, or <Stop>-A on a Sun keyboard.
<Break> on serial console.
sparc64
<L1>-A, or <Stop>-A on a Sun keyboard.
<Break> on serial console.
sun3
<L1>-A, or <Stop>-A on a Sun keyboard.
<Break> on serial console.
vax
<Esc>-<Shift>-D on serial console.
x68k
Interrupt switch on the body.
xen dom0
<Ctrl>-<Alt>-<Esc> on PC style keyboards.
+++++ (five plus signs) on serial console.
xen domU
+++++ (five plus signs) on serial console.
zaurus
<Ctrl>-<Alt>-<Esc>
The key sequence to activate ddb can be changed by modifying “hw.cnmagic” with sysctl(8). If the console is not dedicated to ddb the sequence should not be easily typed by accident. In addition, ddb may be explicitly activated by the debugging code in the kernel if DDB is configured.
Commands can be automatically run when ddb is entered by using options DDB_COMMANDONENTER or by setting ddb.commandonenter with sysctl(8). Multiple commands can be separated by a semi-colon.

The general command syntax is:
command[
/modifier
] address [
,count
]
The current memory location being edited is referred to as dot, and the next location is next. They are displayed as hexadecimal numbers.
Commands that examine and/or modify memory update dot to the address of the last line examined or the last location modified, and set next to the next location to be examined or modified. Other commands don't change dot, and set next to be the same as dot.
A blank line repeats the previous command from the address next with the previous count and no modifiers. Specifying address sets dot to the address. If address is omitted, dot is used. A missing count is taken to be 1 for printing commands, and infinity for stack traces.
The syntax:
,count
repeats the previous command, just as a blank line does, but with the specified count.
ddb has a more(1)-like functionality; if a number of lines in a command's output exceeds the number defined in the lines variable, then ddb displays “--db more--” and waits for a response, which may be one of:
 
 
⟨return⟩
one more line.
 
 
⟨space⟩
one more page.
 
 
abort the current command, and return to the command input mode.
You can set lines variable to zero to disable this feature.
If ddb history editing is enabled (by defining the
options DDB_HISTORY_SIZE=num
kernel option), then a history of the last num commands is kept. The history can be manipulated with the following key sequences:
 
 
<Ctrl>-P
retrieve previous command in history (if any).
 
 
<Ctrl>-N
retrieve next command in history (if any).

ddb supports the following commands:
 
 
address([
expression[
,...
])
]
A synonym for call.
 
 
[
/u
] address[
,count
]
Set a breakpoint at address. If count is supplied, continues (count-1) times before stopping at the breakpoint. If the breakpoint is set, a breakpoint number is printed with ‘#’. This number can be used to delete the breakpoint, or to add conditions to it.
If /u is specified, set a breakpoint at a user-space address. Without /u, address is considered to be in the kernel-space, and an address in the wrong space will be rejected, and an error message will be emitted. This modifier may only be used if it is supported by machine dependent routines.
Warning: if a user text is shadowed by a normal user-space debugger, user-space breakpoints may not work correctly. Setting a breakpoint at the low-level code paths may also cause strange behavior.
 
 
[
/ul
] [
frame-address
][
,count
]
A synonym for trace.
 
 
[
/ul
] [
pid
][
,count
]
A synonym for trace/t.
 
 
[
/ul
] [
lwpaddr
][
,count
]
A synonym for trace/a.
 
 
address([
expression[
,...
])
]
Call the function specified by address with the argument(s) listed in parentheses. Parentheses may be omitted if the function takes no arguments. The number of arguments is currently limited to 10.
 
 
[
/c
]
Continue execution until a breakpoint or watchpoint. If /c is specified, count instructions while executing. Some machines (e.g., pmax) also count loads and stores.
Warning: when counting, the debugger is really silently single-stepping. This means that single-stepping on low-level may cause strange behavior.
 
 
address | number
Delete a breakpoint. The target breakpoint may be specified by address, as per break, or by the breakpoint number returned by break if it's prefixed with ‘#’.
 
 
[
count
]
Prints the contents of the kernel message buffer. The optional count argument will limit printing to at most the last count bytes of the message buffer.
 
 
address
Delete the watchpoint at address that was previously set with watch command.
 
 
[
/modifier
] address[
,count
]
Display the address locations according to the format in modifier. Multiple modifier formats display multiple locations. If modifier isn't specified, the modifier from the last use of examine is used.
The valid format characters for modifier are:
examine bytes (8 bits).
examine half-words (16 bits).
examine words (legacy “long”, 32 bits).
examine long words (implementation dependent)
print the location being examined.
print the location with a line number if possible.
display in unsigned hex.
display in signed hex.
display in unsigned octal.
display in signed decimal.
display in unsigned decimal.
display in current radix, signed.
display low 8 bits as a character. Non-printing characters as displayed as an octal escape code (e.g., ‘\000’).
display the NUL terminated string at the location. Non-printing characters are displayed as octal escapes.
display in unsigned hex with a character dump at the end of each line. The location is displayed as hex at the beginning of each line.
display as a machine instruction.
display as a machine instruction, with possible alternative formats depending upon the machine:
alpha
print register operands
m68k
use Motorola syntax
vax
don't assume that each external label is a procedure entry mask
 
 
pid[
,signal_number
]
Send a signal to the process specified by the pid. Note that pid is interpreted using the current radix (see trace/t command for details). If signal_number isn't specified, the SIGTERM signal is sent.
 
 
[
/p
]
A synonym for next.
 
 
[
/p
]
Stop at the matching return instruction. If /p is specified, print the call nesting depth and the cumulative instruction count at each call or return. Otherwise, only print when the matching return is hit.
 
 
[
/axzodurc
] address [
address ...
]
Print addresses address according to the modifier character, as per examine. Valid modifiers are: /a, /x, /z, /o, /d, /u, /r, and /c (as per examine). If no modifier is specified, the most recent one specified is used. address may be a string, and is printed “as-is”. For example:
print/x "eax = " $eax "\necx = " $ecx "\n"
    
will produce:
eax = xxxxxx 
ecx = yyyyyy
    
 
 
[
/a
][
/n
][
/w
][
/l
]
A synonym for show all procs.
 
 
[
flags
]
Reboot, using the optionally supplied boot flags, which is a bitmask supporting the same values as for reboot(2). Some of the more useful flags:
Value Name Description
0x1 RB_ASKNAME Ask for file name to reboot from
0x2 RB_SINGLE Reboot to single user mode
0x4 RB_NOSYNC Don't sync before reboot
0x8 RB_HALT Halt instead of reboot
0x40 RB_KDB Boot into kernel debugger
0x100 RB_DUMP Dump unconditionally before reboot
0x808 RB_POWERDOWN Power off (or at least halt)
Note: Limitations of the command line interface preclude specification of a boot string.
 
 
[
/bhl
] address value [
mask
] [
,count
]
Search memory from address for value. The unit size is specified with a modifier character, as per examine. Valid modifiers are: /b, /h, and /l. If no modifier is specified, /l is used.
This command might fail in interesting ways if it doesn't find value. This is because ddb doesn't always recover from touching bad memory. The optional count limits the search.
 
 
$variable [
=
] expression
Set the named variable or register to the value of expression. Valid variable names are described in VARIABLES.
 
 
Display information about callouts in the system. See callout(9) for more information on callouts.
 
 
[
/t
]
Display details information about all active locks. If /t is specified, stack traces of LWPs holding locks are also printed. This command is only useful if a kernel is compiled with options LOCKDEBUG.
 
 
Display basic information about all physical pages managed by the VM system. For more detailed information about a single page, use show page.
 
 
[
/clp
]
Display all pool information. Modifiers are the same as show pool.
 
 
[
/a
][
/n
][
/w
][
/l
]
Display all process information. Valid modifiers:
 
 
show process information in a ps(1) style format. Information printed includes: process ID, parent process ID, process group, UID, process status, process flags, number of LWPs, command name, and process wait channel message.
 
 
show each process ID, command name, kernel virtual addresses of each process' proc structure, u-area, and vmspace structure. The vmspace address is also the address of the process' vm_map structure, and can be used in the show map command.
 
 
show each LWP ID, process ID, command name, system call emulation, priority, wait channel message and wait channel address. LWPs currently running on a CPU are marked with the '>' sign.
 
 
show each LWP ID, process ID, process status, CPU ID the LWP runs on, process flags, kernel virtual address of LWP structure, LWP name and wait channel message. LWPs currently running on a CPU are marked with the '>' sign. This is the default.
 
 
Dump the entire AF_INET routing table. This command is available only on systems which support inet.
 
 
Display all breakpoints.
 
 
[
/f
] address
Print the struct buf at address. The /f does nothing at this time.
 
 
[
/f
][
/i
][
/m
][
/t
]
Print all the non-zero evcnt(9) event counters. Valid modifiers:
 
 
event counters with a count of zero are printed as well.
 
 
interrupted counters will be displayed.
 
 
misc counters will be displayed.
 
 
trap counters will be displayed.
If none of /i, /m, or /t are specified, all are shown. You can combine any of these. For example, the modifier /itf will select both interrupt and trap events, including those that are non-zero.
 
 
address
Display information about the vnodes of the files that are currently open by the process associated with the proc structure at address. This address can be found using the show all procs /a command. If the kernel is compiled with options LOCKDEBUG then details about the locking of the underlying uvm object will also be displayed.
 
 
address
Display information about a lock at address. This command is only useful if a kernel is compiled with options LOCKDEBUG.
 
 
Display information about lock statistics. This command is only useful if a kernel is compiled with options LOCKDEBUG.
 
 
[
/f
] address
Print the vm_map at address. If /f is specified, the complete map is printed.
 
 
[
/f
] address
Print the mount structure at address. If /f is specified, the complete vnode list is printed.
 
 
[
/cdv
] address
Print the mbuf structure at address. Valid modifiers:
The mbufs in the chain are NOT followed.
The data is dumped.
Decode the mbuf chain as a packet. It currently supports Ethernet, PPP, PPPoE, ARP, IPv4, ICMP, IPv6, ICMP6, TCP and UDP.
 
 
address
Dump the namecache list associated with vnode at address.
 
 
[
/f
] address
Print the vm_object at address. If /f is specified, the complete object is printed.
 
 
[
/f
] address
Print the vm_page at address. If /f is specified, the complete page is printed.
 
 
Print the current "panic" string.
 
 
[
/clp
] address
Print the pool at address. Valid modifiers:
Print the cachelist and its statistics for this pool.
Print the log entries for this pool.
Print the pagelist for this pool.
 
 
[
/ap
] address | pid
Show information about a process and its LWPs. LWPs currently running on a CPU are marked with the '>' sign.
The argument passed is the kernel virtual address of LWP structure.
The argument passed is a PID. Note that pid is interpreted using the current radix (see trace/t command for details). This is the default.
 
 
[
/u
]
Display the register set. If /u is specified, display user registers instead of kernel registers or the currently save one.
Warning: support for /u is machine dependent. If not supported, incorrect information will be displayed.
 
 
Print the state of the scheduler's run queues. For each run queue that has an LWP, the run queue index and the list of LWPs will be shown. If the run queue has LWPs, but the sched_whichqs bit is not set for that queue, the queue index will be prefixed with a ‘!’.
 
 
[
/ampv
]
Print usage of system's socket buffers. By default, empty sockets aren't printed.
Print all processes which use the socket.
Print mbuf chain in the socket buffer.
By default, a process which uses the socket is printed (only one socket). If /p is specified, the process isn't printed.
Verbose mode. If /v is specified, all sockets are printed.
 
 
Print a selection of UVM counters and statistics.
 
 
[
/i
] [
addr[
,count
]
]
Dumps all the kernel histories if no address is specified, or the history at the address. If /i is specified, display information about the named history or all histories, instead of history entries. If count is specified, only the last count entries will be displayed. Currently the count handling is only performed if a single history is requested. This command is available only if a kernel is compiled with one or more of the kernel history options KERNHIST, SYSCALL_DEBUG, USB_DEBUG, BIOHIST, or UVMHIST.
 
 
[
/f
] address
Print the vnode at address. If /f is specified, the complete vnode is printed.
 
 
[
/f
] address
Print the vnode which has its lock at address. If /f is specified, the complete vnode is printed.
 
 
Display all watchpoints.
 
 
[
/F
] string
Search the symbol tables for all symbols of which string is a substring, and display them. If /F is specified, a character is displayed immediately after each symbol name indicating the type of symbol.
For a.out(5)-format symbol tables, absolute symbols display @, text segment symbols display *, data segment symbols display +, BSS segment symbols display -, and filename symbols display /. For ELF-format symbol tables, object symbols display +, function symbols display *, section symbols display &, and file symbols display /.
To sift for a string beginning with a number, escape the first character with a backslash as:
sifting \386
    
 
 
[
/p
] [
,count
]
Single-step count times. If /p is specified, print each instruction at each step. Otherwise, only print the last instruction.
Warning: depending on the machine type, it may not be possible to single-step through some low-level code paths or user-space code. On machines with software-emulated single-stepping (e.g., pmax), stepping through code executed by interrupt handlers will probably do the wrong thing.
 
 
Sync the disks, force a crash dump, and then reboot.
 
 
[
/u[
l
]
] [
frame-address
][
,count
]
Stack trace from frame-address. If /u is specified, trace user-space, otherwise trace kernel-space. count is the number of frames to be traced. If count is omitted, all frames are printed. If /l is specified, the trace is printed and also stored in the kernel message buffer.
Warning: user-space stack trace is valid only if the machine dependent code supports it.
 
 
[
l
] [
pid
][
,count
]
Stack trace by “thread” (process, on NetBSD) rather than by stack frame address. Note that pid is interpreted using the current radix, whilst ps displays pids in decimal; prefix pid with ‘0t’ to force it to be interpreted as decimal (see VARIABLES section for radix). If /l is specified, the trace is printed and also stored in the kernel message buffer.
Warning: trace by pid is valid only if the machine dependent code supports it.
 
 
[
l
] [
lwpaddr
][
,count
]
Stack trace by light weight process (LWP) address rather than by stack frame address. If /l is specified, the trace is printed and also stored in the kernel message buffer.
Warning: trace by LWP address is valid only if the machine dependent code supports it.
 
 
[
/p
]
Stop at the next call or return instruction. If /p is specified, print the call nesting depth and the cumulative instruction count at each call or return. Otherwise, only print when the matching return is hit.
 
 
address[
,size
]
Set a watchpoint for a region. Execution stops when an attempt to modify the region occurs. size defaults to 4.
If you specify a wrong space address, the request is rejected with an error message.
Warning: attempts to watch wired kernel memory may cause an unrecoverable error in some systems such as i386. Watchpoints on user addresses work the best.
 
 
address
Describe what an address is.
 
 
[
/bhlBHL
] address expression [
expression ...
]
Write the expressions at succeeding locations. The unit size is specified with a modifier character, as per examine. Valid modifiers are: /b, /h, and /l. If no modifier is specified, /l is used.
Specifying the modifiers in upper case, /B, /H, /L, will prevent ddb from reading the memory location first, which is useful for avoiding side effects when writing to I/O memory regions.
Warning: since there is no delimiter between expressions, strange things may occur. It's best to enclose each expression in parentheses.
 
 
[
/modifier
] address[
,count
]
A synonym for examine.

The "glue" code that hooks ddb into the NetBSD kernel for any given port can also add machine specific commands to the ddb command parser. All of these commands are preceded by the command word machine to indicate that they are part of the machine-specific command set (e.g. machine reboot). Some of these commands are:

Switch to another cpu.
Print cpu information about the ``struct cpuinfo''.
Given a trap frame address, print out the trap frame.
Print lwp information about the ``struct lwp''.
Print PTE information.
Print system registers.
Set or clear a hardware watchpoint. Pass the address to be watched, or watchpoint number to clear the watchpoint. Optional modifiers are “l” for load access, “s” for store access, “1” for 8 bit width, “2” for 16 bit, “3” for 24 bit, “4” for 32 bit, “5” for 40 bit, “6” for 48 bit, “7” for 56 bit, “8” for 64 bit.

Switch to another cpu.

Switch to another cpu.

Given a trap frame address, print out the trap frame.

Without an address the default trap frame is printed. Otherwise, the trap frame address can be given, or, when the “l” modifier is used, an LWP address.

Switch to another cpu.

Without a vector, information about all 256 vectors is shown. Otherwise, the given vector is shown.

Dump CP0 (coprocessor 0) register values.
Print the physical address for a given kernel virtual address.
Print out the Translation Lookaside Buffer (TLB). Only works in NetBSD kernels compiled with DEBUG option.

Print process MMU context information.
Print PA->VA mapping information.
Reset the system.
Display the contents of the trapframe.
Display instruction translation storage buffer information.
Set the DCR register. Must be between 0x00 and 0x3ff.
Display user memory. Use the “i” modifier to get instruction decoding.

Print BAT registers and translations.
Print MMU registers.

Print TLB entries.
Print cache entries.
Print switch frame and trap frames.
Print kernel stack usage. Only works in NetBSD kernels compiled with the KSTACK_DEBUG option.

Switch to another cpu.
Enter the Sun PROM monitor.
Display some information about the LWP pointed to, or curlwp.
Display information about the “struct pcb” listed.
Display the pointer to the “struct vm_page” for this physical address.

Print process context information.
Switch to another cpu.
Print data translation look-aside buffer context information.
Display data translation storage buffer information.
Display information about the listed mapping in the kernel pmap. Use the “f” modifier to get a full listing.
Extract the physical address for a given virtual address from the kernel pmap.
Dump the FPU state.
Print instruction translation look-aside buffer context information.
Display instruction translation storage buffer information.
Display a struct lwp
Display information about the “struct pcb” listed.
Attempt to change process context.
Display the pointer to the “struct vm_page” for this physical address.
Display physical memory.
Display the pmap. Use the “f” modifier to get a fuller listing.
Display some information about the process pointed to, or curproc.
Enter the OFW PROM.
Display the “struct pv_entry” pointed to.
Reset the machine and enter prom (do a Software Initiated Reset).
Dump the window stack. Use the “u” modifier to get userland information.
Display full trap frame state. This is most useful for inclusion with bug reports.
Display trap state.
Display or set trap trace information. Use the “r” and “f” modifiers to get reversed and full information, respectively.
Set or clear a physical or virtual hardware watchpoint. Pass the address to be watched, or “0” (or omit the address) to clear the watchpoint. Optional modifiers are “p” for physical address, “r” for trap on read access (default: trap on write access only), “b” for 8 bit width, “h” for 16 bit, “l” for 32 bit or “L” for 64 bit.
Print register window information. Argument is a stack frame number (0 is top of stack, which is used when no index is given).

Drop into monitor via abort (allows continue).
Exit to Sun PROM monitor as in halt(8).
Reboot the machine as in reboot(8).
Given an address, print the address, segment map, page map, and Page Table Entry (PTE).

Switch to another cpu.

ddb accesses registers and variables as $name. Register names are as per the show registers command. Some variables are suffixed with numbers, and may have a modifier following a colon immediately after the variable name. For example, register variables may have a ‘u’ modifier to indicate user register (e.g., $eax:u).
Built-in variables currently supported are:
dumpstack
If non-zero (the default), causes a stack trace to be printed when ddb is entered on panic.
fromconsole
If non-zero (the default), the kernel allows to enter ddb from the console (by break signal or special key sequence). If the kernel configuration option
options DDB_FROMCONSOLE=0
is used, fromconsole will be initialized to off.
lines
The number of lines. This is used by the more feature. When this variable is set to zero the more feature is disabled.
maxoff
Addresses are printed as 'symbol'+offset unless offset is greater than maxoff.
maxwidth
The width of the displayed line. ddb wraps the current line by printing new line when maxwidth column is reached. When this variable is set to zero ddb doesn't perform any wrapping.
onpanic
If greater than zero (the default is 1), ddb will be invoked when the kernel panics. If the kernel configuration option
options DDB_ONPANIC=0
is used, onpanic will be initialized to off, causing a stack trace to be printed and the system to be rebooted instead of ddb being entered. Setting onpanic to -1 suppresses the stack trace before reboot.
radix
Input and output radix.
tabstops
Tab stop width.
tee_msgbuf
If explicitly set to non zero (zero is the default) all ddb output will not only be displayed on screen but also be fed to the msgbuf. The default of the variable can be set using the kernel configuration option
options DDB_TEE_MSGBUF=1
which will initialize tee_msgbuf to be 1. This option is especially handy for poor souls who don't have a serial console but want to recall ddb output from a crash investigation. This option is more generic than the /l command modifier possible for selected commands as discussed above to log the output. Mixing both /l and this setting can give double loggings.
panicstackframes
Number of stack frames to display on panic. Useful to avoid scrolling away the interesting frames on a glass tty. Default value is 65535 (all frames), useful value around 10.
All built-in variables are accessible via sysctl(3).

Almost all expression operators in C are supported, except ‘~’, ‘^’, and unary ‘&’. Special rules in ddb are:
 
 
identifier
name of a symbol. It is translated to the address (or value) of it. ‘.’ and ‘:’ can be used in the identifier. If supported by an object format dependent routine, [
filename:
]function[
:line number
], [
filename:
]variable, and filename[
:line number
], can be accepted as a symbol. The symbol may be prefixed with symbol_table_name:: (e.g., emulator::mach_msg_trap) to specify other than kernel symbols.
 
 
number
number. Radix is determined by the first two characters: ‘0x’ - hex, ‘0o’ - octal, ‘0t’ - decimal, otherwise follow current radix.
 
 
dot
 
 
next
 
 
address of the start of the last line examined. Unlike dot or next, this is only changed by the examine or write commands.
 
 
last address explicitly specified.
 
 
name
register name or variable. It is translated to the value of it. It may be followed by a ‘:’ and modifiers as described above.
 
 
a binary operator which rounds up the left hand side to the next multiple of right hand side.
 
 
expr
expression indirection. It may be followed by a ‘:’ and modifiers as described above.

reboot(2), options(4), crash(8), reboot(8), sysctl(8), cnmagic(9)

The ddb kernel debugger was written as part of the MACH project at Carnegie-Mellon University.
October 12, 2018 NetBSD-current