notes

libstdc++ debug info on Ubuntu 12.04 i386:

 * /usr/share/gcc-4.6/python or equivalent needs to be added to PYTHONPATH to
   get STL debugging working. Should be added to .profile so gVim picks it up.

 * libstdc++6-4.6-dbg only includes debug info and not the libstdc++ sources.
   Might need to get those from gcc-4.6.3.

   - Need to build libstdc++ to get source files in the right locations (in the
     build directory). The layout differs from that in the gcc source dir, and
     perhaps files are generated as well.

   - Install gcc-4.6.3 dependencies:
     $ sudo apt-get install libgmp3c2 libmpfr-dev libmpc-dev libc6-dev flex bison

   - Look into 'apt-get source' as an alternative. Might include patches, etc.

 * Need to map paths in libstdc++6-4.6-dbg's debug info to the source directory.
   Absolute paths can be fixed with 'set substitute-path'. How do you deal with
   relative paths?

   - Relative paths use per-object-file directory (DW_AT_comp_dir; see also DW_AT_name).

Debug info on Gentoo:

 * http://www.gentoo.org/proj/en/qa/backtraces.xml
 * installsources, splitdebug
 * FEATURES=installsources, FEATURES=nostrip
 * /usr/debug
 * debugedit fixes up paths

File I/O optimization:

 * posix_fadvice and linked man pages

C89 spec:

 * http://flash-gordon.me.uk/ansi.c.txt
 * http://web.archive.org/web/20050207005628/http://dev.unicals.com/papers/c89-draft.html

iostream types:

 * off_type is streamoff for char/wchar_t specializations. For example,
   basic_istream<char/wchar_t>::off_type =
     traits::off_type                    =
     char_traits<char/wchar_t>::off_type =
     streamoff

 * pos_type is either streampos or wstreampos by the same logic, but these
   are guaranteed to be the same by 27.2:

   - streampos = fpos<char_traits<char>::state_type> = fpos<mbstate_t>
   - wstreampos = fpos<char_traits<wchar_t>::state_type> = fpos<mbstate_t>

  Therefore, pos_type is fpos<mbstate_t>.

 * streamsize is basically ssize_t.

 * pos_type (fpos<mbstate_t>) can be offset by a streamoff (27.4.3.2). Two
   pos_type objects can be subtracted to get a streamoff.

 * streamsize and streamoff can be converted to each other. pos_type and
   off_type can be converted to each other.

 * "Stream operations that return a value type traits::pos_type return
   P(O(-1)) [fpos<mbstate_t>(streamoff(-1))] as an invalid value
   to signal an error."

   - Okay to compare against -1 to check for error, since
     fpos<mbstate_t>(streamoff(-1)) is guaranteed to be convertible to a streamoff
     with the value -1 (27.4.3.2).

printf cheat sheet:

 * Non-'n' versions return the number of characters written excluding the
   terminating null character.

 * 'n' versions return number of characters that would have been written,
   excluding the terminating null character.

 * A negative value is returned upon failure.

 * printf and wprintf can both print wide characters. With printf, %ls is
   used.

 * (non-C99) %n$ and *n$ can be used to specify that argument n should be
   used for the value.

   - printf("%3$*2$.*1$d", 3, 5, 7) prints 7 with field width 5 and precision 3

 * Conversion specification format:
   % <flags> <field width> <precision> <length modifier> <conversion specificer>

   - flags:
     #   - Use alternate form: 0 added for 'o'; "0x"/"0X" added for 'x'/'X';
           decimal point always displayed for floating-point numbers; and
           trailing zeros not removed for g/G
     0   - Zero-pad numbers on the left. Ignored if precision is specified.
     -   - Left-justify within field. Overrides 0.
     ' ' - Put blank before positive numbers
     +   - Put '+' before positive numbers. Overrides ' '.
     '   - (non-C99) Use locale's thousand's grouping character (LC_NUMERIC
           category)
     I   - (non-C99) Use locale-specific digits

  - field width:
    - If the output has fewer characters than the field width, it is padded on
      the left with either spaces or 0's (for numbers). For left-justification,
      it is always padded with spaces.
    - Never truncates
    - '*' fetches value from next argument, which must be int

  - precision:
    - '.' followed by decimal digits
    - For integers, minimum number of digits to write
    - For floating-point numbers, significant digits to write with g/G,
      and digits to write after the radix character for other
      conversion specifiers
    - For strings, maximum number of characters to write
    - If missing or negative, taken to be zero
    - '*' fetches value from next argument, which must be int

  - length modifier:
    - Specifies length for following conversion specifier.
    hh - Signed/unsigned char  for integer conversion, signed char  for 'n'
    h  - Signed/unsigned short for integer conversion, signed short for 'n'
    l  - Signed/unsigned long  for integer conversion, signed long  for 'n',
         wint_t for 'c', wchar_t* for 's'
    ll - Signed/unsigned long long for integer conversion, signed long long for
         'n'
    L  - Long double for floating-point number
    j  - (u)intmax_t for following integer conversion
    z  - (s)size_t   for following integer conversion
    t  - ptrdiff_t   for following integer conversion

  - conversion specifier:
    - If integer 0 printed with precision 0, output is empty.
    d/i - int
    o   - unsigned int as octal
    u   - unsigned int as decimal
    x/X - unsigned int as hex with lower/uppercase characters
    e/E - double as [-]d.ddde+/-dd. Defaults to precision 6.
          e/E determines case of 'e'. Exponent has at least two
          digits.
    f/F - double without scientific notation. Defaults to precision
          6. No decimal point with precision 0. f/F determines case
          for infinity and NaN.
    g/G - double with fancy formatting. (Like either 'e' or 'f'.)
    a/A - double as hex with lower/uppercase characters
    c   - int as character. With 'l', wint_t as character
    s   - const char*. Writes at most 'precision' characters.
          With 'l', expects const wchar_t*, and precision is the
          maximum number of _bytes_ to write.
    p   - void* as if by %#<length>x
    n   - Number of characters written so far, stored into pointer location
          of type determined by length modifier
    m   - (Glibc extension) strerror(errno). Takes no argument.
    %   - Literal '%'

scanf cheat sheet:

 * Returns EOF on eof and error, and the number of input items successfully
   matched and assigned (i.e., not including suppressed assignments) otherwise.
   It's sketchy whether a matched %n increments the count; it's probably wiser
   to check if variable values changed after the scanf.

 * A sequence of whitespace characters (isspace()) matches any amount of
   whitespace.

 * (non-C99) %n$ can be used to specify the position of the argument to write
   into.

   - "%2$5[0-9]%1$d" reads at most five digit characters + null into the second
     argument and an integer into the first.

 * Conversion specification format:
   % <* (suppress assignment)> <a/m (non-c99 - allocate buffer)>
     <maximum field width> <type modifier> <conversion specifier>

   - (non-C99) With 'a' or 'm', address of pointer expected and buffer
     dynamically allocated. Only for 's' and '[' (and, for 'm', with 'c' with
     some versions). Must be free()d later.

   - The maximum field width specifies the maximum number of characters to
     read. Does not count discarded initial whitespace. The terminating null
     stored in the result is not included in the maximum field width for
     strings.

   - Type modifiers are mostly like for printf, but L can be used for long long.

   - conversion specifier:
     %              - Literal '%'. Discards initial whitespace.
     d              - Reads integer into int
     i              - Reads integer into int, interpreting initial 0x/0X as hex
                      and 0 as oct
     o              - Reads octal integer into int
     u              - Reads integer into unsigned int
     x/X            - Reads hex integer into unsigned int
     a(C99)/f/e/g/E - Reads floating-point number into float
     s              - Reads string + null into char[]. Stops at whitespace.
     c              - Reads <maximum field width> (default 1) characters into
                      char[]. No null is appended. Does not skip leading
                      whitespace.
     [<chars>]      - Reads nonempty sequence of characters from <chars> into
                      char[]. Adds a terminating null. Does not skip leading
                      whitespace. Exclusive if first character is ^. To include
                      literal ']', make it first character after [ or initial
                      ^. Hyphen can be used for range; make last character to
                      include literally.
     p              - Reads void pointer as printed by printf with same
                      conversion specifier
     n              - Writes the number of characters consumed so far into int

GCC and Windows:

  * GCC traditionally supports two exception handling models: DW2 (dwarf2,
    table-based) and SJLJ (longjmp-based). The exception model to use is picked
    when building GCC (--enable-sjlj-exception, the absence of which seems to
    imply DW2); there are no runtime flags to pick one.

  * DW2 exceptions can't propagate through foreign code on Windows. Throwing an
    exception from a callback called from foreign code and attempting to catch
    it at the point of passing the callback's address would be broken for
    example. DW2 exceptions can cross executable boundaries as long as both
    executables use DW2 however.

  * MinGW has been forked into two releases: the mingw.org releases and
    mingw-w64.

    - The mingw.org releases use DW2 and are 32-bit Windows only.

    - mingw-w64 uses SJLJ on 32-bit Windows and (beginning with GCC 4.8)
      SEH-based table-driven exceptions on 64-bit Windows. SEH-based
      table-driven exceptions are patent encumbered on 32-bit Windows: "kietz:
      That we don't provide support for SEH 32-bit is mainly caused by
      patent-issue with Borland (or whatever it is now). July 2014 patent will
      end and then we might consider to implement."

    - mingw-w64 can use multilib, but it's not recommended.

    - mingw-w64 installation:
      http://sourceforge.net/apps/trac/mingw-w64/wiki/Downloading%20and%20installing%20MinGW-w64

  * Exception notes from #mingw on Freenode:

    jon_y: dw2 can cross boundaries, only if you used the libstdc++
           and libgcc dll

    jon_y: SEH is still waiting for patent to end in 2014
    jon_y: there is some SEH support in win64
    ...
    jon_y: patent covers only 32bit implementation

    Ulfalizer: is there something inherent about win64 that makes dw2 difficult
               to port, or has it just not been done?
    jon_y: I think win64 stores exception tables in xdata or pdata sections
    jon_y: something like that
    jon_y: you'll need to ask ktietz in #mingw-w64 at irc.oftc.net

    jon_y: afaik, SEH is supposed to be zero cost like dw2, but works through foreign code

  * Links (mostly exception-related):

    - Why doesn't mingw-w64 gcc support Dwarf-2 Exception Handling?
      http://sourceforge.net/apps/trac/mingw-w64/wiki/Exception%20Handling

    - GCC and exceptions on Windows
      http://gcc.gnu.org/wiki/WindowsGCCImprovements

    - General GCC configuration options (--enable-sjlj-exception)
      http://gcc.gnu.org/install/configure.html

    - libstdc++ configuration
      http://gcc.gnu.org/onlinedocs/libstdc++/manual/configure.html

    - A Crash Course on the Depths of Win32 Structured Exception Handling
      http://www.microsoft.com/msj/0197/exception/exception.aspx

    - TDM-GCC (Based on MinGW/MinGW-w64)
      http://tdm-gcc.tdragon.net/

  * libstdc++ in MinGW calls through to MSVCRT on Windows. Cygwin uses Newlib
    on top of their syscall emulation.

  * MinGW adds a layer on top of MSVCRT to fix things up:
    Zao: It's roughly a libc, implemented in terms of another runtime
    Zao: There's a lot of translation, mangling and other adjustments to get a
         usable runtime out of it
    Zao: Particularly if you want reasonable encoding out of fopen args, and
         stuff like that.
    Zao: Or something like the MS vsnprintf-alikes, which do not behave in a
         standard manner.

  * Programs don't link directly against DLLs on Windows. Instead, they link
    against a (static) "import library" corresponding to the DLL (with a .lib
    suffix). .lib files are not universal and documented, so mingw-(w64) needs
    to to use its own import libraries. import libraries can be generated by
    first running gendef on the DLL to get a .def file, from which the import
    library can be generated using dlltool:

    http://sourceforge.net/apps/trac/mingw-w64/wiki/Answer%20generation%20of%20DLL%20import%20library

    The corresponding msvc facility is 'lib.exe /DEF:'.

    Choice quotes:
    NightStrike: gendef gives you the def file that dlltool can use to make the libX.a
    Zao: tl;dr - if you want to use a system DLL, either use one of our import
         libraries or gendef+dlltool it. If you want to use a third party
         shared library, gendef+dlltool it or build from scratch.

  * Programs might link statically against libc on Windows, and there are
    multiple versions of the DLLs:
    http://stackoverflow.com/questions/2766233/what-is-the-c-runtime-library

C and C++ type sizes in the standards:

  * The (minimum) number of bytes in the object representation of different
    types is not explicitly defined. (It wouldn't make much sense since the
    byte size isn't defined.) It can be inferred from the ranges required to be
    supported.

  * numeric_limits<> (<limits>) refers to <climits> macros.

  * Inferred sizes for integer types (C99, 5.2.4.2.1):

    - char     : 1+ ((U/S)CHAR_MIN/MAX)
    - short    : 2+ ((U)SHRT_MIN/MAX)
    - int      : 2+ ((U)INT_MIN/MAX)
    - long     : 4+ ((U)LONG_MIN/MAX)
    - long long: 8+ ((U)LLONG_MIN/MAX)

  * Misc. <climits> macros

    CHAR_BIT   - Bits in char
    MB_LEN_MAX - Maximum number of bytes in multibyte character for any
                 supported locale

  * Test if char is signed:
    #include <limits.h>

    #if CHAR_MAX == SCHAR_MAX
    /* signed */
    #else
    /* unsigned */
    #endif

  * De facto standards:

    32-bit *nix/Windows: 32-bit int, 32-bit long
    64-bit *nix        : 32-bit int, 64-bit long
    64-bit Windows     : 32-bit int, 32-bit long

stdint.h types and macros (C99):

  * (u)int_leastN_t - Smallest type of width at least N
    (u)int_fastN_t - ~Fastest type of width at least N
    N = 8, 16, 32, 64 required for both

  * (u)intptr_t - Convertible from/to *void. Enables arithmetic. 2+ bytes.

  * (u)intmax_t - Can hold any (un)signed value representible by other
                  types. 8+ bytes.

  * Limit macros (C++ (only 03?) requires __STDC_LIMIT_MACROS to be defined
    before stdint.h is included for these):
    - (U)INT<N>_MIN/MAX - Constant across implementations. Assumes 2's
                          complement.
    - (U)INT_LEAST<N>_MIN/MAX
    - (U)INT_FAST<N>_MIN/MAX
    - (U)INTPTR_MIN/MAX
    - (U)INTMAX_MIN
    - PTRDIFF_MIN/MAX (implies ptrdiff_t is 2+ bytes)
    - SIG_ATOMIC_MIN/MAX
    - SIZE_MAX (implies size_t is 2+ bytes)

  * Constant macros (C++ (only 03?) requires __STDC_CONSTANT_MACROS to be defined
    before stdint.h is included for these):
    - (U)INT<N>_C(value) expands to literal of type (u)int_least<N>_t
    - (U)INTMAX_C(value) expands to literal of type (u)intmax_t

Github:

  * Pull requests are available as refs (that aren't fetched automatically) on
    the repo on which the pull request is made. To get them, you can do e.g.
    'git fetch origin refs/pull/4/head'.

GDB:

  * Attaching to process that is not a direct parent of the current process
    requires 'echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope' on
    Ubuntu:
    http://askubuntu.com/questions/143561/why-wont-strace-gdb-attach-to-a-process-even-though-im-root

Python:

  * __pypy__.builders.{String,Unicode}Builder

ASM snooper plugin:

  * Use dwarfdump?

ALSA:

  * Seems the File plugin won't work unless the period size is set explicitly.

  * Parameters to plugins (supplied in the device string) can be found in
    /usr/share/alsa/alsa.conf . Parameters can be named. Examples:

      file:foo,raw
      file:FILE=foo,FORMAT=raw

    Relevant doc: http://www.alsa-project.org/alsa-doc/alsa-lib/confarg.html

Memory dumping:

  * http://serverfault.com/questions/173999/dump-a-linux-processs-memory-to-file

libav:

  * Ubuntu 13.10 uses libav 0.8, corresponding to 4b63cc18bc.

  * Examples: libavcodec/api-example.c, libavformat/output-example.c

  * opt_default() receives codec-specific options such as 'preset' and 'tune'
    and puts them in a global AVDictionary 'codec_opts'. This is then used to
    initialize ost->opts, which is passed to avcodec_open2(). avcodec_open2()
    processes the codec-specific options and passes them to X264_init() in
    the X264Context instance avctx->priv_data.

  * ffmpeg options without arguments are always booleans.

  * 'formats' lists available container formats, 'codecs' available codecs.
    'help' lists codec-specific options.

  * The AVCodec struct for x264 is ff_libx264_encoder. Its 'defaults' member
    is used to initialize the AVCodecContext in
    avcodec_get_context_defaults3(), which is called from
    avcodec_alloc_context3(), which is called from avformat_new_stream() if a
    codec is supplied.

  * Encoding for YouTube:

    - Command: ffmpeg -i input -c:v libx264 -preset slow -crf 18 -c:a copy -pix_fmt yuv420p output.mkv

      - 'codec' and 'c' are synonymous. 'v' is a stream specifier in 'c:v',
        matching all video streams ('c:v:0' would select only the first video
        stream).

      - 'preset', 'tune', and 'crf' are codec-specific options - passed in an AVDictionary.

      - 'pix_fmt' is a generic option.

Linux Standard Base:

  * lsb_release -v: LSB version against which the system is compliant.

  * lsb_release -rs: Short release number string

Custom Ubuntu kernel:

  * Can use https://wiki.ubuntu.com/KernelTeam/GitKernelBuild to generate
    .deb packages instead

  * Building and installing:

    - make menuconfig - also pulls in (distribution-provided) defconfig

    - make localmodconfig -- disables modules not currently loaded. Probably
      only good for reducing the initramfs size.

    - make && make modules_install && make install

      - 'make install' makes use of the distribution-provided installkernel
        utility, which also builds an initramfs (meaning the modules must
        already be installed) and adds a Grub menu entry.

  (The steps below are redundant with 'make install', but just for
  reference.)

  * Creating the initramfs:

    - sudo update-initramfs -c -k 4.1.0-custom

    - Placed in /boot/initrd.img-4.1.0-custom

  * Grub configuration:

    - https://help.ubuntu.com/community/Grub2

    - update-grub uses /etc/default/grub and /etc/grub.d/ to generate
      /boot/grub/grub.cfg

    - Add a file like /etc/grub.d/11_my_custom_kernel, with contents like
      the following (derived from the configuration files above):

      #!/bin/sh
      cat <<END
      menuentry 'Custom kernel' {
          recordfail
          load_video
          gfxmode \$linux_gfx_mode
          insmod gzio
          insmod part_msdos
          insmod ext2
          set root='hd0,msdos1'
          if [ x\$feature_platform_search_hint = xy ]; then
            search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  506306c8-9bc3-4315-a580-42e51661fcab
          else
            search --no-floppy --fs-uuid --set=root $GRUB_DEVICE_UUID
          fi
          linux /boot/custom root=UUID=$GRUB_DEVICE_UUID ro \$vt_handoff
          initrd /boot/initrd.img-4.1.0-custom
      }
      END

git format-patch:

  * 'git format-patch -M -n -s -o outgoing origin/master' is the recommended
    Buildroot format

  * -M: detect renames

  * -n: number patches even if there's only one

  * -s: add Signed-off-by line

  * -o <dir>: add formatted patches to <dir>

  * --in-reply-to=Message-Id: Make the first mail (or all the mails with
    --no-thread) appear as a reply to the given Message-Id, which avoids
    breaking threads to provide a new patch series

  * origin/master: add commits from (but not including) the first shared
    ancestor commit of origin/master and master up to the tip of master

git send-email:

  * sudo apt-get install git-email

  * Can optionally take the same parameters as git format-patch instead of
    specifying a directory

  * --from=<address>: sendemail.from is used if this isn't specified

  * --to a@a --to b@b

  * --(b)cc a@a --(b)cc b@b

  * --in-reply-to=Message-Id: Send emails as replies to the given Message-Id

  * --compose: add an introductory message for patch series. --subject
    specifies its subject.

  * [sendemail]
            smtpencryption = tls
            smtpserver = smtp.gmail.com
            smtpuser = yourname@gmail.com
            smtpserverport = 587

Patch workflow:

  * git format-patch --cover-letter -v2 -s -o patches origin/master

  * git send-email --chain-reply-to --to <foo@bar> --cc <baz@foo> patches

  * Combined version:

    - git send-email --annotate --chain-reply-to --to ... origin/master -v2 -s --cover-letter

email notes:

  * RFCs: http://www.lsoft.com/manuals/Maestro/2.1/Users/WebHelp/Appendix_D_Email_Related_RFCs.htm

  * Header fields: http://ietf.org/rfc/rfc2822.txt

Kernel documentation:

  * Documentation/development-process/

  * https://www.kernel.org/doc/

  * https://www.kernel.org/doc/ols/2008/ols2008v2-pages-7-18.pdf

  * http://kerneltrap.org/Linux/Decoding_Oops

  * Full history: http://www.kernel.org/pub/linux/kernel/people/landley/linux-fullhist.tar.bz2

  * make htmldocs - Documentation/DocBook/index.html

  * make installmandocs

  * Documentation/kbuild/makefiles.txt has a step-by-step overview of
    the build process

Kernel debugging:

  * CONFIG_DEBUG_FS

    - Likely to already be compiled in and mounted on /sys/kernel/debug

    - mount -t debugfs none /sys/kernel/debug

    - Documented in  Documentation/DocBook/filesystems/debugfs.html

  * CONFIG_DYNAMIC_DEBUG (dyndbg)

    - Uses debugfs

    - Documentation/dynamic-debug-howto.txt

    - Dynamic control of pr_debug(), dev_dbg(), print_hex_dump_{debug,bytes}()

    - /sys/kernel/debug/dynamic_debug/control

  * Documentation/development-process/4.Coding has a list of debugging
    tools and techniques

  * lockdep

    - Kernel hacking -> Lock Debugging

  * sparse

    - Documentation/sparse.txt

    - Checks e.g. __user pointer verification

    - Can also do type handling as well as locking checks

    - 'make C=1' runs sparse on recompiled C files, 'make C=2' runs sparse on C
      files regardless of whether they need to be recompiled or not

  * Documentation/fault-injection/fault-injection.txt

  * Coccinelle

    - sudo apt-get install coccinelle (suggested by $ spatch)

    - make coccicheck

Kernel messages:

  * <linux/printk.h>

    - pr_debug/notice/info/warn/err(), etc.

      - pr_debug() is compiled out unless DEBUG is defined or
        CONFIG_DYNAMIC_DEBUG set

    - Documentation/printk-formats.txt

  * <linux/device.h>

    - dev_err/warn/info/dbg/vdgb(), etc.

      - These also include device-specific information

Kernel modules:

  * Documentation/kbuild/{modules,makefiles,kbuild}.txt

  * Installed into /lib/modules/$(KERNELRELEASE)/ by 'make modules_install'.
    $(KERNELRELEASE) matches $(uname -r) when running the kernel.

  * /lib/modules/$(uname -r)/ contains modules for a particular kernel, along
    with the following files, many of which are generated by depmod(8):

    - build: symlink to kernel directory

    - modules.alias: aliases specified directly by the module (?)

    - modules.builtin: built-in kernel modules

    - modules.dep: module dependencies

    - modules.devname: special device names provided by modules

    - modules.order: The order modules appear in makefiles. Used by modprobe
      to resolve aliases that match multiple modules.

    - modules.softdep: module soft dependencies

    - modules.symbols: symbols provided by modules

  * EXPORT_SYMBOL_(GPL)() makes a symbol visible to modules

  * Utilities (all symlinks to kmod):

    - lsmod(8): currently loaded modules (via /proc/modules)

    - modinfo(8): prints information about a module from
      /lib/modules/$(uname -r). An alternative kernel /lib/modules/
      subdirectory can be specified with -k.

    - insmod(8): insert module by filename

    - rmmod(8): remove module by name

    - modprobe(8): insert/remove module with dependencies

      - Looks in /lib/modules/$(uname -r) (unless overriden with -S)

      - Inserts by default, removes with -r. Removing also removes unused
        dependencies.

      - By default, modprobe is quiet if nothing needs to be done.
        --first-time changes this.

      - --show-depends shows dependencies of a module.

      - modprobe.d(5) contains configuration files

        - 'alias' gives an alternate name for a module

        - 'install' runs custom commands to insert a module. Deprecated due
          to complicating automatic dependency determination.

        - 'options' gives (additional) options to pass to the module when
          loading it

        - 'remove' is similar to 'install' for custom removal

        - 'softdep' gives modules to attempt to install or remove before
          inserting the module

  * Compilation flags

    - Documentation/kbuild/makefiles.txt

    - e.g., 'ccflags-$(DEBUG) := -DDEBUG' and 'make DEBUG=y' to enable
      pr_debug() output. This applies to all module sources in the directory.

    - The subdir- variants, e.g. subdir-ccflags-y, also apply to subdirectories

    - CFLAGS_foo.o applies to a specific file

USB:

  * Host/device (master/slave) architecture. The host is responsible for all
    data transfers over the bus. Devices can only signal that they need
    attention. This allows devices to be kept simple.

  * A single physical device can expose multiple logical devices, called
    device functions. Such a physical device is called a composite device.

    - A connection from the host to a particular logical device is called a
      pipe. The logical device is the endpoint. Devices lists their endpoints
      during enumeration.

    - A message pipe is for control (simple commands, etc.) transfer.

    - A stream pipe is a unidirectional pipe used for data transfers in one
      of three modes:

      - Isochronous tranfer: fixed rate, possibly lossy

      - Interrupt transfers: quick response, bounded latency, e.g. for input
        devices

      - Bulk transfer: large reliable transfers with few latency requirements

    - An endpoints is addressed with a (device_address, endpoint_number) tuple.
      Endpoint 0 is used for for device configuration.

    - To initiate data transfer, the host sends an IN or OUT packet
      (specializations of a TOKEN packet) with the target's tuple

  * Device classes define the functionality of an USB device. Communicated
    to the host to load appropriate drivers.

  * A and B connector ends

    - Only A end can provide power. A standard-A plug is usually plugged into
      the host.

  * Data transfer modes: Low/Full/High/Super/Super+. Super+ is only supported
    by USB 3.0.

  * USB On-The-Go

    - Can act both as host and device

  * Versions:

    - 1.0 1996 Low Speed (1.5 Mb/s), Full Speed (12 Mb/s)
    - 1.1 1998
    - 2.0 2000 High Speed (480 Mbit/s)
    - 3.0 2008 SuperSpeed (5 Gb/s)
    - 3.1 2014 SuperSpeed+ (10 Gb/s)

    - USB Type-C (reversible plug)

  * Documentation/usb/hotplug.txt

  * Simple USB driver: drivers/usb/misc/usbled.c

  * USB keyboard driver: drivers/hid/usbhid/usbkbd.c

  * USBDEVFS_RESET (DocBook/usb/usbfs-ioctl.html) might be able to simulate an
    unplug/replug

udev(7) notes:

  * 'u' is for 'userspace', in contrast with devfs, which ran in the kernel

  * Handles permissions, userspace notification, and persistence (via symlinks,
    e.g. /dev/disk/by-uuid/*)

  * Actual node creation is handled by devtmpfs these days

  * Now bundled as part of systemd

  * Receives notifications from the kernel via netlink

  * Rules:

    - System-wide: /lib/udev/rules.d
    - Runtime: /run/udev/rules.d
    - Local administration: /etc/udev/rules.d

  * mdev is a lightweight alternative from BusyBox

  * Hotplug module loading is probably handled by

    ENV{MODALIAS}=="?*", RUN{builtin}+="kmod load $env{MODALIAS}"

    in /lib/udev/rules.d/80-drivers.rules

Copying data to/from userspace:

  - include/linux/uaccess.h (u for userspace)

  - copy_from_user() might perform a partial read. See commit d7026e0e43
    (libfs: make simple_read_from_buffer conventional).

  - access_ok() might only check that the range lies below TASK_SIZE, and the
    'type' argument (VERIFY_READ/VERIFY_WRITE) seems to be ignored nowadays.

Kernel development process:

  - Each major release is followed by a new development cycle that starts out
    with an approximately two-week merge window during which new patches are
    integrated into the mainline kernel. Top-level maintainers ask Linus to
    pull patches during this period.

  - At the end of the merge window, an -rc1 kernel is released. Stabilization
    work is done from there on with patches that fix problems.

  - A typical series gets up to somewhere between -rc6 and -rc9 before the
    kernel is considered sufficiently stable and is released. The patch rate
    slows down over time.

  - Not introducing regressions into new kernel releases is prioritized, though
    most kernels go out with a handful of known regressions to avoid delaying
    the release and swamping the next merge window

  - Stable releases are maintained by the "stable team", currently consistent
    of Greg Kroah-Hartman

  - Significant fixes can be added to stable releases. They are often pulled
    from the next development cycle. In 3.x.y, y is the stable release number.

  - linux-next (http://www.kernel.org/pub/linux/kernel/next/) is a snapshot of
    what the kernel is expected to look like after the next merge window.

    * https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git

      * https://www.kernel.org/doc/man-pages/linux-next.html explains how to
        add it as a remote tracking branch. A next-master could be added via
        'git branch next-master linux-next/master'.

  - drivers/staging/ contains drivers and filesystems that are on their way
    into the kernel but need more work

Filesystems:

  - Documentation/filesystems/{Locking,seq_file.txt,vfs.txt}

  - Default behavior for different file_operations:

    * llseek: Defaults to no_llseek when NULL, which returns -ESPIPE.
      noop_llseek can be used instead to let seeks succeed but do nothing.
      default_llseek updates 'f_pos' as expected. generic_file_lseek is for
      "normal local filesystems".

    * open: Simply skipped if NULL - opening the file is still possible

  - open()

    * Translate arguments into open_flags via build_open_flags()

      * MAY_WRITE, etc., are particular permission checks that will
        need to be performed later

    * Create a struct filename from the filename via getname()

    * Allocate a file descriptor with get_unused_fd_flags() (properly read as
      [get_unused_fd]_flags). The 'flags' argument is only for O_CLOEXEC, which
      applies to the file descriptor rather than the open file description.

    * Create a struct file (open file description) via do_filp_open()

    * Path to file_operations::open():

      file_operations::open()
      do_dentry_open()
      vfs_open()
      do_last() (-> handle_truncate())
      path_openat()
      do_filp_open()
      do_sys_open()
      sys_open()

    * handle_truncate to i_size_write():

      i_size_write()
      truncate_setsize()
      simple_setattr()
      notify_change() (protected by i_mutex)
      do_truncate()
      handle_truncate()

  - generic_file_read_iter()

    * Uses the page cache

    * Seems to assume address_space_operations::readpage() is set

  - ramfs implementation:

    * init_ramfs_fs() calls register_filesystem(&ramfs_fs_type), where
      ramfs_fs_type.mount = ramfs_mount

    * ramfs_mount() calls mount_nodev(..., ramfs_fill_super)

    * ramfs_fill_super():

      * Call save_mount_options(), which is required when using
        generic_show_options()

      * Allocate a ramfs_fs_info and stores it in super_block's s_fs_info.
        Only seems to hold the default mode, which can be configured via
        command-line options (ramfs_parse_options()).

      * Initialize various super_block fields, including setting s_op to
        ramfs_ops

      * Call ramfs_get_inode() to get an inode for the root

        * Create a new inode and initialize it, including setting

            inode->i_mapping->a_ops = &ramfs_aops;

          and

            inode->i_op = &ramfs_file_inode_operations;
            inode->i_fop = &ramfs_file_operations;

          for regular files and

            inode->i_op = &ramfs_dir_inode_operations;
            inode->i_fop = &simple_dir_operations

          for directories

sysfs:

  - Documentation/kobject.txt

  - Documentation/filesystems/sysfs.txt

  - Documentation/sysfs-rules.txt

  - Chapter 17 of Linux Kernel Development

  - Chapter 14 of Linux Device Drivers

  - samples/kobject/{kobject,kset}_example.c

  - Documentation/ABI/**/sysfs-*

  - Kobjects map to directories, 'attribute's to files

  - kobj_type - passed e.g. to kobject_init() - holds default attributes (sysfs
    files) in its default_attrs field, and attribute operation callbacks in the
    sysfs_ops field

  - sysfs_create_file(kobj, attr) adds a new attribute to kobj. kobj's
    sysfs_ops (from its 'kobj_type ktype') must be able to handle the new
    attribute

  - attribute
    ---------
    const char *name
    umode_t mode
           .
          /|\
           |
           +-----------------------------------+-------+---- ... (lots more)
           |                                   |       |
    kobj_attribute                             | device_attribute
    --------------                             | ----------------
    show(kobject, kobj_attribute, buf)         | show(device, device_attribute, buf)
    store(kobject, kobj_attribute, buf, count) | store(device, device_attribute, buf, count)
                                               |
           +-----------------------------------+
           |                                   |
           |                              bus_attribute
    bin_attribute                         -------------
    -------------                         show(bus, buf)
    size                                  store(bus, buf, count)
    private
    read(file, kobject, bin_attribute, char*, loff_t, size_t)
    write(file, kobject, bin_attribute, char*, loff_t, size_t)
    mmap(file, kobject, attr, vma)

  - attribute_group
    ---------------
    name
    is_visible(kobject, attribute, int)
    attribute *attrs[]
    bin_attribute *bin_attrs[]

  - samples/kobject/kobject-example.c

    * kobject_create_and_add("kobject_example", kernel_kobj) is short for

      kobj = kobject_create();
      kobject_add(kobj, kernel_kobj, "%s", "kobject_example");
      return kobj;

      * kobject_create() is short for

        kobj = kzalloc(sizeof(*kbobj), GFP_KERNEL);
        kobject_init(kobj, &dynamic_kobj_type);
        return kobj;

      * kobj_type dynamic_kobj_type = {
            .release = dynamic_kobj_release,
            .sysfs_ops = &kobj_sysfs_ops
        };

      * sysfs_ops kobj_sysfs_ops = {
            .show = kobj_attr_show,
            .store = kobj_attr_store
        };

      * kobj_attr_show(kobject *kobj, attribute *attr, char *buf)

        and

        kobj_attr_store(kobject *kobj, attribute *attr, char *buf, size_t count)

        upcast attr to to kobj_attribute and chain through to the
        show()/store() method defined there.

    * sysfs_create_group(example_kobj, &attr_group);

      * Loops over each attribute in attr_group.attrs (in create_files()),
        calling sysfs_add_file_mode_ns() on each

      * Can optionally create a separate directory and specify some visibility
        stuff and binary attributes, but it's not used here

      * SYSFS_PREALLOC

        * Set by __ATTR_PREALLOC(). Only seems to be used by drivers/md/md.c

        * Sets 'prealloc' in struct kernfs_ops to true

        * fs/kernfs/file.c

  - samples/kobject/kset-example.c

    * kset_create_and_add("kset_example", NULL, kernel_kobj) is short for

      kset = kset_create("kset_example", /* kset_uevent_ops */ NULL, kernel_kobj);
      kset_register(kset);
      return kset;

      * kset_create() is short for

        kset = kzalloc(sizeof(*kset), GFP_KERNEL);
        kobject_set_name(&kset->kobj, "%s", "kset_example");
        kset->kobj.parent = kernel_kobj;
        kset->kobj.ktype = &kset_ktype;
        kset->kobj.kset = NULL;
        return kset;

      * kset_register(kset) is short for

        kset_init(kset);
        kobject_add_internal(&kset->kobj);
        kobject_uevent(&k->kobj, KOBJ_ADD);
        return 0;

    * create_foo_obj("foo") is short for

      foo = kzalloc(sizeof(*foo), GFP_KERNEL);
      foo->kobj.kset = example_kset;
      kobject_init_and_add(&foo->kobj, &foo_ktype, NULL, "%s", "foo");
      kobject_uevent(&foo->kobj, KOBJ_ADD);
      return foo;

/proc:

  - Documentation/filesystems/proc.txt has some stuff that's not in proc(5)

  - sysctl(8) simply accesses /sys/

  - prctl(2) controls various other process-related stuff

  - /<pid>/ files:
    * attr/: selinux-related - the source for proc(5) has comments
    * autogroup: the source for the proc(5) man page has comments -
      CONFIG_SCHED_AUTOGROUP
    * auxv: contents of the auxiliary vector passed to processes by the
      kernel's ELF binary loader - getauxval(3)
    * cgroup: the cgroup, from each hierarchy, that the process is in
    * clear_refs: clears the PG_Referenced and ACCESSED/YOUNG bits for
      different types of mappings, as seen through e.g. smaps
    * cmdline: the execve() command line for the process
    * comm: the command name associated with the process. Maybe this holds
      thread names too?
    * coredump_filter: controls which segments are written to core dumps
    * cpuset: the cgroup within the cpuset hierarchy. Maybe a leftover from
      a time when cpusets weren't implemented with cgroups.
    * environ: environment block
    * exe: symbolic link containing the actual pathname of the executed
      command. Can be executed to run the same command again.
    * fd/: one entry per open file descriptor - a symbolic link to the file
      (or containing some related information for special files)
    * fdinfo/: has some more information, e.g. position, per file descriptor
    * uid_map/gid_map: user namespace mapping thingy
    * io: i/o stats - read/written bytes, number of read()/write() calls, etc.
    * limits: resource limits
    * loginuid: someting related to
      audit_get_loginuid()/CAP_AUDIT_CONTROL/CONFIG_AUDITSYSCAL
    * map_files/: symbolic links to memory-mapped files, named by virtual
      address region. Needs CONFIG_CHECKPOINT_RESTORE.
    * maps: process mappings - files and special memory areas
    * mem: provides access to the process's memory through
      open()/read()/lseek()
    * mountinfo: information about mount points in some odd format
    * mounts: currently mounted filesystem. This can vary between processes due
      to mount namespaces. /proc/mounts is a symlink to /proc/self/mounts for
      this reason.
    * mountstats: some more information about the mount points
    * ns/: contains symlinks to various namespaces for the process. Various
      magic can be used on them - namespaces(7)
    * numa_maps: information on NUMA policies and allocations for memory ranges
    * oom_adj: bit-shift applied to the oom_score value for OOM decisions
    * oom_score: higher score means higher likelhood of being selected by the
      OOM killer
    * oom_score_adj: another oom_score adjustment. Perhaps simply added.
    * pagemap: virtual-to-physical mapping (for in-RAM pages) and some flags.
      Needs CONFIG_PROC_PAGE_MONITOR. TODO: /proc/kpageflags, /proc/kpagecount.
    * personality: the personality of the process. Personalities alter some
      behaviors to provide compatibility for programs written for other *nixes.
      See personality(2) and 973f911f ("Remove execution domain support") as
      well.
    * projid_map: seems to be something related to per-"project" disk quotas
    * root: symbolic link to the process's root (changeable with chroot(2))
    * sched/schedstat: "Displays various scheduling parameters" according to
      comment in proc.5.gz
    * sessionid: ?
    * setgroups: seems to indicate whether setgroups(2) is allowed for the
      process. Also settable.
    * smaps: shows memory consumption for each mapping. pmap(1) can be used
      too. Non-file-backed mappings can be dirty too, meaning the version in
      swap is outdated and would need to be updated.
    * stack: kernel stacktrace for the process. Needs CONFIG_STACKTRACE.
    * stat: status information. Used by ps(1).
    * statm: memory usage info
    * status: human-readable version of stat and statm
    * syscall: number and argument registers for the currently executing
      syscall (if any)
    * timers: list of POSIX timers for the process
    * wchan: the symbolic name corresponding to the location in the kernel
      where the process is sleeping

Terms and acronyms:

  - BW: bandwidth
  - CFQ: Completely Fair Queuing
  - fsnotify: in-kernel backend for (d/i/fa)notify
  - GFP: Get Free Page
  - LSM: Linux Security Module
  - namei: sarnold suggested "name to inode"
  - ns: namespace
  - PFN: page frame number - page number in e.g. 4096-byte units
  - pte: page table entry
  - sd (in struct kobject): sysfs dirent