Artifact 67f0ca59a1b2993812681b23a3ad57705357f143da8d4d77d637864ad5ca75e6:

Wiki page [Laptop] by rmarrone 2019-12-11 12:00:22.
D 2019-12-11T12:00:22.436
L Laptop
N text/x-markdown
P 34114d7b182530b78601afff2460bf8443773aecf8c9e39346895f81145ea4d4
U rmarrone
W 20320
# Increasing OpenBSD's Resilience to Power Loss on a Powerful Laptop

## Introduction
I have occasionally had kernel panics when losing power while running
OpenBSD on FFS on softraid CRYPTO on cheap SSDs. The loss of power would
occur either because a laptop battery ran out of power, a laptop battery
fell out (from a ThinkPad X201), or a desktop power cable became unplugged.

After the initial panic, the system would continue to panic after
I would try to access certain parts of the softraid disk.

I never really figured out what the problem was, just that it happened
with this pattern. Rather than fixing it, I got around this issue
by switching one of my computers to FreeBSD
and using GELI and ZFS. (There were also many other reasons
for switching the operating system.)

For my laptop, however, I wanted OpenBSD, as OpenBSD officially supports
its hardware (ThinkPad X240) and OpenBSD has worked much better with this
hardware than FreeBSD has, in my experience.

## Related work
I took inspiration from Marko Cupańá's method on
[How to Increase OpenBSD's Resilience to Power Outages](
Increased resilience is achieved when everything is mounted either
as read-only or as in-memory filesystems.

Marko mentions that he uses this technique for network appliances, presumably
with SD card storage "(such as branch office routers in godforsaken places
where having electricity and Internet access at all is considered a
lucky circumstance)" and he documents an example of a PC Engines apu3b4.

## Present application
Unlike Marko's situation, my situation is a general-purpose laptop with SSD
storage that I mostly use to log in to the other computer and that I sometimes
write things on temporarily when I don't have an internet connection.

It happens that I have pretty good internet connectivity where I am, so
I store most data on the main computer and access it through mosh.
On the other hand, I do want to be able to store some things locally
as I sometimes have periods of a few minutes without internet access and
I sometimes turn off the other computer. I also wanted to keep it closer
to standard OpenBSD so that configuration would be easier for me.

I am mainly trying to prevent corruption of the softraid CRYPTO disk.

## Differences from Marko's configuration
I changed some things to account for the differences in method of use
between Marko and me.

### Root filesystem
I decided to mount most things read-only or in memory but to keep
/ as read-write. I kept / read-write for these reasons.

* This is the most annoying one to set up as read-only. I did not want
  to spend the time on that, and I did not trust myself to do it properly.
* I frequently want to edit files in /etc, at least right now while
  I am still setting things up.
* OpenBSD has the /altroot feature.
* I am not really concerned in part about the hardware's lifespan.
  Marko is, because he uses SD card storage and because it is
  inconvenient for him to travel to wherever the computer is and replace
  the SD card when it goes bad. I am using an SSD rather than an SD
  card, and I don't turn the computer on very much, so I am not worried
  about lifespan of the storage. Furthermore, I have the computer
  with me, so it will be easy to replace the SSD when it goes bad.

### Extra partitions
While Marko puts /var/syspatch in its own non-memory partition, I kept
/var/syspatch in memory, as I can have up to 8 GB of memory in this
laptop. If the size does become an issue, maybe I will move it later.

I did create a read-only /opt partition for large files. So far I have
used it only for a database dump and a corresponding search index.
I search the database frequently and update the database infrequently.

### Frequency of saving
Marko runs a daily cron job to save the contents of the in-memory
filesystem to the SD card. I use an hourly cron job, because

* My edits to the system are more important. More than logging and
  caching, they include things like the present report.
* I am not worried about SD card wear.

I also explicitly save periodically.

## Method
I mostly followed Marko's "Software", "Installation", and "First
single-user mode boot" steps.

I did something similar to "First standard boot", but it is different
enough to warrant explanation.

* I wrote something similar to /root/, but it is shorter,
  and it is saved directly as /etc/rc.shutdown.
* I installed /etc/rc.shutdown in several other places as well.
* I did not modify rc.local, as I did not need to mount / read-only.

## Final configuration
The final configuration can be summarized by the currently-mounted
partitions, the disklabels, and the contents of a few important files.

I am running OpenBSD 6.6 on a ThinkPad X240. It has a ~256GB SSD on
sd0, softraid CRYPTO on sd2, and a keydisk for the softraid CRYPTO
on sd1. My filesystems look like this at boot.

    $ mount
    /dev/sd2a on / type ffs (local)
    /dev/sd2f on /mfs type ffs (local, nodev, nosuid, read-only)
    /dev/sd2d on /usr type ffs (local, nodev, read-only)
    /dev/sd2e on /usr/X11R6 type ffs (local, nodev, read-only)
    /dev/sd2g on /usr/local type ffs (local, nodev, wxallowed, read-only)
    mfs:8611 on /tmp type mfs (asynchronous, local, nodev, nosuid, size=262144 1K-blocks)
    mfs:15822 on /var type mfs (asynchronous, local, nodev, nosuid, size=262144 1K-blocks)
    mfs:20765 on /home type mfs (asynchronous, local, nodev, nosuid, size=1048576 1K-blocks)
    /dev/sd2h on /opt type ffs (local, nodev, wxallowed, read-only)
    $ df -h
    Filesystem     Size    Used   Avail Capacity  Mounted on
    /dev/sd2a      123M   89.4M   27.2M    77%    /
    /dev/sd2f      6.9G    208M    6.3G     3%    /mfs
    /dev/sd2d      3.0G    955M    1.9G    33%    /usr
    /dev/sd2e     1008M    202M    756M    21%    /usr/X11R6
    /dev/sd2g     11.8G    1.6G    9.6G    14%    /usr/local
    mfs:8611       247M   46.5M    188M    20%    /tmp
    mfs:15822      247M   14.4M    220M     6%    /var
    mfs:20765      991M    190M    751M    20%    /home
    /dev/sd2h     24.3G    6.2G   17.0G    27%    /opt

At the time of writing I did not have the keydisk plugged in,
so I report the disklabels only of the SSD and softraid disk.

    $ disklabel -p G sd0
    # /dev/rsd0c:
    type: SCSI
    disk: SCSI disk
    label: SAMSUNG MZ7PD256
    duid: c0fd74026484bd67
    bytes/sector: 512
    sectors/track: 63
    tracks/cylinder: 255
    sectors/cylinder: 16065
    cylinders: 31130
    total sectors: 500118192 # total bytes: 238.5G
    boundstart: 64
    boundend: 500103450
    drivedata: 0 
    16 partitions:
    #                size           offset  fstype [fsize bsize   cpg]
      a:            48.0G               64    RAID                    
      c:           238.5G                0  unused                    
    $ disklabel -p G sd2
    # /dev/rsd2c:
    type: SCSI
    disk: SCSI disk
    label: SR CRYPTO
    duid: 15ea7c8e8a859e25
    bytes/sector: 512
    sectors/track: 63
    tracks/cylinder: 255
    sectors/cylinder: 16065
    cylinders: 6266
    total sectors: 100678763 # total bytes: 48.0G
    boundstart: 64
    boundend: 100663290
    drivedata: 0 
    16 partitions:
    #                size           offset  fstype [fsize bsize   cpg]
      a:             0.1G               64  4.2BSD   2048 16384  1998 # /
      b:             0.1G           257024  4.2BSD   2048 16384  1999 
      c:            48.0G                0  unused                    
      d:             3.0G           514080  4.2BSD   2048 16384 12958 # /usr
      e:             1.0G          6811552  4.2BSD   2048 16384 12958 # /usr/X11R6
      f:             7.0G          8916064  4.2BSD   2048 16384 12958 # /mfs
      g:            12.0G         23599456  4.2BSD   2048 16384 12958 # /usr/local
      h:            24.7G         48773312  4.2BSD   2048 16384 12958 # /opt

Note that the only read-write filesystem in the fstab is /.

    $ cat /etc/fstab
    15ea7c8e8a859e25.a / ffs rw 1 1
    bfb4775bb8397569.b /altroot ffs xx 0 0
    15ea7c8e8a859e25.f /mfs ffs ro,nodev,nosuid 1 2
    15ea7c8e8a859e25.d /usr ffs ro,nodev 1 2
    15ea7c8e8a859e25.e /usr/X11R6 ffs ro,nodev 1 2
    15ea7c8e8a859e25.g /usr/local ffs ro,wxallowed,nodev 1 2
    15ea7c8e8a859e25.h /opt ffs ro,wxallowed,nodev 1 2
    swap /tmp mfs rw,nodev,nosuid,-s256m 0 0
    swap /var mfs rw,nodev,nosuid,-s256m,-P=/mfs/var 0 0
    swap /home mfs rw,nodev,nosuid,-s1g,-P=/mfs/home 0 0

With the above context, I can explain the /etc/rc.shutdown.
Filesystems start out as read-only, with the memory filesystems mounted.
I may mount them read-write for whatever reason, like when I am
installing packages. In case I did not switch them back to read-only,
that happens at shutdown. After than, the memory filesystems are saved
to /mfs.

    $ cat /etc/rc.shutdown
    save() {
      local dir="$1"
      /usr/local/bin/rsync -ax --quiet --delete --no-specials --no-devices "${dir}/" "/mfs${dir}"
    # In case they are mounted read-write, mount read-only.
    for fs in /usr/local /usr/X11R6 /usr /opt; do
      mount -ur $fs
    # Save memory file systems.
    mount -uw /mfs
    save /var
    save /home
    mount -ur /mfs

It may seem unnecessary to mount the filesystems as read-only right
before shutdown. Before shutdown this is not so important, but I use
the same file in other places, and it is important in those places.

For example, I save my work periodically by typing "save" in the shell
to run /etc/rc.shutdown.

    $ fgrep /etc/rc.shutdown ~/.tcshrc
    alias save doas sh /etc/rc.shutdown

I have frequently had computers fail to suspend when
I /etc/rc.shutdown, so I have configured apmd to run it too.

    $ ls -lh /etc/apm
    total 0
    lrwxrwxr-x  1 rmarrone  wheel    16B Nov 19 11:58 hibernate -> /etc/rc.shutdown
    lrwxrwxr-x  1 rmarrone  wheel    13B Nov 19 11:58 resume -> /etc/rc.local
    lrwxrwxr-x  1 rmarrone  wheel    16B Nov 19 11:59 standby -> /etc/rc.shutdown
    lrwxrwxr-x  1 rmarrone  wheel    16B Nov 19 11:59 suspend -> /etc/rc.shutdown

The dmesg could be interesting as well.

    $ cat /var/run/dmesg.boot
    OpenBSD 6.6 (GENERIC.MP) #372: Sat Oct 12 10:56:27 MDT 2019
    real mem = 3951247360 (3768MB)
    avail mem = 3818782720 (3641MB)
    mpath0 at root
    scsibus0 at mpath0: 256 targets
    mainbus0 at root
    bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xbcd3d000 (60 entries)
    bios0: vendor LENOVO version "GIET75WW (2.25 )" date 06/24/2014
    bios0: LENOVO 20AMS1KV00
    acpi0 at bios0: ACPI 5.0
    acpi0: sleep states S0 S3 S4 S5
    acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP2(S4) XHCI(S3) EHC1(S3)
    acpitimer0 at acpi0: 3579545 Hz, 24 bits
    acpiec0 at acpi0
    acpihpet0 at acpi0: 14318179 Hz
    acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
    cpu0 at mainbus0: apid 0 (boot processor)
    cpu0: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz, 798.26 MHz, 06-45-01
    cpu0: 256KB 64b/line 8-way L2 cache
    cpu0: smt 0, core 0, package 0
    mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
    cpu0: apic clock running at 99MHz
    cpu0: mwait min=64, max=64, C-substates=, IBE
    cpu1 at mainbus0: apid 1 (application processor)
    cpu1: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz, 798.16 MHz, 06-45-01
    cpu1: 256KB 64b/line 8-way L2 cache
    cpu1: smt 1, core 0, package 0
    cpu2 at mainbus0: apid 2 (application processor)
    cpu2: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz, 798.17 MHz, 06-45-01
    cpu2: 256KB 64b/line 8-way L2 cache
    cpu2: smt 0, core 1, package 0
    cpu3 at mainbus0: apid 3 (application processor)
    cpu3: Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz, 798.16 MHz, 06-45-01
    cpu3: 256KB 64b/line 8-way L2 cache
    cpu3: smt 1, core 1, package 0
    ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 40 pins
    acpimcfg0 at acpi0
    acpimcfg0: addr 0xf8000000, bus 0-63
    acpiprt0 at acpi0: bus 0 (PCI0)
    acpiprt1 at acpi0: bus -1 (PEG_)
    acpiprt2 at acpi0: bus 2 (EXP1)
    acpiprt3 at acpi0: bus -1 (EXP2)
    acpiprt4 at acpi0: bus -1 (EXP3)
    acpicpu0 at acpi0: C3(200@506 mwait.1@0x60), C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
    acpicpu1 at acpi0: C3(200@506 mwait.1@0x60), C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
    acpicpu2 at acpi0: C3(200@506 mwait.1@0x60), C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
    acpicpu3 at acpi0: C3(200@506 mwait.1@0x60), C2(200@148 mwait.1@0x33), C1(1000@1 mwait.1), PSS
    acpipwrres0 at acpi0: PUBS, resource for XHCI, EHC1
    acpitz0 at acpi0: critical temperature is 200 degC
    acpibtn0 at acpi0: LID_
    acpibtn1 at acpi0: SLPB
    acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
    acpicmos0 at acpi0
    tpm0 at acpi0: TPM_ addr 0xfed40000/0x5000, device 0x0000104a rev 0x4e
    acpibat0 at acpi0: BAT0 model "45N1773" serial 15792 type LION oem "SANYO"
    acpibat1 at acpi0: BAT1 model "45N1137" serial   102 type LION oem "SANYO"
    acpiac0 at acpi0: AC unit offline
    acpithinkpad0 at acpi0
    "PNP0C14" at acpi0 not configured
    "PNP0C14" at acpi0 not configured
    "PNP0C14" at acpi0 not configured
    "INT340F" at acpi0 not configured
    acpivideo0 at acpi0: VID_
    acpivout at acpivideo0 not configured
    acpivideo1 at acpi0: VID_
    cpu0: using VERW MDS workaround (except on vmm entry)
    cpu0: Enhanced SpeedStep 798 MHz: speeds: 2701, 2700, 2600, 2400, 2300, 2100, 2000, 1800, 1700, 1600, 1400, 1300, 1100, 1000, 800, 756 MHz
    pci0 at mainbus0 bus 0
    pchb0 at pci0 dev 0 function 0 "Intel Core 4G Host" rev 0x0b
    inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x0b
    drm0 at inteldrm0
    inteldrm0: msi
    azalia0 at pci0 dev 3 function 0 "Intel Core 4G HD Audio" rev 0x0b: msi
    azalia0: No codecs found
    xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x04: msi, xHCI 1.0
    usb0 at xhci0: USB revision 3.0
    uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00 addr 1
    "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured
    em0 at pci0 dev 25 function 0 "Intel I218-LM" rev 0x04: msi, address 28:d2:44:e4:f9:ac
    azalia1 at pci0 dev 27 function 0 "Intel 8 Series HD Audio" rev 0x04: msi
    azalia1: codecs: Realtek ALC292
    audio0 at azalia1
    ppb0 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xe4: msi
    pci1 at ppb0 bus 2
    rtsx0 at pci1 dev 0 function 0 "Realtek RTS5227 Card Reader" rev 0x01: msi
    sdmmc0 at rtsx0: 4-bit, dma
    ehci0 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x04: apic 2 int 23
    usb1 at ehci0: USB revision 2.0
    uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
    pcib0 at pci0 dev 31 function 0 "Intel 8 Series LPC" rev 0x04
    ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x04: msi, AHCI 1.3
    ahci0: port 0: 6.0Gb/s
    scsibus1 at ahci0: 32 targets
    sd0 at scsibus1 targ 0 lun 0: <ATA, SAMSUNG MZ7PD256, DXM0> naa.5002538500000000
    sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin
    ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x04: apic 2 int 18
    iic0 at ichiic0
    spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM PC3-12800 SO-DIMM
    isa0 at pcib0
    isadma0 at isa0
    pckbc0 at isa0 port 0x60/5 irq 1 irq 12
    pckbd0 at pckbc0 (kbd slot)
    wskbd0 at pckbd0: console keyboard
    pms0 at pckbc0 (aux slot)
    wsmouse0 at pms0 mux 0
    wsmouse1 at pms0 mux 0
    pms0: Synaptics clickpad, firmware 8.1, 0x1e2b1 0x940300
    pcppi0 at isa0 port 0x61
    spkr0 at pcppi0
    vmm0 at mainbus0: VMX/EPT
    umodem0 at uhub0 port 4 configuration 1 interface 1 "Lenovo N5321 gw" rev 2.00/0.00 addr 2
    umodem0: data interface 2, has CM over data, has break
    umodem0: status change notification available
    ucom0 at umodem0
    umodem1 at uhub0 port 4 configuration 1 interface 3 "Lenovo N5321 gw" rev 2.00/0.00 addr 2
    umodem1: data interface 4, has CM over data, has break
    umodem1: status change notification available
    ucom1 at umodem1
    umb0 at uhub0 port 4 configuration 1 interface 6 "Lenovo N5321 gw" rev 2.00/0.00 addr 2
    umodem2 at uhub0 port 4 configuration 1 interface 9 "Lenovo N5321 gw" rev 2.00/0.00 addr 2
    umodem2: data interface 10, has CM over data, has break
    umodem2: status change notification available
    ucom2 at umodem2
    ugen0 at uhub0 port 4 configuration 1 "Lenovo N5321 gw" rev 2.00/0.00 addr 2
    uvideo0 at uhub0 port 8 configuration 1 interface 0 "J31E8IAE0 Integrated Camera" rev 2.00/10.04 addr 3
    video0 at uvideo0
    umass0 at uhub0 port 11 configuration 1 interface 0 "Generic USB3.0 Card Reader" rev 3.00/15.32 addr 4
    umass0: using SCSI over Bulk-Only
    scsibus2 at umass0: 2 targets, initiator 0
    sd1 at scsibus2 targ 1 lun 0: <Generic, STORAGE DEVICE, 1532> removable serial.05e30749000000001532
    sd1: 1876MB, 512 bytes/sector, 3842048 sectors
    uhub2 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev 2.00/0.04 addr 2
    vscsi0 at root
    scsibus3 at vscsi0: 256 targets
    softraid0 at root
    scsibus4 at softraid0: 256 targets
    sd2 at scsibus4 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006>
    sd2: 49159MB, 512 bytes/sector, 100678763 sectors
    root on sd2a (15ea7c8e8a859e25.a) swap on sd2b dump on sd2b
    inteldrm0: 1366x768, 32bpp
    wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
    wsdisplay0: screen 1-5 added (std, vt100 emulation)

## Final notes
I did some other things to reduce the risk of power loss and consequent
problems with softraid CRYPTO.

* I removed the wifi card, to save power.
* I purposely chose this laptop for its two batteries. This makes for long
  battery life and for a reduced risk of batteries falling out, especially
  because one of them is inside the case.
* I purposely got a laptop with an i7 processor, as those use less
  power than the other options for this laptop. I don't know if that matters so
  much though.

It was easy for me to set this up, and I have found it very convenient
to use. I am no longer worried about what will happen if I lose power
on my laptop.

Z 83bce6aaf5282868c798e56077c05b90