neitzels irrelevant irregular intarweb notes

Upgrading the RouterOS on MikroTik devices is a simple affair:

• You check for new firmware;
• if there is any, review the release notes;
• download, install and reboot;
• update the bootloader in a separate step,
• followed by a second reboot.

All this is usually done in five minutes. So far I never had any issues caused by an RouterOS update.

Which makes you buy more MikroTik gear. The youngster in my home is...

lowell.marshlabs.gaertner.de

[neitzel@lowell] > /system routerboard print 
       routerboard: yes
    board-name: hAP lite
         model: RouterBOARD 941-2nD
     serial-number: 7C2C07C4455B
     firmware-type: qca9531L
  factory-firmware: 3.36
  current-firmware: 6.46.6
  upgrade-firmware: 6.46.6

This is a cheap (22 EUR), small wireless router/switch/access point serving my kitchen. Permanently attached nodes are NCD X terminal terminal fz and the DAB+/FM/Internet/LAN-Media radio gaga.

Lowell is supposed to replace the small cisco switch lab there but, as of now, they all still share the window sill:

Updates vs. IPv6-only

I have run out of my public IPv4 addresses at home long ago. Because lowell is currently mostly just operating as an access point it doesn't need any layer-3 address except for management. And so it became my first IPv6-only node, without any IPv4 address at all.

This was all fine. Until I tried the first RouterOS upgrade.

As tcpdump showed the upgrade process will resolve the server name

download.mikrotik.com has address 159.148.172.226
download.mikrotik.com has address 159.148.147.204
download.mikrotik.com has IPv6 address 2a02:610:7501:4000::226
download.mikrotik.com has IPv6 address 2a02:610:7501:1000::196

which apparently would serve both the current and the vintage protocol flavours. The hAP though will first try an IPv4 server, notice that that network is unreachable, and... give up. What a shame!

This short-coming is particularly disappointing because the RouterOS can transfer data via IPv6 when asked manually:

[neitzel@lowell] /file> /tool fetch url="http://hackett.6.ml.gaertner.de/index.html" output=user
      status: finished
  downloaded: 0KiB
    data: All my friends and I are crazy. That's the only thing that 
          keeps us sane.

[neitzel@lowell] /file>

D'oh!

Workaround: for RouterOS updates, I temporarily /ip dhcp-client enable 0. Do the upgrade dance, and /ip dhcp-client disable 0 again.

Not nice but there are worse things in life.

Updates vs. Flash Size

Hey, let's just spend fifteen minutes on upgrading all three MikroTik gadgets, I thought around 8pm. When I went into bed, it was around 5am.

The upgrades went without a hitch on the two larger devices, billy and hall but on lowell strange things would happen. The "download" step went fine but the "reboot for install" step would end up in the same old package versions as before (6.46.2), with the download new version (6.46.6) purged from the /file area. Repeated attempts didn't help.

WLKIKIV, as we say here, and a quick /log print shows the problem:

not enough disk space.

The "hdd" flash memory is indeed much more constrained on lowell:

% echo billy hall lowell | \
> xargs -n1 -Ixx ssh xx /system resource print | \
> grep -E 'hdd|board-name'
       free-hdd-space: 107.3MiB
      total-hdd-space: 128.0MiB
           board-name: RB2011L
       free-hdd-space: 109.0MiB
      total-hdd-space: 128.0MiB
           board-name: CRS125-24G-1S-2HnD
       free-hdd-space: 7.1MiB
      total-hdd-space: 16.0MiB
           board-name: hAP lite
%

The size of the stock "combo" release packages is now approaching half of the 16.0MiB disk size:

% echo 2 6 | xargs -n1 -I X lynx -head -dump \
> https://download.mikrotik.com/routeros/6.46.X/routeros-smips-6.46.X.npk |\
> grep Length
Content-Length: 7651050
Content-Length: 7700154
%

During an upgrade, both the old and new version have to sit side-by-side on the disk, the filesystem structure needs some space, the config needs some space, ... the official documentation is asking for 2 MB spare capacity. After this download though I was down to the last 44 KB(!) on the disk.

A few months ago, in the same situation, I found a surplus support-dump I could delete to gain enough breathing space. No such luck tonight.

With the current RouterOS "Stable" images, things have now simply become too tight for a stock "hAP lite" and similar devices. without much extra config/data on its flash medium to upgrade to newer stock RouterOS versions. To be frank, this is major surprise if not a disgrace.

Me vs. the Web Forum

The official RouterOS documentation doesn't address this problem.

Grudgingly I dived into the "community support". I simply hate sifting through web fora, no matter which ones. It took hours.

Yes, I was not the only one with the problem. There were messages about the problem without any followup at all; there were quite a handful of wrong explanations; there was even a bit of ad-hominem and mud-slinging.

It did pointed me to the proper solution though:

By default, have their software installed from a "combined routeros package" which contains a selection of individual feature packages. It should not happen but the combined package can become to big for smaller platforms. You have then to switch over to deal with the packages individually, selecting you own mix.

The Solution

My first idea was to delete a few of the 6.46.2 packages which I currently don't use in order to create the space for the complete new kit.

Turns out that you cannot /system package uninstall anything when everything comes from the "combined routeros".

The only way forward is this:

1. Make an extra backup of your configuration beyond of what the automatic reboot/reset backup is providing. The commands are simple and the demands on precious flash space are small:

   [neitzel@lowell] > /system backup save
   [neitzel@lowell] > /export compact file=cfg-mn
   [neitzel@lowell] > /file print where type!=directory
    # NAME                   TYPE             SIZE CREATION-TIME       
    0 cfg-mn.rsc             script         6.4KiB may/27/2020 03:18:50
    1 auto-before-reset.b... backup        19.0KiB jan/02/1970 02:42:11
    2 lowell-20200527-031... backup        30.1KiB may/27/2020 03:14:53
   

2. Download the "Extra packages" kit matching your hardware from https://www.mikrotik.com/download.

   This kit does not contain just "extra" packages for the more obscure features as the title suggests to me. Instead, the filename is much more appropriate: all_packages-smips-6.46.6.zip. This zip contains the ten packages which comprise the "combined" = "Main" package (=routeros-smips-6.46.6.npk), and only three extra pkgs: multicast, openflow, tr069-client. (A full listing is below.)

3. Download this .zip archive elsewhere and extract the .npk packages.

4. Use scp, ftp or RouterOS' /tool fetch to copy a subset of the packages into the /file area for installation. Everybody needs the system package which weighs in with 5.5 MB alone. Another essential package for me is ipv6 (196 KB) to be able to access/manage the hAP-lite at all. dhcp might be that thing for you, and in that case you also need security (155 + 307 = 462 KB). And since security is also required for ssh access, I used that, too. These four pkgs already total at 6+ MB, enough to get nervous.

5. Reboot to install these packages.

6. You only get the few selected new packages.

   All packages/features from the old version get removed. The result is not a mix of old and updated packages.

   Your new reduced feature set will load your old configration as much as possible. Settings for now missing features will be lost. For example, without the wireless pkg, I lost my WLAN definition.

   Luckily, you didn't skip the the first step, saving your config, did you?

7. With the old version's packages gone, you have now plenty of disk space for the other new packages. Install as much as you want by copying them to the /file area and rebooting.

8. With all wanted new packages in place, you can now reload your configuration:

   /system backup load name=lowell-20200527-0314.backup
   

   or

   /import cfg-mn.rsc
   

   As of now I haven't figured out which is better in which case. I suppose that either would do for me.

I believe you can choose between these two strategies:

• Exercise some restraint and aim at "below 7 MB for everything", so that future upgrades are completely painless. The standard /system package update process should download only those packages you have in use.

  In my case, this would be: system, ipv6, wireless, security, dhcp. As of now, this already totals in 7149168 bytes aka 6.8MiB. Hrrmmm....

• If you prefer a "all packages" setup, you will have to go through the "update to/with minimal package set / add extras later" on every single update. The only ease is that you can get rid of ballast before doing the upgrade: /system package uninstall will now work. You can then do the (minimal) upgrade and re-add non-minimal packages afterwards. Again, this requires the download of the "Extras" .zip-file. And, of course, the backup of your configuration.

I am wondering how all this will pan out for me. I'll try to automate the "all packages" updates, i.e. the second approach.

Summary

For reference, here is my current lowell installation and sizes of the corresponding all_packages:

neitzel 373 > unzip -l all_packages-smips-6.46.6.zip
Archive:  all_packages-smips-6.46.6.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
    69713  05-14-20 12:14   advanced-tools-6.46.6-smips.npk
   155729  05-14-20 12:14   dhcp-6.46.6-smips.npk
   147537  05-14-20 12:14   hotspot-6.46.6-smips.npk
   196689  05-14-20 12:14   ipv6-6.46.6-smips.npk
    57425  05-14-20 12:14   mpls-6.46.6-smips.npk
    36945  05-14-20 12:14   multicast-6.46.6-smips.npk
    49233  05-14-20 12:14   openflow-6.46.6-smips.npk
   258129  05-14-20 12:14   ppp-6.46.6-smips.npk
    69713  05-14-20 12:14   routing-6.46.6-smips.npk
   307281  05-14-20 12:14   security-6.46.6-smips.npk
  5330220  05-14-20 12:14   system-6.46.6-smips.npk
   114769  05-14-20 12:14   tr069-client-6.46.6-smips.npk
  1159249  05-14-20 12:14   wireless-6.46.6-smips.npk

[neitzel@lowell] > /system package print
Flags: X - disabled
 #   NAME                    VERSION                    SCHEDULED              
 0   security                6.46.6
 1   ipv6                    6.46.6
 2   dhcp                    6.46.6
 3   advanced-tools          6.46.6
 4   system                  6.46.6
 5   wireless                6.46.6
 6   hotspot                 6.46.6
 7   mpls                    6.46.6
 8   multicast               6.46.6
 9   openflow                6.46.6
10   ppp                     6.46.6
11   routing                 6.46.6
12   tr069-client            6.46.6

[neitzel@lowell] > /system reso print
           uptime: 8h49m20s
          version: 6.46.6 (testing)
           build-time: Apr/27/2020 10:32:16
     factory-software: 6.28
          free-memory: 7.7MiB
         total-memory: 32.0MiB
              cpu: MIPS 24Kc V7.4
        cpu-count: 1
        cpu-frequency: 650MHz
         cpu-load: 0%
       free-hdd-space: 7.0MiB
      total-hdd-space: 16.0MiB
  write-sect-since-reboot: 215
     write-sect-total: 194269
           bad-blocks: 0%
    architecture-name: smips
           board-name: hAP lite
         platform: MikroTik

The 22,- EUR are dirt cheap but my time isn't. Automating the the "all packages" updates will certainly be a worthwile learning experience.

How about 44,- EUR for a non-lite hAP? Or a 50,- hAP ac lite? Flash is still sized at 16MiB but you can add a USB stick. Would that help? I couldn't find any statements on this in the manual or product brochures.

If not, then the entire hAP/cAP/wAP range of "16 MB Flash" MikroTik products does not really have a future in the "Stable" RouterOS track for consumers. MikroTik must resolve this issue somehow.

The RB951Ui-2HnD comes at 80,- EUR and with 128MiB NAND storage. This would definitely remove the upgrade pains albeit at a noticeable price increase.

I used to run plan9 on my old pentium1 laptop and pentium1 PCs in the last millenium. In fact, I had bought specific hardware such as 3com 3C589 or Orinoco Wavelan PCMCIA devices supported by plan9.

I never had any any luck to run plan9 (or variants such as 9atom) on more modern gear. My only successful attempt to run plan9 on a 1000H Eeepc was with Russ Cox' 9vx.

This day started off with some googling after bhyve plan9, took a turn via proxmox plan9, and ended with another read of the 9front pages. Since I'm running now pretty decent hardware capable of running qemu, this is now a viable option:

This is just an initial boot. The actual installation has to wait for another day. I am still a complete klutz with using qemu and had to google how to get my mouse back. (Luckily, another machine was at hand.)

read: nftables, Chapter 4 & 5

A good bug report goes a long way... Matthew Sporleder fixed the netbsd-8.x repositories within just a few hours. Thanks!

With the recent upgrade of hackett on the netbsd-8 stable track to "8.2-and-a-bit-later" I also switch the pkgin(1) repository from the 8.1 subdirectory to the 8.2 one. This was on May, 2nd.

For the first nine days after, pkgin update wouldn't find any new catalogue but eventually, on May 11th, there was one.

pkgin upgrade then greeted me with something similar to this:

log 167 # pkgin upgrade
calculating dependencies...done.

30 packages to refresh:
  p5-Socket6-0.29nb1 p5-IO-Socket-INET6-2.72nb5 dehydrated-0.6.5
  p5-IO-CaptureOutput-1.1105 p5-Email-Valid-1.202nb3 Markdown-1.0.1nb7
  p5-Net-Domain-TLD-1.75nb3 p5-Net-SMTP-SSL-1.04nb3 automake-1.16.1nb1
  p5-TimeDate-2.30nb6 p5-Authen-SASL-2.16nb7 p5-Net-IP-1.26nb7
  p5-GSSAPI-0.28nb10 p5-Mozilla-CA-20180117nb2 p5-Net-LibIDN-0.12nb11
  p5-Digest-HMAC-1.03nb9 m4-1.4.18nb2 autoconf-2.69nb9 ytalk-3.3.0nb1
  tig-1.2.1nb3 tcsh-6.22.02 rlwrap-0.43nb3 pcre2-10.34 lzo-2.10 lz4-1.9.2
  libxml2-2.9.10nb1 libuuid-2.32.1 libunistring-0.9.10 iperf-2.0.5nb1
  gmake-4.2.1nb1

19 packages to upgrade:
  scrollz-2.2.3nb8 screen-4.8.0nb1 python37-3.7.7 perl-5.30.2 p5-Net-SSLeay-1.88
  p5-Net-DNS-1.23 p5-MailTools-2.21 p5-IO-Socket-SSL-2.067 p5-Error-0.17029
  openvpn-2.4.8nb2 nghttp2-1.40.0nb2 libidn-1.35 libffi-3.3nb2 iperf3-3.7nb1
  git-docs-2.25.4 git-base-2.25.4 fossil-2.10nb2 curl-7.69.1 bash-5.0.16nb1

1 package to install:
  heimdal-1.5.3nb24

30 to refresh, 19 to upgrade, 1 to install
14M to download, 15M to install

proceed ? [Y/n]

I welcome the 19 packages to upgrade -- that's why I run the command.

I was very suprised by the refresh list, though. Why do need pkgs to get refreshed even when the pkg is already on board, with the exact same version? Different dependencies? Loss of local cache files? Whatever the reason may be, I was particularly surprised of the ytalk pkg appearing in this list. I had installed that package just five days earlier. And yes, at exactly ytalk-3.3.0nb1 already.

Be that as it may, the real problem showed up when proceeding with this upgrade:

proceed ? [Y/n] y
p5-Authen-SASL-2.16nb7.tgz                    100%   24KB  24.3KB/s   00:00    
download error: p5-Authen-SASL-2.16nb7 size does not match pkg_summary

This error message is not new to me. I have seen it before (last summer) and suspect some inconsistencies within the Fastly CDN hosting the NetBSD repositories.

The relevant commands to debug this:

pkgin 75 > pwd
/var/db/pkgin
pkgin 76 > sqlite3 pkgin.db 
SQLite version 3.17.0 2017-02-13 16:02:40
Enter ".help" for usage hints.
sqlite> .mode line
sqlite> select *  from remote_pkg where pkgname like 'p5-Authen-SASL' ;
          PKG_ID = 6173
     FULLPKGNAME = p5-Authen-SASL-2.16nb7
         PKGNAME = p5-Authen-SASL
         PKGVERS = 2.16nb7
      BUILD_DATE = 2020-04-01 03:57:23 +0000
         COMMENT = Perl module to handle SASL authentication
         LICENSE = gnu-gpl-v2 OR artistic
PKGTOOLS_VERSION = 20091115
        HOMEPAGE = https://metacpan.org/release/Authen-SASL
      OS_VERSION = 8.0
     DESCRIPTION = 
         PKGPATH = security/p5-Authen-SASL
     PKG_OPTIONS = gssapi
      CATEGORIES = security perl5
        SIZE_PKG = 119267
       FILE_SIZE = 24892
           OPSYS = NetBSD
      REPOSITORY = http://cdn.Netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/8.2/All
sqlite> ^D

pkgin 76 > echo 0 1 2 | xargs -n1 -I XX lynx -head -dump \
? http://cdn.Netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/8.XX/All/p5-Authen-SASL-2.16nb7.tgz |\
? grep Length
Content-Length: 24900
Content-Length: 24892
Content-Length: 24900

So there's the "size does not match". Others on the netbsd-users@ mailing list are seeing this download error, too, and I just joined in with my findings.

If I tweaked things correctly, this entry should show up beneath a header with month/day time (not just time) when viewed on it own "permalink" page.

fred.marshlabs.gaertner.de is now running DragonflyBSD-5.8.1, a bugfix release from three days ago.

quick builds did the job. Total time for this source upgrade including everthing from git pull to merging /etc files, rebooting, updating pkgs, and a final make initrd: some 45min.

fred is an Intel E6550 Core2 Duo CPU with exactly that: 2 cores, running at 2.33GHz, has 4 GB RAM, and 230 GB spinning rust.

Two days ago I was looking for some multi-user collaboration software with these properties:

1. multi-user chat without line buffering (unlike IRC)
2. sharing of a shell session
3. console-based (as opposed to: requiring X11 or even a state-of-the-performance-limits web browser)

I know how to use screen(1) or tmux(1) to this effect but these programs are not easy to use for everybody:

Multi-user screen sessions are by default very independent and you need some extra coordination of terminal sizes and windows/regions viewed. Multi-user tmux sessions can be started to be auto-coordinated in these respects, but all users then also share the same cursor, and so user A cannot comment easily on actions of user B. In both cases, an extra telphone conversation can compensate. But if you are only online, all users are better quite familiar with screen/tmux.

My "aptitude search ..." (debian) and "pkgin search ..." (netbsd) commands led me to the ytalk package. This is patterned after the old BSD talk(1) program and even relies on the original (n)talkd for initiating conversations. It is pimped in two respects:

1. It is multi-user (and not just for two users).
2. Users can start a shell in their pane (and not just "talk").

This fits the bill for me perfectly.

On top of all that, ytalk is still pretty easy to use, even for newcomers. Actions and options can be controlled through two simple menues. There is no need to learn control sequences. Also, pane sizes are automatically coordinated between the participants.

It also turned out that ytalk is pretty old: the first version was released in 1993. It even wasn't new to me: I had it installed on miles.marshlabs.gaertner.de in 2002. Back then, though, it still had some significant trouble spots, limiting the use of shells. So I had forgotten about it.

Like the original talk, ytalk it is network-enabled: you can talk with users on other hosts. The prerequisites for this are:

• Each host has the talkd service enabled, and not just listening on 127.0.0.1.
• Each host is reachable on legacy IP (IPv4). talk/ytalk won't run over IPv6.
• No NAT is involved. Just NATting the traffic between the talkds won't do: These packets just coordinate the session between the caller and callee, and their adresses are communicated inside of the session setup packets.

Having said this, ytalk will be fine where all users are on the same host anyway. Also, hosts behind some NAT gateway can still talk to other hosts within a VPN confederation.

Good keyboards do not die. They just get a bit dirty.

Here's a quick run through the marshlabs, hunting for all those Cherry G80ies with their original DIN plugs still in active use. For the youngsters: we are talking about these:

G80-DIN PS/2-adaptered into the IGEL thin client / host susan.marshlabs.gaertner.de:

G80-DIN PS/2-adaptered into the NCD X terminal fz.marshlabs.gaertner.de:

G80-DIN straight into into the i486-DX66 miles.marshlabs.gaertner.de:

Front side: 3 * Pentium1 (133 Mhz), backside: 3 DIN kbd sockets:

Nerds move with boxes dedicated to gear. Let's home in into the bottom one:

(The small adaptor there is for a PS/2 keyboard and a DIN computer socket. Allowing you to take modern kbd and adaptor USB to PS/2 to DIN to miles.)

Update in the evening: a quick tally in our office rooms (tech and sales = 14 people) yields five DIN-plugged keyboards still in active use in the company.

The highlights there:

Some work in the aftermath of Sunday's DSL troubles:

Management LAN

To recap: my "CPE machinery" consists of

1. A Fritz!BOX WLAN 3170 in modem mode. (The routing/NAT and WLAN features are not active.) The FB provides 4 ethernet ports.

2. A MikroTik RouterBoard 2011 (billy.marshlabs.gaertner.de) is acting as PPPoE client and is the actual Internet gateway for the marshlabs.

3. The PPPoE packets between the RB-2011 and FB-3170 travel over a dedicated 7 meter ethernet cable.

The FB-3170 ist still manageable via IP. I defined its IP address to be 192.168.77.1/24 instead of the default 192.168.178.1/24 (which my neighbor's WLAN already uses -- another FritzBox in da house, and within reach :-). For trouble-shooting on Sunday/Monday, I moved my laptop close to the FB and plugged an extra cable into one of the three remaing free LAN ports.

Today, I reconfigured the RB-2011 to have things a bit more convenient.

• Before: the PPPoE was defined on top of Ethernet port with the cross-link to the FB. This uplink port (number 10) was not part of any other bridge. (Bridges are the RouterBoard-RouterOS way to tie ports together.)

• After: there is now a new small "cpe-bridge" defined on the RouterBoard, with ports 9 and 10 as members. So port 9 becomes another option to attach to the FB management LAN, right next to my desk. Packets travel over the existing 7m crosslink cable. The pppoe-client interface had to be moved a little bit: now it sits on top of the cpe-bridge, not anymore on top of the single port.

With this setup, it was already possible to keep the laptop at the desk when connecting to the FB-3170.

Even more luxury was possible by making the RB-2011 an active player in and router for the mgmt LAN: just add a IP address to the bridge. I decided on the static 192.168.77.2/24 instead of some DHCP assignment from the FB.

I then tested two alternatives to connect to the FB-3170 (192.168.77.1/24) from my real LAN (217.13.64.128/26):

1. Let the RouterBoard do the work and hide my LAN via NAT behind its 192.168.77.2 bridge address:

   [neitzel@billy] /ip firewall nat> print 
   Flags: X - disabled, I - invalid, D - dynamic 
    0    chain=srcnat action=masquerade dst-address=192.168.77.1 
         out-interface=cpe-bridge log=no log-prefix=""
   

2. Let FB3170 do the work: even in "modem-only" mode, it will take additional static routes to extend the "LAN" side.

Of course I settled on the latter. Let's avoid NAT wherever possible.

New firmware

My first actual "management action" today was to install new firmware to the FB-3170. I went from 49.04.24 to 49.04.58. The release notes promised "more stable DSL". Well, it turns out that the downstream now syncs at only 10.700 Kbps instead of 11.300 as before. (Nope, there is no Go Faster! option; I could just throttle things further down.)

And there is now an "energy monitor". Fancy.

On the lucky side: they didn't nuke the modem-only operational mode.

TODO: in four weeks, check if April and May differ noticably in the RIPE ATLAS measurements. Will atlas.marshlabs.gaertner.de aka p2781.probes.atlas.ripe.net be more reachable than before?

alexis.marshlabs.gaertner.de is an OLinuXino ARM-Board. It still runs the original Debian wheezy distribution which, by now, is "a bit" long in the tooth.

There's a reason for that old system. In earlier years, the special MALI-400 GPU support tied to the 3.4 linux kernel prevented an upgrade.

In the recent past, though, the mainlining efforts have made huge steps forward.

Today, I downloaded the Olimex-provided images for their "Armbianish" Debian-buster and Ubuntu-bionic releases, and even more current versions are afoot. Another candidate for the upgrade is Arch Linux ARM.

I need to find out if its possible to use a btrfs root filesystem. This would be just great for an extensive systemd-nspawn/machinectl setup.

I still have 90 GB unassigned on the 120 GB SSD and two unused partition entries in the MBR, set aside from the start for such an upgrade.

I don't dare to do the tests during the week: The board serves as XDCP, font server, and TFTP server for the X terminal in the kitchen, as well as for the telephone (tftp). It's also the DNS server for the marshlabs. So, I wouldn't mind a rainy weekend.

What a waisted day.

My DSL-CPE elected to reboot itself every three or four minutes. Ping packets run as usual initially, with 44ms RTT into the office, then times ramp into the hundreds, thousands, ... and between 10000ms and 20000ms the CPE reboots.

It's a lowly Fritz!Box 7130 operated just as an ADSL2+ modem, not as router. (FritzBoxen do a forced NAT. I run public IP space at home.)

Turning the machine of for a short (10 min) or long (5 hours) period didn't change anything. However, since exactly(!) midnight everything was OK again.

Let' see how this develops. I had just a single reboot today. A matching replacement power-supply is at hand, too: my small 5" VGA LCD monitor delivers 12V/1A, too. (Hey, 1.2A even.)

Thanks to my friendly upstairs neighbor who let's me use her WLAN & internet connection in times like these.

And yes: It's always a good idea to choose something other than the default vendor LAN addresses. Being connected both to my neighbor's WLAN 192.168.178.0/24 and on the LAN side to my CPE's 192.168.178.0/24 managment LAN just doesn't cut it.

Makes me wonder: can we already use the IPv6 fe80:...%if notation for RFC-3927 169.254.0.0/16 IPv4 link-local nets?

One of the open problems shifting my "marshlabs" mail to hackett was how to deal with HTML mails. (I mean you, Paypal, and all the non-tech friends naively using webmail accounts and/or smart-phones.)

NetBSD's mail(1) (aka Mail and mailx) originates from Kurt Shoen's BSD-mail 8.1 but has been brought into the current millenium with basic support for MIME. It will

1. take care of transfer-encoded messages (such as base64 or quoted-printable),
2. lets you access/save individual parts of a MIME message, and
3. lets you define handlers for specific MIME content-types.

(If you need to know: nope, OpenBSD and FreeBSD are still sticking to the orginal.)

The basic pipeline to process a message (as stated in the man page):

mail -> MIME-decoder -> MIME-hook -> pager -> screen

The man page also provides an example how to deal with HTML parts:

set mime-body-text-html="+/usr/pkg/bin/lynx -force_html -dump -stdin"

The "+" is a flag to prefer the HTML version over others in a multipart/alternative mail. The -dump option makes lynx not to behave interactively but to push the rendered message to stdout where it gets passed on to the pager stage of the pipeline.

Of course an interactive lynx would be much nicer. It would allow you to use any embedded links. However, just leaving off the -dump option didn't cut it. It works as bad as

lynx https://gaertner.de/~neitzel/nb/ | less

does -- you end up with just an empty page.

Today I dived further into the mail(1) manpage and found a workaround:

set mime-body-text-html="lynx -stdin". Before displaying an HTML mail, either set pager-off (NetBSD-mail-specific) or unset crt (old-school) to suppress the pager stage.

Lynx can be used interactively then. Toggling the pager off and on again, is a nuisance, though. It would be nicer to have that as another flag in the handler.

Strongly related: The currently defined flags for a mime-hook are:

• !: use a sub-shell instead of doing an exec(3).
• +: mark as preferred alternate type
• -: disable the transfer-decoding stage

I already know that the "-" feature, i.e. an easy activation/deactivation of the transfer-encoding decoder is important. Gunnar Ritter's fabulous Heirloom-mailx (aka nail) provides two variations of the classic | command to filter a message: pipe (= |) and Pipe (preserve headers and MIME sections), and you can use set raw/unset raw to toggle the transfer decoding. I need all of these, and quite often.

With NetBSD, the | won't do the transfer-decoding, ever. You need to provide it yourself, as in

& | base64 -d | grep foo

You get the decoding but also the (abridged) mail header with this command:

& set enable-pipes
& p . | grep foo

I wasn't able to make the write command work with a |foo command destination.

Summary:

1. I should hack bit more on the NetBSD mail source and add a "no-pager" flag for the mime-hooks.

2. I need to install the metamail package and tie that in, too. It's the oldest MIME support kit I am aware of, and I have existing setups with various other MUAs. Thi should make more things work rather quickly, without relying on MUA-specific MIME-hooks.

i just updated hackett.marshlabs.gaertner.de on its netbsd-8 STABLE track. We are now at "8.2 and a bit". (The last update had been on 2019-12-27.)

Maybe I even knew this in an earlier life already, but I just accidently ran across the little detail that lynx(1) is also capable of both reading and posting news articles via NNTP.

A quick test, posted via lynx, ingested via inews, transfered via UUCP via tcp6 between two Cnews hosts, displayed via trn(1):

ds.test #48                                  (1)
Xref: marshlabs.gaertner.de gds.test:48
Newsgroups: gds.test
Path: marshlabs.gaertner.de!gaertner.de!usenet
From: neitzel@marshlabs.gaertner.de
Subject: test
Sender: usenet@gaertner.de (Mr. News)
Organization: marshlabs
Message-ID: <q9po79.FD3@gaertner.de>
X-Nntp-Posting-Host: hackett.marshlabs.gaertner.de
Date: Sat, 2 May 2020 15:54:45 GMT

one
two
tree

End of article 48 (of 48) -- what next? [npq]

For security reasons and old age, trn strips off any 8-bit and control characters (by default) and cannot cope with UTF-8 or any other multi-byte encodings (by design). Maybe lynx is a light-weight, terminal-based aid in these cases. Initial testing shows: perhaps, but not right away. (There are many lynx encoding knobs to fiddle around with.)

read: nftables, Chapter 3