And now for something completely different:
The newest addition to the labs is not 7+ years old but spanking new gear: i5-9400, 32GB RAM, 500GB SSD, ... and very, very quiet.
I usually don't put stickers to my machines. The Abteilung-fuer-Redundanz-Abteilungdeserves an exception to this rule: their sticker not only matches the colour scheme of the mini-tower/shoebox enclosure perfectly but the six wings also hint at the six cores in the i5-9400.
The machine comes with a big monitor (2560x1440 on 27") and is my new home office workplace.
This replaces the the prior work setup of my Lenovo X220 laptop + 1280x1024 external monitor. (Which is way too noisy when used the whole day long.)
I had to shoot the pictures right away because such a clean desk is veryrare with me. It was only possible because I shifted all of the entropy to the neighbouring desk:
Both desks feature a row of 2cm holes in the back. These were originally used as air vents for the heating elements the desks covered in the 70ies. Now they are great for much of the cabling.
The cables will be tied up nicely once things have settled.
I still need to learn how to route the audio via the DisplayPort instead of the analog LineOut/Headphone jacks. Also, having the keyboard USB-hubbed off the monitor means that I have to re-run my
xmodmap ~/,Xmodmap
every time after switching the monitor off/on. (Probably a job for udev; for now, I let the screensaver do the work.)
I intend to run hubert as a 24/7 server+workstation. I decided on Proxmox-VE/Debian as OS, extended with the desktop services and applications I need at the hypervisor level.
Hubert
should inherit many tasks from
alexis
, the Olinuxino ARM board which provided the 24/7 services for the last years. (XDMCP, bootp/tftp, X11 xfs, DNS, NTP, lpd). Once
fz
, the NCD X Terminal in the kitchen can boot/work from
hubert
, I can give alexis its badly needed system upgrade.
Upgrading the RouterOS on MikroTik devices is a simple affair:
All this is usually done in five minutes. So far I never had any issues caused by an RouterOS update.
Which makes you buy more MikroTik gear. The youngster in my home is...
[neitzel@lowell] > /system routerboard print
routerboard: yes
board-name: hAP lite
model: RouterBOARD 941-2nD
serial-number: 7C2C07C4455B
firmware-type: qca9531L
factory-firmware: 3.36
current-firmware: 6.46.6
upgrade-firmware: 6.46.6
This is a cheap (22 EUR), small wireless router/switch/access point serving my kitchen. Permanently attached nodes are NCD X terminal terminal
fz
and the DAB+/FM/Internet/LAN-Media radio
gaga
.
Lowell
is supposed to replace the small cisco switch lab there but, as of now, they all still share the window sill:
I have run out of my public IPv4 addresses at home long ago. Because
lowell
is currently mostly just operating as an access point it doesn't need any layer-3 address except for management. And so it became my first IPv6-only node, without any IPv4 address at all.
This was all fine. Until I tried the first RouterOS upgrade.
As
tcpdump
showed the upgrade process will resolve the server name
download.mikrotik.com has address 159.148.172.226
download.mikrotik.com has address 159.148.147.204
download.mikrotik.com has IPv6 address 2a02:610:7501:4000::226
download.mikrotik.com has IPv6 address 2a02:610:7501:1000::196
which apparently wouldserve both the current and the vintage protocol flavours. The hAP though will first try an IPv4 server, notice that that network is unreachable, and... give up. What a shame!
This short-coming is particularly disappointing because the RouterOS cantransfer data via IPv6 when asked manually:
[neitzel@lowell] /file> /tool fetch url="http://hackett.6.ml.gaertner.de/index.html" output=user
status: finished
downloaded: 0KiB
data: All my friends and I are crazy. That's the only thing that
keeps us sane.
[neitzel@lowell] /file>
D'oh!
Workaround: for RouterOS updates, I temporarily
/ip dhcp-client enable 0
. Do the upgrade dance, and
/ip dhcp-client disable 0
again.
Not nice but there are worse things in life.
Hey, let's just spend fifteen minutes on upgrading all three MikroTik gadgets, I thought around 8pm. When I went into bed, it was around 5am.
The upgrades went without a hitch on the two larger devices,
billy
and
hall
but on
lowell
strange things would happen. The "download" step went fine but the "reboot for install" step would end up in the same old package versions as before (6.46.2), with the download new version (6.46.6) purged from the
/file
area. Repeated attempts didn't help.
WLKIKIV, as we say here, and a quick
/log print
shows the problem:
not enough disk space.
The "hdd" flash memory is indeed much more constrained on
lowell
:
% echo billy hall lowell | \
> xargs -n1 -Ixx ssh xx /system resource print | \
> grep -E 'hdd|board-name'
free-hdd-space: 107.3MiB
total-hdd-space: 128.0MiB
board-name: RB2011L
free-hdd-space: 109.0MiB
total-hdd-space: 128.0MiB
board-name: CRS125-24G-1S-2HnD
free-hdd-space: 7.1MiB
total-hdd-space: 16.0MiB
board-name: hAP lite
%
The size of the stock "combo" release packages is now approaching half of the
16.0MiB
disk size:
% echo 2 6 | xargs -n1 -I X lynx -head -dump \
> https://download.mikrotik.com/routeros/6.46.X/routeros-smips-6.46.X.npk |\
> grep Length
Content-Length: 7651050
Content-Length: 7700154
%
During an upgrade, both the old and new version have to sit side-by-side on the disk, the filesystem structure needs some space, the config needs some space, ... the official documentation is asking for 2 MB spare capacity. After this download though I was down to the last 44 KB(!) on the disk.
A few months ago, in the same situation, I found a surplus support-dump I could delete to gain enough breathing space. No such luck tonight.
With the current RouterOS "Stable" images, things have now simply become too tight for a stock "hAP lite" and similar devices. without much extra config/data on its flash medium to upgrade to newer stock RouterOS versions. To be frank, this is major surprise if not a disgrace.
The official RouterOS documentation doesn't address this problem.
Grudgingly I dived into the "community support". I simply hate sifting through web fora, no matter which ones. It took hours.
Yes, I was not the only one with the problem. There were messages about the problem without any followup at all; there were quite a handful of wrong explanations; there was even a bit of ad-hominem and mud-slinging.
It did pointed me to the proper solution though:
By default, have their software installed from a "combined routeros package" which contains a selection of individual feature packages. It should not happen but the combined package can become to big for smaller platforms. You have then to switch over to deal with the packages individually, selecting you own mix.
My first idea was to delete a few of the 6.46.2 packages which I currently don't use in order to create the space for the complete new kit.
Turns out that you cannot
/system package uninstall
anything when everything comes from the "combined routeros".
The only way forward is this:
Make an extra backup of your configuration beyond of what the automatic reboot/reset backup is providing. The commands are simple and the demands on precious flash space are small:
[neitzel@lowell] > /system backup save
[neitzel@lowell] > /export compact file=cfg-mn
[neitzel@lowell] > /file print where type!=directory
# NAME TYPE SIZE CREATION-TIME
0 cfg-mn.rsc script 6.4KiB may/27/2020 03:18:50
1 auto-before-reset.b... backup 19.0KiB jan/02/1970 02:42:11
2 lowell-20200527-031... backup 30.1KiB may/27/2020 03:14:53
Download the "Extra packages" kit matching your hardware from https://www.mikrotik.com/download.
This kit does
notcontain just "extra" packages for the more obscure features as the title suggests to me. Instead, the filename is much more appropriate:
all_packages-smips-6.46.6.zip
. This zip contains the ten packages which comprise the "combined" = "Main" package (=
routeros-smips-6.46.6.npk
), and only three extra pkgs:
multicast
,
openflow
,
tr069-client
. (A full listing is below.)
Download this .zip archive elsewhere and extract the .npk packages.
Use scp, ftp or RouterOS'
/tool fetch
to copy a subset of the packages into the
/file
area for installation. Everybody needs the
system
package which weighs in with 5.5 MB alone. Another essential package for me is
ipv6
(196 KB) to be able to access/manage the hAP-lite at all.
dhcp
might be that thing for you, and in that case you also need
security
(155 + 307 = 462 KB). And since
security
is also required for
ssh
access, I used that, too. These four pkgs already total at 6+ MB, enough to get nervous.
Reboot to install these packages.
You onlyget the few selected new packages.
Allpackages/features from the old version get removed. The result is nota mix of old and updated packages.
Your new reduced feature set will load your old configration as much as possible. Settings for now missing features will be
lost. For example, without the
wireless
pkg, I lost my WLAN definition.
Luckily, you didn't skip the the first step, saving your config, did you?
With the old version's packages gone, you have now plenty of disk space for the other new packages. Install as much as you want by copying them to the
/file
area and rebooting.
With all wanted new packages in place, you can now reload your configuration:
/system backup load name=lowell-20200527-0314.backup
or
/import cfg-mn.rsc
As of now I haven't figured out which is better in which case. I suppose that either would do for me.
I believe you can choose between these two strategies:
Exercise some restraint and aim at "below 7 MB for everything", so that future upgrades are completely painless. The standard
/system package update
process should download only those packages you have in use.
In my case, this would be: system, ipv6, wireless, security, dhcp. As of now, this already totals in 7149168 bytes aka 6.8MiB. Hrrmmm....
If you prefer a "all packages" setup, you
willhave to go through the "update to/with minimal package set / add extras later" on every single update. The only ease is that you can get rid of ballast before doing the upgrade:
/system package uninstall
will now work. You can then do the (minimal) upgrade and re-add non-minimal packages afterwards. Again, this requires the download of the "Extras" .zip-file. And, of course, the backup of your configuration.
I am wondering how all this will pan out for me. I'll try to automate the "all packages" updates, i.e. the second approach.
For reference, here is my current
lowell
installation and sizes of the corresponding all_packages:
neitzel 373 > unzip -l all_packages-smips-6.46.6.zip
Archive: all_packages-smips-6.46.6.zip
Length Date Time Name
-------- ---- ---- ----
69713 05-14-20 12:14 advanced-tools-6.46.6-smips.npk
155729 05-14-20 12:14 dhcp-6.46.6-smips.npk
147537 05-14-20 12:14 hotspot-6.46.6-smips.npk
196689 05-14-20 12:14 ipv6-6.46.6-smips.npk
57425 05-14-20 12:14 mpls-6.46.6-smips.npk
36945 05-14-20 12:14 multicast-6.46.6-smips.npk
49233 05-14-20 12:14 openflow-6.46.6-smips.npk
258129 05-14-20 12:14 ppp-6.46.6-smips.npk
69713 05-14-20 12:14 routing-6.46.6-smips.npk
307281 05-14-20 12:14 security-6.46.6-smips.npk
5330220 05-14-20 12:14 system-6.46.6-smips.npk
114769 05-14-20 12:14 tr069-client-6.46.6-smips.npk
1159249 05-14-20 12:14 wireless-6.46.6-smips.npk
[neitzel@lowell] > /system package print
Flags: X - disabled
# NAME VERSION SCHEDULED
0 security 6.46.6
1 ipv6 6.46.6
2 dhcp 6.46.6
3 advanced-tools 6.46.6
4 system 6.46.6
5 wireless 6.46.6
6 hotspot 6.46.6
7 mpls 6.46.6
8 multicast 6.46.6
9 openflow 6.46.6
10 ppp 6.46.6
11 routing 6.46.6
12 tr069-client 6.46.6
[neitzel@lowell] > /system reso print
uptime: 8h49m20s
version: 6.46.6 (testing)
build-time: Apr/27/2020 10:32:16
factory-software: 6.28
free-memory: 7.7MiB
total-memory: 32.0MiB
cpu: MIPS 24Kc V7.4
cpu-count: 1
cpu-frequency: 650MHz
cpu-load: 0%
free-hdd-space: 7.0MiB
total-hdd-space: 16.0MiB
write-sect-since-reboot: 215
write-sect-total: 194269
bad-blocks: 0%
architecture-name: smips
board-name: hAP lite
platform: MikroTik
The 22,- EUR are dirt cheap but my time isn't. Automating the the "all packages" updates will certainly be a worthwile learning experience.
How about 44,- EUR for a non-lite hAP? Or a 50,- hAP ac lite? Flash is still sized at 16MiB but you can add a USB stick. Would that help? I couldn't find any statements on this in the manual or product brochures.
If not, then the entire hAP/cAP/wAP range of "16 MB Flash" MikroTik products does not really have a future in the "Stable" RouterOS track for consumers. MikroTik must resolve this issue somehow.
The RB951Ui-2HnD comes at 80,- EUR and with 128MiB NAND storage. This would definitely remove the upgrade pains albeit at a noticeable price increase.
I used to run plan9 on my old pentium1 laptop and pentium1 PCs in the last millenium. In fact, I had bought specific hardware such as 3com 3C589 or Orinoco Wavelan PCMCIA devices supported by plan9.
I never had any any luck to run plan9 (or variants such as 9atom) on more modern gear. My only successful attempt to run plan9 on a 1000H Eeepc was with Russ Cox' 9vx.
This day started off with some googling after
bhyve plan9
, took a turn via
proxmox plan9
, and ended with another read of the
9front
pages. Since I'm running now pretty decent hardware capable of running
qemu
, this is now a viable option:
This is just an initial boot. The actual installation has to wait for another day. I am still a complete klutz with using qemu and had to google how to get my mouse back. (Luckily, another machine was at hand.)
read: nftables, Chapter 4 & 5
Two days ago I was looking for some multi-user collaboration software with these properties:
I know how to use
screen(1)
or
tmux(1)
to this effect but these programs are not easy to use for everybody:
Multi-user
screen
sessions are by default very independent and you need some extra coordination of terminal sizes and windows/regions viewed. Multi-user
tmux
sessions can be started to be auto-coordinated in these respects, but all users then also share the same cursor, and so user A cannot comment easily on actions of user B. In both cases, an extra telphone conversation can compensate. But if you are only online, all users are better quite familiar with screen/tmux.
My "
aptitude search ...
" (debian) and "
pkgin search ...
" (netbsd) commands led me to the
ytalk
package. This is patterned after the old BSD
talk(1)
program and even relies on the original
(n)talkd
for initiating conversations. It is pimped in two respects:
This fits the bill for me perfectly.
On top of all that,
ytalk
is still pretty easy to use, even for newcomers. Actions and options can be controlled through two simple menues. There is no need to learn control sequences. Also, pane sizes are automatically coordinated between the participants.
It also turned out that
ytalk
is pretty old: the first version was released in 1993. It even wasn't new to me: I had it installed on
miles.marshlabs.gaertner.de
in 2002. Back then, though, it still had some significant trouble spots, limiting the use of shells. So I had forgotten about it.
Like the original
talk
,
ytalk
it is network-enabled: you can talk with users on other hosts. The prerequisites for this are:
talkd
service enabled, and not just listening on 127.0.0.1.talk
/
ytalk
won't run over IPv6.talkd
s won't do: These packets just coordinate the session between the caller and callee, and their adresses are communicated inside of the session setup packets.Having said this,
ytalk
will be fine where all users are on the same host anyway. Also, hosts behind some NAT gateway can still talk to other hosts within a VPN confederation.
One of the open problems shifting my "marshlabs" mail to
hackett
was how to deal with HTML mails. (I mean
you, Paypal, and all the non-tech friends naively using webmail accounts and/or smart-phones.)
NetBSD's
mail
(1) (aka
Mail
and
mailx
) originates from Kurt Shoen's BSD-mail 8.1 but has been brought into the current millenium with basic support for MIME. It will
(If you need to know: nope, OpenBSD and FreeBSD are still sticking to the orginal.)
The basic pipeline to process a message (as stated in the man page):
mail -> MIME-decoder -> MIME-hook -> pager -> screen
The man page also provides an example how to deal with HTML parts:
set mime-body-text-html="+/usr/pkg/bin/lynx -force_html -dump -stdin"
The "+" is a flag to prefer the HTML version over others in a
multipart/alternative
mail. The
-dump
option makes lynx not to behave interactively but to push the rendered message to
stdout
where it gets passed on to the
pager
stage of the pipeline.
Of course an interactive lynx would be much nicer. It would allow you to use any embedded links. However, just leaving off the
-dump
option didn't cut it. It works as bad as
lynx https://gaertner.de/~neitzel/nb/ | less
does -- you end up with just an empty page.
Today I dived further into the
mail(1)
manpage and found a workaround:
set mime-body-text-html="lynx -stdin"
. Before displaying an HTML mail, eitherset pager-off
(NetBSD-mail-specific) orunset crt
(old-school) to suppress the pager stage.
Lynx can be used interactively then. Toggling the pager off and on again, is a nuisance, though. It would be nicer to have that as another flag in the handler.
Strongly related: The currently defined flags for a mime-hook are:
!
: use a sub-shell instead of doing an
exec(3)
.+
: mark as preferred alternate type-
: disable the transfer-decoding stageI already know that the "
-
" feature, i.e. an easy activation/deactivation of the transfer-encoding decoder is important. Gunnar Ritter's fabulous Heirloom-mailx (aka
nail
) provides two variations of the classic
|
command to filter a message:
pipe
(=
|
) and
Pipe
(preserve headers and MIME sections), and you can use
set raw
/
unset raw
to toggle the transfer decoding. I need all of these, and quite often.
With NetBSD, the
|
won't do the transfer-decoding, ever. You need to provide it yourself, as in
& | base64 -d | grep foo
You get the decoding but also the (abridged) mail header with this command:
& set enable-pipes
& p . | grep foo
I wasn't able to make the
write
command work with a
|foo
command destination.
Summary:
I should hack bit more on the NetBSD
mail
source and add a "no-pager" flag for the mime-hooks.
I need to install the
metamail
package and tie that in, too. It's the oldest MIME support kit I am aware of, and I have existing setups with various other MUAs. Thi should make more things work rather quickly, without relying on MUA-specific MIME-hooks.
Maybe I even knew this in an earlier life already, but I just accidently ran across the little detail that
lynx(1)
is also capable of both reading and posting news articles via NNTP.
A quick test, posted via lynx, ingested via inews, transfered via UUCP via tcp6 between two Cnews hosts, displayed via
trn(1)
:
ds.test #48 (1)
Xref: marshlabs.gaertner.de gds.test:48
Newsgroups: gds.test
Path: marshlabs.gaertner.de!gaertner.de!usenet
From: neitzel@marshlabs.gaertner.de
Subject: test
Sender: usenet@gaertner.de (Mr. News)
Organization: marshlabs
Message-ID: <q9po79.FD3@gaertner.de>
X-Nntp-Posting-Host: hackett.marshlabs.gaertner.de
Date: Sat, 2 May 2020 15:54:45 GMT
one
two
tree
End of article 48 (of 48) -- what next? [npq]
For security reasons and old age, trn strips off any 8-bit and control characters (by default) and cannot cope with UTF-8 or any other multi-byte encodings (by design). Maybe lynx is a light-weight, terminal-based aid in these cases. Initial testing shows: perhaps, but not right away. (There are many lynx encoding knobs to fiddle around with.)
read: nftables, Chapter 3
Testing for this year's MeKa event showed that my old Cisco Catalyst WS-C2940-TF8-S switches (
shake
,
rattle
, and
roll
) are very picky about the SFP modules they accept. Cisco-branded only, please!
That policy by Cisco from around 2003 caused much hoopla in the industry, and Cisco was effectively forced to accept any MSA-compliant transceivers with with later IOS versions (12.2ish).
These switches are running something around
IOS 12.1(22)EA14
, though, and they won't accept a
service unsupported-transceiver
command. They
doaccept an
no errdisable detect cause gbic-invalid
but it doesn't help in any way even for Agilent SFPs. (The port is just stuck and remains so, even when putting in a proper Cisco SFP instead. A cold restart is necessary.)
So in the aftermath of the meeting, I ebayed two GLC-SX-MM SFPs with them bridges on (19.98 EUR for the pair, incl. shipping). Just to have a few more fibre options in my lab...