The Sipeed NanoKVM is a pretty neat device. Level 1 Techs review:

The procedure to update firmware however, is a little obscure.

The system has two components:

1. The Firmware - the embedded linux image that boots the device; and
2. The NanoKVM application - which runs the Web UI and does video encoding etc etc.

The application itself can be updated within the Web UI, but the firmware itself is a little more invasive. The officially supported way to do this is to pull out the microSD card and reflash the entire device. If you’ve got the device installed somewhere remote, or can’t easily get the device apart, this is a pain in the butt. So, here’s a bit of a hacky way to update the device without actually pulling it apart.

Step 1: Grab the latest firmware image from the github releases page. At the time of writing this, the latest firmware is 1.3.0.

Step 2: Mount the image file to get access to the goodies inside:

1
2
3
4

$ mkdir -p tmp/boot tmp/rootfs
$ sudo kpartx -a 20241120_NanoKVM_Rev1_3_0.img
$ sudo mount /dev/mapper/loop0p1 tmp/boot
$ sudo mount /dev/mapper/loop0p2 tmp/rootfs

Step 3: Get the NanoKVM ready. Log in via SSH and then do the following - making sure not to reboot from now on, or you will have to reflash the SD manually…

Make sure you do this on the NanoKVM - not your desktop!

1

# rm -fR /boot/*

Step 4: Back on your desktop, copy everything from the mounted image to the NanoKVM:

1
2
3

$ scp tmp/boot/* root@<kvm ip>:/boot/
$ cd tmp/rootfs/
$ sudo rsync -axv --delete-before . root@<kvm ip>:/

Step 5: Validate that your filesystem on the NanoKVM looks good - if you mess up here, you’re back to flashing the SD card anyway.

On the NanoKVM, ensure that your boot partition looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# ls -l /boot/
total 11728
-rwxr-xr-x    1 root     root      11553428 Jan 10 17:31 boot.sd
-rwxr-xr-x    1 root     root        439808 Jan 10 17:31 fip.bin
-rwxr-xr-x    1 root     root             0 Jan 10 17:31 gt9xx
-rwxr-xr-x    1 root     root          3621 Jan 10 17:31 logo.jpeg
-rwxr-xr-x    1 root     root           132 Jan 10 17:39 resolv.conf
-rwxr-xr-x    1 root     root             0 Jan 10 17:31 usb.dev
-rwxr-xr-x    1 root     root             0 Jan 10 17:31 usb.disk0
-rwxr-xr-x    1 root     root             0 Jan 10 17:31 usb.rndis0
-rwxr-xr-x    1 root     root            28 Jan 10 17:31 ver
-rwxr-xr-x    1 root     root             1 Jan 10 17:31 wifi.pass
-rwxr-xr-x    1 root     root             1 Jan 10 17:31 wifi.ssid
-rwxr-xr-x    1 root     root             0 Jan 10 17:31 wifi.sta

Make sure the / path looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# ls -l /
total 100
lrwxrwxrwx    1 root     root             7 Sep 20 06:00 bin -> usr/bin
drwxr-xr-x    2 root     root         16384 Jan  1  1970 boot
drwxr-xr-x    2 root     root         32768 Jan  1  1980 data
drwxr-xr-x   11 root     root          4780 Jan 10 17:39 dev
drwxr-xr-x   19 root     root          4096 Jan 10 17:39 etc
drwxr-xr-x    6 root     root          4096 Jan 10 17:39 kvmapp
lrwxrwxrwx    1 root     root             7 Sep 20 06:00 lib -> usr/lib
lrwxrwxrwx    1 root     root             3 Sep 20 06:00 lib64 -> lib
lrwxrwxrwx    1 root     root            11 Sep 20 06:00 linuxrc -> bin/busybox
drwx------    2 root     root         16384 Nov 13 01:58 lost+found
drwxr-xr-x    2 root     root          4096 Sep 20 06:00 media
drwxr-xr-x    5 root     root          4096 Sep 20 06:00 mnt
drwxr-xr-x    2 root     root          4096 Sep 20 06:00 opt
dr-xr-xr-x  140 root     root             0 Jan  1  1970 proc
drwx------    3 root     root          4096 Jan 10 17:39 root
drwxr-xr-x    8 root     root           440 Jan 10 17:37 run
lrwxrwxrwx    1 root     root             8 Sep 20 06:00 sbin -> usr/sbin
dr-xr-xr-x   11 root     root             0 Jan  1  1970 sys
drwxrwxrwt    5 root     root           180 Jan 10 17:39 tmp
drwxr-xr-x    7 root     root          4096 Sep 20 06:00 usr
drwxr-xr-x    5 root     root          4096 Sep 20 06:00 var

If all goes well, reboot the IP KVM and it should boot, then start loading the app back onto the device. After a little wait, the device should work as normal again but with an updated firmware…

Step 6: Clean up your PC…

1
2
3
4

$ cd ../..
$ sudo umount tmp/boot tmp/rootfs
$ sudo kpartx -d 20241120_NanoKVM_Rev1_3_0.img
$ rmdir tmp/boot tmp/rootfs

I hope this saves the hassle that is disassembling the NanoKVM to get at the microSD card!

Starting with BIND 9.19, you can now set up DNS over TLS in the forwarders option.

You can use this in Fedora now by installing the bind9-next packages instead of bind.

Configuring this is quite simple, the example below uses Google, Quad9 and Cloudflare as upstream DNS servers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

tls cloudflare-tls { remote-hostname "one.one.one.one"; };
tls quad9-tls { remote-hostname "dns.quad9.net"; };
tls google-tls { remote-hostname "dns.google"; };
options {
    ...
    forwarders port 853 {
        1.1.1.1 tls cloudflare-tls;
        1.0.0.1 tls cloudflare-tls;
        2606:4700:4700::1111 tls cloudflare-tls;
        2606:4700:4700::1001 tls cloudflare-tls;

        9.9.9.9 tls quad9-tls;
        149.112.112.112 tls quad9-tls;
        2620:fe::fe tls quad9-tls;
        2620:fe::9 tls quad9-tls;

        8.8.8.8 tls google-tls;
        8.8.4.4 tls google-tls;
        2001:4860:4860::8844 tls google-tls;
        2001:4860:4860::8888 tls google-tls;
    };
};

Customise the above however you like to disable IPv6 servers, or a certain upstream provider.

Keep in mind that all traffic for upstream DNS will now go to port 853 on the target upstream.

The Linus Tech Tips channel ShortCircuit recently did a video on an NTP time source card. Video here for reference:

It’s really nice to have your own clock source, but when you start digging into the price of those units, saying you might not get change from $13,000 isn’t a lie. Of course, that’s for the highest end cards - but do we really need that for a small user? What if you could get most of the way there for about $50 USD installed?

Good news - you can.

The idea is to use one of these GPS recievers with a little bit of level conversion and hook it straight into your serial port. It’s exactly the same method as when I did this about 8 years ago - however the tooling has changed a little since then - so lets revisit this topic with some modern tools.

We’ll be using chronyd as our NTP time server - as its pretty much the default everywhere these days, along with gpsd and its tools to configure the module properly.

What you’ll need:

• UBX-M8030-KT type GPS reciever
• Dual Channel RS232 to TTL module
• DB9 serial port connector
• 5v power source

Looking at the GPS module, you’ll see it’s pretty straight forward. TX / RX for serial data, PPS for the pulse, and power.

Here’s a quick picture of my plug wiring. In my setup, my RS232 port allows me to inject 5vDC on pin 9 of the DB9. You probably won’t have this, so you’ll need to supply 5vDC to the VCC pad on the converter - which also connects to the GPS Vcc - and one of the GND pads on the board. It’s important that the serial port ground, the power supply ground, and the level converters grounds are all connected.

On the hardware side, that’s pretty much it - so now, lets set up the software.

I installed mine on a Proxmox server - so everything here is based on a Debian install. These tools are generic, so search for them on your distro.

Firstly, install the required packages:

1
2

# apt-get update
# apt-get install gpsd gpsd-tools setserial chrony

Now, to configure gpsd, edit the file /etc/default/gpsd, and make its contents as below. Subsitute your serial port instead of /dev/ttyS0.

1
2
3
4
5
6
7
8
9
10

# Devices gpsd should collect to at boot time.
# They need to be read/writeable, either by user gpsd or the group dialout.
DEVICES=""

# Other options you want to pass to gpsd
GPSD_OPTIONS="-n"
OPTIONS="-s 38400 -F /run/gpsd.sock /dev/ttyS0"

# Automatically hot add/remove USB GPS devices via gpsdctl
USBAUTO="false"

Now we’re going to want to edit the gpsd systemd service to add some commands to initialise the GPS module on startup.

Do this via systemctl edit gpsd and add in the following - again, alter your serial port as required:

1
2
3
4
5
6
7

[Service]
ExecStartPre=/usr/bin/setserial /dev/ttyS0 low_latency
ExecStartPre=/usr/bin/ubxtool -f /dev/ttyS0 -P 18 -s 9600 -S 38400
ExecStartPre=/usr/bin/ubxtool -f /dev/ttyS0 -P 18 -s 38400 -p MODEL,2
ExecStartPre=/usr/bin/ubxtool -f /dev/ttyS0 -P 18 -s 38400 -e BINARY
ExecStartPre=/usr/bin/ubxtool -f /dev/ttyS0 -P 18 -s 38400 -d NMEA
ExecStartPre=/usr/bin/ubxtool -f /dev/ttyS0 -P 18 -s 38400 -e PPS

Finally, we set up chronyd to use both the GPS and PPS output. Edit /etc/chrony/chrony.conf and add this at the bottom:

1
2
3
4
5

refclock SHM 0 refid GPS offset 0.600 delay 0.2
refclock SHM 1 refid PPS offset 0.0 delay 0.0

server pool.ntp.org iburst
allow 10.0.0.0/24

In this config, we deliberately add an error to the GPS lines. This is because the NMEA data can be quite regular, and in some cases people have seen it being preferred as a source over the PPS. Inducing an error here will ensure that we always use the PPS source.

The last two lines allow your network to use the chronyd instance as an NTP source and sets an external reference to start against.

Now to configure these services to start on boot, and start them now:

1
2

# systemctl enable gpsd chronyd
# systemctl restart gpsd chronyd

If you then watch the logs in journald, you’ll see something like this:

1
2
3
4
5
6
7

systemd[1]: Starting chrony.service - chrony, an NTP client/server...
chronyd[1084]: chronyd version 4.3 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
chronyd[1084]: Frequency -12.140 +/- 0.070 ppm read from /var/lib/chrony/chrony.drift
chronyd[1084]: Using right/UTC timezone to obtain leap second data
chronyd[1084]: Loaded seccomp filter (level 1)
systemd[1]: Started chrony.service - chrony, an NTP client/server.
chronyd[1084]: Selected source PPS

I then use this command to watch what’s going on: watch "chronyc tracking; echo; chronyc sources; echo; chronyc sourcestats"

You can check the accuracy in this by looking at this bit:

1
2
3
4

Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
GPS                         7   3    90    +19.721     89.013   -382ms   664us
PPS                        64  31  1001     -0.000      0.071   -147ns    48us

Job done. Enjoy your very cheap stratum 1 NTP server.

If you’re like me, you’ve got a home lab with a dozen or so virtual machines doing all sorts of things - and each of them are pulling down updates from somewhere on the internet.

What if you could have a single endpoint for all VMs to reference? That way, updates that are common would be distributed to all systems at LAN speeds after the first download.

Introducing - mod_cache for Apache :)

Assuming you’re already running Apache somewhere, you can start mapping part of the local path structure to a remote endpoint.

Drop the following into /etc/httpd/conf.d/mod_cache.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

CacheEnable	            	disk /fedora
CacheRoot	            	/var/cache/httpd/fedora
CacheMaxFileSize        	524288000
CacheDefaultExpire      	14400
CacheDetailHeader       	on

# common caching directives
CacheQuickHandler       	off
CacheLock	            	on
CacheLockPath	        	/tmp/mod_cache-lock
CacheLockMaxAge	        	5
CacheHeader	            	On

# cache control
#CacheIgnoreNoLastMod   	On
#CacheIgnoreCacheControl	On
   
# unset headers from upstream server
Header unset Expires
Header unset Cache-Control
Header unset Pragma

ProxyRequests	        	Off
ProxyPass   	        	/fedora http://dl.fedoraproject.org/pub/fedora
ProxyPassReverse        	/fedora http://dl.fedoraproject.org/pub/fedora

UseCanonicalName        	On

When in use, this will map http://my.apache.host/fedora to the Fedora mirror, and cache all responses and downloaded files.

The cache won’t automatically clean itself though - so we need a systemd service to clean things up over time. Create the file /etc/systemd/system/http-cache-clean.service as follows:

1
2
3
4
5
6
7
8
9
10

[Unit]
Description=Apache cache cleaner
After=network-online.target

[Service]
Type=forking
ExecStart=/usr/sbin/htcacheclean -d 60 -i -l 5G -p /var/cache/httpd/fedora/

[Install]
WantedBy=multi-user.target

This will limit the cache size to 5Gb and remove the oldest files first.

There is one gotcha when using this with Fedoras updates and dnf - zchunk. I believe this is because mod_cache doesn’t work on partial content requests - which is how zchunk functions.

To get around this, we can disable zchunk in the DNF configuration file /etc/dnf/dnf.conf. I also disable deltarpm - as its quicker to download the file from the LAN cache than it is to rebuild a drpm update.

1
2
3
4
5
6
7
8
9
10

[main]
gpgcheck=True
installonly_limit=3
clean_requirements_on_remove=True
best=False
skip_if_unavailable=True
max_parallel_downloads=10
fastestmirror=True
zchunk=False
deltarpm=0

We can then point the yum repo file to the local apache server - for example, part of /etc/yum.repos.d/fedora-updates.repo:

1
2
3
4
5
6

[updates]
name=Fedora $releasever - $basearch - Updates
#baseurl=http://download.example/pub/fedora/linux/updates/$releasever/Everything/$basearch/
#metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch
baseurl=http://my.apache.host/fedora/linux/updates/$releasever/Everything/$basearch/
enabled=1

For years, I’ve run a set of Logitech Z-5500 speakers into an optical port on my PC. It gives good quality 5.1 audio, and supports AC3 + DTS digital passthrough as well as 44, 48, and 96khz bitrates.

The problem is, the speakers go into a ‘sleep’ mode where it takes nearly a second to bring the amp back online to play audio - so notification sounds are often not played at all.

To correct this, in the past, I’ve run a simple systemd service using sox to output a sine wave that is below the audible level like so: /usr/bin/play -q -n -c2 synth sin gain -95

Now however, we can do this directly within pipewire itself.

Firstly, we need to identify the output device using pw-top. Play some audio, and look for which sink it is being played on - eg:

1
2
3
4
5
6
7
8
9
10
11
12
13

S   ID  QUANT   RATE    WAIT    BUSY   W/Q   B/Q  ERR FORMAT           NAME                                                                                                                                                                   
S   28      0	   0    ---     ---   ---   ---     0                  Dummy-Driver
S   29      0	   0    ---     ---   ---   ---     0                  Freewheel-Driver
S   36      0	   0    ---     ---   ---   ---     0                  Midi-Bridge
S   42      0	   0    ---     ---   ---   ---     0                  alsa_output.usb-Kingston_HyperX_Cloud_Stinger_Core_Wireless___7.1_000000000000-00.analog-stereo
S   49      0	   0    ---     ---   ---   ---     0                  alsa_input.usb-Kingston_HyperX_Cloud_Stinger_Core_Wireless___7.1_000000000000-00.mono-fallback
R   40   1024  48000  32.3us   4.3us  0.00  0.00    0    S16LE 2 48000 alsa_output.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_3__sink
R  106   1024  48000  20.5us   5.1us  0.00  0.00    0    F32LE 2 48000  + Brave
S   50      0	   0    ---     ---   ---   ---     0                  alsa_output.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_1__sink
S   51      0	   0    ---     ---   ---   ---     0                  alsa_output.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio__sink
S   52      0	   0    ---     ---   ---   ---     0                  alsa_input.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_2__source
S   53      0	   0    ---     ---   ---   ---     0                  alsa_input.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_1__source
S   54      0	   0    ---     ---   ---   ---     0                  alsa_output.pci-0000_2f_00.1.hdmi-stereo

In my case, the audio device is alsa_output.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_3__sink.

Now we create a file at ~/.config/wireplumber/main.lua.d/spdif-noise.lua with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

rule = {
  matches = {
    {
      { "node.name", "matches", "alsa_output.usb-Generic_USB_Audio-00.HiFi_5_1__hw_Audio_3__sink" }
    },
  },
  apply_properties = {
    ["dither.noise"] = 2,
    ["node.pause-on-idle"] = false,
    ["session.suspend-timeout-seconds"] = 0
  }
}

table.insert(alsa_monitor.rules,rule)

You’ll need to swap the name

Restart pipewire now: systemctl --user restart pipewire.service.

Now, when your first sound plays, pipewire will continue to output sub-audible noise to keep everything alive - which is a much better solution than using sox!