Francesco Servida's Blog

Last week I had a simple idea, upgrade a xiaomi openwrt router to a whitebox running esxi, vyos and a couple other applications.

My current homelab setup was a site-to-site using Wireguard between the openwrt router and a USG3 with DHCP relayed from the remote (openwrt) site to the main site’s DCs, both sites are behind NAT (thanks to ISP routers without bridging -_- ) and thus wireguard is the best choice.

Checking vyos doc, wireguard is present, vyos has a dhcp-relay functionality. All good, should be as easy as installing vyos and configuring everything, right? Riiight?

Well no… apparently vyos uses ISC-DHCP-Relay (and server), and there is a known issue with virtual tunnels and the interfaces dhcp-relay can associate with: https://phabricator.vyos.net/T4280

Doh’, what to do?
Let’s switch to IPSec-VTI, maybe IPSec is better supported and it is a bug with Wireguard.
…
…
Hours later, the IPSec-VTI tunnel was up between the two sites (not pertinent for this guide, if you are configuring it between to routers behind NAT, play around with the “authentication remote-id” and “authentication id” options so that the IKE and IPSEC SA work).
Let’s change the interface the the vti one andddddddd nothing. SAME ERROR!

Ok, not possible that no-one had this error before,  let’s scavenge the ISC repo, and maybe look at alternative free routers…

So, apparently pfsense and opnsense have the same problem and actually hide the WG interfaces instead of actually trying to patch the problem. Thanks…

However a couple people on the pfsense and ISC repo posted patches supposedly to make it work with tunnels, althouth none were integrated.
Fine, let’s compile and test the package manually.
Nothing, the error is always there, given the high “interface type” number I imagine it is combined with some VyOS weirdness.

At this point I spent too much time to just give up, although I did want to move a DC to the remote site and have local dhcp I did not want to lose the ability to have a remote DHCP backup in case I had to do maintenance.

Some googling afterward, Eureka! there is a package that does only dhcp relaying: dhcp-helper. Although it cannot run at the same time as ISC (obviously the listen port is the same), I can just drop all DHCP server functions on the router and handle all VLANs from the DHCP server on DC.
Luckily VyOS has an easy option to configure additional packages when building an image:

./configure –custom-package PACKAGENAME

It doesn’t work!  For some reason in my build environment (docker vyos-build pulled from docker hub) I have to run

python3 scripts/build-config –custom-package PACKAGENAME

in order for the options to work. Anyway, easy fix.

Let’s add “dhcp-helper” and try to configure it somehow with boot scripts afterward. Easy right now? riiight?

Not even. dhcp-helper declares isc-dhcp-server and isc-dhcp-relay as conflict in the .deb configuration. So vyos-build tries to install it, cannot install isc-dhcp* and fails with dependency errors.

This is were the hacked way starts…

In desperation I decide that surely the conflicts of a deb package can be edited.

• I download the latest deb for amd64 of dhcp-helper.
• unpack the deb:
  • dpkg-deb -b tmp_dhcp/ dhcp-helper_1.2-2_amd64.deb
• edit the tmp_dhcp/DEBIAN/control
  • to remove the conflicting packages
  • to rename the package to dhcp-helper-vyos, isc-dhcp-server and relay also have a conflicting option set which would stop this package from being installed.
• edit tmp_dhcp/etc/default/dhcp-helper
  • to already match my configuration, maybe it will be easier to configure, notably in server mode (-s) and only excluding (-e) the WAN interface, this way it will listen on everything else and would not need much configuration changes.
• repack the deb:
  • dpkg-deb -b tmp_dhcp/ dhcp-helper-vyos_1.2-2_amd64_noconflicts.deb

At this point, I follow vyos documentation and put the package in the “./packages” folder, rerun the build, attach the iso to a VM andddd nothing… apparently something is wrong and the .deb is not picked up… maybe only .deb files of declared packages can be used…

The last hope at this point is to host my own repository, add the package and use the custom-package option at build time with my custom package name. It was the last option before abandoning the project anddd… it worked!

I put the package in a new folder, followed this guide to create the required files:

• dpkg-scanpackages . /dev/null > Release
• dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

and uploaded the three files to my static server under the /debian path.

In vyos-build:

• python3 scripts/build-config –custom-package “dhcp-helper-vyos” –custom-apt-entry “deb [trusted=yes] https://STATIC_SERVER/debian/ ./”
• sudo make iso

Boot the iso…
Binary installed? Check!
Can the service be enabled and started?
Check!
Does the service survive a reboot? Check! Apparently this does not need anything particular for persistence.

Does it work?
Yes it does!

Finally, too many hours later this is solved!
I’d juste hope that this is fixed upstream someway, by fixing ISC (but the relay is EOL) or by including both packages and leaving the option to fully disable ISC and just use dhcp-helper for the relay.

If you decide to go this route ensure you “delete service dhcp server” and “delete service dhcp relay” before enabling dhcp-helper or else it will not work.

Well that’s it for the story! (and for my doc, in case I ever need to update vyos),