I think I finally got all the pieces working on my Linux router. The last step was have it switch automatically to 4G when the main internet was down. Since I'm using NetworkManager I enabled an option that checks for connectivity[archived], and adds 20000 to the route metric if it can't access a given URI.

Linux networking favors routes with lower metrics, so this should re-route all packages through 4G automatically when the main internet is down.

A cool use case listed in the docs[archived]:

This has the purpose to give a better default-route to devices that have global connectivity. For example, when being connected to WWAN and to a Wi-Fi network which is behind a captive portal, WWAN still gets preferred until login.