tldr: Configuring a massive wireguard VPN for both intratilde and intertilde communications is relatively easy and provides massive improvements in security and usability.
One day in 2019, I parked my big rig at my favorite Ohio truck stop and took my dogs for a walk. I was putting a bag of their poop in a dumpster like a good citizen when I peeked into the bin and saw a black desktop computer. It was a gaming PC in near-working order. When I got a load to Texas, I brought it to my friends house and we fixed it up and connected it.
Thus, our shared server was born. We upgraded the hardware and started running services like gameservers and groupware on it. I thought about extending its services to family and friends and friends of friends, maybe even recruiting some fellow nerds to help with administration. Then I discovered tildes in the end of 2020. I'm either going to extend our server into a full tilde or wiggle my way into an admin role on an existing tilde.
The first thing I noticed is a lack of VPN on most tildes. We used wireguard heavily on our server and this kind of network overlay provided numerous advantages. In this article, I will "sell" wireguard as a useful tech for use in tildes. Then I will discuss potential network topologies and use cases for both intra-tilde and inter-tilde communications. This is the vast bulk of this article: describing the massive benefit. Finally, I will provide some unanswered questions and a launchpoint for future discussions.
If you haven't used wireguard before, your opinion of it may be that, "it's like OpenVPN, but faster, I guess." You're right, but it's a big deal. Have you ever gotten lag on your game and said "Oh wait, let me disconnect from my VPN," to get a speed boost? Pre-wireguard VPNs had too much overhead to provide a proper drop-in replacement for a clearnet connection.
I've run netcat speedtests to compare UDP transmission speeds over the same network path using wireguard and a straight IP connection. The VPN connection was able to transmit something like 93% of the data in the same amount of time. (The percentage will differ wildly depending on how many clearnet hops apart the two hosts are. I got these numbers using a connection from my laptop in Colorado to our server in Texas).
This means that there's no longer any reason to even provide public connections to internal services: you can completely close ports to the internet, which vastly reduces the attack vector on a server. Imagine, lots of tildes are currently running services (like nextcloud) that are protected by nothing more than a user's potentially awful password. On our dumpster-dived server, we only opened ports 80, 443, and 25 (70, 1965 coming soon). IMAP, SSH, nextcloud, gameservers, etc were all closed to the public internet. Since wireguard clients are available for mac, linux, android, windows, and iphone, we didn't have any problem checking or sending emails from our phones or whatever.
Wireguard runs at the kernel level and is managed, mostly, by standard unix tools (ip link, ip addr, ip route, iptables). This makes it easy to apply different rules to different zones, like how you would manage WAN and LAN on your home router. With wireguard, we can create an arbitrary network topology with different rules for intra- and inter-tilde IP traffic, regardless of the underlying link layers. Using your OS's built-in firewall. Opps, I'm drooling.
Now let's do some use cases. Assume a standard client-server model, with a wireguard overlay network on the 10.0.77.0/24 IPv4 space.
Each of the clients A-E, drawn with circles, are laptops and phones of users. They may be behind firewalls. Their NAT router might not let them open ports. It might be a cell phone with wildly varying IP addresses and no port forwarding whatsoever.
The square in the center is our tilde server. It has a static IP and can open ports freely.
So what can we do using the wireguard network that wasn't possible with the clearnet? Let's say I'm user A and an admin of the tilde.
I can pipe all my internet traffic through the tilde (like what most people use VPNs for). This provides protection against snooping on public wifi AND, by combining all our outgoing connections, protects us from ISP snooping. (Because our cleartext traffic is all pooled through the single exit node, making each of us harder to surveil).
I can run services on a client machine, like (B). Say B is a desktop computer with a beefy GPU that can handle better gameservers than the poor tilde running on a VPS. The tilde can handle everyone's connection to B, and B can take on the gameserver's GPU load
Same as above, but with B providing a mighty API to a small PWA served by the tiny tilde. For example, a hash-cracking service. The tilde can serve a small upload form on a html page, which uploads to a hashcat instance running on B. No need to set up any kind of digest auth because the service can be firewalled off to trusted users only.
Maybe I'm doing development with another user, and I want to share my localhost dev server.
Why is it still so damn hard to share large files? Even tildes' temporary file hosting services don't allow very large file uploads. With wireguard, I can share a file directly to another user's machine using netcat! It's encrypted, it's way faster than scp/rsync, and doesn't involve any upload/download to an intermediate server.
Users can ping their computers from their phones and vice versa, without having to set up their own VPN or whatever.
Local gopher services are now encrypted.
Seriously, this list could go on forever. Dream sweet dreams of what is possible on a VPN tilde
Let's look at what's possible with an intertilde topology.
Using the same conventions as above, we can see three interconnected tildes on a wireguard network of wireguard networks. The "clearnet" cloud represents open ports to the clearnet; it is assumed that all hosts have an upstream connection to the clearnet.
Note that some users are in multiple tildes. For example, user C might have IPs 10.0.77.4 AND 10.0.42.8 and can connect to their different tilde servers using 10.0.77.1 and 10.0.42.1. On unix, this kind of routing is easy. And this user can theoretically connect to any service on any tilde server or client, if all the nodes on a route there will allow it.
The three tildes' admins in this diagram have been assigned the 10.0.X.0/24 namespace, where X is 42, 69, and 77. I will refer to them as ~42, ~69 and ~77, respectively.
~69 has not implemented any intratilde VPN, but they have joined the tildenet. All their services open to the clearnet. Maybe this is a public tilde with public admission and a low-risk threat model.
~42 has intranet VPN services, and some clearnet services. Maybe they have some services available to both, so users can use an encrypted wireguard connection to a gameserver and still invite outsiders to the same gameserver on a clearnet-opened port.
~77 has all ports closed to the clearnet. They don't have a public webserver with a landing page or anything. This might be a tilde you manage between your trusted friends, verified users at your company or school, or a tilde of hacktivists sharing resources and code.
Since there is no central tilde, tildes can manage and block connections between themselves (again, using their built-in firewalls). Tilde admins can report nefarious tildes and connect or disconnect between each other freely. In this way, bad tildes can be completely removed from the network through unanimous exclusion.
A final advantage to having an intertilde network is that global tildeverse services can be hosted across multiple tildes using redundancy technologies like round-robining.
I've had great success using domains names with custom TLDs on my own wireguard network with both self-hosted nameservers and modification of /etc/hosts. Since I use the ki9.us domain, I use .ki9 as my wireguard TLD. That way I can host my website publicly at www.ki9.us and my development server privately at www.ki9.
What I'm trying to say is, with custom DNS servers (only available on the network), we can have arbitrary sub-domaining in addition to arbitrary subnet addressing. Since the tildeverse already lives at tildeverse.org, why not use .tildeverse? This would allow for
For the tildiverse:
<service>.tildeverseFor tildes:
<tilde>.tildeverse
<service>.<tilde>.tildeverse
For users:
<user>-<client>.<tilde>.tildeverse
<service>.<user>-<client>.<tilde>.tildeverse
For example:
chat.tildeverse for a VPN connection to tilde.chat round-robin.
chat.tildeverse.org could cname to tilde.chat for a public connection
ctrl-c.tildeverse for VPN connections to the main ctrl-c.club server
nextcloud.tilde-club.tildeverse for tilde.club's nextcloud service
ki9-phone.tilde-pw.tildeverse to ping user my phone on any cell/wifi/satellite network via the tilde.pw server.
urt.ki9-desk.cyberia.tildeverse for a connection through cyberia tilde to an UrbanTerror gameserver running on user ki9's desktop. It could possibly be opened to the public at urt.ki9-desk.cyberia.tildeverse.org if cyberia.tildeverse.org was willing to forward to that machine from the open internet
We could also use the .tilde domain for this, since it's not available on the clearnet anyway.
We basically have three options for internal IP addressing, ranging from easiest to most future-proof: IPv4, both, or IPv6. We can provide IPv6 internally even if the underlying network (our ISP) does not provide IPv6 because the wireguard connection just tunnels through the external IPv4.
IPv4 only is easiest to set up and provides short, easy-to remember addresses. However, there are limitations to the number of hosts. (Let's not do NAT internally, thanks)
Both provides easy-to-remember addresses as well as an easy transition to IPv6 when we run out of addresses.
IPv6 only is the most future proof, allowing for broad control over sub-networking and sub-sub-networking. It doesn't provide easy-to-remember addresses though.
If we provide IPv4, let's use the entire 10.X.X.X space and split the subnet mask at CIDR /20. This would allow for 4,094 tildes with 4,094 clients each. That would let a tilde with ~1000 users allocate 4 addresses to each user device.
Since wireguard's only been around for a few years, and I've been using it for a few years, I guess I'm an expert! I am ready to help build this thing, if folks are interested. The game plan, I imagine, would be:
Establish tildeverse RFCs for the network's address and domain conventions.
Reassign the current tildeverse ~Net IP addresses to use the full 10.X.X.X space (there seems to only be four tildes in the net right now so we can nip this in the bud)
Provide resources (a wiki?) to teach tilde admins how to set up wireguard on their tilde, including articles for various intratilde and intertilde topologies.
I'm sure other people have input. I'm just getting into IRC, so feel free to hit me up on tilde.chat IRC at #tildenet.
^ 2021/02