Discovery

Discovery is the glue that connects an Endpoint to something we can dial. Discovery services resolve EndpointIDs to either a home Relay URL or direct-dialing information. Endpoint discovery is an automated system for an Endpoint to retrieve addressing information. Each iroh endpoint will automatically publish their own addressing information with configured discovery services. Usually this means publishing which Home Relay an endpoint is findable at, but they could also publish their direct addresses.

​

Discovery Services

There are four different implementations of the discovery service in iroh. By default, iroh uses DNS discovery to resolve EndpointIDs to addresses. Local and DHT discovery are disabled by default and must be explicitly configured if you want to use them.

​

The motivation

We want iroh to establish connections with as little friction as possible. Our first big push toward chipping away at this goal was adding holepunching into iroh. Now, devs no longer need to worry about opening up ports on their servers / firewalls or be resigned to only creating connections to computers inside their local NAT. But even with holepunching, you need to know where to dial. Dialing an endpoint in iroh needs either an IP address to talk to, or the URL of a relay to which the remote endpoint is connected. To make things easier, you can use tickets early-on. Tickets are easily encodable bytestrings that contain an endpoint ID, socket addresses and a relay URL. Tickets work well, but they are long - and they expire: People change networks or relays, and then old tickets cannot find you anymore. So while tickets work well in some scenarios, they are not frictionless or universally usable. But what if you didn’t need to know the relay URL or socket address in order to dial an endpoint? Could we enable iroh users to dial peers using the least amount of information possible, ie can we enable dialing only by EndpointID, and no other address data?

​

The solution

We had two “guiding lights” while doing research on global endpoint discovery: First, we needed to see a path forward that would allow for a fully distributed topology, even if our first solution had a federated structure. If we (or some contributor) wanted to create or opt into a fully p2p version of global endpoint discovery in the future, that needed to work in tandem with whatever federated solution we came up with. Second, it was very important for us as an organization that we do not invent protocols and specs unnecessarily. Can we instead leverage standards that have stood the test of time (and scrutiny) in novel ways to solve our problem? It turns out, we can! And we can do it using one of the oldest and most dependable technologies we have on the internet: DNS. Using the DNS standard along side Pkarr (public-key addressable resource records), we now have global endpoint discovery in iroh!

​

The approach

We are using two standards: the domain name system (DNS) and Pkarr. DNS is typically used for resolving domain names into their associated IP addresses. But DNS actually relates domain names to resource records, of which IP addresses are one type. Pkarr allows us to publish DNS records that resolve elliptic curve keys (not domain names) to resource records instead. Our EndpointIDs in iroh are elliptic curve public keys. This means we can publish an association between iroh EndpointIDs and some resource records. Importantly, these records are signed, so that you can verify that the record was actually published by the endpoint with the given EndpointID. As long as iroh has an EndpointID and its associated relay URL (the address of the relay server that endpoint uses to hole-punch and proxy relay packets), we can dial that endpoint. So the Pkarr packet currently only needs to contain the EndpointID and the relay URL of its preferred relay server (which we call its “home relay”). When Pkarr publishing is enabled on your iroh endpoint, your endpoint will create a Pkarr packet with its EndpointID and relay URL, sign it, and defaults to publishing on an iroh-dns server that is run by the n0 team. From there, others can discover your dialing information by resolving your EndpointID using regular DNS. It’s worth noting that others must still learn your endpoint ID for this to work. The following sections describe the format of the Pkarr publishing records and endpoint discovery via DNS queries in greater detail.

​

Endpoint discovery via DNS

When connecting to an unknown EndpointId, the DNS discovery mechanism in iroh will perform a DNS query to discover relays or addresses for the endpoint. The DNS discovery is configured with a origin domain (which defaults to dns.iroh.link, a server run by n0). iroh will then perform a DNS query through the configured DNS resolver (which defaults to using the host system’s nameservers): _iroh.<z32-endpoint-id>.<origin-domain> TXT

• _iroh is the record name defined in this spec
• <z32-endpoint-id> is the z32 encoding of the 32-byte long EndpointId (which is a string of 52 characters)
• <origin-domain> is the configured origin domain
• TXT is the queried record type

The returned TXT records must contain a string value of the form key=value, as defined in RFC1464. This spec defines the following attributes:

• relay=<url>: The home relay for this endpoint, e.g. https://euw1-1.derp.iroh.network.
• addr=<addr> <addr> .. A space-separated list of socket addresses for this iroh endpoint. Each address is an IPv4 or IPv6 address with a port (e.g. 1.2.3.4:7367 or [::1]:3521)

​

Endpoint announces via pkarr

Endpoints announce their address information in Pkarr signed packets. The TXT records, as described below, are added to the answers section of a DNS server. Their name must be _iroh.<endpoint-id>. The encoded endpoint ID must be the root name, no other origin but . (the single dot) is permitted. Those packets are published to a Pkarr relay server, which is a HTTP service handling PUT requests with the signed packets. iroh’s Pkarr server is iroh-dns-server, which serves the received records over DNS. Pkarr packets can also be published onto the bittorrent mainline DHT, as specified by Pkarr (this is not yet supported in iroh natively, see below for our plans). DNS servers that support this spec will receive these Pkarr signed packets, check their signature and format validity, and then serve each contained record, with the DNS server’s configured origin domain appended to all record names.