fluencelabs/aqua-book
1
0
Fork 0
mirror of https://github.com/fluencelabs/aqua-book synced 2024-12-04 15:20:19 +00:00
Code Issues Projects Releases Wiki Activity

GitBook: [#81] aqua-dht: delete outdated note

Browse Source
This commit is contained in:
folex 2021-11-26 15:18:10 +00:00 committed by gitbook-bot
parent 21c33c87db
commit d4ccdba410
No known key found for this signature in database
GPG Key ID: 07D2180C7B12D0FF
1 changed files with 28 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

62
libraries/aqua-dht.md
View File

@ -4,7 +4,7 @@ description: Aqua implementation of DHT and PubSub
# @fluencelabs/aqua-dht
`aqua-dht` provides PubSub API that can be used for service discovery and event delivery.
`aqua-dht` provides PubSub API that can be used for service discovery and event delivery. 
## Releases
@ -17,35 +17,30 @@ For the API reference, take a look at [pubsub.aqua](https://github.com/fluencela
## Terminology
* **@fluencelabs/aqua-dht** - Aqua library on NPM. Provides high level and low-level APIs to develop custom Aqua scripts. Suits more advanced use-cases.
{% hint style="success" %}
Both **aqua-dht-ts** and **aqua-dht** can be used with TypeScript and JavaScript.
{% endhint %}
* **PubSub** - short for Publish-Subscribe. A pattern for messaging and discovery. **Subscribers** create **subscriptions** on a **topic** so that **publishers** send messages to subscribers.
* **Kademlia** - algorithm for organizing a peer-to-peer network in such a way that peers can find each other in no more than O\(logN\) hops, where N is the size of the network.
* **PubSub **- short for Publish-Subscribe. A pattern for messaging and discovery. **Subscribers** create **subscriptions** on a **topic **so that **publishers** send messages to subscribers.
* **Kademlia** - algorithm for organizing a peer-to-peer network in such a way that peers can find each other in no more than O(logN) hops, where N is the size of the network.
* **topic** - `string` with associated `peer_id` and a list of **subscribers**. Associated `peer_id` can be thought of as a **topic creator**.
* **topic creator** - `peer id` that created the topic. Other users can't create a topic with the same name.
* **subscriber** - a peer that has called `subscribe` on a **topic**, optionally with an associated **relay\_id** and **service\_id**. Any peer can be a subscriber, including nodes and clients.
* subscriber's **relay\_id** - subscriber is available on the network through this relay.
* **topic creator **- `peer id` that created the topic. Other users can't create a topic with the same name.
* **subscriber **- a peer that has called `subscribe` on a **topic**, optionally with an associated **relay\_id **and **service\_id**. Any peer can be a subscriber, including nodes and clients.
* subscriber's **relay\_id** - subscriber is available on the network through this relay. 
{% hint style="info" %}
When a **subscriber** doesn't have a publicly available IP address, e.g. the client peer is a browser, it connects to the network through a relay node. That means that _other_ peers only can connect to that **subscriber** through a relay.
When a **subscriber** doesn't have a publicly available IP address, e.g. the client peer is a browser, it connects to the network through a relay node. That means that _other_ peers only can connect to that **subscriber** through a relay. 
In that case `subscribe` must be called with the`relay_id`, so other peers can reach the **subscriber**.
In that case `subscribe` must be called with the`relay_id`, so other peers can reach the **subscriber**. 
{% endhint %}
* subscriber's **service\_id** - id of the service provided by that subscriber. Sometimes a subscriber may want **publishers** of a specific **topic** to call functions on this **service\_id** in order to be able to distinguish among service calls.
* **subscription** - a **DHT record** associated with a **topic.** Holds information about a **subscriber**.
* **subscription lifetime** - **subscriptions** have two are evicted \(removed\) [24 hours](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L39) after their creation or after [1 hour](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L38) of being unused.
* **subscriptions limit** - each **topic** can have at most [20 subscribers](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L41). Each new subscriber will remove an old one, following a LIFO principle.
* **publisher** - any peer that has retrieved subscribers of a topic and calls functions on these subscribers
* subscriber's **service\_id** - id of the service provided by that subscriber. Sometimes a subscriber may want **publishers **of a specific **topic** to call functions on this **service\_id** in order to be able to distinguish among service calls.
* **subscription **- a **DHT record** associated with a **topic. **Holds information about a **subscriber**.
* **subscription lifetime **- **subscriptions **have two are evicted (removed) [24 hours](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L39) after their creation or after [1 hour](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L38) of being unused.
* **subscriptions limit **- each **topic** can have at most [20 subscribers](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L41). Each new subscriber will remove an old one, following a LIFO principle.
* **publisher **- any peer that has retrieved subscribers of a topic and calls functions on these subscribers
* **AquaDHT** - a service that provides low-level DHT API. `pubsub.aqua` is built on top of it.
* **DHT key** - a low-level representation of a topic
* **DHT record** - a low-level representation of a subscriber, see [Record](https://github.com/fluencelabs/aqua-dht/blob/9a729611c6da4930b5f145696bfdc975d1227e77/aqua/dht.aqua#L18) in Aqua
* **host value** - a **DHT record** with `peer_id` being that of a node. When a node is subscribed to a topic via `subscribeNode` or `initTopicAndSubscribeNode`, the **subscription** is a **host value**. Host values live much longer \([10 days](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L40)\) than other **DHT records**. [See Subscribe to a topic](aqua-dht.md#subscribe-to-a-topic) for details.
* **DHT key **- a low-level representation of a topic
* **DHT record **- a low-level representation of a subscriber, see [Record](https://github.com/fluencelabs/aqua-dht/blob/9a729611c6da4930b5f145696bfdc975d1227e77/aqua/dht.aqua#L18) in Aqua
* **host value** - a **DHT record** with `peer_id` being that of a node. When a node is subscribed to a topic via `subscribeNode` or `initTopicAndSubscribeNode`, the **subscription** is a **host value**. Host values live much longer ([10 days](https://github.com/fluencelabs/aqua-dht/blob/9a72961/service/src/main.rs#L40)) than other **DHT records**. [See Subscribe to a topic](aqua-dht.md#subscribe-to-a-topic) for details.
* **script caller** - peer that executes a script by sending it to the network. In Aqua it's `%init_peer_id%`
* **node** - usually a Fluence node hosted in the cloud. Nodes are long-lived, can host WebAssembly services and they participate in the Kademlia network.
* **node **- usually a Fluence node hosted in the cloud. Nodes are long-lived, can host WebAssembly services and they participate in the Kademlia network.
## How To Use Aqua-DHT
@ -55,7 +50,7 @@ There are [several simple examples](https://github.com/fluencelabs/aqua-dht/tree
### Create A Topic
Before subscribing to a topic is possible, that topic must be created. That's exactly what `initTopic` does.
Before subscribing to a topic is possible, that topic must be created. That's exactly what `initTopic` does. 
Here's a rather simplistic example in Aqua:
@ -130,7 +125,7 @@ subscribers = await findSubscribers(peerB, topic);
```
{% hint style="info" %}
There is`initTopicAndSubscribeBlocking and initTopicAndSubscribe.`
There is`initTopicAndSubscribeBlocking and initTopicAndSubscribe. `
`Blocking` version waits until at least a single write is done. The latter version is "fire and forget": it `awaits`instantly, but doesn't guarantee that write has happened.
{% endhint %}
@ -139,13 +134,13 @@ There is`initTopicAndSubscribeBlocking and initTopicAndSubscribe.`
#### `initTopicAndSubscribeNode` & `subscribeNode`
These two functions work almost the same as their non-`Node` counterparts, except that they subscribe **node** instead of a **caller**. This is useful when you want to subscribe a **service** hosted on the **node** on a topic.
These two functions work almost the same as their non-`Node` counterparts, except that they subscribe **node** instead of a **caller**. This is useful when you want to subscribe a **service** hosted on the **node **on a topic.
Such subscriptions live 10 times longer than these by `subscribe`, because services can't renew subscriptions on their own.
#### Renew Subscription Periodically
After a normal \(non-host\) subscription is created, it must be used at least once an hour to keep it from being marked **stale** and deleted. Also, peers must resubscribe themselves at least once per 24 hours to prevent subscription **expiration** and deletion.
After a normal (non-host) subscription is created, it must be used at least once an hour to keep it from being marked **stale** and deleted. Also, peers must resubscribe themselves at least once per 24 hours to prevent subscription **expiration **and deletion.
While this collection schedule may seem aggressive, it keeps keep PubSub up-to-date and performant as short-lived client-peers, such as browsers, can go offline at any time or periodically change their relay nodes.
@ -219,7 +214,7 @@ func send_everyone(relay: PeerId, topic: string, event: Event):
### Handle Function Calls
`subscribeToEvent` function from the [Fluence JS SDK](https://github.com/fluencelabs/fluence-js) allows JS/TS peers to define their API through services and functions.
`subscribeToEvent` function from the [Fluence JS SDK](https://github.com/fluencelabs/fluence-js) allows JS/TS peers to define their API through services and functions. 
Let's take the `SubscriberAPI` from the previous example and extend it a little:
@ -237,7 +232,7 @@ service SubscriberAPI:
Let's save this file to `subscriber_api.aqua` and compile it
```text
```
npx aqua -i . -o src/generated
```
@ -268,7 +263,7 @@ await registerSubscriberAPI(service_id, {
If your app requires to have more than 20 subscribers on a single topic, then it's time to think about a custom WebAssembly service that would store all these subscriptions in memory or on disk. Basically a simple subscriptions "directory".
With such a service implemented and deployed, you can use `pubsub.aqua` to subscribe that "subscriptions directory" service to a topic. Depending on your app's architecture, you might want to have several instances of "subscriptions directory" service.
With such a service implemented and deployed, you can use `pubsub.aqua` to subscribe that "subscriptions directory" service to a topic. Depending on your app's architecture, you might want to have several instances of "subscriptions directory" service. 
The code to get all subscriptions from "directory" services might look something like this in Aqua:
@ -298,10 +293,10 @@ func dir_subscribers(relay: PeerId, topic: string) -> [][]Record:
Fluence nodes participate in the Kademlia network. Kademlia organizes peers in such a way that given any key, you can find a set of peers that are "responsible" for that key. That set contains up to 20 nodes.
That set is called "neighborhood" or "K-closest nodes" \(K=20\). In Aqua, it is accessible in `aqua-lib` via the `Kad.neighbourhood` function.
That set is called "neighborhood" or "K-closest nodes" (K=20). In Aqua, it is accessible in `aqua-lib` via the `Kad.neighbourhood` function.
The two most important properties of the Kademlia neighborhood are:
1\) it exists for _any_ key
The two most important properties of the Kademlia neighborhood are: \
1\) it exists for _any_ key \
2\) it is more or less stable
A lot of DHTs rely on these properties to implement data replication. So does AquaDHT.
@ -329,7 +324,7 @@ This ensures that data is replicated across several peers.
#### At rest
Subscriptions are also replicated "at rest". That is, once per hour all **stale** values are removed, and non-**stale** values are replicated to all nodes in the neighborhood.
Subscriptions are also replicated "at rest". That is, once per hour all **stale** values are removed, and non-**stale** values are replicated to all nodes in the neighborhood. 
This ensures that even if a neighborhood for a topic has changed due to some peers go offline and others join the network, data will be replicated to all nodes in the neighborhood.
@ -338,4 +333,3 @@ For advanced users accustomed to AIR scripts:
There's an implementation of "at rest" replication for AquaDHT [on GitHub](https://github.com/fluencelabs/fluence/tree/2320204/deploy/builtins/aqua-dht/scheduled)
{% endhint %}