GitBook: [2.0.0] 24 pages modified

SUMMARY.md

@@ -11,7 +11,8 @@
     * [Marine Repl](knowledge_aquamarine/marine/marine-repl.md)
     * [Marine Rust SDK](knowledge_aquamarine/marine/marine-rs-sdk.md)
 * [Tools](knowledge_tools.md)
-* [Builtin Services](knowledge_node_services.md)
+* [Node](node/README.md)
+  * [Builtin Services](node/knowledge_node_services.md)
 * [Security](knowledge_security.md)
 * [Tutorials](tutorials_tutorials/README.md)
   * [Deploy A Local Fluence Node](tutorials_tutorials/tutorial_run_local_node.md)

knowledge_aquamarine/README.md

@@ -1,22 +1,8 @@
 # Aquamarine
 
-Aquamarine is a programming language and executable choreography tool for distributed applications and backends. Aquamarine manages the communication and coordination between services, devices, and APIs without introducing any centralized gateway and can be used to express various distributed systems: from simple request-response to comprehensive network consensus algorithms.
+Fluence's Aquamarine stack is comprised of Aqua and Marine. Aqua is a programming language and runtime environment for peer-to-peer workflows. Marine, on the other hand, is a general purpose runtime that executes hosted code on nodes, whereas Aqua facilitates the programming of workflows composed from hosted code. In combination, Aqua and Marine enable any distributed application.
 
-At the core of Aquamarine is the design ideal and idea to pair concurrent systems, and especially decentralized networks, with a programing and execution tool chain to avoid centralized bottlenecks commonly introduced with [workflow engines](https://en.wikipedia.org/wiki/Workflow_engine) and [Business rule engines](https://en.wikipedia.org/wiki/Business_rules_engine). This not only makes Aquamarine the rosetta stone of the Fluence solution but also a very powerful generic coordination and composition medium.
+At the core of Aqua is the design ideal and idea to pair concurrent systems, and especially decentralized networks, with a programming and execution tool chain to avoid centralized bottlenecks commonly introduced with workflow engines and business rule engines. To this end,  Aqua manages the communication and coordination between services, devices, and APIs without introducing a centralized gateway and can be used to express various distributed systems: from simple request-response models to comprehensive network consensus algorithms.
 
-## Background
 
-When we build systems, we need to be able to model, specify, analyze and verify them and this is especially important to concurrent systems such as parallel and multi-threaded systems. [Formal specification](https://en.wikipedia.org/wiki/Formal_specification) are a family of formal approaches to design, model, and verify system. In the context of concurrent systems, there are two distinct formal specification techniques available. The state oriented approach is concerned with modeling verifying a systems state and state transitions and is often accomplished with [TLA+](https://en.wikipedia.org/wiki/TLA%2B). Modern blockchain design, modeling, and verification tend to rely on a state-based specification.
-
-An alternative, complementary approach is based on [Process calculus](https://en.wikipedia.org/wiki/Process_calculus) to model and verify the sequence of communications operations of a system at any given time. [π-Calculs](https://en.wikipedia.org/wiki/%CE%A0-calculus) is a modern process calculus employed in a wide range of applications ranging from biology to games and business processes.
-
-Aquamarine, Fluence's distributed composition language and runtime, is based on π-calculus and provides a solid theoretical basis toward the design, modeling, implementation, and verification of a wide class of distributed, peer-to-peer networks, applications and backends.
-
-## Language
-
-[Aquamarine Intermediate Representation](https://github.com/boneyard93501/docs/tree/a512080f81137fb575a5b96d3f3e83fa3044fd1c/src/knowledge-base/knowledge_aquamarine__air.md) \(AIR\) is a low-level language modeled after the [WebAssembly text format](https://developer.mozilla.org/en-US/docs/WebAssembly/Understanding_the_text_format) and allows developers to manage network peers as well as services and backends. AIR, while intended as a compile target, is currently the only Aquamarine language implementation although a high level language \(HLL\) is currently under active development.
-
-## Runtime
-
-The Aquamarine runtime is a virtual machine executed by the [Fluence Compute Engine](https://github.com/boneyard93501/docs/tree/a512080f81137fb575a5b96d3f3e83fa3044fd1c/src/knowledge-base/knowledge_fce.md) \(FCE\) running not only on every Fluence network peer but also on every frontend client. The distributed runtime availability per node not only aids in decentralized service discovery and execution at the same level of decentralization of the network, which is of significant importance. Moreover, with running execution scripts on both the client and the \(remote\) nodes, a high degree of auditability and verifiability can be attained.
 

knowledge_aquamarine/marine/README.md

@@ -2,8 +2,6 @@
 
 [Marine](https://github.com/fluencelabs/marine) is a general purpose WebAssembly runtime favoring Wasm modules based on the [ECS](https://en.wikipedia.org/wiki/Entity_component_system) pattern or plugin architecture and uses Wasm [Interface Types](https://github.com/WebAssembly/interface-types/blob/master/proposals/interface-types/Explainer.mdhttps://github.com/WebAssembly/interface-types/blob/master/proposals/interface-types/Explainer.md)  \( IT\) to implement a [shared-nothing](https://en.wikipedia.org/wiki/Shared-nothing_architecture) linking scheme. Fluence [nodes](https://github.com/fluencelabs/fluence) use Marine to host the Aqua VM and execute hosted Wasm services.
 
-Todo: we could really do with diagram
-
 The  [Marine Rust SDK](https://github.com/fluencelabs/marine-rs-sdk) allows to hide the IT implementation details behind a handy procedural macro `[marine]` and provides the scaffolding for unit tests.
 
 

node/README.md

@@ -0,0 +1,11 @@
+# Node
+
+The Fluence protocol is implemented as the Fluence [reference node](https://github.com/fluencelabs/fluence) which includes the
+
+* Peer-to-peer communication layer
+* Marine interpreter
+* Aqua VM
+* Builtin services
+
+and more.
+

node/knowledge_node_services.md

@@ -0,0 +1,403 @@
+# Builtin Services
+
+## Overview
+
+Each Fluence peer is equipped with a set of "built-in" services that can be called from Aquamarine and fall into the following namespaces:
+
+1. _peer_ - operations related to connectivity or state of a given peer
+2. _kad_ - Kademlia API
+3. _srv_ – management and information about services on a node
+4. _dist_ – distribution and inspection of modules and blueprints
+5. _script_ – to manage recurring scripts
+6. _op_ – basic operations on data deprecated - namespace for deprecated API Below is the reference documentation for all the existing built-in services. Please refer to the JS SDK documentation to learn how to easily use them from the JS SDK
+7. _deprecated_ - namespace for deprecated API
+
+Please note that the [`fldist`](../knowledge_tools.md#fluence-proto-distributor-fldist) CLI tool, as well as the [JS SDK](../knowledge_tools.md#fluence-js-sdk), provide access to node-based services.
+
+## API
+
+### peer is\_connected
+
+Checks if there is a direct connection to the peer identified by a given PeerId
+
+* **Arguments**:
+  * PeerId – id of the peer to check if there's a connection with
+* **Returns**: bool - true if connected to the peer, false otherwise
+
+Example of a service call:
+
+```scheme
+(call node ("peer" "is_connected") ["123D..."] ok)
+```
+
+Initiates a connection to the specified peer
+
+* **Arguments**
+  * _PeerId_ – id of the target peer
+  * [_Multiaddr_](https://crates.io/crates/multiaddr) – an array of target peer's addresses
+* **Returns**: bool - true if connection was successful
+
+Example of a service call:
+
+```scheme
+(seq 
+    (call node ("op" "identity") ["/ip4/1.2.3.4/tcp/7777" "/ip4/1.2.3.4/tcp/9999"] addrs)
+    (call node ("peer" "connect") ["123D..." addrs] ok) 
+)
+```
+
+### peer get\_contact
+
+Resolves the contact of a peer via [Kademlia](https://en.wikipedia.org/wiki/Kademlia)
+
+* **Arguments**
+  * _PeerId_ – id of the target peer
+* **Returns**: Contact - true if connection was successful
+
+```rust
+// get_contact return struct
+Contact { 
+    peer_id: PeerId,
+    addresses: [Multiaddr] 
+}
+```
+
+Example of a service call:
+
+```scheme
+(call node ("peer" "get_contact") ["123D..."] contact)
+```
+
+#### peer identify
+
+Get information about the peer
+
+* **Arguments**: None
+* **Returns:**  _external address_
+
+```javascript
+{ "external_addresses": [ "/ip4/1.2.3.4/tcp/7777", "/dns4/stage.fluence.dev/tcp/19002" ] }
+```
+
+Example of service call:
+
+```scheme
+(call node ("peer" "identify") [] info) peer timestamp_ms
+```
+
+### peer timestamp\_ms
+
+Get Unix timestamp in milliseconds
+
+* **Arguments**: None
+* **Returns**: _u128_ - number of milliseconds since 1970
+
+Example of service call:
+
+```scheme
+(call node ("peer" "timestamp_ms") [] ts_ms)
+```
+
+### peer timestamp\_sec
+
+Get Unix timestamp in seconds
+
+* **Arguments**: None
+* **Returns**: _u64_ - number of seconds since 1970
+
+Example of service call:
+
+```scheme
+(call node ("peer" "timestamp_sec") [] ts_sec)
+```
+
+### kad neighborhood
+
+Instructs node to return the locally-known nodes in the Kademlia neighborhood for a given key
+
+* **Arguments**: _key_ – the peer ID \(PeerId\) of the node
+* **Returns**: _peers_ – an array of PeerIds of the nodes that are in the Kademlia neighborhood for the given hash\(key\)
+
+Example of service call:
+
+```scheme
+(call node ("dht" "neighborhood") [key] peers)
+```
+
+Please note that this service does _not_ traverse the network and may yield incomplete neighborhood.
+
+### srv create
+
+Used to create a service on a certain node.
+
+* **Arguments**:
+  * blueprint\_id – ID of the blueprint that has been added to the node specified in the service call by the dist add\_blueprint service. 
+* **Returns**: service\_id – the service ID of the created service.
+
+Example of service call:
+
+```scheme
+(call node ("srv" "create") [blueprint_id] service_id)
+```
+
+### srv list
+
+Used to enumerate services deployed to a peer.
+
+* **Arguments**: None
+* **Returns**: a list of services running on a peer
+
+Example of service call:
+
+```scheme
+(call node ("srv" "list") [] services)
+```
+
+### srv add\_alias
+
+Adds an alias on service, so service could be called not only by service\_id but by alias.
+
+* **Argument**: alias - settable service name service\_id – ID of the service whose interface you want to name.
+* **Returns**: alias id
+
+Example of service call:
+
+```scheme
+(call node ("srv" "add_alias") [alias service_id])
+```
+
+### srv get\_interface
+
+Retrieves the functional interface of a service running on the node specified in the service call.
+
+* Argument: service\_id – ID of the service whose interface you want to retrieve. 
+* Returns : an interface object of the following structure:
+
+```typescript
+{ 
+    interface: { function_signatures, record_types },
+    blueprint_id: "uuid-1234...",
+    service_id: "uuid-1234..." 
+}
+```
+
+Example of service call:
+
+```scheme
+(call node ("srv" "get_interface") [service_id] interface)
+```
+
+### dist add\_module
+
+Used to add modules to the node specified in the service call.
+
+* Arguments:
+
+  * bytes – a base64 string containing the .wasm module to add. 
+  * config – an object of the following structure
+
+  ```javascript
+  {
+    "name": "my_module_name"
+  }
+  ```
+
+Example of service call:
+
+```scheme
+(call node ("dist" "add_module") [bytes config] hash)
+```
+
+### dist list\_modules
+
+Get a list of modules available on the node
+
+* Arguments: None
+* Returns: an array of objects containing module descriptions
+
+  ```javascript
+  [ 
+      { 
+          "name": "moduleA",
+          "hash": "6ebff28c",
+          "config": { "name": "moduleA" }
+      } 
+  ]
+  ```
+
+Example of service call:
+
+```scheme
+(call node ("dist" "list_modules") [] modules)
+```
+
+### dist get\_module\_interface
+
+Get the interface of a module
+
+* Arguments: hash of a module
+* Returns: an interface of the module \( see _srv get\_interface \)_ mple of service call:
+
+```scheme
+(call node ("dist" "get_interface") [hash] interface)
+```
+
+### dist add\_blueprint
+
+Used to add a blueprint to the node specified in the service call.
+
+* Arguments: blueprint – an object of the following structure
+
+  ```javascript
+  {
+      "name": "good_service",
+      "dependencies": [ "hash:6ebff28c...", "hash:1e59875a...", "hash:d164a07..." ] 
+  }
+  ```
+
+  ```text
+    Where module dependencies are specified as [_blake3_](https://crates.io/crates/blake3) hashes of modules
+  ```
+
+* Returns: Generated blueprint id
+
+Example of service call:
+
+```scheme
+(call node ("dist" "add_blueprint") [blueprint] blueprint_id)
+```
+
+### dist list\_blueprints
+
+Used to get the blueprints available on the node specified in the service call.
+
+* Arguments: None
+* Returns: an array of blueprint structures.
+
+A blueprint is an object of the following structure:
+
+```javascript
+{ 
+    "id": "uuid-1234-...", 
+    "name": "good_service", 
+    "dependencies": [ "hash:6ebff28c...", "hash:1e59875a...", "hash:d164a07..." ] 
+}
+```
+
+Example of service call:
+
+```scheme
+(call node ("dist" "list_blueprints") [] blueprints)
+```
+
+#### script add
+
+Adds a given script to a node. That script will be called with a fixed interval with the default setting at approx. three \(3\) seconds.
+
+Recurring scripts can't read variables from data, they must be literal. That means that every address or value must be specified as a literal: \(call "QmNode" \("service\_id-1234-uuid" "function"\) \["arg1" "arg2"\]\).
+
+* Arguments:
+  * _script_ – a string containing "literal" script 
+  * _interval_ – an optional string containing interval in seconds. If set, the script will be executed periodically at that interval. If omitted, the script will be executed only once. All intervals are rounded to 3 seconds. The minimum interval is 3 seconds.
+* Returns: uuid – script id that can be used to remove that script
+
+Example of service call:
+
+* Without an interval parameter value, the script executes once:
+
+  ```text
+  (call node ("script" "add") [script] id)
+  ```
+
+* With an interval parameter value _k_ passed as a string, the script executes every _k_ seconds \(21 in this case\)
+
+  ```scheme
+  (call node ("script" "add") [script "21"] id)
+  ```
+
+### script remove
+
+Removes recurring script from a node. Only a creator of the script can delete it
+
+* Arguments: _script id_ \(as received from _script add_\)
+* Returns: true if the script was deleted and false otherwise
+
+Example of service call:
+
+```scheme
+(call node ("script" "remove") [script_id] result)
+```
+
+### script list
+
+* Arguments: None
+* Returns: A list of existing scripts on the node. Each object in the list is of the following structure:
+
+  ```javascript
+  { 
+      "id": "uuid-1234-...",
+      "src": "(seq (call ...",  // 
+      "failures": 0,
+      "interval": "21s",
+      "owner": "123DKooAbcEfgh..."
+  }
+  ```
+
+  Example of a service call:
+
+```scheme
+(call node ("script" "list") [] list)
+```
+
+### op identity
+
+Acts as an identity function. This service returns exactly what was passed to it. Useful for moving the execution of some service topologically or for extracting some data and putting it into an output variable.
+
+Example of service call:
+
+```scheme
+(call node ("op" "identity") [args] result)
+```
+
+### deprecated add\_provider
+
+Used in service aliasing. \_\*\*\_Stores the specified service provider \(provider\) in the internal storage of the node indicated in the service call and associates it with the given key \(key\). After executing add\_provider, the provider can be accessed via the get\_providers service using this key.
+
+* Arguments:
+
+  * key – a string; usually, it is a human-readable service alias. 
+  * provider – the location of the service. It is an object of the following structure:
+
+  ```javascript
+  { 
+      "peer": "123D...", // PeerId of some peer in the network
+      "service_id": "uuid-1234-..." // Optional service_id of the service running on the peer specified by peer 
+  }
+  ```
+
+Example of service call:
+
+```scheme
+(call node ("deprecated" "add_provider") [key provider])
+```
+
+### deprecated get\_providers
+
+Used in service aliasing to retrieve providers for a given key.
+
+* Arguments: _key_ – a string; usually, it is a human-readable service alias. 
+* Returns: an array of objects of the following structure:
+
+  ```javascript
+  { 
+      "peer": "123D...", // required field
+      "service_id": "uuid-1234-..." // optional field
+  }
+  ```
+
+Example of service call:
+
+```scheme
+(call node ("deprecated" "get_providers") [key] providers)
+```
+

tutorials_tutorials/add-your-own-builtin.md

@@ -1,6 +1,6 @@
 # Add Your Own Builtins
 
-As discussed in the [Node](../knowledge_node_services.md) section, some service functionalities have ubiquitous demand making them suitable candidates to be directly deployed to a peer node. The [Aqua distributed hash table](https://github.com/fluencelabs/fluence/tree/master/deploy/builtins/aqua-dht) \(DHT\) is an example of builtin service. The remainder of this tutorial guides you through the steps necessary to create and deploy a Builtin service.
+As discussed in the [Node](../node/knowledge_node_services.md) section, some service functionalities have ubiquitous demand making them suitable candidates to be directly deployed to a peer node. The [Aqua distributed hash table](https://github.com/fluencelabs/fluence/tree/master/deploy/builtins/aqua-dht) \(DHT\) is an example of builtin service. The remainder of this tutorial guides you through the steps necessary to create and deploy a Builtin service.
 
 In order to have a service available out-of-the-box with the necessary startup and scheduling scripts, we can take advantage of the Fluence [deployer feature](https://github.com/fluencelabs/fluence/tree/master/deploy) for Node native services. This feature handles the complete deployment process including