Abstract

This guide introduces developers to HyperBEAM's distributed computing framework through hands-on device extension. Learn how to leverage Erlang/OTP architecture and the Converge Protocol to create custom devices. Beginners will gain practical experience through a calculator device demo, understanding NIFs (Native Implemented Functions) and WASM port communication patterns.

Chapters

1. Introduction to HyperBEAM

2. Converge Protocol : the root of device call logic and path

3. Building a Simple Calculator Device

4. Conclusion

1. Introduction to HyperBEAM

HyperBEAM is a distributed computing platform built on Erlang/OTP that enables message-based device interactions through the Converge Protocol. Let's explore its architecture and how to extend it with custom devices.

1.1 Application Structure

The heart of HyperBEAM lies in its application configuration, defined in hb.app.src. This file serves as the blueprint for the entire application, specifying dependencies, registered processes, and environmental configurations. It orchestrates how different components interact within the system. It's the backbone of the HyperBEAM system, ensuring smooth operation and seamless communication between different components.

{application, hb, [
    {description, "HyperBEAM - Distributed Computing Platform"},
    {vsn, "0.0.1"},
    {registered, []},
    {mod, {hb_app, []}},
    {applications, [
        kernel,
        stdlib,
        cowboy,
        gun,
        jiffy
    ]},
    {env, []},
    {modules, []}
]}

1.2 Build Configuration

HyperBEAM utilizes Rebar3 as its build tool, providing a robust build and dependency management system. The rebar.config file manages everything from compilation settings to test configurations, ensuring consistent builds across different environments. This configuration handles both Erlang code and native implementations, supporting multiple programming languages like Rust and C++.

To use Rebar3 with HyperBEAM:

1. Install Rebar3 following the instructions at rebar3.org

2. Common commands:

   • rebar3 compile - Build the project

   • rebar3 eunit --module=dev_calculator - Run unit tests

   • rebar3 shell - Start an interactive Erlang shell

   • rebar3 release - Create a release

   • rebar3 clean - Clean build artifacts

2. Converge Protocol : the root of device call logic and path

The Converge Protocol is HyperBEAM's innovative messaging system that forms the backbone of all device interactions. It implements a standardized way of communication between different components of the system, using a message-based approach that ensures consistency and reliability.

The protocol uses a dual-message structure (M1 and M2) that provides context and payload information respectively. This design allows for flexible device interactions while maintaining strict type safety and validation. Messages are structured as maps containing binary keys, making them both efficient and easy to extend.

For details, refer to the Converge Protocol documentation.

Therefore, the device call path orchestrates distributed computation through four critical phases:

1. Initial Resolution: The system first resolves the device identifier and determines the appropriate module to handle the request. hb:converge(Message1, Message2) -> {Status, Message3}

2. Message Validation: Messages are validated against expected formats and types.

3. Device Execution: The resolved device handles the request, potentially involving native code execution.

4. Result Processing: Results are formatted and returned through the system.

3. Building a Simple Calculator Device

In this section, we'll walk through the process of building a simple calculator device using HyperBEAM. This device will demonstrate how to extend HyperBEAM with custom logic by sending calculation messages (such as sum or divide) to the calculator device. The device processes these calculations using C source code and returns the results to HyperBEAM.

3.1 Prerequisites

To compile the entire codebase on Ubuntu 22.04, ensure that Erlang and other related environments are correctly configured. For the convenience of developers, we have provided the necessary information at this link. If you encounter any issues during installation, please contact us promptly.

3.2 Codes added and modified

1. Add calculator to preload_devices in hb_opts.erl

   preloaded_devices =>
       #{
           ...
           <<"test-device@1.0">> => dev_test,
           <<"calculator@1.0">> => dev_calculator
           ...
       },
   
   

2. Adding calculator C++ related files

       1. native/dev_calculator_nif/dev_calculator.cpp : actual execute file
       2. native/dev_calculator_nif/dev_calculator_nif.cpp : communicate with erlang
       3. native/dev_calculator_nif/dev_calculator_log.cpp : print logs
   
   

3. Adding calculator erlang related files

       1. src/dev_calculator_nif.erl : direct call c++ program
       2. src/dev_calculator.erl : integrate with Converge Protocol
   
   

4. Compile configuration file

       rebar.config : add c++ compile statement
   
   

3.3 Run & Test

1. Compile the whole codebase

   rebar3 compile
   
   

2. Run the calculator unittest with debug info

   HB_DEBUG=1 rebar3 eunit --module=dev_calculator
   
   

3. If you see the following info, it means the test is passed. Congratulations!

Please note that all above source codes are available in the APUS Network Github Repository.

4. Conclusion

This guide demonstrates HyperBEAM's capabilities through its Erlang/OTP architecture and Converge Protocol, exemplified by our simple demo calculator device implementation that validates NIF and WASM port communication patterns. As shown in the tutorial, developers can seamlessly integrate heterogeneous computing units using standardized protocols.

While APUS network faces GPU challenges like deterministic execution and communication latency, HyperBEAM addresses these through WASM port optimization and emerging standards like WASI-NN. These efforts advance HyperBEAM's vision of "enabling any device to join distributed networks effortlessly" paving the way for next-generation distributed GPU computing solutions.