NGC

Getting started

So, you decided to give it a try? At the current stage NGC is in Alpha release. Only limited amount of users have access to it.

NGC is accessed through remote workspace called sandbox. If you don't have an access to it you may try to request it at our developers discord server Code in sandbox is managed by git version control system. Sandbox can use multiple git repositories (private or public) and there is also one special - official community sandbox. This is the default repo when setting up your sandbox.

Setting up your sandbox for the first time

The next thing to do after you have been approved to use the sandbox is to create and connect to it. Sandbox is accessed through OpenVPN private network. Please download and install the OpenVPN client side software on your machine:

Windows users: https://openvpn.net/client-connect-vpn-for-windows/
MacOS users: https://openvpn.net/client-connect-vpn-for-mac-os/
Linux users: Install directly from terminal with
sudo apt-get install openvpn

In order to connect to VPN you need to have your client certificates which we share as encrypted with GPG keys. Before creating a sandbox it is required for you to setup your GPG keys. Grab it and install directly from GnuPG download section. Then, please follow the instructions from GitHub documentation to setup your keyring. Skip uploading your public key to GitHub and instead upload it to your GitLab account in db.nomocode.io here. Please note key you setup must match email in your GitLab account.

When this is done, you can create your sandbox by issueing the following command in discord ngc channel:

!ngc sandbox start

When your sandbox is ready you will recieve notification from Sandbox bot. In the message there will be links to your OpenVPN client certificates. Download one based on which operating system you have and then decrypt it with GPG ( here as an example user named Charlie ):

gpg --output Charlie.ovpn --decrypt Charlie.ovpn.gpg

Load your OpenVPN profile ( decrypted client certificates ) to OpenVPN Connect application and connect to VPN. For Linux users connect from terminal ( substitude config path accordingly ):

sudo openvpn --config /path/to/your/certs/Charlie.ovpn

After successfully connecting to VPN network open the following link in your web browser

http://sandbox.ngc

Then open console in code server and init your workspace with the following command

ngc init

You will be asked to generate your ssh keypair. Just follow the instructions and upload your public ssh key to your GitLab account. Next, when everything is done. Open ~/workspace directory.

You are now ready to go! Just as a starting example you can build and explore this documentation project inside your sandbox. Copy and run the following command in terminal

cd projects/docs && ngc build

Then, visit http://docs.ngc

When you are finished with your development in sandbox just disconnect from VPN network and issue the following command in discord ngc channel:

!ngc sandbox stop

Next time connection

The above instructions were just one time setup, next time when you will want to enter your sandbox just type in discord ngc channel:

!ngc sandbox start

Then wait for notification from Sandbox bot and connect to VPN and open sandbox in your browser.

When finished you can stop it with:

!ngc sandbox stop

Destroying your sandbox

You may decide at some point you don't want to use your sandbox anymore. You can destroy your sandbox completely which is also equivalent to removing all of your data stored in sandbox - you will no longer be able to connect to sandbox with your current VPN certificates and you will need to through the first time setup instructions once again. In order to destroy the sandbox type the following command in discord ngc channel:

!ngc sandbox destroy

Templates

NGC uses Jinja2 templating engine to leverage code reusability to another level. With Jinja2 templating almost every project file can be a template ( excluding project.ngc.yaml and values.yaml ). In practice it means you can create not only projects but also project templates making your application configurable and flexible. Templates are stored in templates directory and/or directly within user project. Templates in templates/projects are collection of files which together form a project basis. This basis can be then inherited or composed in user project allowing to lay the common ground across multiple projects.

How it works in NGC?

When project is being built every source file is copied into build context. Build context is just a temporary build directory. Source files are copied starting from very bottom of the project inheritence - first files from inherited project templates are copied then the user project itself ( all in the order of inheritence ). If there are multiple files with the same name in the same relative directory, then files are overwritten in favor of child ( same behaviour like overwritting class method in any programming language ). After all project files are copied into build context, Jinja2 engine is started and it will render all project files with *.template file extension. Templates are rendered using variables stored in values.yaml file. NGC is looking for this file in the project root directory. When multiple variables.yaml are provided ( both in project template an user project ) values stored by this file file are merged together. Values from base/template project can be overwritten in a same manner template files are. Please note once again: values.yaml and project.ngc.yaml are only files that cannot be templated.

This is good but I still don't know how Jinja2 works

Here we are just discussing how Jinja2 fits into NGC. If you are new to Jinja2 templating please refer to the official documentation, as the Jinja2 usage is out of the scope of the NGC documentation.

Projects

Projects in NGC are user fullstack applications deployed independently. The purpose of functions and project templates are essentially to be used in user projects. User projects are stored in projects directory and they are created with microservices architecture in mind utilizng Docker containerization. Eventually, each project is encapsulated in docker containers allowing to bring into project any tool external libraires or frameworks users like. Users can actually forget about all NGC related stuff and develop application as usual they would without NGC.

User projects are split into two types based on the number of microservices: single or multi application project

Single application project

Single application user project, or just single app project in short, are projects which deploy only one microservice i.e. docker container for example a node.js running discord bot. Single app projects are requires project.ngc.yaml file to be present at project root directory. More on project.ngc.yaml will be discovered later in this documentation. Because every NGC project is built as docker container also a Dockerfile can but there side by side to the project.ngc.yaml. However, If Dockerfile is not present a default one will be inserted with http static server running on nginx. Note, you can also derive Dockerfile from base project ( project template ) - we are talking here about files which will ultimately land into the build context meaning files across whole project inheritence tree count.

Multi application user project

This kind of project requires multiple microservices and as you already may get our point we are talking about apps and microservices interchangeably. In NGC we call microservice an app and project can of course have multiple apps. Multi-app project contains apps folder in the project root directory. Apps folder contains another folders each for one application which are essentially just a collection of single app user projects. Apps are always single application projects meaning they cannot have another apps folder inside. Structure of Multi app project is therefore flat and it is just a composition of equally permitted apps.

Remember, user projects can only be created in projects top level workspace directory. Project templates are as the name suggest just templates and they have to inherited in user project to be used ( they behave like an abstract class concept from programming )

Functions

Functions in NGC workspace are smallest independent piece of code. If stored in libs directory, they can be reused across multiple projects within workspace. There exist special rules/constraints function implementation must meet to be stored in libs directory.

Language

Only TypeScript functions are supported right now. In future this will be extended for other languagues

Name convention

File name ( excluding extension ) needs to match the function name being implemented in this file.

One function per file

Exactly one function should be implemented in one file.

Importing and exporting convention

Function needs to be exported using "export default" and export clausure have to immiedietly preceed function implementation. Also, only default importing is allowed. Function alias in import statetemt doesn't need to match imported function name.

No implicit types

Every function argument need to have it's type explicitly written. Same applies for the function return value

Always documented

Functions have to be documented using jsdoc like comments format. The format is extended to include additional tags used by NGC:

@brief - function description
@speak - nomospeak sentence for function usage
@inline - tells NGC this is an inline function
@notest - indicates function has no automated tests ( supress warnings )

Examples:

 import Random_getRandom from "./getRandom"
 
 /**
  * @brief Returns a pseudorandom integer between 0 and max ( max not inclusive )
  * @param max not inclusive right bound ( max value ) of the generated integer
  * @return generated pseudorandom integer
  * @speak get random integer less than {max}
  * @notest
  * @inline
  */
 export default function getRandomInt(max:number): number {
   return Math.floor(Random_getRandom() * max);
 }


Workspace overview

What is workspace

Workspace is just a git repository with a special structure that enables NGC to operate on it. NGC does not care if your workspace is hosted on GitLab, GitHub, Bitbucket or other VCS. Additionally, there exists official community workspace called sandbox. This workspace is public and contains open source projects, libraries and templates everyone can benefit from. This is also the default workspace being initialized in your sandbox upon creation.

Workspace structure

As mentioned, NGC relies on special structure of the workspace and we will briefly discuss it below. There are few mandatory top level directories that need to exist to call any git repository a NGC workspace.

libs

This directory contains all of the common functions organized within subfolders. These functions are common meaning they can be reused in any project within workspace. Functions are written in TypeScript but in future other languages will be supported. Each function has it's own file meaning only one function can be stored in single file. Apart from functions, libs folder also store tests for functions. Tests are executed using jest javascript test framework. Tests have to end with *.test.ts extension and are also written using TypeScript.

templates

Templates folder holds common Jinja2 templates which can be reused in projects. Templates are split into files and projects. Template files are just independent single files wheras templates projects holds whole projects structure and can be used as any project foundation.

projects

This directory keeps all projects within single workspace. We name project an independent fullstack application. From architectural point of view project is a collection of microservices that work together as a signle system ( project ).

filters

Filters directory contains implamentations for custom Jinja2 filters

Scripting language

Every user project ( or app in case of multi app project ) includes project.ngc.yaml special file. This file is a mix of project configuration and source code entries. Content of this file is yaml formatted and we call it NGC scripting languague. Apart from obvious project configruation statements in this file we also include top level entry of the code. This means every application will start executing the code written project.ngc.yaml first ( unless you decide to keep your code separate ).

So why this scripting language even exists? In fact developers can use any other language they want in any app and still it will be totally fine. The reason we introduce this language is to have one abstration above all other languages which gives are several advantages in the long run.

Firstly, we are no longer obliged to use only TypeScript for shared code in libs. In future we will be support more and more programming languages and by introducing scripting languages acts like a adapter. We will be able to use either programming language and even automatically reimplement some of the existing functions into another languages.

Secondly, by using abstract scripting language we can easly track functions dependencies, create intelligent CI pipelines which is building and running tests only for exactly the changed or affected code.

No more imports. You don't need to think of them. Functions are automatically resolved when you start using them. Additinal, external libraries are automatically added.

Code optimization. By using scripting language you don't produce dead code. Only functions which are actually used are included. Moreover, you can benefit from code minification and obfuscation, functions inlining and more.

Foundation for spoken language which will allow users with less programming knowledge to contribute to the code, or program in GUI mode.

Scripting language reference

Scripting language in project.ngc.yaml includes both project configuration and code statemets:

from

Project configuration statement which defines project base template. Expected value is a relative path to project template starting from templates directory. For example:

from: Docker/web

Tells NGC this project is based on Docker/web template project

do

Top level "do" keyword is a entry point for code execution. Expected value is a list of scripting language instructions to execute

Example:

 do:
  - var:
   name: counter
   value: 0
  - Arithmetic/add:
   args:
    left: counter
    right: 1
   return: counter


var

Defines new variable. Expects 2 properties:

name: name of the variable

value: value of the variable

Example:

  - var:
   name: counter
   value: 0


Function call

You can call any shared libs function by specifing it's relative path in libs directory ( without file extension ). Function call expects args property to be set in case any function argument needs to be passed and return property in case you want to save function result. Args is not an array but key, value pairs of function arguments names and passed values. Example:

 - Arithmetic/add:
  args:
   left: 2
   right: 1
  return: counter


function

Defines new function. Function definition requires name and do properties. If function takes arguments, set also args which consists of key-value paris defining argument name and its type

 - function:
  name: printMessage
  args:
   msg: string
  do:
   - Console/Log:
    args:
     text: msg


if

If conditional. Executes code if condition is true. Optionally, executes code if condition is false in else block. conditional expression is set in condition key and right now support only plain javascript.

 - if:
  condition: counter > 10
  do:
   - Console/Log:
    args:
     text: |
      "Counter has reached it's maximum"
  else:
   - Console/Log:
    args:
     text: |
      "Counter not yet saturated"


while

While loop executes block of code as long as condition is true. Structure is very simillar to if statement except there is no else block

 - while:
  condition: counter < 10
  do:
   - Console/Log:
    args:
     text: |
      "Counter value is" + counter
   - Arithmetic/add:
    args:
     left: counter
     right: 1
    return: counter