Turbulent Dynamics developes Maching Learning, MPI and iOS applications for both High Performance Computing (Supercomputing), edge devices (Xavier, Raspberry PI, MyriadX) and MacOS. Minimising system admin workload is not trivial so this guide was created to try setup a common dominator for all projects.
Turbulent Dynamics developes Maching Learning, MPI and iOS applications for both High Performance Computing (Supercomputing) edge devices (Xavier, Raspberry PI, MyriadX) and MacOS. Minimising system admin workload is not trivial, as different devices require a different stack, especially edge devices, and sometimes sudo is not available (on HPC systems). This drives out environment and app choices.
- Avoid sudo installs by using Brew for basic tools.
- Avoid sudo and allow multiple versions of apps using Spack (also compiles all dependencies giving performance advantages).
- Use containers where possible (Edge devices struggle or are unable).
- Use Python Venv, for ML Tensorflow and tools.
| Device | Use Case | Notes |
|---|---|---|
| HPC System | Training ML and Large Scale MPI apps 100s nodes | Sudo not available |
| Workstations (with AMD GPU) | Training ML | |
| Workstations (with Nvidia GPU) | Training ML, rebuilding Xavier/Nano and MPI app testing | Nvidia SDK limits to Ubuntu 18.04 |
| MacOS (AMD GPU) | Visualisations in Metal and iOS apps | Develop in Swift |
| NVIDIA Xavier/Nano | ML Inferencing | Limited to Cuda 10.0, Tensorflow 1.14 |
| MyriadX (Intel Compute Stick) | ML Inferencing | OpenVINO limits to Ubuntu 16 |
| Raspberry Pi | ML Inferencing |
Simple utility to check if OpenMP, MPI and cuda are working as expected. Diagnostics OpenMP, MPI, GPU
- Coding guidelines
- Vector Identifiers The vectors are numbered differently than usual LBM implementations
- Item Identifiers The cells in the outer shell of the lattice grid has been given an identification
- Visualisation 1000 cubes