Category Archives: librism

64x64x16colours (Sweetie16) PixelArt with Pixelorama “β-karoten – We know whom will be eat”

→ Version en français ici

β-karoten - We know whom will be eat

I participated to the LoveByte Battleground demoparty> that runned this week-end, by posting a drawing last week. Sadly/funnily , there was few mistake ^^:
* I made a 64×64 pixels picture instead of a 128×128 one. The palette for the competition was 16 colours Sweetie16 (the default one on FOSS, TIC-80 fantasy console (Source code)).
* My second mistake is I’ve uploaded a first version, and few hours later another one using (FOSS) source code) on Debian on (FOSH) RISC−V (Specifications, there are lot of free or not implementations) emulator of FOSS Qemu (Source code, Git instance), the first upload worked fine but the second one didn’t work (maybe because my installation of Netsurf doesn’t manage javascript)^^. I also performed in a livecoding match (256 bytes and 25 minutes limits). Result of my poor production, Commented Live coding session record.

This is made with FOSS Pixelorama (Source code), itself made on FOSS Godot game engine (Source code). I use FOSS Arch Linux OS. Also made a ArchLinux AUR package pixelorama-git after pixelorama package (for git version, I would like to use v0.9, still not out, only v0.8 was available). There are pixelorama package (last stable, compiling from source), and pixelorama-bin package (from developers binary tarball). Pixelorama is a Pixel art picture and animation editor. I believe I discovered Pixelorama thanks to blog Librearts.org.

The name is “β-karoten – We know whom will be eat”.

Pixelorama editor screenshot
Screenshot of Pixelorama

Telegram, autocollants animés Lottie, Python et Glaxnimate

Sommaire

* Introduction
* Un bon flux de travail
* Publication sur le web (SVG)
* Bot Telegram

Introduction


Je suis un grand fan de l’interface de l’application de chat Telegram. Si le serveur est fermé, l’application bureau et Android est ouverte, l’interface est bien pensée, intuitive, relativement légère et plein de fonctionnalités intéressantes par rapport aux autres applications de ce type. Il serait d’ailleurs sans doute intéressant de la patcher pour une compatibilité Jabber/XMPP. Il a été le premier à ma connaissance, à utiliser Lottie, dès juillet 2019 (et l’API pour bot qui va avec) un format d’animation vectoriel dont je rêvait depuis des décennies, devenu un standard de fait et ouvert. Les animations de cette page ont été crées avec Glaxnimate, un outil pour réaliser des animations au format Lottie, que l’on peut également exporter au format SVG animation. Les auteurs de Lottie fournissent des JavaScript pour le web, la bibliothèque libre, rlottie de Samsung (en C++, avec version WASM) permet de les jouer et comporte des adaptations pour différents langage de programmation. Qt intègre sa lecture par défaut et il existe différents outils pour en créer.


Le logiciel libre multiplate-forme d’animation Glaxnimate permet d’en réaliser avec une interface graphique d’un type assez classique pour les logiciels d’animation. Son auteur à également fait une bibliothèque python, python-lottie (AUR : python-lottie-git, pip : Lottie), permettant d’en générer algorithmiquement et les 2 permettent de convertir différents formats (export SVG animé, Lottie JSON, page HTML toute prête, Telegram, mpeg4, PNG, wepb, gif, et je dois en oublier), et en entrée de vectoriser des gif animés, etc… Le logiciel d’animation vectoriel 2D Synfig à également un exportateur, ainsi que Blender3D, fait également par l’auteur de Glaxnimate. Les choix ne manquent donc pas pour en créer.


Glaxnimate permet de faire facilement des animations au format Lottie, dont le format .TGS, une version aux spécifications volontairement plus limitées et compressée en gzip de Telegram. Le format de ce dernier est plus compact mais comportes quelques contraintes, cela n’a pas empêché d’avoir quelques merveilles depuis plus de 2 ans :
* format 512×512 pixels
* 3 secondes
* 64 Kio par autocollant animé
* Le tout peut être mis dans un paquet d’animations (sticker pack) regroupant différentes expressions.

Pourquoi 512×512 pixels alors que c’est un format vectoriel ? L’application cliente Telegram rendra dans cette dimension l’animation et la compressera en local, pendant la première boucle, en animation WebP de 512×512 pixels pour la conservation en cache. Cette dimension est déjà grande pour les certains écrans mobiles. Cela permet d’économiser la bande passante (animation vectorielle compacte transférée), d’avoir la haute qualité du vectoriel (généralement calculé par GPU, des accélérateurs SVG étaient déjà présent sur les téléphones Symbian des années 2000), et d’avoir des boucles d’animation qui ne prennent pas trop de CPU/GPU après la première boucle, ni trop de mémoire (compression WebP). WeChat/微信 a choisit des GIF animés il y a une bonne décennie, les GIF sont probablement convertit en MPEG aujourd’hui ? La messagerie instantanée Discord, populaire en Extrême-Occident, utilise aussi le format Lottie depuis juillet 2021.


Pour référence, l’animation du haut fait :
* 1109 octets (1,1 Ko) en TGS (format Telegram compressé binaire)
* 4682 octets (4.5 Ko) en json uglifié (plus d’indentations ni re retours chariots)
* 508054 octets (500 Ko) en WebP
* 9710 octets (9.6 Ko) en SVG animé (il y avait un path inutile en trop, je sais pas pourquoi).
* 1804 octets (1.7 Ko) en SVGZ animé (SVG compressé via gzip), il peut être plus intéressant de laisser le serveur compresser via brotli (Nginx, Apache) ou de pré-compresser (.br) Il est bien supporté par les navigateurs
* 6114 octets (6.1 Ko) en SVG, sortie optimisée Inkscape, on peut encore gagner, aujourd’hui si l’option de précision du nombre de chiffres après la virgule est inférieure à 3, elle reste à 3. Dans les parties animation non gérées on se retrouve à six 0 (1.000000 ou 0.000000). On peut encore l’optimiser à la main en attendant de l’intégrer (voir les sed plus bas), soit :
* 5827 octets (5.8 Ko) en SVG, retravaillé un peu à la main.
* 5559 octets (5.5 Ko) en supprimant des groupes intermédiaires inutiles et en remplaçant 1.0 par 1


Un des principaux problèmes à la sortie optimisée de SVG depuis Glaxnimate est que Lottie est basés sur une forte utilisation des groupes, tandis-que SVG permet d’effectuer de nombreuses opérations équivalentes dur les objets eux-mêmes. Ainsi les matrices de transformation sont situées dans des groupes contenant les objets. Voir l’affichage XML d’Inkscape ci-contre. Cela demande donc aujourd’hui du travail manuel sur les fichiers. Il doit être possible d’automatiser ça.

Il semble que SVGO ai d’autres méthodes encore plus efficaces, mais basé sur du node.js, mais il casse complétement les animations. Je n’ai pas trop confiance, en raison de beaucoup de mauvaises habitudes autour de Node. Et le plugin Inkscape, inkscape-svgo ne déroge pas à ses nouvelles habitudes, si l’on essaie de le compiler avec le Makefile inclus, pas de test d’installation existante, récupération forcée de node 11 (x86_64, perd le multi-architecture), et recompilation de toutes les dépendances. On se retrouve une fois de plus avec une extension énorme de dizaines de méga-octets (75 Mo, contre 145 Mo pour Inkscape et 5 Mo pour la bibliothèque SVGO elle même), alors que ça devrait faire que quelques Ko grand maximum.

Un bon flux de travail

* Créer l’objet sous Inkscape
* L’animer avec Glaxnimate
* Exporter en SVG
* Ouvrir de nouveau avec Inkscape pour nettoyer (sortir des groupes ce qui n’en a pas besoin,
* Faire une sortie optimisée
* Retravailler à la main.

Pour la dernière partie, cela oriente vers deux solutions pour l’automatisation :
* Patcher la sortie d’Inkscape optimisée
* Faire un optimiseur couche 2

On peut déjà pas mal gagner avec :

sed s/1.00000/1/g <input.svg | sed s/0.00000/0/g | sed 's/0000;/;/g' >output.svg

Un ou 2 chiffre de précision (à la place de 3 ou 4 aujourd’hui de Scour), devraient également suffire.

SVGcleaner est meilleur que Scour dans différents cas, mais ne supporte pas et n’a pas l’intention de supporter les animations (complexe à gérer, peut facilement tourner à la catastrophe).

Publication sur le web (SVG)

Comme le SVG animé est infiniment plus simple à intégré pour le web je préfère cette option. Le mieux est de ne pas compresser en SVGZ, mais seulement d’optimiser le fichier SVG puis de le compresser en Brotli (.br) et éventuellement en gzip pour des très vieux navigateurs.

L’intégration dans une page est très simple, elle se fait comme pour toute image :

<img src="/chemin/du/fichier_animation.svg" align="right" width="200" height="250">

C’est ce que j’ai fait sur cette page, le navigateur la prend automatiquement en charge.

Pour la compression, j’utilise les max voici un moyen simple d’avoir tous les fichiers d’un dossier compressé à côté de la version non compressée :

ls --ignore=*.gz --ignore=*.br | while read file
do
 brotli -q 11 -c <"${file}" >"${file}".br
 gzip -9 -c <"${file}" >"${file}".gz
done

* 5559 murphy_anim0.animoptiv5.svg
* 1303 murphy_anim0.animoptiv5.svg.br
* 1543 murphy_anim0.animoptiv5.svg.gz
* 4682 murphy.json
* 976 murphy.json.br
* 1149 murphy.json.gz

Pour forcer le chargement d’un des 2, si le navigateur supporte (si il supporte les 2, Brotli (.br) sera utilisé, sous Nginx (après avoir compilé le patch Brotli, les règles dans le fichier de configuration sont simples :

brotli_static on;
gzip_static on;

Sous Apache, Brotli est présent par défaut dans les versions 2.4, mais c’est un peu plus compliqué.

Pour tester, le mieux est d’utiliser la commande curl:

curl -I -H 'Accept-Encoding: gzip, deflate, br' https://host.net/fichier.svg

Si le brotli est bien activé, dans la réponse, vous verrez :

content-encoding: br

Bot Telegram

La bibliothèque python-lottie tombe bien, je voulais me mettre à Python, notamment pour micropython dans l’embarqué et pour du travail d’admin sys & réseau, où il est de plus en plus utilisé, avec l’outil de déploiement de parc Ansible, et d’autres outils système de base. Environ une année d’expérience de génération procédurale en Lua me donnait envie d’en faire d’en d’autres langages (je réserve une surprise pour bientôt). Il existe plusieurs bibliothèques de bot pour Telegram en python, dont Python Telegram Bot, qui colle assez bien à l’API officielle de Telegram et Telethon (Documentation) qui permet de communiquer directement en MTproto, le protocole de Telegram, plutôt que de passer par la couche HTTP, cela réduit d’une couche et permet un contrôle plus poussé des échanges. Voir le comparatif Bot API (HTTP vs MT Proto).


On ne trouve pas encore beaucoup d’exemples avec Python Telegram bot avec lequel j’ai commencé, mais en fouillant un peu dans la documentation, on trouve ce qu’on veut. En mixant la bibliothèque et python-lottie pour la génération procédurale, Glaxnimate pour la génération affinée à la main et le bot on a de quoi s’amuser. J’ai donc fait un premier bot en python-telegram-bot, suites aux suggestions de Glax, l’auteur de Glaxnimate qui m’a fait connaître le module python, Telethon, qu’il utilise pour son bot similaire, Glaxcomm, générant des Lottie/TGS à la volée. Je vais refaire mon bot et l’emmener vers ce que je voulais faire. Mon but étant aussi de pouvoir l’utiliser avec d’autres protocoles que Telegram.

Glax à fait un bon ensemble d’exemples de génération procédurale de base pour comprendre l’utilisation, bonne documentation sur le format Lottie, exemples de scripts de stickers

J’ai fait un bot simple en partant des exemples de la bibliothèque python-telegram-bot, mais il manquait des explications complètes (et je n’ai pas trouvé sur le net) sur la façon d’utiliser le file_id, méthode recommandée par Telegram, plutôt que d’envoyer plusieurs fois le même fichier. En fait ce file_id est récupéré au premier envoie. Donc, l’idéal est de stocker dans une base ou fichier sur disque l’id des fichiers qui ont déjà été envoyés (et reçu) au premier envoie réussi). Cet exemple crée un fichier contenant l’id à côté de l’image, c’est sans doute un peu plus bourrin que de conserver tous les ID dans une base (sql ou TOML) :

if os.path.exists(idfile):
  fid = open(idfile)
  stickid = fid.read()
  fid.close()
  msg = update.message.reply_sticker(stickid)
else
  msg = update.message.reply_sticker(open(stickfile, 'rb'))
  fid = open(idfile, 'w')
  fid.write(msg['sticker']['file_id'])
  fid.close()

Android without Google and GNU/Linux with APK

Android will move from Linux kernel and open specs APK format to a more closed package format and to their closed source Fuschia kernel going away from Free Open Source software (FOSS) that make Google, Android, ChromeOS grow, but also that are used on most of the internet servers, on most of supercomputer in the world, on most spacecrafts, on most internet boxes, TVboxes, etc…

GNU/Linux on your phone now

At the same time GNU/Linux distributions for mobile grow a lot the 2 recent years, thanks to availability of PinePhone for developers for about 2 years. There are now at least 15 Linux distributions for phone and applications maturated a lot, including since few month MegaPixels hardware accelerted camera application, that use GL and other processor to accelerate rendering/computing of what camera receive.

PostMarketOS is a GNU/Linux distribution available for a bunch of available market phones, but not everything is managed an all devices.

GNU/Linux with Android APK compatibility layer

Just like there is Wine to run Windows application on GNU/Linux, there are also several systems allowing to run Android APK on your phone if needed. Jusr as Wine, beside the possibility of running Android in a qemu VM, you can also run these application inside your the standard GNU system.

AnBox allow to run APK/Android application inside a snap packaged container.

Google goes away from FOSS, afraid by the lost of market share, and they probably loose a lot from this, as Microsoft does by the past. Chinese company, thanks to Trump restrictions, embrace more FOSS.

Huawei the maker of the probably highest quality phones, that was going in first seller position just ahead Samsung, when Trump blocked it, already very active in Linux kernel and community for years, made HarmonyOS for their devices, a system, that continue tu uses Linux kernel for their power devices, and LiteOS for their low end/old devices. LiteOS is a FOSS embedded devices oriented light kernel. They keep the EMUI User Interface they used previously on Android, moving step by step away from it, and at the same time keep a compatibility layer with current APK open-source format.

Another Chinese company, JingOS, made high quality tablets, based on GNU/Linux distribution, that derivates from Ubuntu, with KDE tools and Android APK application compatibility using Anbox. And they are active with KDE and open source communities. Few demos here and here.

Removing closed source spywares from your current Android system as a transition phase

Beside long time F-droid Free software apk app repository (I use it for year as main source of APK).

You can remove Android pre-installed (memory+cpu+bandwidth thieves) applications, including NSA compliant one like Google, Amazon, FaceBook, Microsoft ones using from a computer with adb. This allowed me to have less bloated phone and to gain few more days of on-battery usage.

List packages:

adb shell pm list packages

See which one are related to facebook:

adb shell pm list packages facebook

For easier copy paste and have only the package name without package: prefix

adb shell pm list packages | cut -d: -f 2

The corresponding application, can be checked by using in a web browser, this does not work for some system applications, for example, for com.google.android.videos:

https://play.google.com/store/apps/details?id=com.google.android.videos

Remove never at least first seen, never used application:

# FaceBook spyware

adb shell pm uninstall --user 0 com.facebook.katana
adb shell pm uninstall --user 0 com.facebook.system
adb shell pm uninstall --user 0 com.facebook.appmanager

# Microsoft stuff
adb shell pm uninstall --user 0 com.microsoft.office.excel
adb shell pm uninstall --user 0 com.microsoft.office.onenote
adb shell pm uninstall --user 0 com.microsoft.office.powerpoint
adb shell pm uninstall --user 0 com.microsoft.office.word
adb shell pm uninstall --user 0 com.microsoft.skydrive # data thief

adb shell pm uninstall --user 0 com.google.android.apps.docs # Google Drive (data thief)
adb shell pm uninstall --user 0 com.google.android.apps.photos # Google photos (data thief)
adb shell pm uninstall --user 0 com.google.android.googlequicksearchbox # Google Search box (behavior thief)
adb shell pm uninstall --user 0 com.google.android.gm # GMail
adb shell pm uninstall --user 0 com.google.android.talk # Hangouts
adb shell pm uninstall --user 0 com.google.android.music # Google Play Music
adb shell pm uninstall --user 0 com.google.android.videos # Google Play Film & Series
adb shell pm uninstall --user 0 com.google.android.youtube # Youtube application

To be checked

adb shell pm uninstall --user 0 com.google.android.gms # Services Google Play

adb shell Dumpsys a dumper and debugger

adb shell dumpsys allow to dump application before removing them and reinstalling it after that.

I still didn’t tried that, will update it after my tests

adb shell dumpsys package package.name # dump package state
adb pm enable —user 0 nom_de_paquet

Linux syscall and RISC-V assembly

Sample of RISC-V assembly code

Syscall in Linux kernel, is an interface to access to kernel basic functions. They are described in section 2 of man pages. The introduction is in man 2 syscall (indirect system call), and the list of functions are described in man 2 syscalls. ”’Update:”’ System Calls in lectures of official Linux kernel documentation including “Linux system calls implementation”, “VDSO and virtual syscalls” and “Accessing user space from system calls”

This article follow previous one about RISC-V overall progress and available tools to play with, I will try to make a short article here about Linux syscall usage and the RISC-V assembly case.

Table of Content

* Description section of the man page
* Getting the list of function and how to access them
* Passing parameters
* Function number and registers of return values
* Return values and error code
* Compiling and executing on virtual environment
* Update: Bronzebeard assembler and its baremetal environment for real hardware

Description section of the man page

* syscall() is a small library function that invokes the system call whose assembly language interface has the specified number with the specified arguments. Employing
* syscall() is useful, for example, when invoking a system call that has no wrapper function in the C library.
* syscall() saves CPU registers before making the system call, restores the registers upon return from the system call, and stores any error returned by the system call in errno(3).
* Symbolic constants for system call numbers can be found in the header file .

You can find here function, like access to files open/close/read/write/flush, access to sockets, ioctl, uid, gid, pid, messages, ptrace, restart system, etc…

Getting the list of function and how to access them

As far I know, now only a part of syscall functions are accessible easily in assembly, they are defined in /usr/include/unistd.h, and function numbers assigned in ABI are defined in /usr/include/asm-generic/unistd.h.

The more practical match I found is using /usr/include/asm-generic/unistd.h to see which function are available and there respective manpage for the function header definition. For example:
* asm-generic: #define __NR_read 63
* man 2 read: ssize_t read(int fd, void *buf, size_t count);

The ABI with RISC_V as defined in man 2 syscall in section Architecture calling conventions use the registers following 2 tables rules.

Passing parameters

The second table in this section of the man page shows the registers used to pass the system call arguments.

Arch/ABI      arg1  arg2  arg3  arg4  arg5  arg6  arg7  Notes
──────────────────────────────────────────────────────────────
riscv         a0    a1    a2    a3    a4    a5    -

Here are the arguments in the order of the function definition, for example, in read (63) function:

ssize_t      read  ( int fd, void *buf, size_t count );
a0(result) = a7(63)( a0(fd),  a1(*buf),    a2(count) )

For remember, 3 standard I/O file descriptors are STDIN=0, STDOUT=1, STDERR=2, the other are used when opened a file with open and closed by close.

So we set the arguments as this. x0 is the always 0 register:

        addi  a0, x0, 0       # Set STDIN as sources
        la    a1, buffer_addr # load address of helloworld
        addi  a2, x0, 3       # reaad 3 bytes

Function number and registers of return values

And the first one give the register in which put the function number, that will receive return value and errno (error value)

Arch/ABI    Instruction           System  Ret  Ret  Error    Notes
                                  call #  val  val2
───────────────────────────────────────────────────────────────────
riscv       ecall                 a7      a0   a1   -

So for the read function, we need to put 63 (as found in /usr/include/asm-generic/unistd.h in register a7, and registers and following registers will receive the return values, a0 will receive system call result, and a1 an error message (the errno value).

ssize_t read(int fd, void *buf, size_t count);
a0    =  63 (    a0,       a1,         a2)

So can set them as this:

        addi  a7, x0, 63     # set called function as read()
        ecall                # call the function

Return values and error code

After the man page of read(2):
* On success, the number of bytes read is returned
* On error, -1 is returned, and errno is set to indicate the error.

So we can test first the return value of a0 and if a0 < 0 then we jump to part for display the error message, else we can simply display a OK message. So for the branching part, RISC-V in its super reduced set only have < (lt) and <= (le) comparators, you just need to swap registers to compute > (gt) and >= (ge), but this avoid lots more of transistors.

    addi a3,x0,0           # x3=0
    blt  a1,a3, error_seq  # if x1<0 branch to error_seq

We so use here the syscall write function (64) defined as:
* asm-generic: #define __NR_write 64
* man 2 write: ssize_t write(int fd, const void *buf, size_t count);
* So, registers: a0=1 (STDOUT), a1=*buf, a2=count, a7=64 (function number)

And we will finish with exit() syscall, defined as:
* asm-generic: #define __NR_exit 93
* man 2 exit: noreturn void _exit(int status);
* So, registers: a0=return code, a7=93 (function number)

    la    a1, ok           # load address (pseudo code) of ok string
    addi  a2, x0, 3        # set length of text to 3 (O + K + \n)
    addi  a7, x0, 64       # set ecall to write function
    ecall                  # Call the function

    addi  a0, x0, 0        # set return code to 0 (OK) for exit (93) function
    j     end              # unconditional jump to end before quit

error_seq:
    la    a1, error        # load address (pseudo code) of error string
    addi  a2, a2, 0x30     # add 0x30 (0 ASCII code) to the error code
    sb    a2, 7(a1)        # put the (byte) value at position 7 of Error string (before \n)
    addi  a2, x0, 10       # set now length of our string
    addi  a7, x0, 64       # set ecall to write function
    ecall                  # Call the function

    addi  a0, x0, -1       # set return code to -1 (error) for exit (93) function
end:
    addi    a7, x0, 93     # set ecall to exit (93) funciton
    ecall                  # Call linux to terminate the program

.data:
ok:     .ascii "OK\n"
error:  .ascii "Error:  \n"

RISC-V Longan nano

Compiling and executing on virtual environment

If you don't have a RISC-V hardware (can be found as low as 3€ now), you need to have a cross compiler and qemu for emulating instructions, or a whole system installed.

Packages needed for compiling on ArchLinux x86 or ARM for example.

sudo pacman -S riscv64-linux-gnu-gcc  riscv64-linux-gnu-glibc riscv64-elf-binutils riscv64-elf-binutils riscv64-elf-gcc riscv64-elf-gdb

Newlib is a lightweight RV32 (RISC-V 32bits) lightweight library for bare metal that can be used instead of a whole GNU system on embedded devices with low memory capacity (as Longan nano, less than 8€ with screen, see picture below, or 3€ Sipeed RV): riscv32-elf-newlib.

I made a simple shell script to don't have to remember the commands to assemble the code from an x86 platform (work also on ARM or RISC-V one) that take the .s as argument:

name=$1
riscv64-linux-gnu-as -march=rv64imac -o ${name}.o ${name}.s
riscv64-linux-gnu-ld -o ${name} ${name}.o

You can add a strip but better to avoid it if you need to debug it:

riscv64-elf-strip --strip-all ${name}

And it can be executed on non RISC-V platforms by using qemu-riscv64, if it doesn't depend on libraries or if you have them installed, it allow you to test it without having a full RISC-V system installed, qemu is so fantastic. On ArchLinux it is available in package qemu-arch-extra:

qemu-riscv64 ${name}

And can be disassembled (will probably use different instruction than your assembly code, due to RISC-V assembly pseudo-instructions:

riscv64-linux-gnu-objdump -d ${name}

Bronzebeard assembler and its baremetal environment for real hardware

RISC-V Longan nano

Update: Bronzebeard is an assembler with light baremetal environment builder for RISC-V, GD32VF103 as Longan Nano (about 8€ with a screen, as pictured on this article pictures) and Wio (similar board with an added ESP8266 SoC). I made an AUR package of Bronzebeard, and someone made a Mandelbrot set demo in 918 bytes pure RISC-V assembly. You can find some other example in the source of Bronzebeard. gd32vf103inator is a set of tools for GD32V, to manage from a simple random text editor.

RISC-V overall progress

WordPress is so cumbersome (brut long text in SQL DB and other misconception (WTF?), buggy (especially to multilingual content missing) and hard to maintain on long term that I didn’t posted for long time I want to migrate. You can test the new log engine (here specialised in TIC-80 256 bytes code on https://256b.popolon.org/.

I continued my travel toward RISC-V I started as said in a previous post in may 2018.

RISC-V Benefits

Among the benefit of RISC-V beside other implementations:
* Open source and without license fee availability, allowing everyone to participate, implement and have full specifications
* Highest modularity for a processor in specifications, You can reduce the core to only the set of functions you want for a specific tasks, allowing less transistor/more compact specialized cores, and then multiply cores with some specialized in some kind of tasks.
* Vector extension

As example of specific task oriented cores:
* You don’t need a SIMD if you have a more powerful DSP unit, and you can implement this DSP unit using a RISC-V core with Vector extension, that is more efficient that a simple SIMD for this kind of task.
* JIT extension could be useful for some dynamic web server using scripting language (as PHP, Ruby, JS), but less useful for static web server (PCRE-jit computation in servers like nginx is marginal a core could be dedicated on a processor with many cores without this extension). Alibaba Group (Chinese Amazon) developed it’s own RISC-V implementation for its web servers).
* The European processor Initiative started in 2018, with first goal to uses United Kingdom made ARM core for main processor (at this time ARM had solid build/compilation/system ecosystem) with several kind of computing accelerators based on RISC-V, called EPAC (European Processor Accelerators)(a.is). It is developed at Barcelone with several partners inside European Union(a.is), Sadly as most European union project it will help USA industry for manufacturing, (GlobalFoundries) instead of maintain, sustain and develop local knowledge, experience and workforce for independence, employment and smaller carbon footprint shipments (as China and United-States do). The first test version is a 22nm FPGA, the production one will be 12nm ASIC, and among the objectives, there will be to highest throughput while not going above the 1GHz top barrier, to achieve higher energy efficiency.

Operating System working on RISC-V in June 2021

First efforts with toward embedded operating systems as first available RISC-V chips was in this domain. As the RISC-V embedded market is mature, more effort are made toward mobile and desktop Operating Systems.

Several Linux distribution already work on RISC-V now. Next Debian Bulleye (release August 14 2021, already in beta), Fedora have a RISC-V flavor already working on the BeagleV Starlight open-source board and its StarFive JH7100 processor, and there is a port of Yocto.

For reference, here is instructions about making the RISCV64 virtual machine on Debian Wiki, and a more general guide to make Linux run on Qemu, RISC-V.

You can find the Debian RISC-V system image on this page, just click on to download (direct link). This image is made with Debian Quick Image Baker (dqib)

To run it, you also need OpenSBI (Open Source Supervisor Binary Interface) needed to load the bootloader (Documentation, Prebuild deb packages) I used deb package on Archlinux:

ar x ../opensbi_0.9-1_all.deb
SBIPATH=`pwd`
(cd /; sudo tar xf "$SBIPATH/data.tar.xz")

And the Uboot for Qemu. I personally just extract and the directory that also contain, the “artifacts” Debian image.

By default image is in Qcow format, that is more compact for unused blocks, but also really slower.
You can convert it easily by:

qemu-img convert -f qcow2 -O raw image.qcow2 image.raw

Here is the boot.sh to launch it (you can replace image.raw by image.qcow2, if you have a verypowerfull system and only place count: I use 2 cores and 512M of RAM. For huge compilation I restart it with more mem (1536M is generally enough, but more means more cache and faster compilation).

cores=2
mem=512M
qemu-system-riscv64 -machine virt -cpu rv64 -smp ${cores} -m ${mem} \
  -device virtio-blk-device,drive=hd -drive file=image.raw,if=none,id=hd \
  -device virtio-net-device,netdev=net -netdev user,id=net,hostfwd=tcp::2222-:22 \
  -bios /usr/riscv64-linux-gnu/lib/opensbi/generic/fw_jump.elf \
  -kernel ../qemu-riscv64_smode/uboot.elf \
  -object rng-random,filename=/dev/urandom,id=rng -device virtio-rng-device,rng=rng \
  -nographic -append "root=LABEL=rootfs console=ttyS0"

The login/pass are root/root.

you can connect by ssh:

ssh -p 2222 root@127.0.0.1

and rsync files via:

rsync -a --rsh="ssh -p 2222" root@127.0.0.1:path/to/rsync .

I already managed to compile TIC-80 on it and run it on Qemu:

.

There is also BareDOOM a port of DOOM that run on RISC-V emulated in TinyEmu (source code in the same page, there is also a fork with additional features on GitHub, a system emulator for the RISC-V and x86 architectures), JSLinux allow to run it in a browser (There is also a Quake port on RISC-V K210 microcontroller based Sipeed Maix Bit board). There is also a patched version of buildroot for K210 (no-mmu), for building Linux on this platform.

There is an important Note in system images from Fabrice Bellard site to make binary working with TinyEMU:


- TinyEMU only supports raw boot loader and kernel images. So after
  building riscv-pk or the Linux kernel, you must convert the ELF
  image to a raw image with:

  riscv64-unknown-linux-gnu-objcopy -O binary bbl bbl.bin
  
  riscv64-unknown-linux-gnu-objcopy -O binary vmlinux kernel.bin

A port of Haiku has already well progressed, also use TinyEMU for the port.

Update 2 July 2021: Another RISC-V (RV32 only) emulator is Nemu (github) it is a part of XiangShan (香山) RISC-V open processor from Chinese Academy of Sciences (OpenXiangShan (Gitee)) made by Chinese Academy of Sciences, the processors should have the performances of an ARM Cortex-A72/A73 in july 2021 with a 28nm process in first batch (Yanqi hu / Yanqi Lake / 雁栖湖), and performances of an Intel i9-10900K in second batch ( Nanhu / South Lake / 南湖) in fall 2021, with 14nm process, both manufactured by TSMC, and it already run GNU/Linux distribution Debian. See also(中文).

Another Virtual Machine emulator for RISC-V is JuiceVM supporting RV64IMASU. It is lightweight, support small UART serial I/O and MMU

Available RISC-V Hardware in June 2021

Several developers have received a first beta version of BeagleV RISC-V board. Here are demos (on youtube sadly) of first tries of different devs at beginning of may "BeagleV RISC-V Computer (Beta)" (unboxing), "BeagleV from BeagleBoard.org Fedora image Live Booting - 1" (booting), now BeagleV give a prepared Fedora with XFCE working on HDMI.


I have now 2 embedded microcontroller oriented RISC-V Board from Sipeed, one 32bits and one MaixDuino SipeedM1 (microPython (by default), dual-core 64bits+AI module) of the young hackers, Shenzhen based, SiPeed company. The last one support lot of embedded systems including default MicroPython based MaixDuino system (and cool plateforrm.io interface, Arduino, PlatformIO, microPython, OpenMV, FreeRTOS, et TensorFlow, etc..

The Nezha board of SiPeed, is a bit more expansive and with lower specs in first version, but very promising for the future Nezha: Your first RV64 Linux SBC for IoT~ (video Youtube).

Huawei Hi3861 is oriented toward their LiteOS kernel that is used in their new Android compatible HarmonyOS.

The BL602/BL604 de Bouffalo Lab (RISC-V WiFi & Bluetooth 5.0 LE) based on ESP32-C (RISC-V variant of ESP32, pin2pin compatible) is a 100 % open source BootLoader & driver (no blob needed), thanks to the work of Lup Yuen Lee (李立源). He wrote lot of interesting articles about the process of making the driver and a book about this chip, and several article around Ecosystem, including using Rust, How to simulate it in WebAssembly, etc.

China Science Academy is currently working on optimizing Firefox (SpiderMonkey) and Chrome/ium (Webkit) engines for RISC-V 2000 laptops with equivalent power of a Cartex-A78 should be available at then end of 2022

At the same time the port of true GNU/Linux on embedded devices like phones progress fast, on ARM architecture, with specialized distributions like PostMarket OS (and it's derivative Alpine Linux), or more general distributions, including Debian, Manjaro (ArchLinux derivative) etc. Port of KDE Plasma and GTK4/GNOME40 (especially GTK4.1+ with very efficient OpenGL acceleration, are of great help in this area. PinePhone from Pine64.org community is really an active and efficient hub for these ports. This phone (developer oriented in current state) is at the same time cheap (<200€) have hard switch for all network features, and every part can be easily changed. Those project are all ready to be ported on RISC-V versions of this kind of devices. Pine64 is already working on RISC-V opensource bl602 driver (see above) and a RISC-V (GD32VF103TB) running a smart/programmable iron solderer, and a future RISC-V based mother board to replace the current ARM one on the PinePhone could be totally realistic. See also LINux on MOBile for news on the port of Linux on Mobile devices.

An interesting post (and blog) about RISC-V assembly programming from Stephen Marz, another blog about RISC-V assembly from Daniel Mangun and one from ieee.org Build a RISC-V CPU From Scratch. A simple HelloWorld example, with deep explanations, including Linux RISC-V Application Binary Interface (ABI), to start with RISC-V assembly, based on GNU Assembler (support RV32 & RV64). There is also a Python made, limited, RISC-V assembler. A complete tutorial in French for starting with the Sipeed Longan Nano, (available for ~5 € with LCD screen).

HTML5 RISC-V Interpreter/simulator. There is also RARS a really slow/unusable Java RISC-V simulator (pacaur -S rars), jupiter (with link to other simulators) (pacaur -S jupiter) is another one, and spike (pacman -S spike) is a c made simulator. Ripes (pacaur -S ripes-git) is a very advanced one, with multiple level of pipe, data/instruction cache states, etc... Venus with online simulator in Kotlin language.

Tutorial serie about designing RISC-V processor in VHDL.

I hope that with progress I made in real-time programming and different new programming languages in last few years (Lua, some bit of RISC-V assembly, JavaScript), I will make quickly some RISC-V demo on embedded board.