Category Archives: programmation

Playing with button and LEDs on RISC-V based ESP32-C3 NodeMCU board with ESP-IDF (FreeRTOS)

Table of Content


* Introduction
* Circuit
** Components
** Breadboard
** Choose GPIO ports and their board pins

** LED part
** Resistors
** Switch button part
* The Software
** Initialisation
** Main loop
** ISR (Interrupt Service Routine)
** Debouncing
*** ESP timer

Introduction

After ArchLinux upgrade from python 3.9 to 3.10, tools need to be reinstalled by:

cd ~/esp/esp-idf
git pull
git submodule update --init --recursive
./install.sh esp32c3

If you never used ESP-IDF, you can read the previous introduction article to ESP-IDF on RISC-V based ESP32-C3, how to install it and start environment for compiling and flashing code. I also wrote article about using ESP32-C3 with Apache NuttX POSIX OS, but it will be useless here.

This article is about, on ESP32 (more specifically a less than 3.5€ ESP32-C3 based NodeMCU board, but it should work about the same way on other ESP based boards) :
* How to blink an external LED using GPIO, including how to know LED needed voltage, amperage, and compute needed resistor, with several possible means.
* Explanations about resistors values colours bands and computation of parallel mounted resistors. I also give link to free and open source softwares I wrote to help to compute resistors (depending on led, and desired intensity).
* How to connect an external switch to GPIO, and which resistor is needed. How to receive and manage it’s state a good way. By debouncing physical human pressure on switch, and use software interruption (that’s more easy that it could sounds).
* How to blink included RGB LED and stop/start it by using switch, an asynchronous way.

Hardware: The Circuit


After 20 years without practising electronics and searching about how to make the circuitry, for the LED and for the button, I found several one for each part. I finally found a some article that give explanations for this board, but using Arduino, and with lot of deep errors. Too strong resistor, not at the good place, after gathering of informations of lot of sources and after made lot of tests and looking back to source, I decided to write a complete tutorial for real beginners like me. This could help me to understand again all needed bases when needed, and I hope it will be useful for other people too.

Components

A basic tool to test and learn on circuits is a breadboard, it allow to test without needing to solder anything. It can so also be used by children, as 3,3 V is not dangerous at all.

* We need also a 5mm LED, that can be found on old electronics circuit or are really cheap, we use a RED that is perfect to represent an ON or OFF state and with 1,8V can be managed by 3,3V board.
* We also need a switch button. A cap is more comfortable but not required to make it work.
* We also need a resistor, if you have a 75 ohms (75Ω), or a 100Ω + a 330Ω, that’s perfect, else a 100 ohm alone is just nice, above 100 and until 330Ω there will still have light, but it could not be very bright.

You can also use a 10KΩ resistor for the switch, called pull-up (or pull-down) to have a more perfect signal, but external resistor is not required for this kind of board, as it includes internal pull-up/pull-down resistors, that is more efficient and possible to enable or disable by software with the esp_err_t gpio_set_pull_mode(gpio_num_t gpio_num, gpio_pull_mode_t pull). We will not need to use the Open drain mode, used with SPI, where pull-up/pull-down is used as changing active/inactive signal.

Breadboard

half width Breadboard
Breadboard is a very convenient board that allow to test circuits without soldering. Any kind of components can be easily inserted and removed, and contact just work. The holes follow the weird electronics standard space of 2.54mm. This is because 2.54 mm is 1 inch of the imperial units, as modern electronics was made in the US, and this is the only one country that still uses this very complex units system. System International (SI), made during French Revolution, is far easier for any kind physics computation.


half width Breadboard with electric links drawn
In brown, electric links hidden inside breadboard

The plastic made board contains underlying electric links. They are divided on three parts, large central part and two side part.
* The two side part are just long line generally used for Ground (GND) and Voltage (V or Vcc). Blue (for GND) and red (for Vcc) lines are drawn to help to avoid to mix both VCC and GND on the same line and so avoid short circuits. Having one VCC+GND backbone one each side is just perfect for microcontrollers that can have both 5V and 3.3V or to have easier access to each side of the board. More efficient boards only use 3.3V.
* In the middle part, electric links are perpendicular to the side ones, separated in the middle, and are used to place electronics components. Some hole counters every 5 holes, and Letter at both extremities of the breadboard are wrote to copy a circuit from a board map an easier way.

Choose GPIO ports and their board pins

Here is the pineout of the board, we need to choose ports used for the circuitry, some pins have GPIO including only digital signal (0 or 1), analog support with analog to digital conversion (ADC) input, or digital to analog conversion (DAC) output. Some pins are to add voltage to electronic components (3,3V ports in RED on the picture) Warning to not connect them directly to GPIO port, you could definitively damage your board..


NodeMCU ESP32-C3-32S-kit pineout
NodeMCU ESP32-C3-32S-kit pineout

Here is the pinout mapping for NodeMCU ESP32-C3-32S-kit. Drawing, thanks to J-C François.

You can generally found them in the documentation of your board. founding this kind of schemas is not always easy, but they are often provided y board vendors. List of ESP32-C3 SoC GPIO are on the doc, but every board vendors can choose to map or not them on the board pins.

We choose here GPIO 1 (GPIO_NUM_1 in the API) for the button, and GPIO 2 (GPIO_NUM_2) for the LED. For this specific board, they are respectively second and third pins, on the left side starting from the top.

ADC0_CH1, GPIO1 and GPIO2
In the source code, as we will see later:

#define EXTERN_BUT GPIO_NUM_1 // BUTTON on GPIO1
#define EXTERN_LED GPIO_NUM_2 // LED on GPIO2

LED part

Anode (+) and cathode (-) on a diode symbolWe will use here a red LED. LED means “light-emitting diode”, this is so a special kind of diode that emit light. A diode work only in one direction, start at a determined voltage, and shouldn’t be powered over a top voltage to avoid damages. If it is powered in the wrong direction (and not too much), nothing happen.


Diode and LED symbols
Diodes (at left) and LED (at right, with arrows) symbols.


Anode leg is longer than cathode one
Anode leg is longer than cathode one, the leds rolled by themselves on the flat side.


led bottom ring is flat at the cathode side
led bottom ring is flat at the cathode side

For LED part we need a 330Ω resistor. electronics LED work with different ranges. About 10 to 50mA, 20 to 30mA is generally safe, but warning, sometime less than mA. Generally with lower value than 20mA, the LED will not be very bright. Voltage depend on LED colour. Try to keep in the middle of these values to have enough light and to not burn your LED. The best is to follow the specifications of your LED vendor.
* IR 1,2 to 1.6V
* Red 1,8 to 2.1V
* Orange/yellow 1.9 to 2.2V
* Green 1.8 to 3.1V
* White/UV 3 to 3.4V
* Blue 3 to 3.7V
* RGB, each colour pin has his own colour voltage.

You can test the “Forward Voltage” of a LED with a digital multimeter. Forward because this is the direction where it lights.


Digital multimeter in diode testing position
Digital multimeter in diode testing position

Select the diode mode of the multimeter (as on above picture), and touch the Anode (longer leg, to the rounded side of the LED base) with the red connector, and the cathode (shorter leg, to the flat side of the LED base) with the black connector.

We use a red LED, that consume about 1,8V. The ESP32C3 has 3.3V output, so there is a difference of :

3.3 - 1.8 = 1.5V

We need a resistor to avoid an over-voltage of the LED.

Resistors

After Ohm law, U = RI, where U is voltage (V), R=resistance (Ohms or Ω) and I intensity (A). So :

R = U/I
 1.5/0.02 = 75Ω


We choose the nearest resistor equal or above this result.

The resistors are painted with rings indicating their resistance. There is in general 4 or 5 rings (or bands). That can be read from left to right as: resistance (2 rings), multiplier (one ring) and tolerance % (1 or 2 rings). Here is a good 5 bands calculators.

A good mnemonic-technique to memorize colours order in English is "Bad Beer Rots Our Young Guts But Vodka Goes Well (in) Silver Goblets". for value rings they start by 0, then 1, etc..., for multiplier by 1, then 10, etc....

Black 0 0 x1
Brown 1 1 x10 ±1%
Red 2 2 x100 ±2%
Orange 3 3 x1000=x1K
Yellow 4 4 x10K
Green 5 5 x100K ±0.5%
Blue 6 6 x1000K=x1M ±0.25%
Violet 7 7 x10M ±0.1%
Gray 8 8 x100M ±0.05%
White 9 9 x1000M=1G x1
Gold ±5%
Silver x0.01 ±10%

Warning, to the light when you look at the bands, some bands can be confused, and it can have disastrous consequences, especially if that's for the multiplier.


Resistors with flash at left and ambient light shadow at right
Resistors with flash at left and ambient light shadow at right. As we can see on the light blue resistor at left, the second painted ring is orange, where it seems brown with the shadow of the natural light et right.

If you have some doubt about their value, you can still use a digital multimeter on their resistance position displayed by a greek Omega character (Ω).


Digital multimeter in resistor testing position
Digital multimeter in ohm (resistor) testing mode position

Some suggest 330 ohms with 5V, this is really too much, and even for 3.3V, that result in

1.5/330 = 0.01A = 1mA

1mA is 1/20th of ideal light, this is still light up but with far less intensity.

I have a 47ohms resistor that would result in a too bit too high value:

1.5/47 = 31.91mA

See this video (the difference is visible on the ambient light, as camera focus on the light. Here 330 ohms (resulting to 1mA) and 100 ohms (resulting to 15mA) resistors are connected in parallel, a better solution (see below), and then 330 ohms only, we can see the ambient light change only as phone sensor wasn't in HDR mode. This difference of light means that the LED is not powered enough.

I made a simple resistor calculator tool with TIC80 in Lua You can use it online or download it to use on your computer or phone, it's Open Source with GPLv3 license:



Screenshot of resisor_calculator

28 december 2021 update: I discovered after this post that the famous free and open source electronic design suite, KiCad contains a tool called PCB Calculator that allow to compute lot of electronics related things, including resistors, and has a resistor color table. The version 6.0.0 of Kicad had been released on 25 december and contains several years of work huge improvements.

Another Ohm law say that When linking resistors in parallel called "parallel_resistor.lua", the computation is:

1/R = 1/R₁ + 1/R₂ + ... + 1/R₉ + ...

So

R = 1 / (1/R₁ + 1/R₂ + ... + 1/R₉ + ...)

And we have with 330Ω and 100Ω resistors in parallel:

R = 1 / (1/100 + 1/330) = 1 / 76.744Ω
I = 1.5/76.744 = 0.01954A = 20mA

That's just near perfect.

This formula make everything in one path:

I = U * 1 / (1/R₁ + 1/R₂ + ... + 1/R₉ + ...)

So here:

1.5 / (1/100 + 1/330)

I also made a simple command line calculator tool for resistors in parallel, available here https://framagit.org/popolon/electronhelp beside the tic80 single resistor calculator. I still need to implement, parallel calculation in tic80 version, and unique resistor calculation in command line version.

Switch button part

Pull-up resistor
For the button, the principle is just a switch, that cut current by default and make a short-circuit when the button is pressed. This switch is on a circuit between one of the GPIO pin and the ground (GND) pin. Due to the signal noise, a pull-up/pull-down resistor is needed to stabilize the signal. Pull-up/pull-down resistor is just a very low current that goes out of short circuit current. A 10KΩ resistor connected to the 3.3V pin is used for this. It is called pull-up if the resistor is connected to the button circuit between the button and the GPIO (3.3V too) pin, or pull-down if it is connected between the button and the ground.

Today most MCU boards, including ESP32-C3 include internal pull-up/pull-down resistors. They can be activated by software.

The legs connexion inside switch should be counter-intuitive at first, but legs on the same side are unconnected, each leg is connected with the one on the opposite side. To be clear, on the schema about the pull-up resistor, the two left legs are always connected together and the the two right legs are connected together, the button make the connection between left legs and right legs.

The Software

We want to manage:
* A main loop that blink the RGB LED of the board
* An external red LED that light when blinking is off.
* A switch button, that can at anytime, asynchronously, switch between these two states. This button can't be locked down, so we change state at each time it is push down. Releasing it doesn't have any effect.

To manage all this an asynchronous way, we need interrupts.
* Timer interrupt for waiting between LED blink steps
* Interrupt when on/off switch is pushed down, to change state.
* At this level we need also another asynchronous timer interrupt used for what is called debouncing. When the button is push down, it physically bounds, and the contact is on/off several time. The same effect is produced at the electrical level but the analogue current, is managed by electronic components, we can see at our level the current with just digital logical 1 (on) or 0 (off) state.

There is an included example with queueing in ESP-IDF, but it was not very clear for me about it's goal. After managing the switch, I looked it again, it allow to test queuing and interrupt, just by connecting 2 pins to 2 other pins, two pin send signal, the two other receive it, following by serial terminal output to understand what is running, is also of great value. Anyway, after searching a lot about how to use switch button, I found this topic helped me for the software part but was not complete, especially for debouncing. So I wrote here a working one and how it works. All the functions about GPIO of ESP-IDF are described here.

I made a copy of the esp-idf/examples/get-started/blink example as a starting project. See the previous article about ESP-IDF for this and how to compile it.

Just copy the blink example, and replace blink/main/blink.c by my version, that you can download here.

You can as an helper compile and flash it a first time, you will then only have to relplace the blink.c file and compile and flash it again.

To initialise the environnement for an ESP32-C3:

. $HOME/esp/esp-idf/export.sh
idf.py set-target esp32c3

Then to build and flash it (you can only apply build of flash depending on your needs) :

idf.py build flash

Initialisation

Set the state = 1 meaning internal LED will blink and RED Led the opposite way will be off.
* Choose used GPIO, reset all GPIO, set LED ones as output, button as input
* Initialise interrupt and timer fur button, we set it as POSEDGE (positive edge), this is the time when the current move up from 0 to 1, when liaison contact become active inside button.

In the default blink project, the blue intern led is defined as BLINKING LED, we instead start from it's internal GPIO, and the same for other internal colours of the internal RGB LED (respectively R=3. G=4. B=5). I use here connector GIO1 pin for the button and GPIO2 for the external LED (line 19).

#define RED_GPIO   GPIO_NUM_3
#define GREEN_GPIO GPIO_NUM_4
#define BLUE_GPIO  GPIO_NUM_5

#define EXTERN_BUT GPIO_NUM_1 // BUTTON on GPIO1
#define EXTERN_LED GPIO_NUM_2 // LED on GPIO2

We define a variable called state that define the current state of the machine at 1. 1 = on, 0 = off (line 31):

// set initial state to blinking led on
static int state = 1;

We need to first reset all the pin we use (line 61):

  gpio_reset_pin(RED_GPIO); // reset internal RGB LED GPIO
  gpio_reset_pin(GREEN_GPIO);
  gpio_reset_pin(BLUE_GPIO);
  gpio_reset_pin(EXTERN_LED); // reset external GPIO
  gpio_reset_pin(EXTERN_BUT);

We then create the timer, by sending to the appropriate function the previously defined structure (line 67).

  // create the timer
  esp_timer_create(&debounce_timer_args, &debounce_timer);

We set all the characteristics of the button GPIO. Could be set outside of the code, and send it to the gpio_config function (line 70).

  if (ISR_MODE == 1) {  // Interrupt mode
    gpio_config_t btn_conf;
    btn_conf.intr_type = GPIO_INTR_POSEDGE;  // Interrupt at Posedge only
    btn_conf.mode = GPIO_MODE_INPUT;           // Use INPUT mode
    btn_conf.pin_bit_mask = EXTERN_BUT_MASK;   // MASK of the button pin
    btn_conf.pull_up_en = GPIO_PULLUP_DISABLE;    //Disable pullup
    btn_conf.pull_down_en = GPIO_PULLDOWN_ENABLE; //Enable pulldown
    gpio_config(&btn_conf); // send config
  } else {               // Naive mode
    gpio_set_direction(EXTERN_BUT, GPIO_MODE_INPUT);
  }
  printf("button configured\n");

We now set the interruption associated with the button at the GPIO EXTERN_BUT to the isr_button_pressed() handler function (line 83).

  if (ISR_MODE == 1) {
    gpio_install_isr_service(ESP_INTR_FLAG_DEFAULT); // install GPIO interrupt
    gpio_isr_handler_add(EXTERN_BUT, isr_button_pressed, (void*) EXTERN_BUT); //Add handler of interrupt
    printf("Interrupt configured\n");
  } // end of ISR initialistaion

Then we set the OUTPUT direction of the LED and change the external red LED to 1 - state = 0, or off, we don't set the 3 internal LED at off, as they will be set just after, at the beginning of the loop (line 89)

  /* Set the LED GPIO as a push/pull output */
  gpio_set_direction(BLUE_GPIO, GPIO_MODE_OUTPUT);
  gpio_set_direction(RED_GPIO, GPIO_MODE_OUTPUT);
  gpio_set_direction(GREEN_GPIO, GPIO_MODE_OUTPUT);
  gpio_set_direction(EXTERN_LED, GPIO_MODE_OUTPUT);
  printf("LED output configured\n");

  gpio_set_level(EXTERN_LED, 1 - state); // 1-1=0 1-0=1

Main loop

In this infinite loop (while(1)), we basically light one colour of the LED, wait a bit using interrupt, to avoid power consumption, then light off the LED. Some informations bout the current state are send to the serial console like the state of the system, the begining of the loop, and when the internal red LED is lighted on.

We choose here to make the loop and to send the state value to the LED, then to wait, then to change to 0 value the LED to stop lighting it. It would have be better to stop the loop as soon as the system is off, ant to wait to be at on again to loop.

The loop start by printing the current state and light off RGB (line 98):

while(1) {
  printf("starting cycle by turning off the LEDs\n");
  gpio_set_level(RED_GPIO, 0);
  gpio_set_level(GREEN_GPIO, 0);
  gpio_set_level(BLUE_GPIO, 0);

then if we are in naive mode without interruption get the GPIO of the button state with gpio_get_level() function, and in any case display it. This allow to see if we currently connected the button to the appropriate GPIO pin. We then wait 1 second (1000 millisecond) all LEDs off (line 104):

  if ( ISR_MODE == 0 ) { // naive mode
    state = gpio_get_level(EXTERN_BUT);
  }
  printf("state=%d\n",state); // display current state on console
  vTaskDelay(1000 / portTICK_PERIOD_MS); // wait with lights off

We then light the blue LED, wait 200 milliseconds (0,2 s) and turn it off (line 110):

  gpio_set_level(BLUE_GPIO, state); // light on blue if state up
  vTaskDelay(200 / portTICK_PERIOD_MS);
  gpio_set_level(BLUE_GPIO, 0);     // light off blue

Then we do the same with the green LED (line 114), then the red (line 118), and then the green at the same time, lighting a yellowish colour (line 122). We wait for the last time 200ms (line 124) and the cycle restart by turning off all the LEDs (line 100):

  gpio_set_level(GREEN_GPIO, state);// light on green
  vTaskDelay(200 / portTICK_PERIOD_MS);
  gpio_set_level(GREEN_GPIO, 0);    // light off green

  gpio_set_level(RED_GPIO, state);  // light on red
  vTaskDelay(200 / portTICK_PERIOD_MS);
  gpio_set_level(RED_GPIO, 0);      // light off red

  gpio_set_level(RED_GPIO, state);  // light on red
  gpio_set_level(GREEN_GPIO, state);// and green => yellow!!
  vTaskDelay(200 / portTICK_PERIOD_MS); // end loop
}

ISR (Interrupt Service Routine)

gpio_isr_register() to register interruption function.

gpio_install_isr_service() and gpio_isr_handler_add().

Interrupt allocation. One of interesting aspect to know is that you can keep interruptions in IRAM (Instruction RAM) and use datas in DRAM (Data RAM), allowing less latency in interrupt and keep them independent from flash read/write. About Memory in ESP32-C3), IRAM and DRAM can be read/write in parallel.

Example without interruption.

Debouncing


This article illustrate the problem of bouncing and 2 methods for debouncing with examples on FPGA, and that's really more simple to implement on an FPGA than on a general purpose microprocessor or microcontroller.

The main app already use the main timer vTaskDelay, so we can't use it as I first done, else it will change the return address of the function, and so break the main loop. We could create a new xApp, but the more elegant way, is to use ESP Timer, it will solve all our problems an easy way.

ESP timer

We will follow the second one, and use an High Resolution Timer, ESP Timer for this. ESP-IDF include an example in examples/system/esp_timer/. This is not that we care about high resolution, ms is good enough here, but, this has the advantage to manage the timer by interruption and to easily avoid conflicting call, as this is needed due to bounces.

We need to create the structure to be sent to the function , that initialize the interrupt, and so the header of the callback function must be predefined (line 34):

// button manager with debounce timer definition
static esp_timer_handle_t debounce_timer;
static void test_button(void* arg);
const esp_timer_create_args_t debounce_timer_args = {
  .callback = &test_button,
  /* argument specified here will be passed to timer callback function */
  .arg = NULL,
  .name = "test_button"
};

The timer function, is called by an interruption at the end of the timer. It verify if the button is still pushed, and then switch state (1-0=1, 1-0=1). The external LED is set at the opposite of the actual state, if board LED blinks, then external RED LED is off (line 44).

static void test_button(void* arg) {
  if (gpio_get_level(EXTERN_BUT) == 1)
  {
    state = 1 - state;
    gpio_set_level(EXTERN_LED, 1 - state);
  }
}

The isr_button_pressed callback function is called when the button is pushed down. (at the positive edge). Due to the bounce it can be called several times for one human analogue pressure. We still need to double check we are really at push down state (high voltage) at the end of a short delay, else goes out. We then verify that the timer wasn't started before. If it wasn't (!) the case then we start once the timer that will start test_button function after 1 millisecond (1000 microseconds). This avoid bounce but still allow quick repeated push (for fast Morse code typing, of furious action games) (line 52):

static void isr_button_pressed(void* args) {
  if (gpio_get_level(EXTERN_BUT) == 0) 
    return; // we only want to manage pushed button
  if (!esp_timer_is_active(debounce_timer)) { // start only if the timer isn't active
    ESP_ERROR_CHECK(esp_timer_start_once(debounce_timer, 1000)); // 1ms = 1Kµs
  }
}

p.s.: I found an article about another method for debouncing specifically with RTOS capabilities too.

Short and practical introduction to FPGA, Verilog and Verilator and few words about SystemVerilog

→ Version en français ici

Warning, I’m a less than one month beginner in Verilog, Verilator et FPGA, studied as a hobby, if there are some mistakes I will correct it. You can contact me on The Fediverse.

Table of Content


* Introduction
** Real world full process
** Things to know and understand
** How to code a FPGA
* Verilog
** Values
** Types
** Gates
** Modules
** Simple example, writing an “and” gate
** Initial and always blocks
* Simple example with Verilator
** Make the test with Verilator
** Tracing example and GTKWave
** About Verilator examples
** Basic practical example with Verilator
* Further reading

The text in strong are here to help diagonal reading.

Introduction

Still continue to go to lower layers with the world of FPGA (Field Processors Gateway Array). This is a reprogrammable development electronic tool used to build and test processor. After the processor is validated in FPGA, you can start to build ASIC (Application-specific integrated circuit), actual hardwired processors that we use every-days in our computing devices. FPGA are also used as is in several industrial appliance (avionics, audio or video processing, etc…) for their parallelism, so the fact they are faster than a general purpose ASIC and a piece of software in these cases, and the ability to update them easily in case of problem. This post is a little introduction about FPGA, the popular IEEE standard Verilog HDL (“Hardware description language”) language and how to test it with free and open source software (FOSS) Verilator simulator. If you want to use VHDL, GHDL is FOSS simulator for VHDL.

Real world full process

The steps to implement a circuitry on an FGPA after it’s design are (Opensource softwares are given as reference for each step):
* Implementation of the logic (on an HDL, “Hardware description language”), so Verilog here, any text or code editor can be used (I use amphibian VIM and NeoVIM and sometime Geany light GUI but powerful IDE.
* SimulationVerilator here, making testbench, it’s already a good step in verification too.
* Formal verificationYoSYS is an open source verification and synthesizer system, so it’s more formal verification than Verilator. We will not go through this point here.
, Synthesis — making routes and placement of logic blocks, with physical and time constraints (see basic principle and today implementations) blocks from HDL script and output a bitstream in the FPGA format, YoSYS again.
* FlashingopenFPGAloader make the job on most FPGA.
* Real world testing on FPGA.

We will only test on simulator here, for understanding the basis. When you are ok, with this, you can go further, choose a FPGA, and start working with it.

Things to know and understand

The main concept to understand and be able to manipulate to build a basic integrated circuit are:
* 0, 1: electricity moving across a wire or not, that can be intrepted as 1 and 0 sate, true and false, etc…
* Boolean operators (for logical gateway), that is the main tool to manipulate them. You don’t need to have other notions of electronics. Most basics one are OR, AND, NOT, and XOR (exclusive OR). You can made all this operator with a NAND gateway (AND+NOT), but this not the more efficient way of doing them. If you master algebraic properties of booleans and know how to simplify them, you will be able to produce simpler circuit and do more efficient one, but it is not required to make circuitry.
* ALU (arithmetics logical unit), allowing simple additions
* flip-flop: special circuitry used to memorize information and change it depending on input and clock edge.
* latch: sequential logic circuitry does not depend on clock
* clock: Time, operation are time dependant and need to be synchronized to have a coherent behaviour. Clocks are like the drummer in a music band, It give the rhythm basis necessary to every time related construction, other instruments, singers, and dancers follow his time signals. Called clock signals here.
* Multiplexers (MUX) that can aggregate several lines in one, and demultiplexer that made the opposite.
These functions are all integrated in each logical block of a FPGA.

Other important part of a FPGA are:
* One to few PLL (Phase-locked loop) that allow to have secondary clocks ticks at frequencies multiples of the main clock or to sync on external clocks.
* A flash memory that allow to keep the bitstream on the board after a shutdown, it will be automatically loaded at start.
* Lots of peripherals I/O blocks to communicate with the outside world.

How to code a FPGA

There are three main methods for developing FPGA circuitries, and both method can be mixed:
* Drawing them with graphical tool representing this logical doors and other basic tools mentioned (flip-flop, latch, MUX, etc).
* Using a HDL (hardware description language), like Verilog or VHDL for the most famous. Compilers are here to build all the stuff behind this languages, as there are assembler to transform human easily readable assembly code to machine code, or higher level language (C/C++, Python, Lua, JavaScript, FORTRAN, BASIC, Pascal, Camel, LISP, etc) compiler or interpreter to convert them to machine code. There are also higher level language like Chisel, that are higher level description languages.
* Using OpenCL (Open Computing Language), designed mainly for intensive computing, that can be spread simultaneously on GPGPU and CPUs.

When using HDL, tools are here to automatically computing the routing circuitry for you. Tools like VTR (Verilog To Routing) and NextPNR, optimize their placement for you.

Symbiyosys is a verification tool.

OpenFPGAloader is a FOSS software used to program/flash/fuse the FPGA itself.

You can do very interesting things only few hours after start study of HDL, it allows to design simple circuitry with code, near from some higher level languages, with variables, constants, registers, conditional tests and basic booleans transformations, additions and bits manipulations. A register is already itself a circuit of several, basic logical gates. They are generaly made as ASIC inside FPGA, to accelerate circuit and reduce power usage.

Verilog

So we choose here Verilog (Official IEEE Std 1364-2001 specifications), it is well spread with lot of tools, VHDL is also a very famous one, anyway it is always interesting to look at alternatives, that could have interesting aspects. I choose it as a starting language, because used in few projects and examples I’m interested to, and, there are all the FOSS tools needed to write, simulate, verify. There are also several light RISC-V implementations for embedded (SERV and PicoRV32 (Github) (including Sipeed Lichee Tang (20€ FPGA board) implementation (I actually choose a 12€ Sipeed Tang Nano 4K as first board, The Tang Nano (without 4K) cost 3€ but only have 1000 logical blocks. There are also the very convenient and efficient Verilator simulator and VTR (Verilog to Routing, route optimizer.

Values

By defaults, numbers are integers, it is possible to force the numbers of bits (before the single quote ' character) and choose signed (s character, signed meaning able to use minus sign) or unsigned (default). Values can be in b binary, o octal, h hexadecimal, and even force d decimal (case by default). The datas can also be given in string format.

32'hFFFF00FF  // 32 bits unsigned hexadecimal opaque yellow (full red+green+alpha) RGBA
8'b00101010   // 42 (8bit unsigned binary)
8'b0010_1010  // 42 (8bit unsigned binary) easier to read version
8'sb1010_1010 // -86 (8bit signed binary)
-8'sd42       // -42 (8bit signed decimal)
12'o0754      // -rwxr-x-r-- rights (12 bits unsigned octal) on Unix compatible filesystems.
"Hello"       // a character string

Constants can be defined by preceding them by simple quote followed by define 'define:

'define myvalue -8'sd42;

Types

There are two different class of types, network and logic. We cover here :
* wire, that is of network type, just implementing a wire between components.
* reg, that is a logical type, register, that can be set or unset to memorize values.

wire w; // a simple wire between 2 components called w
reg r0; // a 1 bit memory called r0 (Register 0)

Value of a wire is set dynamically by the components linked to it, but it is possible to hard set them a permanent value with the assign function:

assign a = 1;

There are also higher level types that will use several of previous items, as for example:

integer i,j; // Two signed 32bits integers called i and j
real num;    // one 64bits float called num
time t;      // one 64bits unsigned integer representing time (in clock ticks)
realtime rt; // one 64bits float representing time

About time, it is interesting to know that #integer can be used to wait n clock ticks. No semi-column (;) must be put after the value, instead the instruction that follow the delay is generally wrote on the same line. For example:

#20 out = a & b;// wait 20 ticks then compute out value from a AND b

Simple bit elements, wire and reg can be organized in vectors, their bits are noted from MSB (most significant bit) to LSB (less significant bit).

wire [7:0] Bus0; // a 8 bits bus called Bus0 connected to the system
reg [31:0] R0;   // a 32 bits register called R0

It is possible to use arithmetic expressions to define them, in this cases, using array (see bellow) is better:

reg[8*256:1] string;   // 256 bytes string, * to simplify usage, start from 1, as will not be accessed by bit
reg[16*16-1:0] sprite; // a 16×16 monochrome sprite, better to use -1 and start by 0

After these declaration, we can access to unique bits or groups of bits the following way:

Bus0[0];   // Access to LSB of Bus0
Bus0[7:5]; // Access to 3 MSB of Bus0 (defined as 8 bits from 0 to 7)

Verilog also allows to concatenate several bits ranges to another one. For example, here, to convert from 16 bit Little Endian to 16 bit Big Endian data format:

BigEndian[15:0] = {LittleEndian[7:0], LittleEndian[15,8]};

If the destination bus is 16 bits here, then [15:0] is not mandatory for the left part as we concatenate two 8 bits values).

And all this types can be organized in arrays.

Both complex types (in the general meaning, not mathematics complex numbers) and vectors can be organized in arrays.

reg[7:0] cursor[0:7];            // 8x8 pixels monochrome cursor bitmap, Here, cursor is the name
reg[15:0] sprite[0:15];          // 16*16 pixels monochrome sprite bitmap
reg[31:0] palette[0:15];         // 16 RGBA8 colours palette
reg[31:0] FrameBuffer[0:307199]; // 640×480 RGBA8 screen frame-buffer

Arrays are noted from their lower address to higher one.

Gates

Here is available logical gates (or operators) in the language, they are bitwise, but can be applied bit-to-bit to all bits of a bit vector:

~a     // NOT
a & b  // AND
a | b  // OR
a ^ b  // XOR
a ~^ b // XNOR, can also be wrote ^~

Gates Look-up tables

There are also two bitwise shift operators available:

>> // Right shift
<< // Left shift

Modules

Verilog is organized by module (like classes in object programming, or an integrated circuit on a PCB). A module has:
* A module name
* It’s interface between its inside and outside world (think to public variables or methods in object programing or IC pins in electronics), is a list of ports defined between parenthesis as for functions in a functional language module module_name (wire a, wire b, wire out); ... endmodule.
* By default, declaration are logic blocks, executed in parallel.
* For executing code sequentially and conditionally, always (and initial in testbenchs) sub-block must be used (see below for more details).

Simple example, writing an “and” gate

AND
Basic operators are already present in Verilog, but see here how it could be wrote. In Verilog boolean functions, the left argument is the output. So an and gate could be reimplemented this way:

module and(f,a,b); // The ; is not an error here
  output f;
  input a,b;

  assign f = a & b; // ^ is an AND gate
endomdule

This example should be put in an "and.v" file.
* The default type is wire
* Since Verilog 2005, input output can be put in the module header like this:

module and(output f, input a,b);

This example is a permanent unconditional gate, you can use them to do full circuitry, but Verilog is an high level language allowing to code more complex things a simple way. The always blocks is here for more complex gatewares.

Initial and always blocks

There are two types of blocks to know to start in Verilog:
* always block can be synthesized, so used in circuitry and executed each time conditions are met, so triggered by an event given between parenthesis always @(event).
* initial block can’t be synthesized, so used in testbench only and executed only one time and unconditionally. They allow to give initial values of the test. It is possible to have several initial blocks. This also allow to test/prototype quickly a simple function with text output, before put in a circuitry always block.

They both allow to:Posedge and negedge on clock signal
* execute code sequentially
* if else conditionals
* while, for, loops

A typical event to trigger an always can be, for example, a positive clock edge (posedge) or negative clock edge (negedge)

always @(posedge clk)
  ...
end

but can be a conditional logic test

always @(a or b)
  ...
end

Simple example with Verilator

A simple example given with Verilator:

module our (clk);
  input clk;  // Clock is required to get initial activation
  always @(posedge clk)
    begin $display("Hello World"); $finish;
  end
endmodule

Here, the always block is triggered at first positif clock edge (posedge), it print ($display) an “Hello World” string on the standard output, and finish ($finish) the simulation.

It is possible to test and validate lot of things with Verilator before going to further in FPGA process (Formal verification, converting to bitstream (that can be very slow with routing process) and flashing the FPGA). A good start is to make a copy of the above given example, and to modify the top.v file.

Make the test with Verilator

The example are installed on:
* Arch Linux or Manjaro: /usr/share/verilator/examples/
* Debian or Ubuntu: /usr/share/doc/verilator/examples/

So to test it, simply copy the directory to your home directory, where you will have write access, and then, build and execute with a simple make (Verilator transpile it in C and then compile it, this allow to have a very fast simulation):

On Arch Linux based distribution:

cp -a /usr/share/verilator/examples/make_hello_c ~/

On Debian based distribution:

cp -a /usr/share/doc/verilator/examples/make_hello_c ~/

Then on any distribution:

cd ~/make_hello_c
make

make allows to rebuild from the sources if needed, and then execute the testbench immediatly.

The output of the bench itself:

code>-- RUN ---------------------
obj_dir/Vtop
Hello World!
- top.v:11: Verilog $finish
-- DONE --------------------

The following output text (I didn’t paste here) is about compilation and suggestiong to use next tutorial.

Tracing example and GTKWave

GTKwave is companion tool to Verilator. It allows you to see the chronogram resulting from the simulation, to understand the temporal functioning of the circuit, and to see what could be the source of a possible problem. It uses the .vcd (Value Change Dump) file, for tracing. It is used as source to draw the chronogramme (see make_tracing_c example, output can be found in the logs/ subdir. The file is logs/vlt_dump.vcd, created after using the make command).

To use the tracing example with GTKwave, you need to install the gtkwave package, then, as for first example, duplicate the directory to a place you can write into:
* On Arch Linux based distribution:

cp -a /usr/share/verilator/examples/make_tracing_c ~/

* On Debian based distribution:

cp -a /usr/share/doc/verilator/examples/make_tracing_c ~/

Then on any distribution:

cd ~/make_tracing_c
make
gktwave logs/vlt_dump.vcd

GTKwave

The text output during the make phase give interesting informations too:

-- RUN ---------------------
obj_dir/Vtop +trace
[1] Tracing to logs/vlt_dump.vcd...

[1] Model running...

[1] clk=1 rstl=1 iquad=1234 -> oquad=1235 owide=3_22222222_11111112
[2] clk=0 rstl=0 iquad=1246 -> oquad=0 owide=0_00000000_00000000
[3] clk=1 rstl=0 iquad=1246 -> oquad=0 owide=0_00000000_00000000
[4] clk=0 rstl=0 iquad=1258 -> oquad=0 owide=0_00000000_00000000
[5] clk=1 rstl=0 iquad=1258 -> oquad=0 owide=0_00000000_00000000
[6] clk=0 rstl=0 iquad=126a -> oquad=0 owide=0_00000000_00000000
[7] clk=1 rstl=0 iquad=126a -> oquad=0 owide=0_00000000_00000000
[8] clk=0 rstl=0 iquad=127c -> oquad=0 owide=0_00000000_00000000
[9] clk=1 rstl=0 iquad=127c -> oquad=0 owide=0_00000000_00000000
[10] clk=0 rstl=1 iquad=128e -> oquad=128f owide=3_22222222_11111112
[11] clk=1 rstl=1 iquad=128e -> oquad=128f owide=3_22222222_11111112
[12] clk=0 rstl=1 iquad=12a0 -> oquad=12a1 owide=3_22222222_11111112
[13] clk=1 rstl=1 iquad=12a0 -> oquad=12a1 owide=3_22222222_11111112
[14] clk=0 rstl=1 iquad=12b2 -> oquad=12b3 owide=3_22222222_11111112
[15] clk=1 rstl=1 iquad=12b2 -> oquad=12b3 owide=3_22222222_11111112
[16] clk=0 rstl=1 iquad=12c4 -> oquad=12c5 owide=3_22222222_11111112
*-* All Finished *-*
- sub.v:29: Verilog $finish
[17] clk=1 rstl=1 iquad=12c4 -> oquad=12c5 owide=3_22222222_11111112

You can see here, each steps of the execution with the clock (clk) switching between 0 and 1 state.

The time is in tick, and in testbench, can be defined by timescale directive, that allow more precise resolution granular time. We don’t need to use it for now.

The .vcd (Value Change Dump) file is defined in top.v:

$dumpfile("logs/vlt_dump.vcd");

And you will have all the step in the logs/annotated/ dir and a coverage file in logs/coverage.dat:

-- COVERAGE ----------------
verilator_coverage --annotate logs/annotated logs/coverage.dat
Total coverage (2/31) 6.00%
See lines with '%00' in logs/annotated

-- DONE --------------------

The last one is defined in sim_main.cpp.

About Verilator examples

In each example, there is a small c++ code interface, modifying c++ code isn’t needed for basic testings. When basic concept are understood, c++ part can be tuned to interface with system libraries, to simulate devices communications, audio, graphics, etc.

In these simples starting examples, the most interesting ones for beginning are all available in Verilog:
* make_hello_c Verilog Hello World with Makefile
* make_hello_sc Verilog Hello World using SystemC with Makefile
* cmake_hello_c Verilog Hello World with cmake.
* cmake_hello_sc Verilog Hello World using SystemC with cmake.

SystemC is set of C++ classes and macros which provide an event-driven simulation interface, simulate concurrent processes in a real-time environment.

The other examples are (with their variants cmake|make and c|sc):
* make_tracing_c for a better understanding at start and for debugging purpose.
* make_protect_lib For creating a DPI protected library. DPI (Direct Programming Interface), is an interface between SystemVerilog and functions in a language like C or C++.

System Verilog is an advanced evolution of Verilog so it’s also a HDL and sometime view as a HVL (Hardware Verification Language). It add several types, including block types, has some C aspects, and can be programmed in object-oriented programming.

Some common file extensions with Verilog, System Verilog and related simulators are:
* .v as Verilog.
* .vc (verilog ???) or .f as File, used for large projects, containing arguments to give to Verilator (or other simulator), including, flags, includes directory, linked libs for the simulation.
* .vcd as Value Change Dump file, contains tracing output of simulation for analysis (see below).
* .vo as Verilog Output file.
* .sv as System Verilog,

Basic practical example with Verilator


So we will reuse the simple make example (you can choose cmake example instead in this case you will have to call cmake instead of make.

I copied make_hello_c example in my work directory:

cp -a /usr/share/verilator/examples/make_hello_c verilator_test
cd verilator_test

and then just edited top.v ( vim, emacs, gedit, or any text editor of your choice can be used) and replaced it’s content.

Example of simple module after my test, I would like to understand how to access to a simple (verilog) registers, different kind of datatypes, including sprite one (the one displayed beside the title of this section), you can download this version of top.v here:

module top;
 reg [8*11:1] str1;
 reg [8*25:1] str2;  // filled by spaces at left
 reg a,b,c;
 reg [7:0] sprite[0:7];  // use one byte by line
 reg [15:0] sprite2;  // sprite as bitfield
 integer i;

 initial begin
   str1 = "Hello World"; // string initialisation tests
   str2 = "Hello World";

   a = 1'b1; // bit initialisation tests
   b = a^1;
   c = a^0;

   sprite[0] = 8'b10011000; // sprites initialisation tests
   sprite[1] = 8'b00100100;
   sprite[2] = 8'b01000010;
   sprite[3] = 8'b10011001;
   sprite[4] = 8'b10011001;
   sprite[5] = 8'b01000010;
   sprite[6] = 8'b00100100;
   sprite[7] = 8'b00011000;

   sprite2[ 7:0] = 8'b10011000;
   sprite2[15:8] = 8'b00100100;

   $display ("str1 = %s", str1); // strings display
   $display ("str2 = %s", str2);

   $display ("a = %d", a);       // bits display
   $display ("b = a^1 = %d", b);
   $display ("c = a^0 = %d", c);

   for ( i=0; i<8; i=i+1) begin  // sprites display
    $display ("sprite[%2d] = %b",i,sprite[i]);
   end
   for ( i=0; i<2; i=i+1) begin  // try to read bitranges, including out of bounds
    $display ("sprite2[8*i(%1d) +: 8] = %8b",i,sprite2[8*i +:8]);
    $display ("sprite2[8*i(%1d) -: 8] = %8b",i,sprite2[8*i -:8]);
    $display ("sprite2[4*i(%1d) -: 8] = %8b",i,sprite2[4*i -:8]);
   end
 end 
endmodule

To test it, just, apply a make, here in blue are comment of the output added in blog post only:

make
-- VERILATE & BUILD -------- 
[...]  This part is the compilation log
-- RUN ---------------------   This part is interesting one
obj_dir/Vtop
str1 = Hello World
str2 =               Hello World    str2 with filled 25 chars
a = 1
b = a^1 = 0
c = a^0 = 1
sprite[ 0] = 10011000    Eight lines of the sprite
sprite[ 1] = 00100100
sprite[ 2] = 01000010
sprite[ 3] = 10011001
sprite[ 4] = 10011001
sprite[ 5] = 01000010
sprite[ 6] = 00100100
sprite[ 7] = 00011000
sprite2[8*i(0) +: 8] = 10011000      some bitrange access tests
sprite2[8*i(0) -: 8] = x0010010      We go out of registers here with x displayed
sprite2[4*i(0) -: 8] = x0010010
sprite2[8*i(1) +: 8] = 00100100
sprite2[8*i(1) -: 8] = 01001100
sprite2[4*i(1) -: 8] = xxxxx001

Further reading

Relatively complete documentation online:

About Verilog:
* A complete Verilog manual at Chip Verify
* Verilog TUTORIAL for beginners on ReferenceDesigner.com
* Verilog overview in few presentation screens (PDF) at euler.ecs.umass.edu
* Verilog Quick Reference Card.pdf
*
Verilog HDL Quick reference Guide
* French: Verilog syntax at hdl.telecom-paristech.fr

Testbenchs, SystemC and System Verilog:
* How to write a basic verilog TestBench, using a C++ template. The site FPGA tutorial has 3 main subjects, VHDL, Verilog & System Verilog.
* Tutorials about Systemverilog and SystemC.
* Some tutorials about System Verilog.
* French: System Verilog in 13 minutes on sen.enst.fr
* Verilog Simulation with Verilator and SDL using SDL library to simulate VGA graphics output, based on Verilator and some part of SystemVerilog.

Some interesting FPGA projects:
* From Nand To Tetris, all the step to do in few exercises a computer based on a 16 bit CPU, using only NAND gate (interesting game), then building a Tetris with this small system. Their use their own HDL, but there are already some port as examples, as this one in Verilog using Ikarus Verilog simulator (older free software simulator), one in Verilog for opensourced Lattice ICE40 FPGA (tested on an open harware Olimex board) and Opensource tools (YoSYS as synthesizer), in Verilog for De0-nano (Altera Cyclone IV FPGA).
*
Tang nano MIDI Sounder, a MIDI synthesizer, with nice 8 bits like sound.
* (Japanese) SERV RISC-V procesossor for Tang Nano
* (Japanese) Tetris for Tang Nano in Verilog
* ZipCPU a blog for this CPU with a nice Verilator tests bench examples, using C++ templates for generic testbench.
* UART, Serial Port, RS-232 Interface FPGA implementation (both in Verilog and VHDL).
* Consolite, a light console in Verilog for FPGA, assembly programmable. An emulator is also available.
* FloPoCoFPU for FPGA devcelopped by INRIA.

For Free software loving hackers:
* Apicula project to open Gow1n FPGA array bitstreams

64x64x16colours (Sweetie16) PixelArt with Pixelorama “β-karoten – We know whom will be eat”

→ Version en français ici

β-karoten - We know whom will be eat

I participated to the LoveByte Battleground demoparty> that runned this week-end, by posting a drawing last week. Sadly/funnily , there was few mistake ^^:
* I made a 64×64 pixels picture instead of a 128×128 one. The palette for the competition was 16 colours Sweetie16 (the default one on FOSS, TIC-80 fantasy console (Source code)).
* My second mistake is I’ve uploaded a first version, and few hours later another one using (FOSS) source code) on Debian on (FOSH) RISC−V (Specifications, there are lot of free or not implementations) emulator of FOSS Qemu (Source code, Git instance), the first upload worked fine but the second one didn’t work (maybe because my installation of Netsurf doesn’t manage javascript)^^. I also performed in a livecoding match (256 bytes and 25 minutes limits). Result of my poor production, Commented Live coding session record.

This is made with FOSS Pixelorama (Source code), itself made on FOSS Godot game engine (Source code). I use FOSS Arch Linux OS. Also made a ArchLinux AUR package pixelorama-git after pixelorama package (for git version, I would like to use v0.9, still not out, only v0.8 was available). There are pixelorama package (last stable, compiling from source), and pixelorama-bin package (from developers binary tarball). Pixelorama is a Pixel art picture and animation editor. I believe I discovered Pixelorama thanks to blog Librearts.org.

The name is “β-karoten – We know whom will be eat”.

Pixelorama editor screenshot
Screenshot of Pixelorama

Telegram, Lottie animated stickers, Python and Glaxnimate

→ Version en français ici

Table of Content

* Introduction
* A good workflow
* Publication on the web (SVG)
* Telegram Bot

Introduction


I’m a fan of Telegram chat application interface. If the sever is closed source, desktop and mobile applications are open sources, the interface is very well designed and thinked, intuitive, relativly light and full of interesting functionnalities, compared to other applications of this kind. It could be good to patch to have a Jabber/XMPP compatibility. It is the first, as far I know, to use Lottie, since Jully 2019 (and the bot API that goes with it), an animated vector format that I dreamd of for decades, that become an de facto open standard. Animations of this page have been created with Glaxnimate, a tool to make Lottie format animations, as main goal, but that can also be used to export SVG animation. The authors of Lottie published some JavaScript for the World Wide Web, Samsung, made a free and open source rlottie C++ library, as well as WASM version, that allow to play them and has binding for several programming languages. Qt has an included Lottie reader, and there are several tools to create them.


The multiplatofmr free software Glaxnimate allow to make them with a Graphical User Interface, familiar with people creating animation on computers. His main author also made a Python library, python-lottie (AUR: python-lottie-git, pip: Lottie), that allow procedural generation of them. Both alow to convert them to several formats (export animated SVG, Lottie JSON, HTML page all ready, Telegram, mpeg4, PNG, wepb, gif, and I probably forgot some other), and as input to vectorize animated gif, etc… The 2D vector animation Synfig also have a Lottie exporter, but also Blender3D, a plugin made by Glaxnimate author (distributed with python-lottie) for this last one. There are so several choices to create them.


Glaxnimate allow to easily create animations in Lottie format, including .TGS, a version with more limited specifications and gzip compressed for Telegram. This format is more compact but has some constraints, that didn’t stop some artists to make very great artworks since 2 years:
* 512×512 pixels
* 3 secondes
* 64 KB by animated sticker..
* Animated stickers can be grouped in a sticker pack corresponding to several expressions (like emoticons/emoji).

Why for 512×512 pixels with a vector format? Telegram client application render the animation in this dimension. This dimension is already big for some mobile screens. This vector format allow to reduce bandwidth usage and to have a high quality animation (generaly computed by the GPU, SVG accelerators was already in Nokia and Symbian made phones in 2000s). WeChat/微信 choose animated GIF about one decade ago. GIF are probably converted in MPEG nowaday? The instant messaging Discord, populare in Far West, also use the Lottie format since Jully 2021, following Telegram after 2 years.


As a refecence, top of this page animation is:
* 1109 bytes (1,1 KB) in TGS (binary compressed Telegram format)
* 4682 bytes (4.5 KB) in JSON uglified (no more indentation, or carriage return)
* 508054 bytes (500 KB) in WebP
* 9710 bytes (9.6 KB) in animated SVG (there was an useless animated path, I don’t know why for).
* 1804 bytes (1.7 KB) in ainmated SVGZ (gzip compressed SVG). It is more interesting today to let the server compress using Brotli format (Nginx, Apache) or to precompress them (.br). It is really well supported by web browsers
* 6114 bytes (6.1 KB) in SVG, Inkscape optimised compressed (based on Scour)), it’s possible to reduce size more. Today if the option to reduce digits after comma (floats) is less than 3, it will be still kept at 3 digits. In animation parts still not managed there are six 0 (1.000000 or 0.000000 for example). It is still possible to optimize by hand until thes options are added (See sed usage, below), so :
* 5827 bytes (5.8 KB) in SVG, a bit hand finished.
* 5559 bytes (5.5 KB) by removing some useless intermediate groups (warning to don’t break all, try group by group) and replacing numbers like 1.0 by 1


One of the main problems of optimized SVG output from Glaxnimate is that Lottie heavily use groups, where SVG allow to made most of these operation in the objects themselves. So transformation matrix are all in groups containing the objects, instead of in objects themselves. you can see at right the XML source inf Inkscape. This still ask lot of handcrafted work on files nowaday. It should be possible to improve it.

It looks like there are more efficient methods withSVGO, but based on Node.js, and it breaks animations, some of its optimisation methods could be used in python one. I don’t like Node at all due to bad habits of lot of devs around Node. The Inkscape plugin, “inkscape-svgo” is still in this spirit, if we try to compile it with included Makefile, it doesn’t test if Nodes modules are already installed but forcee download Node JS version 11 (and x86_64 only, so not multi-architecture), and recompile all the dependenccies. So one more time with Node, we wast lot of earth limited resources, wit an extention of tens of Megabytes (75 MB, where Inkscape is 145 MB and the Node SVGO library itself is only 5 MB), where it should only be few KB.

A good workflow

* Create object with Inkscape
* Animate it with Glaxnimate
* Export it in SVG format (don’t forget to also save a Glaxnimate format file)
* Open the new file with Inkscape to clean it (extract some groups that are not needed
* Export to optimized output
* Finish by hand.

For the last part there are 2 solution for have an automated one:
* Patch Inkscape optimized output.
* Do a second path optimizer.

Some space can be saved with :

sed s/1.00000/1/g <input.svg | sed s/0.00000/0/g | sed 's/0000;/;/g' >output.svg

One or two digit precision (instead of 3 or 4 today with Scour)), should be good enough.

SVGcleaner is better than Scour in several cases, but itdoesn’t support animations and don’t want to (it’s complex to manage, can easily become a nightmare).

Publication on the web (SVG)

So as I just said, as animated SVG is really more easy to integrate in a web page, after lot of tries with Lottie, I finnaly choose this option. The best is to not compress with SVGZ, but to optimizer the SVG file then to compress with Brotli (.br) and maybe to have a gzip version beside for oldest web browsers.

Its integration in a web page is really simple, it can be done as other pictures format with image:

<img src="/chemin/du/fichier_animation.svg" align="right" width="200" height="250">

That’s what I done in this page, every browser automatically manage them.

Note : It could be interesting to integrate it inline to manipulate an easier way with Javascript. For simple animated illustration, maybe just animated/zoomed/rotated object, without inside details modifications, image is perfect.

For compressing I use the max compression, here is a simple method to compress all files of a single directory beside their uncompressed version:

ls --ignore=*.gz --ignore=*.br | while read file
do
 brotli -q 11 -c <"${file}" >"${file}".br
 gzip -9 -c <"${file}" >"${file}".gz
done

* 5559 murphy_anim0.animoptiv5.svg
* 1303 murphy_anim0.animoptiv5.svg.br
* 1543 murphy_anim0.animoptiv5.svg.gz
* 4682 murphy.json
* 976 murphy.json.br
* 1149 murphy.json.gz

To force loading one of the 2 files format, if it is supported by borwer (I mean if it supports both), Brotli (.br) will be used, with Nginx (after compiled it with the Brotli patch, rules in the Nignx configuration files are realy simples:

brotli_static on;
gzip_static on;

With Apache, Brotli is here by default in versions 2.4, but it’s a bit harder to manage.

To test their support, Curl can be used:

curl -I -H 'Accept-Encoding: gzip, deflate, br' https://host.net/fichier.svg

If brotli is really activated, you will receive the following line in the answer:

content-encoding: br

Telegram Bot

The Python-Lottie library come just right for me, I would like to study Python, including for micropython, used in embedded world and for my work of System and Network administrator, where it is more and more used, replacing step by step Perl, and with some tools like Ansible an automation paltform used to manage computer farms, as well as more and more other basic system tools. After about one year of experience with procedural generation in Lua give me some willing to do this with other languages too (I’m currently preparing a suprise for soon). There are several bot libraries for Telegram in Python, including Python Telegram Bot, that follow well the official Telegram API, and Telethon (Documentation) that allow to communiatte directly in MTproto, the Telegram protocole, instead of using HTTP layer. This reduce one layer and allow a more fine control of exchanges. You can read here about the differences between Bot API (HTTP vs MT Proto).


There are still not too much examples with Python’s Telegram bot, that I started with, but by digging a bit in documentation, it is possible to find what’s needed. By mixing Python-Lottie for procedural generation, Glaxnimate for hand made animation, and the bot there is some interesting possibilities that open. I so made a first Python-Telegram-Bot bot. After some exchanges with Glax, the author of Glaxnimate, he let me know the Telethon Python module, that it uses for its bot, that has some similarities with what I want. It’s called Glaxcomm, and generate Lottie/TGS on the fly. I hope I will manage (time+willing) to rebuild my bot with this module and I hope I can get it where I want it. My next goal would be to be able to use it with other protocols, including ActivityPub (W3C specifications).

Glax made a good set of basic procedural generation examples that greatly help to understand it’s usage, a good documantation about Lottie format, stickers scripts examples.

I started by make a simple bot from python-telegram-bot library examples, but complete explanation about how to use file_id was missing (and I didn’t found them around the net). This is the methode Telegram promote to to send file, by it’s Id instead of sending several times the same file. The file_id is given by Telegram when a file is sent. So ideal method is to keep the ID of already succefuly sent files in a database or a file on the disk. This example create a file containing the ID beside image file, this is probably a bit better to have thei ID in a database (SQL or TOML) :

if os.path.exists(idfile):
  fid = open(idfile)
  stickid = fid.read()
  fid.close()
  msg = update.message.reply_sticker(stickid)
else:
  msg = update.message.reply_sticker(open(stickfile, 'rb'))
  fid = open(idfile, 'w')
  fid.write(msg['sticker']['file_id'])
  fid.close()

Lua, TIC-80, LÖVE,etc: Introduction to particles systems and games

Version en français ici
default code in TIC-80

A good language for easily developing games, interactive content and procedural art is Lua scripting language. This is a simple functional language with some limited oriented object language features. It is, due to its simplicity and bytecode compiling at start by default, one of the lightest and fastest script language. Some integrations like LÖVE media/games engine and API (very powerful), and TIC-80, that is more limited fantasy-computer inspired first by PICO-8 (also use Lua). They allow quick prototyping, for a final product in the same language or a later port in another language. Lua is also used as plug-in systems language, in lot of games, and tools, including desktop applications (like Blender), web application (like MediaWiki behind Wikimedia) or embedded world. In this domain, popular and open-source drone board control BetaFlight or Open-TX open source radio-command for commands like Taranis one. There is a complete online documentation of Lua on the official website. It is possible to embbed C libs/function in Lua programs with libffi. It as been created first for Python, CFFI, and there is FFI support for PHP too now. It also possible to embed Lua scripts in C program. I also just discovered when I wrote this article (thanks to the author of TIC-80, that there is also PicoC, a simple C language interpreter that so allow to control more finely/low level data structures. The binary size is about the same than Lua interpreter.

An interactive demo of how trigonometric functions workSo, after few years of looking time to time this language and tools, I started to play a bit more in end of 2020, and in few month I can said I made lot of progress in real-time programming. Doing and even finishing light games. I Needed to study again basic trigonometry (follow link for a simple interactive explanation), basic vector algebra and few other mathematics fun things, that I personally consider like puzzles games.

Banner of Falacy Gorx, pseudo 3d game, using lot of tables
I also wrote a short making-off article on itch.io in march, 2021, during a game jam competition (instead of coding ^^). All of this motive me to write more articles tutorials about real-time/vector and procedural generation programming. I will try to write a series explaining methods I used. I will try to keep it as simple as possible to everyone, but some basic knowledge in general programming and mathematics could help a lot in this field as in life in general.

So I start with a short introduction about what I use the most in my development, arrays of elements and some randomness:
* Tables and their common basic management related to animation logic
* Creating a table
* Procedural generation of the content of a table
* A simple example for cleaning a table
* Variation and cleaning of table depending on tests
* Compacting code a bit
* Procedural generation of the content of a table
* Simple graphic particle system example

Tables and their common basic management related to animation logic

Some general table usage in interactive applications

Most things are managed, for scalability, in tables, main actors (like players characters), objects, particles, active agents, etc. Object themselves, also often contains tables of sub-elements, thing as example about the parts of a caterpillar body.

So, most structures have generally 4 vital logic functions that can be managed different ways:
* Cleaning of the table, used when tables elements are no more used, but it is also wise to use it to initialise new general state, like changing from main menu to game for example.
* Initialisation of table (generally including cleaning if needed) and adding 1 to n elements
* Update of the table, states of elements, including their mechanical & physical relationship, positions, their state of drawable, interacting with other objects, and finally removing or creating a new element in the table if needed. All these parameters depend on lot of various critters.
* Output of the content of the tables (display, sound, etc).

You can test these examples in Lua command line interpreter by typing lua in a shell, and then copy-paste them, or by putting them in a file.lua, and calling it by:

lua file.lua

The last examples use TIC-80, that can be used in web browser and natively on several systems, including, Linux, Mac, Windows, Android, Rpi, etc. There is a Pro version, but I never used its functionalities, it’s good to help the developer that is very nice and worked a lot of it. The examples can also easily be ported to other environments.

Creating a table

We will use here an objs[] table (a table with name objs, short of objects). In Lua, by default, table start at 1 when you fill it by objs={elt1,elt2}, but it is still possible to have a 0 indexed element by using objs[0]=elt0. The table can be (re-)initialized empty with objs={}. If it wasn’t empty, Lua garbage collector will clean it. As I coded a lot in low level languages, such as C/C++ and assembly, I’m not a big fan of garbage collectors, but that’s really convenient for quick prototyping.

Content of the table can be simply printed with a classical for loop. For the table called obs, #objs allow to know the number elements currently availablee in the table.

objs={"ball","cube","player"}
for i=1,#objs do
 print(objs[i])
end

Procedural generation of the content of a table

particules systems for clouds, mountains, characters, table of elemets for dragonIn procedural generation, the best friend of Chaos also called Nature, is random function, that generate randomly numbers. In Lua core, math.random is dedicated to this. it accept a range as parameters, limited to integer and with crescent values only. We can as example, decide to generate a number between 1 to 6 included, as the total number of elements we want. If you try several times this function, it will display random value included between 1 and 6 each time. Like if you launched a classical 6 faces cubic dice.

print(math.random(1,6))

In an interactive or animated program, we generally want to create random object, and then varying a more managed way during a period of time. So we fill a table with random values, that can be reused later. We need to first (re-)initialize objs to an empty array, so it can be filled by the for loop.

objs={}
for i=1,math.random(1,6) do
 objs[i]={lt=math.random(1,50)}
end

We so generated a 1 to 6 random number of random 1 to 50 numbered assigned to a parameter we called lt parameter instead of the table directly. This kind of element can be accessed 2 ways in Lua, objs[i].lt or objs[i]["lt"], the second one can be useful in some situations, but we will see this later in another tutorial.
so now the table is filled, we can print values several times, they will be kept the same:

for i=1,#objs do
 print("objs["..i.."]="..objs[i].lt)
end

We use the Lua string concatenation here (symbol ..) to have more information about which element we see printed.

A simple example for cleaning a table

It is important for cleaning a table or removing elements from it to do the loop from last to first element index, as when an element is deleted, all following items in the table will have decremented index. We could then stride over some elements and remove elements we don’t want to.

for i=#objs,1,-1 do
 table.remove(objs,i)
end

In the table.remove() standard Lua function, arguments are the name of the table and the index of the element to remove.

Variation and cleaning of table depending on tests

It can be a good habit to use a short local pointer variable to the element parsed to have a shorter code inside the loop, as it will il most case be accessed a lot. We remove here an element when lt (short for lifetime) is gone to 0, else we decrement it.

for i=#objs,1,-1 do
 local o=objs[i]
 if o.lt<=0 then
  table.remove(objs,i)
 else
  o.lt=o.lt-1
 end
end

The o variable can’t be passed to table.remove(), as it is used as a pointer to an element of the table, not as the name of the table, and the second argument is an index of the table, not either a pointer to the element as o is.

We have now the general base of a particles system.

All particle systems work with generation, often using at least a bit of randomness, and lifetime of each particles as a base, other criteria can changes depending on kind of particles.

Compacting code a bit

I generally add the local variable on the for...do line to have a more readable/compact code and changes behind if...then or else keywords to have a more compact and still readable code:

for i=#objs,1,-1 do local o=objs[i]
 if o.lt<=0 then table.remove(objs,i)
 else o.lt=o.lt-1 end
end

And Lua allow to assign a function pointer to a variable, so I generally use the following trick to have a more compact and clear code :

m=math rnd=m.random

So m is assigned to math and then rnd to m(ath).random.

Warning: on constraint is to not use m variable in the same variable scope than m.* assignation. It’s so better to made this assignation at the beginning, so m can be used freely after.

Simple graphic particle system example

In this example we will simply add random x (horizontal axis) and y (vertical axis) position around a starting central point 120,70 we choose a variation of -15 to +15 pixels in each direction:

objs[i]={lt=rnd(10,30),x=120+rnd(-15,15),y=70+rnd(-15,15)}

Basic particle systemand made them randomly move of a length from 0 to 1 pixel and in direction left/right and up/down. So, we have:
For x-axis
* -1 = left
* 0 don’t move
* +1 = right
For y-axis
* -1 = up
* 0 don’t move
* +1 down

o.x = o.x+rnd(-1,1) o.y = o.y+rnd(-1,1)

In Lua functions are defined by function function_name(arg1,arg2,...) and finish by end.
In the case of TIC-80 for example, that has the advantage to have all embedded in one executable, the function TIC() is a function called periodically, at each new frame, allowing to made time related content (so animation) easily. The function cls(color) is used to clean the screen with color at index “color” as TIC-80 use a 16 colors palettes.

The place of the particles are drawn by circ() (circle) function.

The definition of the function is:

circ(X center, Y center, radius, colour)

We place the center of the circle at the coordinates of the object, it has here a radius of 1 and use the colour index 0, that is black by default with TIC-80.

circ((o.x, o.y, 1, 0)

The current lifetime that the particle stille have is printed just at the right of the particle here for the demonstration. In TIC-80 print(), is used to place screen on graphic screen, and trace() on console.

The part of the print arguments we use are defined the following way:

print(text, X start, Y start, color, fixed font, scale, small font)

So we place as text, the current lifetime of the particle, at the center position of the object (o.x, o.y) with an horizontal variation of +2 pixel (so right) and horizontal of -2 pixel (so top). We choose a fixed size (true), scale of 1, and use small font (true):

print(o.lt, o.x+2, o.y-2, 2, true, 1, true)

In lua we can define several variables in one line, by crossing names of variables and assignations. For example, here, x=5 y=4 can be wrote x,y=5,4.

m=math rnd=m.random
t=0
objs={}
for i=1,rnd(5,8) do
 objs[i]={lt=rnd(10,30),x=120+rnd(-15,15),y=70+rnd(-15,15)}
end
function TIC()
 cls(12)
 for i=#objs,1,-1 do local o=objs[i]
  if o.lt<=0 then table.remove(objs,i)
  else o.lt,o.x,o.y=o.lt-1,o.x+rnd(-1,1),o.y+rnd(-1,1)
   circ(o.x,o.y,1,0)
   print(o.lt,o.x+2,o.y-2,2,true,1,true)
  end
 end
 t=t+1
end

In the Exemple to download we put the generation in a function generate() and call it when table content is null (#objs==0) as a simple loop generation.

function generate()
 for i=1,rnd(5,8) do
  objs[i]={lt=rnd(10,30),x=120+rnd(-15,15),y=70+rnd(-15,15)}
 end
end

function TIC()
 ...
 if #objs==0 then generate()end
 t=t+1
end

If you want to see the particles without their numbers, you can simply comment the print line. To comment code in Lua, simply add two dash at the beginning of the comment.

   -- print(o.lt,o.x+2,o.y-2,2,true,1,true)