Logging Preprocessor

Notes

Currently only for Windows. I am building using GCC 13.1.0 Rev7 MSYS2 on Windows 10. I suspect it will be portable to other Windows machines and C compilers but am not 100 % sure.

Although I don't use Linux at all it seems like it is very popular for embedded development so I am going to be working on porting to Linux over the next few days/weeks but won't promise anything.

At this point everything seems to be working largely as intended. Unfortunately I am not able to test it with hardware as I am travelling. However, testing with a Serial port emulator has been successful.

You need to implement your own device side function of the form send(uint8_t data) or similar, for sending the single byte over serial, the name of this function, i.e. send should be passed into the preprocessor with the argument -R. You probably want to put a line at the top of your source files along the lines of #define [Debug token] to keep your IDE happy. but that will be ignored by the preprocessor.

If you have any comments or find any issues then please feel free to create an account and open an issue, pull request etc. Admittedly they are at the limit of my Git knowledge but I will figure out how to work it.

If you do create an account then:

• Please use a randomised password, this is just a server I run as a hobby, I don't want it to be the reason your super secret password ends up in a data dump.
• It may take a few hours for me to manually approve the account. I have it set up this way to stop any spam accounts, you'd be amazed at the amount of hits this site gets from people fishing.
• You won't be able to create any repo's, I suppose with the manual account acceptance this is maybe not needed but if for whatever reason you want to be able to create repo's then let me know.

Overview

The first half of this program is intended to run before compilation and linking and basically is an extra step of preprocessing. It looks through the supplied input sourch files for a given debug token. This token denotes a debug message intended to be spat out over Serial.

The program creates a temporary edited file and in that file replaces the debug token with a print statement (user selectable) and the verbose message with a unique byte value, i.e. 1.

This unique value and the contents of the verbose message are recorded to a outputs file and the temporary edited file is passed to the next step of preprocessing/compilation (for user to implement).

Once the temporary files are compiled and uploaded to the target device then the second part of the system, the receiver program, is run. This program monitors a given serial port for bytes of data to assocaite with messages as given in the outputs file from before. When bytes of data are received the appropriate message is printed to the console in place of the value.

The intended use case is for verbose debugging/logging of embedded targets with a bare minimum amount of data transfer from the target to the host PC.

I believe something similar is done natively or by a library in Rust and I think I hear about it in a brief comment in "The Amp Hour" podcast.

Usage

Detailed notes on usage of each program are given in the Preprocessor and Receiver READMEs.

I am not really able to test it completely as I don't have any hardware on hand and am travelling.

However, it seems to be working as intended, the only thing that the user will have to do is work it into their toolchain. Likely this will mean calling the preprocessor from their makefile and changing the compilation paths in the makefile to look for files with the temp token, i.e. ".temp.c" rather then ".c".

Next steps

• Testing and bugfixing.
• Possibly adding the ability to print out file names and line numbers with each message. May be a bit redundant as messages can be arbitrarily long.
• Port to Linux.

Future Work

There are some interesting possible extensions once this is all working.

It could serve as a lightweight version of printf, substituting itself in place of printf to output a marker and then raw binary data (of the desired variables to be printed) with the format predetermined and stored in a secondary file and parsed/displayed by a program running on the host PC.

This could be done using a replacement token which can print/send multiple bytes and then correlating each byte with a piece of data.

As an example implementation, not at all valid code but shows the general idea.

The source file line: DEBUGF("The value of loop_counter is %d", loop_counter);

Would become:

Temp file: UART_send(5, loop_counter); or UART_send(5); UART_send(loop_counter);

Output file: 5 : 1 : The value of loop_counter is %d

Where 5 is the unique ID, 1 is the number of integers to be included and %d is used identically to a printf where it inputs the variable to the print.

There would have to be more thought on the details, immediately clear is that the implementation of the replacement token function will have to be such that it can send multiple things, or maybe it will be replaced with mutliple calls to a single byte sending function?

NB

Q: Wouldn't this be far simpler to do in Python?

A: Yes. But hey, I need an excuse to learn C.

Testing

For testing it is best to emulate a COM port with a loopback.

The simplest way I've found to do this is using com0com modem emulator. This sets up two COM ports internally linked so that any data sent into one of the COM ports is spat out into the other. It will be very useful for testing the helper program which parses the output file and displays the debug messages.

To setup the COM ports using the command line utility it is very simple, just put in install portname=COM101 portname=COM102. These ports will show up in device manager under com0com - serial port emulators and can be accessed, for example with pyserial or termite, by the above names, just like any other COM ports.

These ports are persistent through restarts.

Note for Python

Pythons pyserial library is to my mind, a little weird with putting integers into serial.write(). Instead of sending the integer value it will send the integer number of bytes of 0x00. This is kind of a strange way to do it but can be fixed by passing a byte type object into serial.write(). One way this can be done is, for example to send 0x05: serial.write((0x05).to_bytes(1, 'big')). Where the 1 is the number of bytes and big is the endian-ness. If you wanted to send 0x00 0x00 0x05 then you could use serial.write((5).to_bytes(3, 'big')).