Wilkie
/
LoggingPreprocessor
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Working with multiple files able to be used as inputs. All output to a single references file. 

Updated READEMEs to reflect status.
Added things to do regarding temp file names and locations.
This commit is contained in:
Thomas Wilkie 2023-12-03 18:45:30 +09:00
parent 0ae1193e67
commit 94b49019bd
11 changed files with 212 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Makefile
View File

@ -11,4 +11,4 @@ build_ppsr:
gcc -o Preprocessor/app.exe Preprocessor/*.c -I Preprocessor -I ../ReferencesFileInfo.h -Wall --pedantic
run_ppsr:
./Preprocessor/app.exe -R send -D debug
./Preprocessor/app.exe -R send -D debug Test_files/*test.c doesnt_exist.c

51
Preprocessor/GetArguments.c
View File

@ -3,13 +3,12 @@
// Initialises with the default values
argsList application_arguments =
{
.input_file_location = "Test_files/first_test.c",
.references_output_file_location = "References.txt",
.replacement_token = "Send",
.debug_token = "DEBUG"
};
// Makes the argument cases a bit clearer and slightly easier to add
// Makes the argument cases a bit clearer and slightly easier to add more
#define AddArgumentCase(character, variable) case character: \
{\
variable = *(argv + arg_counter + 1);\
@ -22,11 +21,20 @@ argsList application_arguments =
* @param app_arguments Pointer to struct containing all the possible arguments, initialised with defaults
* @param argc The number of argument passed in on the command line
* @param argv List of pointers to the arguments passed in
*
* @return Number of input file path arguments found
*/
void CheckArguments (argsList * app_arguments, int argc, char ** argv)
uint8_t CheckArguments (argsList * app_arguments, int argc, char ** argv)
{
// Skip the first argument on the list, it's just the path
/*
* Skip the first argument on the list, it's just the path.
* If you have more then 255 arguments then there's bigger problems
*/
uint8_t arg_counter = 1;
app_arguments->args_list_input_files_indexs = (uint8_t *) malloc(sizeof(uint8_t) * argc);
uint8_t input_file_list_counter = 0;
while (arg_counter < argc)
{
@ -37,7 +45,6 @@ void CheckArguments (argsList * app_arguments, int argc, char ** argv)
char arg_type = *(next_arg + 1);
switch (arg_type)
{
AddArgumentCase('I', app_arguments->input_file_location);
AddArgumentCase('O', app_arguments->references_output_file_location);
AddArgumentCase('D', app_arguments->debug_token);
AddArgumentCase('R', app_arguments->replacement_token);
@ -45,9 +52,43 @@ void CheckArguments (argsList * app_arguments, int argc, char ** argv)
printf("%s is not a valid argument.\n", next_arg);
break;
}
// Next arg is the argument for the above case and so can be skipped
arg_counter++;
} else
{
// If the arg isn't handled by the above then it is a file path, add it to the list
*(app_arguments->args_list_input_files_indexs + input_file_list_counter) = arg_counter;
input_file_list_counter++;
}
arg_counter++;
}
// Terminating NULL
*(app_arguments->args_list_input_files_indexs + input_file_list_counter) = 0;
return input_file_list_counter;
}
/**
* @brief Prints all the input file paths
*
* @param app_arguments Struct holding all the application arguments passed over the command line
* @param argv List of string input arguments
*/
void PrintInputFilePaths(argsList * app_arguments, char ** argv)
{
printf("Input file paths:\n");
while(1)
{
static uint8_t i = 0;
uint8_t argument_index = *(application_arguments.args_list_input_files_indexs + i);
if (!argument_index)
{
break;
}
printf("%hhu : %s\n", i, *(argv + argument_index));
i++;
}
return;
}

9
Preprocessor/GetArguments.h
View File

@ -8,18 +8,18 @@
/**
* @brief Struct to hold pointers to all the possible arguments
*
* @param input_file_location The file to scan through to find the debug tokens
* @param references_output_file_location The file to write all the token value pairs to
* @param replacement_token The token for the debug token to be replaced by
* @param debug_token The token to be used to mark a debug statment`
*
* @param args_list_input_files_indexs The indexes of any input file paths held in the args list
*
*/
typedef struct
{
char * input_file_location;
char * references_output_file_location;
char * replacement_token;
char * debug_token;
uint8_t * args_list_input_files_indexs;
} argsList;
/**
@ -28,5 +28,6 @@ typedef struct
*/
extern argsList application_arguments;
void CheckArguments (argsList * app_arguments, int argc, char ** argv);
uint8_t CheckArguments (argsList * app_arguments, int argc, char ** argv);
void PrintInputFilePaths(argsList * app_arguments, char ** argv);
#endif

42
Preprocessor/READEME.md
View File

@ -1,38 +1,26 @@
# Purpose
# Preprocessor
This is the program which scans through the code and changes the debug tokens to the desired system call while saving the message externally and associating it with a unique value.
## Usage
Runs over the target file and creates the processed temporary file. Also creates the output reference file which is stored up one directory level.
`./app.exe <target file path> <output file path>`
Runs over the files passed in and outputs to a reference file. There are optional arguments with defaults given below. Any argument not before a "-X" marker will be taken as an input file path.
Keep in mind that (in Windows 10 powershell) if you pass in a path with a wildcard, i.e. "src/*.c" then that path will be expanded by Powershell before being passed to the app. I.e. the app will automatically receive multiple arguments of the expanded file paths.
Keep in mind that the executable's file search is from the directory it is called from and not the directory the executable is located in.
Can be compiled and ran using `make build_ppsr` and `make run_ppsr`.
__NOTE__ The `Debug("Message")` must be indented by at least one space or tab character for everything to work. _To Do:_ Fix this.
| Option | Argument | Default | Description |
|----------------|----------|----------------|-----------------------------------------------------------------|
| Input file | -I | Test_files/first_test.c | Path to input file |
| Output file | -O | References.txt | Path to output references file |
| Debug Token | -D | DEBUG | Token to to mark debug statement |
| Replacement Token| -R | Send | Replacement token for debug to send data |
| | | | |
Can be compiled and ran using `make build_ppsr` and `make run_ppsr`. _To Do:_ Temp file directory and temp file append are not implemented yet.
## TODO
| Option | Argument | Default | Description |
|---------------------|----------|----------------|------------------------------------------------------|
| Output File | -O | References.txt | Path to output references file |
| Debug Token | -D | DEBUG | Token to to mark debug statement |
| Replacement Token | -R | Send | Replacement token for debug to send data |
| Temp File Directory | -T | [1] | Directory to store temp files in |
| Temp File Append | -A | .temp | String to append to files to mark them as temp files |
| | | | |
This kind of works but a few things need to be changed.
The line numbers are probably extraneous but may be useful to make it more human-readable.
There needs to be a header put at the top of the references file of format:
``` c
printf("Version : %hhu.%hhu, Debug messages : %hhu, Max line length : %hu\n",
&references_header->version_major,
&references_header->version_minor,
&references_header->number_debug_messages,
&references_header->max_line_length);
```
This will require keeping track of the length of strings which shouldn't be a problem with strlen().
After all that is done then it needs to be able to take multiple files as inputs.
[1] If this is not specified then temp files are created at the same directory as their corresponding input files. If it is specified then all the temp files will be put into that directory.

189
Preprocessor/main.c
View File

@ -19,8 +19,10 @@ referencesFileHeader references_header_info =
.version_minor = 1
};
HRESULT setupFiles(char * filepath, FILE ** input_file, FILE ** temporary_file);
HRESULT SetupFiles(char * filepath, FILE ** input_file, FILE ** temporary_file);
bool ReadNextLine(FILE * input_file, FILE * temporary_file, FILE * output_file);
HRESULT OpenNextFile(char * filepath, FILE ** input_file, FILE ** temporary_file);
HRESULT OpenInputFile(char * input_file_path, FILE ** input_file, FILE ** temporary_file);
void DebugTokenLine(FILE * temporary_file, FILE * output_file, char * line_copy);
void PrintHeaderInformation(FILE * file, fpos_t file_start);
void PadFile(FILE * file);
@ -28,31 +30,71 @@ void PadFile(FILE * file);
int main (int argc, char ** argv)
{
CheckArguments(&application_arguments, argc, argv);
if (argc > 255)
{
printf("Too many arguments\n");
return 0;
}
if(!CheckArguments(&application_arguments, argc, argv))
{
printf("No input files given\n");
return 1;
}
//PrintInputFilePaths(&application_arguments, argv);
// This is the file to be read from
FILE * input_file = NULL;
// This is the file which will be created with the debug statements changed
/**
* @brief Copy of input file with debug tokens changed.
*
* @note This is created for each input file and is essentially a copy of the file.
* However, it has the debug tokens replaced with the desired function call.
* Each debug token has a unique value which is correlated in the references file.
* This temporary file will have the same name but a .temp extension appended
*/
FILE * temporary_file = NULL;
// This is the reference file which will be generated and is to be passed to the companion program at runtime
// This will create a new file on each run so care should be taken to ensure that it doesn't overwrite an older file
/**
* @brief Output file for references
*
* @note This is the reference file which will be generated and is to be passed to the receiver program at runtime.
* This will create a new file on each run so care should be taken to ensure that it doesn't overwrite an older file.
*/
FILE * output_file = fopen(application_arguments.references_output_file_location, "w");
// This will be needed later to write the header to the file
fpos_t output_file_start;
fgetpos(output_file, &output_file_start);
PadFile(output_file);
// Go through and process each file from the list
while (1)
{
static uint8_t i = 0;
// If argument_index is 0 then the list of arguments is finished
uint8_t argument_index = *(application_arguments.args_list_input_files_indexs + i);
if (!argument_index)
{
break;
}
char * next_file_name = *(argv + argument_index);
if (!OpenInputFile(next_file_name, &input_file, &temporary_file))
{
// Read and process all the lines out of the file
while(ReadNextLine(input_file, temporary_file, output_file));
// Close the input file and its temporary copy
fclose(input_file);
fclose(temporary_file);
}
setupFiles(application_arguments.input_file_location, &input_file, &temporary_file);
while(ReadNextLine(input_file, temporary_file, output_file));
i++;
}
// Finished reading through all the file and editing, now put the header with metadata at the top of the output file
PrintHeaderInformation(output_file, output_file_start);
fclose(input_file);
fclose(temporary_file);
fclose(output_file);
printf("Done\n");
@ -61,49 +103,14 @@ int main (int argc, char ** argv)
/**
* @brief
* @brief Reads the next line of the input file and decides if it has an input token. If it does then sends it to the debug token handler function.
*
* @param filepath
* @param input_file
* @param temporary_file
* @return HRESULT
*/
HRESULT setupFiles(char * filepath, FILE ** input_file, FILE ** temporary_file)
{
// Confirms the destination file exists, opens it for reading and creates a temp file to write into
printf("Copying and reading from %s\n", filepath);
// ".temp" is only 5 characters but the 6th is to include the string terminator
// Open input file for read fopen returns null if input file doesn't exist.
*input_file = fopen(filepath, "r");
if (!*input_file)
{
printf("File not found.\n");
return HFILE_ERROR;
}
uint16_t filepath_length = strlen(filepath);
char * temp_filepath = (char *) malloc(filepath_length + 6);
// Confirm memory assignment has worked, pretty unlikely to fail
if (!temp_filepath) return E_OUTOFMEMORY;
memcpy(temp_filepath, filepath, filepath_length);
const char * temp_name_suffix = ".temp";
memcpy((temp_filepath + filepath_length),temp_name_suffix, 6);
*temporary_file = fopen(temp_filepath, "w");
free(temp_filepath);
return S_OK;
}
/**
* @brief
* @param[in] input_file File to read
* @param[in] temporary_file File to copy out either unedited lines or edited lines with debug token substituted
* @param[in] output_file File to put associations of debug message IDs and the messages
*
* @param input_file
* @param temporary_file
* @param output_file
* @return true
* @return false
* @return true : Another line available to be read in file.
* @return false : EOF.
*/
bool ReadNextLine(FILE * input_file, FILE * temporary_file, FILE * output_file)
{
@ -113,7 +120,7 @@ bool ReadNextLine(FILE * input_file, FILE * temporary_file, FILE * output_file)
// fgets() returns null for EOF
if(!fgets(line, sizeof(line), input_file))
{
printf("End of file\n");
//printf("End of file\n");
line_counter = 0;
return false;
}
@ -162,16 +169,16 @@ bool ReadNextLine(FILE * input_file, FILE * temporary_file, FILE * output_file)
/**
* @brief
* @brief Takes a line with a debug token gives it an ID and prints out the edited line to the temp file and references file
*
* @param temporary_file
* @param output_file
* @param line_copy
* @param[in] temporary_file
* @param[in] output_file
* @param[in] debug_line
*/
void DebugTokenLine(FILE * temporary_file, FILE * output_file, char * line_copy)
void DebugTokenLine(FILE * temporary_file, FILE * output_file, char * debug_line)
{
// Read the tabs/whitespace before the start of the debug token
char * tabs_token = strtok(line_copy, application_arguments.debug_token);
char * tabs_token = strtok(debug_line, application_arguments.debug_token);
// Read and then ignore up to the first double quotes, this will be the debug token
strtok(NULL, "\"");
// Read the debug message into the token
@ -222,22 +229,64 @@ void PrintHeaderInformation(FILE * file, fpos_t file_start)
/**
* @brief
* @brief Adds a lump of whitespace terminated with a newline to the start of the file
*
* @param file
* @note This needs to be done so the header can be added without overwriting data at the end.
* It's a bit of a bodge and relys on the header being no larger than a hard-coded size
* At this point the header structure is
* "Version : %hhu.%hhu, Debug messages : %hhu, Max line length : %hu\n"
* Which gives it a maximum length of <70 characters (64 I believe)
* So the file start will be padded with empty characters
* There can be extra whitespace without any issues
*
* @param file Output references file to add the padding to
*/
void PadFile(FILE * file)
{
/*
* This needs to be done so the header can be added without overwriting data at the end.
* It's a bit of a bodge and relys on the header being no larger than a hard-coded size
* At this point the header structure is
* "Version : %hhu.%hhu, Debug messages : %hhu, Max line length : %hu\n"
* Which gives it a maximum length of <70 characters (64 I believe)
* So the file start will be padded with empty characters
*/
const char newline = '\n';
// Padding length has to be hardcoded into printf
fprintf(file, "%70c", newline);
return;
}
/**
* @brief Opens the file given by the input path creates a temporary file to store the edited file
*
* @param[in] input_file_path Path of input file
* @param[out] input_file Address of pointer to file handler for input file
* @param[out] temporary_file Address of pointer to file handler for output file
*
* @return Result of trying to open files
*/
HRESULT OpenInputFile(char * input_file_path, FILE ** input_file, FILE ** temporary_file)
{
printf("Copying and reading from %s\n", input_file_path);
// Open input file for read fopen returns null if input file doesn't exist.
*input_file = fopen(input_file_path, "r");
if (!*input_file)
{
printf("File not found : %s\n", input_file_path);
return HFILE_ERROR;
}
uint16_t filepath_length = strlen(input_file_path);
// + 6 is to fit the extra ".temp" appeneded to the end along with a NULL termination
char * temp_filepath = (char *) malloc(filepath_length + 6);
if (!temp_filepath)
{
printf("Memory error\n");
return E_OUTOFMEMORY;
}
memcpy(temp_filepath, input_file_path, filepath_length);
const char * temp_name_suffix = ".temp";
memcpy((temp_filepath + filepath_length),temp_name_suffix, 6);
*temporary_file = fopen(temp_filepath, "w");
free(temp_filepath);
return S_OK;
}

31
README.md
View File

@ -18,33 +18,21 @@ For initial development it will be targetted at a single file and give a single
`./app.exe [target_file.c]` Will read through the target file and substitute in the values, printing pairs to the output file.
Detailed notes on usage are given in the Preprocessor and Receiver READMEs.
## Companion Program
There will also need to be a companion program setup to actually read the incoming data stream and parse it.
Initially I think the starting point would be to do it only over serial port but any other data streams could be used.
## Next step
## Next steps
Create a new file for the pairs .
Send the `key : value` pairs to the separate file.
Enable specification of the different tokens, i.e. debug token, replaced token.
Enable more command line options (but keep defaults). Some ideas:
- `-of` -> Output file for the `key : value` pairs
- `-dt` -> Specify the debug token, currently defaults to `DEBUG`
- `-rt` -> Specify the replacement token, this is the function to be called by the target to actually send the value
- `-rd` -> Dynamic replacement token, looks to a defined line in the file to find out the replacement token
- `-ml` -> Max line length in characters
- `-mm` -> Max message length in characters
- `-ib` -> ID bytes, one or two byte unsigned ints for the ID's, would allow extension past 256 unique messages
Message reuse would also be good, allowing identical messages to use the same unique ID, but a bit difficult and super unecessary.
- Testing and bugfixing.
- Possibly adding the ability to print out file names and line numbers with each message. May be a bit redundant as messages can be arbitrarily long.
## Future Work
There are some really interesting possible extensions once this is all working.
As a first step it really should be able to read multiple files and combine them all so you aren't limited to just logging statements in your main file. This could be done by looking at all files in a directory. Probably at this point it would be better to put all the temp files in their own directory though.
There are some interesting possible extensions once this is all working.
It could serve as a lightweight version of `printf`, substituting itself in place of `printf` to output a marker and then raw binary data (of the desired variables to be printed) with the format predetermined and stored in a secondary file and parsed/displayed by a program running on the host PC.
@ -68,12 +56,13 @@ There would have to be more thought on the details, immediately clear is that th
Q: Should this just be done in Python?
A: Yes. But hey, I need an excuse to learn C. Also Python is a tiny bit slower, it's just feels nice to have it all run in a fraction of a second.
A: Yes. But hey, I need an excuse to learn C.
## Testing
For testing I would like to emulate a COM port.
It seems like the simplest way to do this is using [com0com](https://com0com.sourceforge.net) modem emulator.
For testing it is best to emulate a COM port with a loopback.
The simplest way I've found to do this is using [com0com](https://com0com.sourceforge.net) modem emulator.
This sets up two COM ports internally linked so that any data sent into one of the COM ports is spat out into the other. It will be very useful for testing the helper program which parses the output file and displays the debug messages.
To setup the COM ports using the command line utility it is very simple, just put in `install portname=COM101 portname=COM102`.

20
References.txt
View File

@ -1,8 +1,12 @@
Version : 0.1, Debug messages : 7, Max line length : 94
1 : This is the first debug statement.
2 : This is the second debug statement.
3 : Third
4 : Again
5 : Much longer message this time I think it should be able to handle this all without a problem
6 : shorter
7 : This has the debug token in the name and it isn't an issue.
Version : 0.1, Debug messages : 11, Max line length : 94
1 : Another_test.c file
2 : Hello from another
3 : This is the first debug statement.
4 : This is the second debug statement.
5 : Third
6 : Again
7 : Much longer message this time I think it should be able to handle this all without a problem
8 : shorter
9 : This has the debug token in the name and it isn't an issue.
10 : logging_test.c is another file
11 : logging tests

2
Test_files/another_test.c
View File

@ -0,0 +1,2 @@
debug("Another_test.c file");
debug("Hello from another");

2
Test_files/logging_test.c
View File

@ -0,0 +1,2 @@
debug("logging_test.c is another file");
debug("logging tests");

0
Test_files/main.c
View File

0
Test_files/other shit.h
View File