Kevin Day [Wed, 12 Jan 2022 02:30:28 +0000 (20:30 -0600)]
Bugfix: If/else condition logic is incorrect or incomplete.
The if/else logic is not properly traversing.
I believe the original design is to just have if and else be a single set rather than a chain.
At some point I changed this to be chainable (or maybe I am incorrectly thinking that I did).
It made sense to use this if/then chain logic for my unit test files.
The Fake make unit test files are not working as I want them to because if/else chaining is not working.
As a big disclaimer, this does not support nesting.
I continue to refuse to design nesting of the if/else condition logic as I want to keep this as simple as possible.
Which, by the way, this is not simple.
I attempt to cut corners when I did my recent rewrite of the if/else logic.
Redesign this (again) but this time attempt to do this more properly.
The block process state now exclusively uses a special block process state enumeration rather than relying on a synthetic rule type.
The synthetic rule type is now removed.
The goal with this design is to:
1) On syntax error, bail out of the entire if/else chain.
2) On comparison error, continue an if/else chain passes or the end of chain is reached.
3) When any if/else chain passes, all subsequent conditions in the chain are not run.
Replace the "done" condition result with an "error" condition result to more accurately communicate its use.
Isolate the block and block result from the operate processing as much as possible so that all modifications of the variable are done outside of the operate processing.
When the "fail" is set to "exit" the if/else should fail appropriately on syntax error.
When the "fail" is set to "warn" or "ignore", then continue on after skipping the entire chain.
The operate process still needs to check the if/else chain states to determine if it should skip or not.
When an if/else condition terminates unexpectedly due to an end of file (or section) an error is more consistently reported regardless of place in the if/else chain.
This still needs real testing to confirm that all of the possible permutations work as expected.
Unit tests for this will need to be written but this is a long way away from being done.
Kevin Day [Mon, 10 Jan 2022 03:35:03 +0000 (21:35 -0600)]
Update: Implement f_capability unit tests and improve existing unit tests.
Add tests for f_capability functions.
This includes my first go at mocking functions using function wrapping.
It seems that the linker is limited on how it handles function wrapping.
I cannot simply wrap functions pulled in via a library.
To work around this I implemented another Fake build settings file for handling this.
This is not an ideal situation and adds a bit of extra work.
The environment variable settings for LD_LIBRARY_PATH needs to also support the work parameter.
These changes expose a logic flaw in the if/else logic of the Fake program.
That bug is not fixed in this commit, it is simply noted in this commit log.
Kevin Day [Mon, 10 Jan 2022 03:30:55 +0000 (21:30 -0600)]
Bugfix: Fix problems exposed by unit testing and perform cleanups.
f_capability_ambient_set() should not be testing for (!value_flag).
Te flags pointer test should be "flags" and not "*flags".
Explicitly cast capability to void * for cap_free().
Use "value" rather than "code" for "f_capability_value_t".
Expand out CAP_IS_SUPPORTED() and CAP_AMBIENT_SUPPORTED() to explicitly set F_true and F_false.
This is better than relying on the direct result of those macros.
Make sure f_type support "weak" compiler attributes.
Rename the compiler attributes to be more than just for visibility.
Kevin Day [Sun, 9 Jan 2022 03:48:52 +0000 (21:48 -0600)]
Cleanup: Simplify structure using more basic file names and using separate print files.
These are private files and are not intended to be exposed.
This allows for the names to be more generic.
Break out the print functions into their own separate files.
This will also make the code more multi-lingual friendly in the respect that much of the code that might need to be swapped out exists in a single file.
This is still very far from multi-lingual friendly with many print code still not in the isolated print functions file.
Kevin Day [Thu, 6 Jan 2022 05:08:59 +0000 (23:08 -0600)]
Update: Have fss-000e Payload be more specific about payload handling.
Because payload may be anything, especially a binary blob, having it always be last as a requirement makes the most sense.
This then allows me to guarantee comments and other lists may not be after or inside.
This allows for the payload to not have to have comments or delimited code to deal with.
The payload will be a complete virgin binary blob until EOF or a designated length is reached.
Kevin Day [Thu, 6 Jan 2022 04:40:45 +0000 (22:40 -0600)]
Bugfix: Valid comment lines are keeping leading white space.
Any line that ends up being a valid comment line may have white space on the left.
For lists, such as FSS Basic List, the leading white space is being printed.
This is not correct at all.
This is easily solved by getting the last new line and add one.
Update the specification to make the expected behavior clear.
Cleanup comment lines and other related code while its convenient.
Kevin Day [Tue, 4 Jan 2022 04:23:29 +0000 (22:23 -0600)]
Progress: Continue working on the controller program's control functionality.
The f_socket_listen() seems to need its own process or thread.
Give it one.
Make this one as isolated as possible so that it can be forcible exited.
(Because listen() doesn't respect signal handlers.)
Kevin Day [Tue, 4 Jan 2022 04:14:31 +0000 (22:14 -0600)]
Update: Fix bug in socket disconnect, add additional signal functions, and some clean up.
The socket close enumerations are being directly passed to shutdown().
This is not correct because they do not directly map.
Use the size_read and size_write already provided in the socket file.
This allows for the length to be exclusively a write (better practice).
Provide f_signal_wait() and f_signal_wait_until() functions that handle sigwaitinfo() and sigtimedwait() respectively.
Use uint8_t rather than unsigned short.
The fact that there is still a short in use here makes it clear that I have not even glanced at this file in a long long time.
Kevin Day [Sun, 2 Jan 2022 18:43:36 +0000 (12:43 -0600)]
Bugfix: Incorrect mode is being set.
The world/other character is being mixed with the user.
Swap the "on" value for world/other and user.
Always reset the "what" value before starting the condition loop.
Turns out that the previous code does not support the form "u+rw-x".
The following is an alternative way to write this that previously worked "u+rw,u-x".
Make sure both methods are supported by adding an additional check for the "+", "-", and "=".
When adding and removing, the previous opposing bits needs to be reset.
For example "u+rwx-r" should set the "add read" bit and then remove the "add read" bit while setting the "subtract read" bit.
Also absurd forms like the following need to work: "u+rrrrrwx+rwwwxwrwrwrwww" or "u+rwx,g-rwx+wrrr,o+rwx-wwr".
A copy and paste mistake in f_file_mode_to_mode() results in the wrong bits being set for the world bits.
Don't operate on the parameters directly.
Update them only on success.
This ensures a safer design at a cost of slightly more resources being used.
Swap the world and the owner bits for replace to make it more logically consistent in the order of the bits.
Hopefully this will make things slightly less confusing.
Kevin Day [Sun, 2 Jan 2022 04:07:52 +0000 (22:07 -0600)]
Progress: Begin implementing Control support in Controller program.
Numerous structural changes and cleanups.
Things are getting bigger so apply some more organization changes to the project structure.
Start using the pointer constant behavior.
Drafted out Control functionality.
Drafted out Task functionality, which is being considered and will be tested to see if I really want to do (keep) this.
Kevin Day [Sun, 2 Jan 2022 04:03:24 +0000 (22:03 -0600)]
Update: f_file project.
Now that I am moving towards using pointer constants (sch as "int * const"), I can move many of the parameters to the left side.
I believe this sufficiently follows the pattern of having constants on the left and editables on the right even though the data pointed to can be edited.
I feel this allows me to relax the compromise I originally made when following this design paradigm.
Kevin Day [Sun, 2 Jan 2022 04:00:49 +0000 (22:00 -0600)]
Update: Add socket functions and improve existing ones.
It is very clear to me that I had stopped working on the socket code.
Much of the code appears incomplete, including some comments that weren't updated after they were copy and pasted.
This is only preliminary work for only the functionality needed or preceived needed by the Controller program or other existing programs.
There will likely be major work in the future during the 0.7.x development release series.
Kevin Day [Sun, 2 Jan 2022 04:00:02 +0000 (22:00 -0600)]
Update: Add several new status codes and update existing ones.
A new group "_di_F_status_network_" is provided.
This is done in anticipation of a large amount of dedicated network related status codes.
The next minor development release series (0.7.x) is planned to focus heavily on networking.
The networking is highly specialized and will likely have network specific variations of many existing status codes.
Kevin Day [Sat, 1 Jan 2022 02:56:10 +0000 (20:56 -0600)]
Cleanup: Use present tense.
Using past tense is a habit because everything being referred to is generally in the past.
The documentation and comments really should present tense despite this.
There is a lot of code that really needs this done but is not done in this comment.
I just happened to see these and decided to immediately fix them.
Kevin Day [Sat, 1 Jan 2022 02:48:05 +0000 (20:48 -0600)]
Update: Improvements to file processing code.
Code cleanups.
Add initializer to F_file_mode_t.
Have file closures set id to -1 even on error due to documented design of the close() function.
Provide path type for process path related file system operation failures.
Some operations do not distinguish file or directory but instead only operate on the path itself.
In these cases, the path type is now available for use.
Add F_file_found_not to standard error printer.
This allows for the standard error printer to still report the problem rather than a code.
Using the file-specific standard error printer is still the recommended approach.
Kevin Day [Sat, 1 Jan 2022 02:46:13 +0000 (20:46 -0600)]
Update: f_path related improvements.
Return F_data_not rather than F_false to provide a more detailed reason as to the return status of f_path_is().
Explicitly check fo F_true when calling f_path_is() because it now returns F_data_not without the error bit.
Add f_path_is_relative().
Pretty much every path that does not begin with a "/" is a relative path.
This function performs a dirt simple check to achieve this.
This function does not perform any checks to see if the path would be a valid path.
Such decisions are filesystem and are otherwise too situation dependent to be reliable detect in a general manner.
Use f_path_separator_s rather than an explicit "/".
Simplify counter using a pre-increment within the if condition check.
Kevin Day [Sat, 25 Dec 2021 22:38:31 +0000 (16:38 -0600)]
Feature: Add support for "context" IKI in Fakefiles.
This updates the IKI-0002 specification to support "context".
This allows for color context and ideally any future context to make software more accessible.
Update the existing testfiles to utilize this context.
Kevin Day [Fri, 24 Dec 2021 05:49:32 +0000 (23:49 -0600)]
Cleanup: Append "_e" to all enums, update status types, and update status strings.
A new practice is to have "_e" at the end of all enums.
Update all of the existing enums to follow this practice.
I noticed some fixme comments about moving the status codes to a lower level.
Do this.
Update all code accordingly.
The status code strings are only for special cases, so make this its own project directory (for both Status and FSS Status).
These are f_status_string and fll_fss_status_string.
Kevin Day [Thu, 23 Dec 2021 17:43:57 +0000 (11:43 -0600)]
Update: Use 'shell' rather than 'run' in testfile.
I apparently missed this line.
The 'run' operation will attempt to execute based on PATH environment.
The 'shell' operation expects local files, which is what these tests are.
Kevin Day [Thu, 23 Dec 2021 17:38:21 +0000 (11:38 -0600)]
Update: Add tests for f_type_array.
Given the size of the task and my relative inexperience with cmocka, I opted to keep the tests as simple as possible.
This means that my tests are likely not thorough.
These are intended to be good enough for a first pass tests.
After all, I have the entire rest of the FLL projects to write tests for (as well as for the programs).
Kevin Day [Thu, 23 Dec 2021 04:58:47 +0000 (22:58 -0600)]
Update: Move main data into the data make structure.
The goal here is to reduce the number arguments passed to functions in a consistent manner.
This has a downside of having another pointer and the costs of dereferencing it for each access.
Rename the "main" property to "id_main" to be clearer as well as to not conflict with the new "main" property that points to the main data.
Remove a stale header that has no associated function implemented.
Kevin Day [Thu, 23 Dec 2021 03:59:47 +0000 (21:59 -0600)]
Update: Add "and" and "or" operations, operation if-then-else logic, and some cleanups.
Add the "and" and "or" operations to make the "if" and "else" operations more complete.
The current design still does not support directly nested multiple operations underneath an "if" or an "else".
Using an "operate" operation is still the only way to do this.
The reason for this is to keep the logic and design simple.
This has a cost of causing complicated design to be even more complicated than they otherwise could be if I allowed "if" and "else" to contain blocks of commands or even be nested.
While I am at it, this cleans up the if-then-else logic.
This needed to be done for some time.
My original design was patched at some point to add unexpected functionality that I had not originally planned but later realized I needed.
The patch was just a good enough for now.
This resulted in the logic being a bit ugly and confusing.
The new logic is a lot better, but there are still some things that might be confusing.
To that end, I decided to only perform simple tests.
I instead intend to write unit tests that will perform all of the possible combinations that I can reasonably come up with.
This should help me find logic flaws in my current design.
This now uses a structure to pass the process state data.
This further simplifies the design to allow for fewer parameters in the relevant functions.
Several of the duplicated print calls are consolidated into a single print function.
Kevin Day [Thu, 23 Dec 2021 03:40:54 +0000 (21:40 -0600)]
Security: Invalid memory access in interrupt handlers.
When I converted the data_main into a pointer from a value, I failed to remove the references on these variables.
The compiler does not catch these because they are cast to a void *.
Kevin Day [Mon, 20 Dec 2021 05:19:23 +0000 (23:19 -0600)]
Update: Add support for if "not", add support for "parameter".
The "parameter" is already supported under "settings" in a fakefile.
Extend the operations to also support "parameter".
This allows for defining the parameter anywhere within the file and it can be overridden.
There exists several "if" operations that would make sense to have the inverse.
This is now supported via the if "not" operation.
To keep the logic more consistent with the previous design, just extend the existing code to handle "if not" behavior rather than adding new structures.
To achieve this, the pre-processor identifies "not" and then parses that to identify the particular "if" operation that is being negated.
The "if" operation is then change to a new operation type to reflect this.
Anything that already supports an inverse through some means are omitted from this.
This list includes:
- fail (opposite of success).
- success (opposite of fail).
- integer/math comparisons: ==, <>, <, <=, >, >=.
Fix the order of several of the functions that are not alphabetical.
Kevin Day [Sun, 19 Dec 2021 18:36:24 +0000 (12:36 -0600)]
Update: Relocate fake_main_t position in function arguments and make it a constant pointer.
The standard practice is all constants on the left and all updatable variables on the right of a functions parameters.
At some point I switched to passing the structures as a pointer rather than directly.
For these, I made them pointer constants.
That is, the pointer itself is constant but what the pointer is pointing to is not.
This allowed for me to move this type further to the left.
Keeping the main data and the fake data on the left side of the functions is a lot more consistent.
Kevin Day [Sun, 19 Dec 2021 18:18:03 +0000 (12:18 -0600)]
Bugfix: The "if defined parameter .." is not supporting reserved parameters.
There are several reserved parameters that are supposed to be supported.
Add code checking for the reserved words for the "if defined parameter" operation.
Update the documentation to better communicate these reserved words and how they operate.
Kevin Day [Sun, 19 Dec 2021 05:49:40 +0000 (23:49 -0600)]
Security: Segfault when "load_build yes" and "build settings".
When the fakefile settings is setup to have "load_build yes" and the fakefile operations has a build operation like "build settings" a segfault occurs.
This appears to be the result of casting the main path_sources to a constant pointer type from a reference.
Kevin Day [Sat, 18 Dec 2021 17:03:30 +0000 (11:03 -0600)]
Update: Improve IKI support, various cleanups, and a few bug fixes.
Allow for getting just the parameter option or parameter value for the special reserved IKI parameters.
This allows form something like:
define LD_LIBRARY_PATH "build/libraries/shared:parameter:'work:value'"
When populating the special parameters, the parameters not specified are getting saved.
This is incorrect.
Skip parameter that are not specified (f_console_result_none).
Kevin Day [Sat, 18 Dec 2021 02:11:34 +0000 (20:11 -0600)]
Update: IKI improvements, cleanups, and bugfixes.
The IKI project has fallen behind in some of the practices and is more consistent.
- Expose the delimit array to the caller rather than operating on it internally.
- This makes IKI more consistent with FSS and improves extensibility.
- Fix the global string names and macro need to follow the appropriate naming structure.
- Use strings rather than characters in the defines.
- Get rid of some macros, replacing them with functions.
- Use the *_increase() and *_increase_by() functions.
The FSS projects are now passing the delimit management to the callers.
IKI is now updated to do the same.
New data types, such as f_iki_delimit_t, are provided to achieve this.
When there are multiple separators, a colon ':', a valid IKI data might be skipped.
This is happening for two reasons:
1) Incorrect increment of the location after identifying a non-IKI defining colon.
2) The seek function is not stopping on special characters like a colon.
The iki_read program is updated to reflect these changes.
A new structure, called iki_data_t, is provided to simplify the arguments being passed around.
The iki_write program is updated to reflect these changes.
The make program is updated to reflect these changes.
Kevin Day [Wed, 15 Dec 2021 02:39:03 +0000 (20:39 -0600)]
Update: Switch to simple string and cleanup syntax and comments.
The f_string_static_t operations_name array can be converted to just an array of f_string_t.
Then the f_string_range_t operations_range can become f_array_length_t operations_length.
Then the operation_name can be removed (it appears to be unused).
Finally The fl_string_dynamic_partial_compare() can become fl_string_dynamic_partial_compare_string().
Update comments and add additional inline comments to help clarify situations.
Kevin Day [Wed, 15 Dec 2021 02:34:21 +0000 (20:34 -0600)]
Update: Don't bother checking, just always update pointer.
At this point the pointer has been allocated.
If the pointer addresses are the same, then there is no problem.
If they are different, then this properly replaces.
Assigning this just removes the extra step of checking.
Kevin Day [Sun, 12 Dec 2021 15:41:16 +0000 (09:41 -0600)]
Update: Restructure fake settings, moving examples into a new projects directory.
Create a projects directory to store some real replacements of other projects build systems.
The bzip2 build system was used as just an example and is now treated as a real use case.
I am planning on trying to use cmocka to provide unit tests for this project.
The cmocka uses that rather unpleasant cmake.
Provide a cmocka fake build setting file for building cmocka.
Kevin Day [Sun, 12 Dec 2021 05:59:06 +0000 (23:59 -0600)]
Update: Improve performance by removing redundant memset().
The calloc() program is supposed to guarantee 0 filled data.
Either the libc or the kernel know how to optimize this automatically using numerous tricks based on architecture or lack thereof.
This makes calloc() potentially faster than malloc()+memset().
Calling calloc()+memset() is just ridiculous.
Remove the calls to memset() that follow a calloc() call.
This is guaranteed to be a performance increase (but how much? I didn't bother trying to find out).
Kevin Day [Sun, 12 Dec 2021 05:50:23 +0000 (23:50 -0600)]
Update: Use C11's aligned_alloc() by default, but keep posix_memalign() via macro _f_memory_USE_posix_memalign_.
The C11 standard introduced aligned_alloc() making it better practice than posix_memalign().
In case the compiler being used doesn't have aligned_alloc() or the user compiling just wants to posix_memalign() this behavior is preserved via _f_memory_USE_posix_memalign_ macro.
I didn't actually test this beyond confirming that it compiles.
I'm flying blind here.
Kevin Day [Sun, 12 Dec 2021 01:14:05 +0000 (19:14 -0600)]
Cleanup: Utilize 'void' inside of function declarations.
It seems that by adding 'void' (without a parameter variable name) instructs the compiler that this function is not allowed to take arguments.
When the parameters are empty such as '()', the compiler simply disable checking what the parameters are.
By adding void this results in instructing the compiler to verify that there are no parameters.
This increases the code integrity.
This change may be a problem for older C compilers.
Kevin Day [Sat, 11 Dec 2021 22:31:08 +0000 (16:31 -0600)]
Update: Add documentation.
This is yet another reminder to me to try and avoid accidental commits.
I should have already had the documentation written up and be committed along with the initial commit.
Given that this project was accidentally committed before it was ready to, this left the project in less than ideal state.
As a reminder to myself to help encourage avoiding this mistake, I am constantly adding this oops notice to my commits.
With this documentation written, I once more believe that I have wrapped everything up that I need to consider this ready.
I previously thought this was the case, but as is seen by recent previous commits, this was not the case.
Going forward, I plan on investigating writing tests for this project and to use this project as an example of writing tests for the entire FLL probject.
This will hopefully allow me to find any remaining bugs and make this program production ready.
Kevin Day [Sat, 11 Dec 2021 22:06:37 +0000 (16:06 -0600)]
Feature: Support outputting width or combining state of characters.
The width is reported as one of: '0', '1', or '2'.
The following is used for unknown or invalid '?'.
The private use area is consider valid but unknown.
The combining state is reported as either 'C' or 'N'.
The 'N' can be considered either 'Not' or 'No' as the meaning is synonymous in this case.
The to_combining and to_width may be used together.
Now that I know how this is to be implemented, remove unneeded functions.
This is failing for two reasons:
1) Not using the original string data when printing detected invalid characters.
2) Performing the from codepoint check before checking the binary output check in the function utf8_print_character().
Also remove a redundant not zero check in the error print function utf8_print_character_invalid().
the function utf8_print_character_invalid() is a wrapper to utf8_print_character(), where that check is already performed.
Kevin Day [Sat, 11 Dec 2021 03:57:44 +0000 (21:57 -0600)]
Bugfix: Error handling should not exit for certain errors.
When a process signal is being received, F_signal is being set with the error bit.
This should not have the error bit set.
Move the conditional logic inside the appropriate printing functions.
Add utf8_print_character_invalid() for printing an error character.
Invalid UTF-8 fragments should not result in an exit on error.
Instead, these should be handled by either appropriate printing or by setting the is valid property on exit.
Kevin Day [Sat, 11 Dec 2021 03:36:42 +0000 (21:36 -0600)]
Bugfix: Raw formatted print sometimes prints trailing NULL.
A logic flaw is resulting in the last NULL after the max length is reached to be printed.
When the strnlen() calculates the length and the calculated length is the requested max length, the subsequent line attempts to print any NULLs.
This is normally fine, except that it needs to check to make sure that "i" is less than the requested max length.
Kevin Day [Fri, 10 Dec 2021 01:36:39 +0000 (19:36 -0600)]
Bugfix: Do not print leading zero's in large Unicode codepoints.
Also cleanup the code moving some generic print functions into utf8-specific print functions.
Use character rather than text to better communicate that the string is intended to represent a single Unicode character.
Kevin Day [Fri, 10 Dec 2021 01:17:43 +0000 (19:17 -0600)]
Bugfix: Codepoint to Binary is not working.
The wrong variable is being processed.
The codepoint (which is the Unicode representation, such as U+8C78 the codepoint is for the character '豸') is being width checked.
The binary character is what should be getting the width check.
A second situation where the codepoint is not being printed at all is with the files.
It seems that I forgot to finish writing this code (another problem caused by my original accidental commit of this project).
While investigating this I saw some opportunity for some cleanup.
- Move the width detection into a separate function utf8_process_text_width().
- Use character.string[0] instead of *character.string.
- Rename 'character' to 'current' to make more semantic sense (At the time I wasn't sure what to call it and 'text' was already unavailable).
- The 'text' in private-utf8_codepoint.c is now initialized in a simpler way.
Kevin Day [Thu, 9 Dec 2021 05:43:42 +0000 (23:43 -0600)]
Bugfix: Performance is slow due to process signal checks (more).
This is a follow up to the previous commit similarly named.
I forgot to hit the save button for the changes to FSS Basic List files in my editor.
The cost of the system call for checking if a signal is received is more expensive than I have previously imagined.
I also was not sure where I should handle the signals and I arbitrarily put them inside loops.
Reduce the number of checks.
Reduce the number of the system call to check the process signal using modulus math.
The performance difference is most notable when using the byte_dump program.
This focuses on solving the immediate performance bug.
I still have not done any extensive performance investigations and I expect this to still have significant room for improvement.
Kevin Day [Thu, 9 Dec 2021 05:21:57 +0000 (23:21 -0600)]
Bugfix: Performance is slow due to process signal checks.
The cost of the system call for checking if a signal is received is more expensive than I have previously imagined.
I also was not sure where I should handle the signals and I arbitrarily put them inside loops.
Reduce the number of checks.
Reduce the number of the system call to check the process signal using modulus math.
The performance difference is most notable when using the byte_dump program.
This focuses on solving the immediate performance bug.
I still have not done any extensive performance investigations and I expect this to still have significant room for improvement.
Kevin Day [Tue, 7 Dec 2021 03:51:02 +0000 (21:51 -0600)]
Update: Add missing function in f_utf needed for completeness and reduce repeated code.
As per my completeness principle, the f_utf_unicode_string_to() must have the f_utf_character_unicode_string_to() compliment.
This function only allows for ASCII characters to represent the number and returns errors as appropriate for non-ASCII values.
Unicode number values are not treated as the ASCII numbers for representing a Unicode code sequence.
The f_utf_character_unicode_to() and f_utf_unicode_to() now has code reduced by utilizing private_f_utf_character_unicode_to().