Kevin Day [Tue, 10 Dec 2024 03:34:41 +0000 (21:34 -0600)]
Bugfix: Fix handling of --line when not using --whole.
The function `iki_read_process_at()` is incorrectly named and should instead be called `iki_read_process_line()`.
The `--object`, `--content`, and `--literal` parameters with `--line` should print the line number for the output of those commands raather than the output of the file.
The printing of the line from the original file is very confusing and so I am considering this a bug.
Kevin Day [Sun, 8 Dec 2024 01:55:01 +0000 (19:55 -0600)]
Bugfix: The -t/--total in iki_read is confusing and allow for --total with --whole.
The `--total` parameter should print the total number of IKI Variables.
The `--line` parameter is selecting based on the line as if the `--whole` parameter is based (even when it is not).
This makes little sense when in the `--content` that is the default behavior.
This solves part of that confusion by focusing on the `--total` parameter.
Using the `--whole` should also print the total number of lines rather than total number of Variables.
I will need to follow this up with runtime tests.
There are none at this time so a separate commit is needed to add all of the appropriate tests.
Kevin Day [Sat, 23 Nov 2024 21:00:03 +0000 (15:00 -0600)]
Update: The time specification, making it more flexible.
Improve the time specification to grant better control and flexibility on what it means to those implementing and using it.
Provide suggestions regarding IKI and EKI.
Remember, these are suggestions and are not requirements.
Kevin Day [Thu, 21 Nov 2024 04:51:51 +0000 (22:51 -0600)]
Bugfix: IKI and EKI specifications, clarifying backslash escaping.
I made several improvements in my previous commit e1c95441efc531bb27117aa9aa6067436d5b17da.
I added documentation regarding the backslashes similar to how the FSS backslashes are handled.
For the IKI and EKI specifications, this is not the case.
Update the specification to clarify this situation.
Update the examples to better describe this.
Fix the examples to be accurate.
Kevin Day [Thu, 21 Nov 2024 04:50:18 +0000 (22:50 -0600)]
Cleanup: Unit test for f_iki.
Use the version 0.1.0 in the test string so that it no longer gets changed with release changes.
Add the missing undef for `_inline_macro_total_rows`.
Fix the constant status declaration scope.
Kevin Day [Tue, 19 Nov 2024 02:26:01 +0000 (20:26 -0600)]
Bugfix: The IKI and EKI specifications have mistakes and needs clarifications.
The wording of the specification is now greatly improved.
I noticed that some of the specification is actually incomplete.
I removed the incomplete sentence.
Make sure the FSS-000C specification is updated to add allowances for the EKI specification.
The example in both the IKI and the EKI specifications has several mistakes.
Use the grave character (U+0060) for the code to help avoid needing to escape the single quotes.
The IKI inside of a valid IKI Variable do not need to be escaped.
The example that directly states "is escaped to not be treated as IKI Variable" is incorrectly being documented as a valid IKI syntax (a significant mistake on my part).
Add explicit escaping examples to help make the escape behavior more explicit and clear.
I believe the current escaping logic in the IKI processing code is incorrect and will need to be updated and corrected following this commit.
I remembered this time to make the date based on UTC.
The version date in this commit is based on the current UTC Earth day.
Kevin Day [Sun, 17 Nov 2024 23:24:51 +0000 (17:24 -0600)]
Cleanup: The f_iki_read() using digits for wrapped rather than f_true and f_false.
This brings in changes introduced upstream commit 60d4e0c667860574659e9a7966ac0c591054f61a.
The `wrapped` is changed to use `0x0` and `0x1` rather than `F_false` and `F_true`.
These are functionally identical but the code is more consistent when having `0x0`, `0x1`, and `0x2` instead of `F_false`, `F_true`, and `0x2`.
Kevin Day [Fri, 8 Nov 2024 05:00:02 +0000 (23:00 -0600)]
Update: Add FSS Embedded List read depth tests and fix test script files.
Add tests for the FSS-0008 standard using depths.
I generated these test expectations.
I will have to slowly review these over time or fix them when I identify one is wrong.
I did do a test by building the same tests on both the 0.6 and 0.7 and found that they both pass.
I either have correct logic or I have consistently bad logic.
This adds a lot of FSS-0008 depth tests, however, the amount of tests needed is a lot more than what I have done here.
I think this large number is a good start.
Add special cases functions for the script tests.
I noticed that the failure code is incorrectly being set (the logic appears reversed).
The conditional argument used to determine which test being run in the test script is wrong.
Create a new variable called `test_set` and use that.
The `standard` variable can then be extracted from the `test_set`.
The test script incorrectly is missing the `-o` for the Object in some cases.
Kevin Day [Mon, 4 Nov 2024 03:19:29 +0000 (21:19 -0600)]
Feature: Add EKI specification for EKI standard, a slightly updated IKI standard.
Think of EKI as an extended IKI.
This introduces a single change (and its accompanying requirements).
Multiple IKI Vocabularies are now supported.
I have been using IKI in practice for a while now.
A need has risen up where I really want to be able to easily designate multiple IKI Vocabularies for a single IKI Variable.
This new format allows for me to do something more similar to HTML.
I have been using IKI as part of a templating process.
With EKI, unlike IKI, I can now make something both bold and italic.
Whereas with IKI, I would have to create a new type that combines them (like `bold_italic`).
I would then have to create two types if I wanted to be order agnostic, such as `bold_italic` and `italic_bold`.
This would get progressively worse if I added a third or more Vocabularies.
With EKI, I don't add any new IKI Variables and the order does not necessarily matter.
Example: `bold:italic:"My example"` (and `italic:bold:"My example"`).
This would have `bold` and `italic`.
The order should be preserved in case something does care about order.
The order can be ignored if something does not care about order.
Kevin Day [Sat, 5 Oct 2024 03:01:27 +0000 (22:01 -0500)]
Bugfix: Empty Objects are preserving leading white space when in original mode and update runtime tests.
This (unfortunately) adds more logic to processing the case when `--original` is used and the Object is empty.
This makes me wonder if I have to implement the opener range.
I was not happy about adding the `close` for memory and space reasons.
Adding this would introduce more inefficiencies.
I may end up having to add support for processing different types of structures if I want to support the simple Object and Content without the `open` and `close` positions as well as with those positions.
The focus in this commit is only on fixing the printing and does not introduce any of the behavior that I described here.
Update the runtime tests for all of the FSS Read programs.
Some of the tests were missing or were incorrect such that this bug slipped by without detection.
The FSS Payload Read tests now have the (missing) NULL characters added to them.
Kevin Day [Thu, 3 Oct 2024 03:23:18 +0000 (22:23 -0500)]
Update: Next micro version (0.6.13).
The following are the commands that I ran to make this change.
# find build/ level_? specifications/ documents/ licenses/ -type f -exec sed -i -e 's|0\.6\.12|0.6.13|g' '{}' ';'
# find build/ level_? specifications/ documents/ licenses/ -type f -exec sed -i -e 's|^version_micro 12|version_micro 13|g' '{}' ';'
# find level_3/ -name common.h -exec sed -i -e 's|_program_version_micro_s F_string_ascii_1_s F_string_ascii_2_s|_program_version_micro_s F_string_ascii_1_s F_string_ascii_3_s|g' '{}' ';'
# find level_3/ -name common.h -exec sed -i -e 's|_program_version_micro_s_length (F_string_ascii_1_s_length + F_string_ascii_2_s_length)|_program_version_micro_s_length (F_string_ascii_1_s_length + F_string_ascii_3_s_length)|g' '{}' ';'
Kevin Day [Tue, 1 Oct 2024 02:07:43 +0000 (21:07 -0500)]
Update: The bootstrap example script now handles building using fake.
The bootstrap example script is generally for using bootstrap scripts.
I have ended up using this for regular building because it is simple and already written.
The FLL 0.7 development now has build settings for some programs that do not build using the bootstrap without additional parameters.
Add support for the bootstrap script to use `fake` directly if it exists in the PATH for these particular programs.
This is added to the 0.6 stable branch for consistency purposes and essentially does nothing there.
There are no programs in the 0.6 stable branch that utilize this.
Kevin Day [Mon, 23 Sep 2024 01:53:35 +0000 (20:53 -0500)]
Feature: Fake settings specification updates and enable simple golang support.
The Featureless Make is intended to not be language specific since its inception.
I did not know how the languages were to work, so I mostly built this to guarantee C/C++ support.
I found that the `golang` supports some command line based compilation.
This adds support for `golang` in the most basic way.
This support is not well tested.
An `example_go` project is added.
I have only tested the basic build and I have not done anything advanced.
This relaxes some of the requirements.
The `bash` is changed to `shell` to encourage more types of scripts.
The `custom` is added and used as a fallback if `build_language_path` is not used.
I found that there needs to be some additional settings in the `settings` specification.
This updates the specification and adds:
- build_compiler_arguments
- build_compiler_arguments_shared
- build_compiler_arguments_static
- build_compiler_arguments_library
- build_compiler_arguments_library_shared
- build_compiler_arguments_library_static
- build_compiler_arguments_object
- build_compiler_arguments_object_shared
- build_compiler_arguments_object_static
- build_compiler_arguments_program
- build_compiler_arguments_program_shared
- build_compiler_arguments_program_static
- build_language_path
This also fixes some mistakes in the language of the specification.
Kevin Day [Tue, 17 Sep 2024 02:13:26 +0000 (21:13 -0500)]
Update: Refresh firewall iptables rules and relax some of the defaults.
Relax some of the defaults so that there will be less immediate failure.
Provide a lot of the previous `DROP` statements for easy re-assignment by just uncommenting.
Add some specific wrapping tools around IPv6 related rules.
Kevin Day [Sat, 7 Sep 2024 13:45:32 +0000 (08:45 -0500)]
Bugfix: FSS Embedded List Read printing tabbing when using original mode and trim mode.
Do not double-print the before spaces when both original and trim modes are in use.
Print the before even with original and trim are in use when the Object has no size.
Kevin Day [Wed, 4 Sep 2024 05:15:16 +0000 (00:15 -0500)]
Feature: Add support for building under Android.
This is not well tested or well supported.
This does have limited testing under the Termux project environment using CLang.
Add a new configuration option to build and run the project in an Android environment.
The test system had problems with text relocations but then required PIE.
This disables the relro in some spots (libraries) while enabling it in others (programs/executables).
Disable a lot of the functions that are not available or supported.
I have observed several warnings about `DT_FLAGS_1=0x41` being unsupported flag.
That is not a very helpful warning.
I have no idea what `0x41` is.
This causes a problem where the output of programs displays these warnings.
This can break programs that require I/O formatting and processing.
This has been tested to build using `bootstrap.sh` and then using `fake`.
Example build and install process:
# mkdir ~/software
# cd fll-0.6.X/
# ./bootstrap.sh build -m monolithic -m thread -m clang -m android && ./install.sh ~/software
# cd ../fake-0.6.X/
# ./bootstrap.sh build -m monolithic -m thread -m clang -m android -w ~/software && ./install.sh ~/software
# export LD_LIBRARY_PATH=~/software/libraries/shared
# export PATH=~/software/programs/shared
# cd ../byte_dump-0.6.X/
# fake -m monolithic -m thread -m clang -m android -w ~/software && ./install.sh ~/software
Example execution of `byte_dump` of `bash` program (this is a large dump).
# byte_dump -wt 7 $(type -p bash)
Kevin Day [Mon, 2 Sep 2024 23:32:42 +0000 (18:32 -0500)]
Update: Implement Object alignment and trim expanding in FSS Read programs and restructure the program flags.
The program flags for the standard-specific functionality vs the program arguments were previously stored in the same bit space.
This breaks the two parts into the main program flags and the "feature" flags.
Implement support for new functionality where the Object structure can be optionally preserved.
Implement support for new functionality where the tabbing of the Object can be preserved, even when trimmed.
This is a step towards a "pretty print" mode that is more or less an "original print" mode.
The tabbing structure is now well preserved for standards like FSS Extended List (FSS-0003).
The Object alignment can determined from the Object close.
Use the `closes` structure to identify how the Object end is tabbed.
Then use that tabbing for the Object.
When trimming is enabled, still apply that tabbing.
This is done only for standards where such behavior makes sense.
The original print is now better preserved.
Printing an Object can now print the before and after Object characters (usually white space).
The trimming and original print may be used together to produce the results where the is no white space before or after the Object but everything else from the original source is preserved.
This combination of trim and original essentially ensure that the old behavior can be still produced.
I would further note that the processing of the FSS Embedded List (FSS-0008) Objects is not recursively performed.
No trimming or anything like that is applied at this time.
Only the selected depth is recursed into such that the Object and Content are processed.
I made this decision to keep the design as simple as possible.
Should I get the motivation and the time, then I may at some point in the future decided to recursively process and format all of the embedded structures.
I happened to notice duplicate and unused flags.
These are now removed.
Update the runtime tests accordingly.
Add new runtime tests to test the updated behavior.
I used the 0.6 and 0.7 to test these and I compared the results.
There is a discrepancy in some of 0.6 and 0.7 tests.
I will have to follow this commit with additional test fixes as necessary pending further review.
Kevin Day [Sun, 1 Sep 2024 02:34:04 +0000 (21:34 -0500)]
Bugfix: FSS Embedded List Read is not applying delimits when printing in line mode.
Make sure delimits are properly applied.
The `--line` mode is printing the line character by character.
Add a loop to check if the given character is to be printed or not based on the delimits array.
Kevin Day [Sat, 31 Aug 2024 23:47:07 +0000 (18:47 -0500)]
Update: The FSS Lists (FSS-0002, FSS-0003, FSS-0008, etc..) to match recent specification changes and reduce redundancy.
Update the Object read functions to be in compliance with the latest specification changes.
Review and update the code, fixing any observed problems and perform some clean ups.
I noticed that several of the list functions for Object read are almost identical.
Move this logic into into a shared private function to reduce repeated code.
The runtime tests are also updated.
I still need to review the Content read functions.
I probably may be able to also find similar, if not identical, code for the list write functions as well.
I will review the code and create a private function similar to what is done here if deemed possible and reasonable.
Kevin Day [Fri, 30 Aug 2024 02:18:10 +0000 (21:18 -0500)]
Update: The FSS-0002 and FSS-0003 standards, modifying the space after Object rules.
My previous changes did not alter the behavior of the standard.
I spent some time considering this and decied that I should make this change.
The new behavior is that white spaces after the last printable character (aka "graph" character) in a valid Object is no longer considered part of the Object.
I decided to do this because supporting the space after the Object but not the space before the Object is awkward and also makes Object name matching more difficult.
One of the pillars of this project is "human first".
Doing this change makes it easier for a human to use by relaxing the exactness of a match when it comes to white spaces before or after a valid Object.
The specifications specifically include optionally supporting untrimmed Objects that include the white space before or after the Object to help accommodate the previous behavior.
I do not want to add quote support in the Object names here to keep it simple.
This new behavior seems to be a good compromise.
Kevin Day [Thu, 29 Aug 2024 03:27:03 +0000 (22:27 -0500)]
Update: Clarify the FSS-0002 and FSS-0003 standards regarding the white space before and after a valid Object.
I updated my 0.7 code and back ported some of the generated runtime tests.
I discovered problems and realized that the standard could be more clear according to the spaces before and after the Object.
This does not change the FSS-0002 and FSS-0003 standards in any functional way.
The simply clarifies the standard regarding the spaces to make it more clear and reduce the chances for a mistake.
Kevin Day [Tue, 27 Aug 2024 04:30:19 +0000 (23:30 -0500)]
Cleanup: FSS Extended List function variable names and code structure.
Make the variable name more explicit.
This variable name also matches recent 0.7 branch changes.
Move the allocation for the comments range after setting all of the variables.
This should better allow for determining the state should the `f_string_ranges_increase()` fail.
Kevin Day [Sun, 25 Aug 2024 16:36:39 +0000 (11:36 -0500)]
Bugfix: FSS Embedded Read is printing a single line with --line parameter with --object.
The entire Content is being displayed when `--object` is used with `--line`.
The logic for printing the `--line` is entirely missing.
This adds the necessary logic to process the Object (and Content) when the `--line` is used.
Kevin Day [Sun, 25 Aug 2024 04:30:44 +0000 (23:30 -0500)]
Bugfix: Invalid comment handling in FSS Embedded Read.
The comments are not being properly processed because graph_first is not and should be being set to 0x1 at start.
As a result, the first comment is not being treated as a comment.
Make sure to reset the `graph_first` and `line_start` after processing the comment.
The comment starts at the line and so use `line_start` rather than `newline_last + 1`.
Remove dead code where `graph_first == 0x2` is being checked inside a block that requires `graph_first` to be `0x1`.
Update code comments about `graph_first` in several of the fl_fss read sources.
Add additional check for buffer string range when processing comments.
Update the runtime tests as appropriate.
I also noticed that several of these runtime tests use `--line` but are showing multiple lines.
I will address this bug in a separate commit.
Kevin Day [Sun, 25 Aug 2024 03:24:56 +0000 (22:24 -0500)]
Bugfix: FSS Embedded Read has incomplete and incorrect code.
I never noticed that I had not implemented the `--columns` support for FSS Embedded Read.
The `--columns` is now properly supported and has runtime tests.
The `--select` is not always printing a number.
Make sure a number, even if it is `0`, is always printed.
The `--select` now also follows the more strict behavior as implemented in the 0.7 branch
Update the `--total` to match the more strict behavior as implemented in the 0.7 branch.
Kevin Day [Sat, 24 Aug 2024 03:09:39 +0000 (22:09 -0500)]
Update: Always count any --select when using --object for FSS Read programs.
Working on the 0.7 FSS Read programs resulted in me noticing an oversight regarding the `--select` with the `--object`.
The `--object` will always result in showing things even if the `--select` number is infinitely large.
This is a back port of those changes.
I decided to also copy over a lot of the FSS Read code for this from 0.7.
Doing this should help make maintaining both the 0.6 nd 0.7 easier.
I still have work for the FSS Embedded Read program in the 0.7 branch and so I decided to withhold any 0.6 changes for FSS Embedded Read until I finish my changes in 0.7.
Kevin Day [Sun, 18 Aug 2024 01:10:08 +0000 (20:10 -0500)]
Update: Improvements around FSS Read functions regarding correctness.
Update the `verify.sh` script (and associated `testfile`) to print the test name.
Update the `generate.sh` and `verify.sh` scripts to safely pass arguments with spaces using `"$@"`.
Update the FSS Embedded Read to include the backported fixes such as those from commit 104ed789a759a344146431ded54cde9fefb063ec.
- The use of `f_memory_array_increase()` is incorrect in several cases.
- Switch to `f_memory_array_resize()`.
- Add 2 when resizing to account for the depth position but also an additional element as a minor memory allocation optimization.
Update the FSS Basic and FSS Extended Read functions.
- Avoid printing end of Object space when there is no Content.
- For FSS Extended Read, move the print quote into dedicated function.
- Add print Content end empty print function to match the 0.7 code (simplify comparisons between the different versions).
- Utilize the flag practices for `print_state` to be similar to the 0.7 code.
Update the FSS Embedded Read program:
- Security issue regarding incorrect memset value (passed `sizeof(skip)` when instead `sizeof(bool)` should be used).
- The `--line` parameter is not being correctly validated and throws an error when a value is specified.
- Fix bug where empty Content is being improperly handled and '}' is being returned as the Content for empty Content.
- Rebuild the tests to match the 0.6 generation because the 0.7 is invalid (once the 0.7 is fixed, then I will do more extensive comparisons and appropriate fixes).
Kevin Day [Tue, 13 Aug 2024 17:29:03 +0000 (12:29 -0500)]
Bugfix: FSS Extended Read needs to only print new line if Object or Content is printed.
Do not always print the new line for the case when printing Object or Content.
When the Content is being printed and is empty Content, then do not print the new line.
Kevin Day [Tue, 13 Aug 2024 02:18:56 +0000 (21:18 -0500)]
Update: Next micro version (0.6.12).
The following are the commands that I ran to make this change.
# find build/ level_? specifications/ documents/ licenses/ -type f -exec sed -i -e 's|0\.6\.11|0.6.12|g' '{}' ';'
# find build/ level_? specifications/ documents/ licenses/ -type f -exec sed -i -e 's|^version_micro 11|version_micro 12|g' '{}' ';'
# find level_3/ -name common.h -exec sed -i -e 's|_program_version_micro_s F_string_ascii_1_s F_string_ascii_1_s|_program_version_micro_s F_string_ascii_1_s F_string_ascii_2_s|g' '{}' ';'
# find level_3/ -name common.h -exec sed -i -e 's|_program_version_micro_s_length (F_string_ascii_1_s_length + F_string_ascii_1_s_length)|_program_version_micro_s_length (F_string_ascii_1_s_length + F_string_ascii_2_s_length)|g' '{}' ';'
Kevin Day [Sun, 11 Aug 2024 21:42:39 +0000 (16:42 -0500)]
Bugfix: Incorrect runtime tests for FSS Read programs.
I mass updated the FSS Read tests in the 0.7 branch.
I chose the quick route of just using the program to generate the tests.
This requires that I trust the results to be correct.
I figured I would eventually go through each one and make sure that they are correct at a later time.
It is a development branch, after all.
However, back porting these tests to the stable 0.6 branch revealed some bugs.
I reviewed the failing tests and files and I confirmed that the 0.6 branch is getting correct results and that the tests are incorrect.
The runtime tests in the 0.7 development branches will have to be updated and the bugs there will have to be fixed.
Kevin Day [Sun, 11 Aug 2024 21:09:16 +0000 (16:09 -0500)]
Bugfix: Properly back port runtime tests for FSS Read programs.
I failed to notice that the program is the `fss_read` program which does not exist in the 0.6 project.
I ended up testing the `fss_read` program installed on the system.
The tests all appeared to pass because of this when in reality the tests were never being run.
For the `fss_embedded_read` test, just disable that one for now.
There are known problems with the 0.6 branch for that.
I will have to address the problems with `fss_embedded_read` in another release.
The while loop is being checked for the comments length.
The if condition immediately outside that is not being checked.
This check is being performed in the original commit that I backported from.
This is simply a backporting oversight.
Kevin Day [Sat, 10 Aug 2024 03:15:06 +0000 (22:15 -0500)]
Regression: Properly set the start range.
The optimization commit 2c0f62c1226b55b9fb546963e4751bd9c82a245b neglected to copy the `found->array[found->used].start` assignment.
It also failed to copy the error status check.
Kevin Day [Tue, 6 Aug 2024 04:17:59 +0000 (23:17 -0500)]
Security: Missing range checks on comment processing.
This is a back port of the fixes for problem discovered in the 0.7 branch.
The 0.7 program fss_payload_read exposed this issue.
This issue affects multiple programs in the 0.6 branch.
The fss_payload_read such as the runtime test is wrong:
# fss_payload_read -ocn payload level_3/fss_read/tests/runtime/fss_000e/source/test-0002-mixed.fss -t
The output is 1 but should instead be 4.
# fss_payload_read -ocn payload level_3/fss_read/tests/runtime/fss_000e/source/test-0002-mixed.fss | wc -l
Investigating this problem revealed that the comment handling code is failing to perform a range check.
The overflow is causing the stop range to point to some random memory address which is almost always larger than the file.
This results in the count being wrong.
This bug is a security concern.
Add the range check in all places where this range check is missing for the comments.
This is the back port of the improvement from the 0.7 development branch.
Adding the macro neither breaks API nor ABI.
Historically the step was always 3.
I found, over time, that increasing the step greatly to something like 128 could greatly reduce memory consumption and performance in many cases.
In the situation where a large number of small objects are allocated then this number like 128 becomes highly abusive.
The simple low allocation step will only allocate a single unit on the very first allocation.
If the next allocation is on an array that has a size greater than one and less than four (via the tiny define), then the step size is set to four during allocation.
If the next allocation is on an array that has a size greater than four and less than eight (via the small define), then the step size is set to eight during allocation.
If the next allocation is on an array that has a size greater than eight and less than sixty-four (via the large define), then the step size is set to sixty-four during allocation.
In all cases, if the request step is less than the calculated step, then the requested step is used.
For example, if the requested step is twelve, then after eight is allocation, then the next generated step size is twelve rather than sixty-four.
Using some test files, shows the following reduction:
- Old: ~8GB of RAM -> New: ~200MB of RAM.
- Old: ~500MB of RAM -> New: ~20MB of RAM.
Kevin Day [Sun, 4 Aug 2024 00:52:46 +0000 (19:52 -0500)]
Update: Optimize away the isdigit(), isalpha(), isalnum(), and isxdigit().
I did some research and learned that the "is*()" functions can greatly affect performance due to locale and other manners.
I originally used these to allow for well established optimization to take place.
Replace these with some mathematical operations that should increase performance.
This also means no function call on the stack.
This project is already function stack heavy by design and so reducing functions when easy is a great thing.
Start using literal characters rather than the standard strings for the UTF related functions.
In these cases the ASCII expectation is guaranteed.
The ability to override these is also not practical as the meaning should not change.
I have not looked at all of the "is*()" functions and I may address any remaining ones at a later time.
I potentially may also investigate mapping tables to further improve performance.
These math calculations can be used in a lot of the non-ASCII UTF ranges as well.
I opted to not do these just yet given that such work will take a large amount of time.
I have not done any performance analysis yet but I plan to do so.
Kevin Day [Thu, 1 Aug 2024 02:24:23 +0000 (21:24 -0500)]
Bugfix: Add stage setting to standards to prevent build state file conflicts.
The build state stage files are conflicting in some cases.
The old solution to this problem has been observed as insufficient.
The different build settings might have the same exact file name.
I have tossed around the idea of a settings Object such as "stage" in the past but I had previously opted against it.
I now believe that skipping over this was a mistake.
Add a new feature to the standards to fix this bug.
The "stage" value may now be specified.
The fakefile files accept the "stage" setting and passes the result along to any build settings.
The settings files accept the "stage" setting and uses the value by appending it to the stage files.
Only a single value is supported.
The forward and backward slashes are explicitly prohibited.
Other special characters are recommended to be avoided given the possibility of local file system problems.
Rather than erroring out, this slashes are stripped out.
The bootstrap.sh script is updated to support this.
The support for "stage" in the boostrap.sh script is very limited.
Kevin Day [Sat, 13 Jul 2024 04:57:20 +0000 (23:57 -0500)]
Update: Add additional protection around assignment of global.thread.enabled during Controller exit process.
Spend some time trying to ensure that the mutex lock can be achieved and then change the enabled state to disabled.
Add a maximum retry as a fail safe but otherwise keep retrying unless certain error conditions are encountered.
Kevin Day [Sat, 13 Jul 2024 04:28:38 +0000 (23:28 -0500)]
Update: Use copy of flag to avoid possible race conditions with controller_thread_is_enabled().
The thread.enabled is checked multiple times in controller_thread_is_enabled() but this value may have changed.
Make a constant copy and reference that copy so that any changes after the first read will not affect the function results.
Kevin Day [Sat, 13 Jul 2024 03:08:12 +0000 (22:08 -0500)]
Bugfix: Controller parameter type is f_number_unsigned_t and not uint8_t.
The array of codes is an array of f_number_unsigned_t and not uint8_t.
I was probably thinking about the array only needs to be of type uint8_t long, which would be represented by the counter.
I probably then accidentally set the value type to be uint8_t to match that.
Kevin Day [Fri, 12 Jul 2024 03:33:15 +0000 (22:33 -0500)]
Bugfix: Incorrect thread unlock in controller_rule_process().
When the lock is grabbed but a failure occurs an if condition handles that case.
After that block is a second block that then attempts to unlock that same lock.
Given that the lock failed, this lock cannot be in a locked state.
Kevin Day [Thu, 11 Jul 2024 05:21:10 +0000 (00:21 -0500)]
Cleanup: Execute process child id structure.
Make the variable "child" more verbose in its name to make it clear that it is coming from the process.
This makes it more clear why there is a write lock obtained to change this.
Kevin Day [Wed, 10 Jul 2024 02:30:32 +0000 (21:30 -0500)]
Update: Provide fallback should the locks on exit fail to catch and update function using pointer.
The previous commit 49a3d41d1c17734b0a8299b356f1e299386f4a42 removed the forced fallback to avoid potential race conditions.
The reason for having that fallback is to ensure that the exit process is guaranteed and not potentially blocked by a lock.
Provide an alternative that makes several attempts to properly lock on exit.
Only when those extra attempts fails should the failsafe/fallback be used.
Update controller_thread_process_cancel() to have the global data as a pointer.
Kevin Day [Mon, 8 Jul 2024 03:00:09 +0000 (22:00 -0500)]
Bugfix: Controller error printing function has wrong locking.
The locking being used in controller_rule_item_print_error() is based on the special case for when fll_error_print() is used.
The fll_error_print() is not being used so do not use the special case locking.
Kevin Day [Mon, 8 Jul 2024 00:12:48 +0000 (19:12 -0500)]
Security: Remove simulate feature that actually executes a script.
The simulate is setup to actually execute scripts via a fake script execution.
This is not intended to do anything other than to catch problems in the script setup.
I have come to the realization that a malicious actor could setup a custom scripting engine to be executed on simulation.
An oblivious user could then be performing a simulation with the expectation that nothing actually happens while the malicious scripting engine performs some nefarious activity.
Strip out the fake script execution to prevent this behavior.
This is a loss of some functionality but I prefer the safety over this potential bad behavior.
This is not actually needed because a 32-bit long should not overflow with just 1000000000.
The logic I replaced it with was sloppy anyway and left out some cases.
The long representing nanoseconds can store greater than 999999999.
Make sure that is handled before adding to the tv_nsec.
Kevin Day [Sun, 7 Jul 2024 04:35:59 +0000 (23:35 -0500)]
Update: Rewrite timing logic in controller.
Check if both numbers added together are greater than or equal to half a second.
If they are, then an overflow will happen.
Subtract the half seconds instead of handling an overflow.
This does not handle the case where the milliseconds or nanoseconds are greater than one second.
This only handles the cases where they are greater than half a second.
Kevin Day [Sun, 7 Jul 2024 04:33:33 +0000 (23:33 -0500)]
Bugfix: Replace f_signal_mask() with f_thread_signal_mask().
The f_thread_signal_mask() is being used to setup the signals.
If that fails, the unmask process is incorrectly using f_signal_mask() when instead f_thread_signal_mask() should be used.
Kevin Day [Fri, 5 Jul 2024 00:28:00 +0000 (19:28 -0500)]
Bugfix: New line is not being printed because sequence is missing.
The new line parameter is present but the replacement sequence is missing.
This results in the new line not printing.
Add the missing '%r' replacement sequence.
Remove the new line parameter in cases where the new line should not be printed.
Kevin Day [Fri, 14 Jun 2024 02:53:39 +0000 (21:53 -0500)]
Security: Console parameter single short values array is too small.
The short parameters "needs" variable now increases the array size before assignment.
The following command line calls are used to expose the problem and its resolution:
# fss_basic_list_read specifications/fss.txt +Q -cn "Featureless Settings Specifications" | iki_read +Q -w -rrrrrrrr anti-KISS 'anti-<abbr title="Keep It Simple Stupid">KISS</abbr>' ASCII '<abbr title="American Standard Code for Information Interchange">ASCII</abbr>' BOM '<abbr title="Byte Order Mark">BOM</abbr>' FSS '<abbr title="Featureless Settings Specifications">FSS</abbr>' KISS '<abbr title="Keep It Simple Stupid">KISS</abbr>' UTF-8 '<abbr title="Unicode Transformation Format 8-bit">UTF-8</abbr>' URL '<abbr title="Byte Order Mark">URL</abbr>' XML '<abbr title="Extensible Markup Language">XML</abbr>' -WWW character '<code class="code">' "</code>" code '<code class="code">' '</code>' italic '<em class="em">' '</em>'
Kevin Day [Tue, 11 Jun 2024 23:22:10 +0000 (18:22 -0500)]
Bugfix: Enable missing support for grave (backtick) is IKI.
The FSS and IKI standards were previously updated to support grave (backtick).
The FSS read and write functions were updated but I apparently didn't do the IKI.