Kevin Day [Fri, 3 Jul 2020 03:58:27 +0000 (22:58 -0500)]
Bugfix: FSS-0000 and FSS-0001 are handling not explicitly follow standard for quote delimit requirements.
In the case where a quote is provided that could potentially be a valid quote, that quote should only be considered a quote (including for delimited purposes) if a valid whitespace, end of string, or end of line is reached.
The code needs to check after the closing quote to confirm that it is actually a valid closing quote before acting on it.
Update the FSS specification to make this behavior more explicitly clear.
Kevin Day [Thu, 2 Jul 2020 03:36:33 +0000 (22:36 -0500)]
Progress: UTF-8
More UTF-8 improvements.
The Punctuation is complete as far as the "Other" punctuation category is concerned.
(Apparently, there are more groups/sets to do!)
Create a new whitespace functions (is_whitespace_other()) to handle non-whitespace whitespace.
So far, this is only a single character: Ogham space mark ( ).
This Ogham space mark is not a whitespace, it is a graph character that is considered a whitespace.
Add some @todos.
Remove invalid whitespace character().
Add is_digit() vs is_numeric().
The traditional alnum() is referring to alphabet and digit.
With UTF-8 there are non-digit numbers, which can be classified as "numeric".
Therefore, the is_digit() functions are added and intended to be used for the traditional behavior.
The is_numeric() are intended to handle any numeric character.
The is_word() tests are for digits and not for numeric.
Kevin Day [Mon, 29 Jun 2020 04:25:14 +0000 (23:25 -0500)]
Bugfix: IKI Read should not always print content on substitution in whole mode.
When printing in whole mode, it is only printing the content, even when --literal is passed.
What should happen is that when --literal is passed, the entire variable should be printed, even when substitution is used.
The substitution will still occur.
Kevin Day [Mon, 29 Jun 2020 00:09:25 +0000 (19:09 -0500)]
Update: finish implementing IKI Read program.
The resulting design made --expand redundant, so it is removed.
Substitution is implemented and works independent of the --name parameter.
Added a new macro to the FLL project called f_macro_memory_structure_macro_increment().
Too often do I have to create those structures, so here is a macro to perform the increment.
I started to update the rest of the project and then decided not to spend time on that right now.
This, therefore, primarily focuses on getting the macro working in IKI.
IKI Read is now considered usable, except for the UTF-8 support.
The UTF-8 support is dependent on updating a ton of UTF-8 codes.
This will be done at a later time.
The IKI Read program will eventually need to be reviewed for mistakes and cleanups.
Kevin Day [Sat, 27 Jun 2020 23:17:45 +0000 (18:17 -0500)]
Bugfix: when --line is used, lengths passed buffer size aren't handled.
data->line is not a position, do not use it as a position (if (data->line < data->buffer.used) { is doing this).
If the line is exceeds the buffer, return f_data_not and properly handle that return case.
For --total, this means print 0.
For just about everything else, print nothing.
Kevin Day [Sat, 27 Jun 2020 23:03:46 +0000 (18:03 -0500)]
Bugfix: delimit placeholder not being applied in all cases
The final exist should apply any delimits.
When determining the remaining width, make sure that the range or buffer has not yet been exceeded.
Not doing this resulted in an exit with an F_parameter error, further resulting in no delimits being applied.
Kevin Day [Sat, 27 Jun 2020 04:33:04 +0000 (23:33 -0500)]
Progress: IKI Read.
Implement the basics of raw mode.
Raw mode should instead work in conjunction with the content, literal, and object modes so redesign accordingly.
Add some additional documentation to the help section.
Expand and Substitute are mostly ignored thus far and will be implemented at a later time.
The substitution will be copied from the argv as-is, so plan on using static strings instead of dynamic strings.
There will be function argument cleaning to do once I determine how Expand and Substitute will be implemented.
Until then, the arguments are fine as-is.
Kevin Day [Fri, 26 Jun 2020 02:54:12 +0000 (21:54 -0500)]
Bugfix: read overflows the per character scanning
There is a logic mistake where the buffer is incremented too many times before the next loop.
This results in skipping some of the characters in the file and if those skipped characters are part of a vocabulary and content, then it will not be detected.
Kevin Day [Fri, 26 Jun 2020 02:51:59 +0000 (21:51 -0500)]
Cleanup: reserve IKI-0000 and update documentation
The standard IKI-0000 is now to be used to represent anything that follows the IKI standard without explicitly descrribing what its vocabulary is.
All other IKI standards (IKI-0001 and greater) are to be used to represent a specific set of vocabulary terms.
Kevin Day [Thu, 25 Jun 2020 04:00:41 +0000 (23:00 -0500)]
Update: dependencies changes, project level changes, and includes changes.
Review the current dependencies and fix them.
There are some cases where some dependencies are not necessary, such as f_utf.
There are some cases where some dependencies are missing (or newly added).
f_conversion split into f_conversion and fl_conversion.
Update dependencies accordingly.
For simplicity, include the core level_0 project headers, except in a small number of cases.
Add fss header comment in dependencies files.
Add comments in *_array.h headers for level_0 core projects.
Add missing header in f_string.
Replace _di_level_0_parameter_checking_ with _di_level_1_parameter_checking_ where incorrect.
The fl_fss identify functions should instead be in the fll_fss project.
Make sure the IKI projects are in the bootstrap example script.
Remove the fl_console_parameter_to_number_unsigned() and similar functions.
These are just wrappers to another call and is therefore a waste.
Instead, just call the fl_conversion_string_to_number_unsigned() and similar functions.
Kevin Day [Wed, 24 Jun 2020 05:14:04 +0000 (00:14 -0500)]
Bugfix: conversion needs to properly handle '0x' and similar
Invalid Parameter happens when there is no following number after the x in '0x'.
This is because the start and stop range were not being checked after applying the offset.
Kevin Day [Tue, 23 Jun 2020 03:54:54 +0000 (22:54 -0500)]
Progress: Featureless Make.
Begin working on integrating IKI into Fake.
I took advantage of the current code to do some testing with IKI and left some of the testing code for now.
In addition to IKI, I am thinking that I need to add another project for exploding a string into separate arguments with support for escapable single and double quotes.
Kevin Day [Mon, 22 Jun 2020 03:01:52 +0000 (22:01 -0500)]
Feature: add fl_iki project.
This provides fl_iki_read, a function for identifying all iki within a given range.
Whereas the f_iki_read function will only identify the first valid iki variable found within the given range.
Kevin Day [Mon, 22 Jun 2020 02:58:21 +0000 (21:58 -0500)]
Update: add missing dependencies, changes to iki
In a previous commit I seem to have forgotten to include the build settings changes.
Fix some minor mistakes in iki.
In particular, I forgot to increment the range->start after identifying where content would begin (which prevented it from working at all).
Add f_iki_variable to store the range for the entire variable.
This will allow for easy removal when substituting (which is one of the design intentions, so I need to cover this use case!).
Kevin Day [Sun, 21 Jun 2020 23:43:01 +0000 (18:43 -0500)]
Update: UTF-8 improvements
Move some FSS UTF-8 processing code into UTF-8 as it should be more generic.
Add more UTF-8 functions (or at least the stubs).
Begin adding emoji, which is all over the place.
Staring at so many codes, it will be easy to make a mistake.
After all of the UTF-8 functions are believed to be fully implemented an extensive review needs to happen.
Kevin Day [Sun, 21 Jun 2020 01:03:42 +0000 (20:03 -0500)]
Progress: organize headers, simplify fss structures, initial IKI support
Move more headers into the secondary structures to simplify the main header, which will now more often be just for function definitions.
The FSS structures can use a more common base to better allow integration with separate projects that utilize the same basic design.
This is important for IKI project.
Casting to/from fss objects and contents to their respective ranges should now be possible.
Begin adding the IKI support, which is a new FSS standard focused on a simpler/smaller Wiki design.
The goal is not to handle presentation but instead to provide very simple context on certain text blocks based on some vocabulary.
A very common example would be sending a plain text e-mail with a URL in it, the URL could be encapsulated in an IKI vocabulary object to tell both the human and the computer that this is a URL.
For a URL, this is very useful in that no part of the URL need be escaped, except for the closing quote and its respective delimiter.
Example iki URL: url:"http://www.example.com/this does not exist/".
As opposed to http://www.example.com/this%20does%20not%20exist/.
The first case is easier for a human while still remaining simple enough for a machine.
The IKI syntax, at least in its most basic form, is not meant to be a full WIKI formatter.
While this may be accomplished with a sufficiently large enough vocabulary, this seems unnecessary at this time.
The IKI syntax standard itself is yet to be ironed out and ill be updated overtime until I am satisfied.
Kevin Day [Sat, 20 Jun 2020 19:08:03 +0000 (14:08 -0500)]
Bugfix: revert slash improvements in FSS object read
Oops.
When relocating this I completely forgot that an object starting with something other than a double quote or single quote can never be a quoted object.
Therefore only the first slash needs to be delimited.
I need to update the FSS documentation to avoid this oops again.
This will be tacked on to the todo list for when I do my extensive review of the FSS processing code.
Kevin Day [Sat, 20 Jun 2020 16:46:24 +0000 (11:46 -0500)]
Progress: update FSS code, including zero-width updates
Update FSS to consider the recent zero-width improvements.
Making the changes reminded me on how I still need to go through and clean the FSS code up.
As noted in previous commits, the changes into supporting UTF really put the code into an unideal and not well tested state.
This needs extensive review and fixes.
To that end, I felt compelled to fix problems and perform cleanups that I observed.
At least to a small extent.
The basic and extended object reads are identical so move the code into a shared private function.
One behavioral change is to better support fail-through practice.
For FSS Extended, the contents are always terminated by a newline.
However, if there is an unterminated quote, then the entire process would previously fail.
This time, do no accept content for that line and continue on to next content.
Kevin Day [Fri, 19 Jun 2020 20:37:16 +0000 (15:37 -0500)]
Update: add additional f_fss functions and update documentation
Add line counting functions.
Remove unnecessary checks against negative values for unsigned numbers.
(They are theoretically necessary in that if the type was overwritten to a signed type, but for now just ignore that case.)
Finish adding or updating the *_together functions.
Review and update the documentation.
Kevin Day [Thu, 18 Jun 2020 05:27:01 +0000 (00:27 -0500)]
Progress: featureless make
Continue development in featureless make, implementing support for "make" operation.
This expands the f_string project to support additional structures.
This expands the fll_fss project to utilize these additional structures.
I have reviewed and redesigned some of the fll_fss project functions.
The fll_fss project function redesign is incomplete and not fully tested as of this commit.
(I plan on reviewing and updating the *_together functions.)
The current fakefile is a stub/example for testing and will be updated to a correct default fakefile once the "make" operation is complete.
The plan is to have the fakefile provide an example for building th featureless make project using the fakefile.
This would mean that the featuereless make project can be built by the featureless make project using either the "build" or the "make" operation.
Kevin Day [Mon, 15 Jun 2020 04:58:02 +0000 (23:58 -0500)]
Feature: add fss_named and fss_set structures and re-organize the fss headers
Provide fss_named and fss_set structures (and their respective array structures).
These are useful when using FSS types nested within another set (not to be confused with fss_nest which is for a single type with recursion).
There are just too many things in fss.h.
Break it out into multiple headers that are auto-included.
Kevin Day [Mon, 15 Jun 2020 04:54:58 +0000 (23:54 -0500)]
Feature: add macro initializers for static string and string ranges
Creating defines with present lengths and them using these in static strings, sometimes with ranges, has become more common.
Provide the macros to make this simpler and ever be more usable for creating constant static strings and constant string ranges.
Kevin Day [Fri, 12 Jun 2020 02:49:46 +0000 (21:49 -0500)]
Progress: UTF-8 file write should handle case where expanded character is larger than write max
When the 4-byte wide UTF-8 character is expanded into potentally 4 1-byte wide UTF-8 characters, the available bytes being expanded need to fit in the output buffer.
Kevin Day [Mon, 8 Jun 2020 23:29:15 +0000 (18:29 -0500)]
Update: fl_serialized to f_serialize, renaming f_serialized to f_serialize
There is nothing forcing the current functions in fl_serialized from being in f_serialize.
Renamed f_serialized to f_serialize to be more consistent with the function names.
To that end, I have renamed the functions like f_unserialize* to f_serialize_un.
I have been unable to come up with a better name that is also reasonably short.
Kevin Day [Mon, 8 Jun 2020 04:34:49 +0000 (23:34 -0500)]
Update: documentation cleanups, fix observed problems, retun F_data_not without error bit
Organize the return codes.
Fix any probles in the commend documentation that I happened to notice.
This is not complete, there will be follow up commits to address more cleanups.
I noticed that there were some public directory functions incorrectly have 'private_' in their name.
The conversion functions should no longer consider no data an error.
Just report that there was nothing to do.
Kevin Day [Mon, 8 Jun 2020 02:16:51 +0000 (21:16 -0500)]
Update: documentation, also use fss-0002
Fix the documentation to clarify that the --work paramter is, in fact, more different than --prefix parameter.
Update the tree examples to show both the build directory tree and work directory tree.
Switch to using the fss-0002 format for all of the documentation files and specification files.
Kevin Day [Sun, 7 Jun 2020 21:40:24 +0000 (16:40 -0500)]
Update: implement --process parameter
Looks like I forgot to implement this.
The --process parameter is a process name that is prepended to stage file names for the purpose of granting the caller additional flexibility.
Kevin Day [Sun, 7 Jun 2020 19:05:29 +0000 (14:05 -0500)]
Update: bootstrap dependency functionality, fix missing dependencies, also ensure level and monolithic
It seems I wrote a dependency generator to make my life easier and then I went and forgot about it.
Update this to work with the more recent changes to the build settings structure.
While I am at it, generate all of the dependencies and save the generated dependencies.
This exposed a typoe in the dependencies file, now fixed.
Might as well make sure the level and monolithic dependencies are generated as well.
Kevin Day [Sun, 7 Jun 2020 14:07:06 +0000 (09:07 -0500)]
Update: support copying documents, licenses, and specifications per project and do some cleanups
Individual projects, specifically programs (level_3), may have custom documentation, licenses, and specifications.
(With custom licenses being least likely because this project in its entirety is under lgpl.)
Add support to all projects just for good practice.
Cleanup ordering of function names and add spaces before function instantiation open brace.
Kevin Day [Sun, 7 Jun 2020 03:30:17 +0000 (22:30 -0500)]
Update: Featureless Make needs to handle the -d/--define parameter
Rename the parameter from defines to define, because it acts on only a single parameter at a time.
The -d must be specified multiple times to get multiple defines.
When -d/--define is specified, the defines provided by the settings file get ignored.
Continue to pass the defines from the console parameters to the process scripts.
Kevin Day [Sun, 7 Jun 2020 02:32:27 +0000 (21:32 -0500)]
Bugfix: shared library link names are incorrect
The standard link names are:
libfake.so -> libfake.so.0
libfake.so.0 -> libfake.so.0.5.0
Instead, I had:
libfake.so.0 -> libfake.so.0.5
libfake.so.0.5 -> libfake.so.0.5.0
While I am at it, make these links reflect the version_target setting.
If target is major, then standard link names above are followed.
If target is minor, then:
libfake.so -> libfake.so.0.5
libfake.so.0.5 -> libfake.so.0.5.0
If target is micro, then:
libfake.so -> libfake.so.0.5.0
Kevin Day [Sun, 7 Jun 2020 01:21:26 +0000 (20:21 -0500)]
Feature: add version_target setting to Featureless Make and FLL generate script
I noticed that I had a different linker version string between Featureless Make and the FLL generate script.
After reviewing and fixing, I decided to make this a customizable option now called version_target.
The default, traditional behavior, is to use only the major version.
A major version target would link like "-Wl,-soname,libfake.so.0".
A minor version target would link like "-Wl,-soname,libfake.so.0.5".
A micro version target would link like "-Wl,-soname,libfake.so.0.5.0".
Kevin Day [Sun, 7 Jun 2020 00:41:06 +0000 (19:41 -0500)]
Update: change settings structure to follow the Featureless Make structure
The existing design is based on the bootstrap scripts.
The bootstrap script is now updated to the new design so these scripts can now be updated to the new design.
Kevin Day [Sun, 7 Jun 2020 00:38:06 +0000 (19:38 -0500)]
Update: redesign to follow the Featureless Make design
Use the setting names specified by Featureless Make, which are a bit more generalized than the current generate.sh design.
This does not fully follow all of the Featureless Make functions.
Only what is necessary (or already implemented) to bootstrap the entire FLL project is done.