From: Kevin Day Date: Sun, 25 Oct 2026 02:54:39 +0000 (-0500) Subject: Feature: Clarify and handle Unicode combining and joiner characters, focusing on... X-Git-Url: https://git.kevux.org/?a=commitdiff_plain;h=970e039a5ef03aa50f8379ddba13a1694eed306a;p=fll Feature: Clarify and handle Unicode combining and joiner characters, focusing on FSS Read. One of the types of breaking changes and features that I allow are in regards to Unicode and following its standards. I was never really sure how I was going to handle special cases, such as combining and joiner characters. That has now changed and as of this change I now have declared the behavior. This, unfortunately, resulted in a lot of changes and behavioral changes. I have decided that making some breaking changes is the route to go. One of my goals with this massive change set is to try to reduce how often I read the combining and joining characters. There is going to be some performance hit but hopefully I have kept it to a minimal. The crux of the problem is that the combining and joiner characters are after the character being changed rather than before. That means that for every character being processed the code has to then go past it and check for a combining character. I've noticed that most software does not handle combining characters and joiner characters following an end of line character. I was going to treat this just like any other case but given the consistent way in which this is ignored I have decided to ignore combining and joiner characters specifically for end of lines `\n` (U+000A). The specifications have been updated. I do note that I need to clean them up at some future date. I have decided to only focus on reading (**FSS Read**) given the size of this change. The write code will be updated following this. This change set is just too large as it is. This change set did have some huge benefits such as exposing and helping me solve existing bugs and logic flaws. Several of those changes have been extracted and committed separately, such as the "Bugfix: Fix invalid FSS Read unit test expectations. " changes. Another major change is in my commit and branching behavior. In the distant past I had some data lost and lost a lot of work. My immediate solution was to just push everything asap. All in one branch. All together. Incomplete work is prefixed with "Progress:" to represent "work in progress". This can get messy. Switching to branching where I keep using "Progress:" will allow for me to manage the code as work in progress but then merge it in a cleaner manner. Thus this massive commit. The following are my notes collected from various commits in the separate progress branch. Have to add is_combining functions to detect combining for the purpose of FSS processing. Skip past initial place holders in the skip functions. The initial combining returns `F_complete_not_utf_start` only for combining at the start. This should be then handled as a combining. Add a new feature via additional functions for explicitly checking for joiner characters. These joiner characters are not the same as combining characters. This requires various build settings, like stand alone builds, to be updated with one of the following source files added: ``` fll/level_0/private-utf_joiner.c level_0/private-utf_joiner.c private-utf_joiner.c ``` This requires various `config.h` files to now support handling of: ``` #define _di_f_utf_character_is_joiner_ #define _di_f_utf_is_joiner_ ``` I had the option of either adding this and not exposing it through the **API** or exposing it through the **API**. Exposing it through the **API** causes this to be a new feature (which I really want to avoid in general). However, if the code is going to be added (because it is necessary and fixes some bugs), then it makes more sense to just expose it through the **API**. This also fixes incidental missing `_di*_` in some cases. The combining combines into what is effectively a graph character. Treat the joiner characters the same, even though one can join two non-printing characters. New lines must ignore joiners (I have seen this as the standard behavior by a lot of software and decided to go with that to avoid problems). The `is_combining`, `is_joiner`, and `is_combining_joiner` are new features added to help solve these problems. I didn't want to add a cost of auto-checking every character after the first. This means that I cannot have the `is_space`, `is_graph`, or other such functions automate this. Therefore, the logic must be changed to check for these spaces. The code will become more complex but this will not incur as much of a performance cost compared to checking each character 2x+ times. I also decided not to continue on after the first combining and joiner characters. The combining already makes something a graph. The joiner requires more searched and is highly subjective. Therefore, make life easier and just treat all joiners as graph-making for the purposes of processing and syntax. Fix problem inn tests where its failing due to `getline()` reaching the end of the line. If the EOL is reached, then the loop must end. The `f_fss_skip_past_combining()` is replaced with `f_fss_skip_past_combining_joiner()`. The `f_fss_seek_to_eol()` returns `F_okay_eol`. Check that instead of doing a buffer string character comparison where possible. Improve some condition block logic regarding breaks, returns, and continues. Change some of the status error checking to reduce return statements. I have probably flip flopped on this problem numerous times. Looking into how compilers and CPUs like to optimize things, I believe that reducing the return statements will be of benefit vs the probably cleaner immediate returns. This can only be done when it is clear and when the status code being returned (or any state) is identical between calls. It seems that editors make it difficult to add and manipulate combining characters. It is possible my test files have lost combining characters without my awareness. I used commands like the following to help generate some combining characters: ```sh echo "first valid.$(utf8 -cB U+0301)" >> basic-combining_read.txt echo -e " not a valid list{$(utf8 -cB U+0301)\n}" >> extended_list-combining_read.txt ``` Work on basic list combining tests, changing test strategy. The previous strategies that I used were only doing a file of only **Objects** and files of only **Contents**. This worked well for simple things. This ideally better allowed for more isolated unit tests. However, it is easier to write and follow tests of a complete **FSS** file. This means doing both the **Object** and **Content** tests in the same unit test (despite being separate functions). This provides more accurate checks, better testing, and as mentioned easier to write and follow test files. I discovered that copying the file with the combining character at the start of the line (U+0301) causes a space to be added. I have to then manually recreate the file and use my `utf8` program to regenerate the correct test data: ```sh utf8 -cB U+0301 >> test.txt ``` I decided that I needed to expand the `f_fss_skip_past_space()` into `f_fss_skip_past_space_previous()` to expose the `previous` position variable to the caller. I must avoid breaking the **API**/**ABI**, so `f_fss_skip_past_space()` is not changed. This should simplify the embedded read functions. Fix documentation, adding "Must not be NULL." where it is missing. The man pages will need to be re-generated more than likely. Just copy the `f_fss_skip_past_space()` man page and update it manually for now. Add missing `_di_f_utf_is_zero_width_` check on `private_f_utf_character_is_joiner()`. Redesign the FSS identify functions. Always reset the original used length on error. Reduce the number of return statements. Add a check for the combining joiner characters after the `#`. Remove inaccurate comment about depending on `f_utf_is_combining()` from parse function. Utilize the `f_state_t` locally to perform the necessary operations. This is not exposed to the caller to prevent **API** and **ABI** breakage. Future versions of **FLL** (0.9+) are expected to pass the state via the function parameter. Switch to the new practice of using flags to manage special internal states. This no longer guesses success based on `ids.used` being non-zero. Using the internal flags allows accurate handling. Add comments to functions that need `f_state_t` so that future version of **FLL** can properly handle. Rename `start` to `line_start` to be more consistent with other functions. The `line_start` is also more unique. This makes it easier to search. Switch to `fll_flags` from `found_data` for managing flags within a function. Handle basic list combining/joiner at start and fix unit test. For lists, if the start of a line is a combining/joiner, then it must be treated as Content even if it would otherwise be a valid Object on that line. The unit test actually has to handle when `F_okay_eos` or `F_okay_stop` is returned on content read. The basic list doesn't know if the content is a valid object or not. It can be assumed, but the function does not do this. The caller must handle incrementing found.used when either `F_okay_eos` or `F_okay_stop` is returned. Update the comment on this as well. Properly verify that `newline_last` is in fact a new line by checking it against the `begin` position. The basic list already has `begin`, but extended list has `begin` added to perform this check. --- diff --git a/build/disable/level_0/f_fss.h b/build/disable/level_0/f_fss.h index 14f5006c9..287ed0578 100644 --- a/build/disable/level_0/f_fss.h +++ b/build/disable/level_0/f_fss.h @@ -99,8 +99,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +#define _di_f_fss_skip_past_combining_ #define _di_f_fss_skip_past_delimit_ #define _di_f_fss_skip_past_space_ +#define _di_f_fss_skip_past_space_previous_ #define _di_f_fss_state_flag_e_ #define _di_f_fss_status_code_e_ #define _di_f_fss_type_s_ diff --git a/build/disable/level_0/f_utf.h b/build/disable/level_0/f_utf.h index 56677ab3e..e5835e3e8 100644 --- a/build/disable/level_0/f_utf.h +++ b/build/disable/level_0/f_utf.h @@ -22,6 +22,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -57,6 +58,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/level_0/settings b/build/level_0/settings index 6bffca26f..15bd1d9a8 100644 --- a/build/level_0/settings +++ b/build/level_0/settings @@ -107,6 +107,7 @@ build_sources_library environment.c build_sources_library execute.c build_sources_library file.c private-file.c file/common.c file/stream.c build_sources_library fss.c fss/common.c fss/item.c fss/named.c fss/nest.c fss/payload.c fss/quote.c fss/set.c fss/set_quote.c fss/simple_packet.c +build_sources_library private-fss.c build_sources_library iki.c iki/common.c iki/data.c iki/eki.c private-iki.c iki/private-data.c iki/private-eki.c build_sources_library limit.c limit/set.c limit/value.c build_sources_library memory.c memory/array.c memory/arrays.c @@ -131,7 +132,7 @@ build_sources_library string/static.c string/statics.c string/staticss.c build_sources_library string/triple.c string/triples.c string/tripless.c build_sources_library time.c build_sources_library type_array/cell.c type_array/date.c type_array/file.c type_array/fll_id.c type_array/int8.c type_array/int16.c type_array/int32.c type_array/int64.c type_array/int128.c type_array/number_signed.c type_array/number_unsigned.c type_array/pid.c type_array/poll.c type_array/quantity.c type_array/quantitys.c type_array/quantityss.c type_array/range.c type_array/ranges.c type_array/rangess.c type_array/range_double.c type_array/range_doubles.c type_array/range_doubless.c type_array/state.c type_array/status.c type_array/time.c type_array/uint8.c type_array/uint16.c type_array/uint32.c type_array/uint64.c type_array/uint128.c -build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c +build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_joiner.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c build_sources_library utf/common.c utf/convert.c build_sources_library utf/dynamic.c utf/dynamics.c utf/dynamicss.c build_sources_library utf/is.c utf/is_character.c diff --git a/build/monolithic/settings b/build/monolithic/settings index 2364f8ae8..9fac4ba8d 100644 --- a/build/monolithic/settings +++ b/build/monolithic/settings @@ -129,6 +129,7 @@ build_sources_library level_0/environment.c build_sources_library level_0/execute.c build_sources_library level_0/file.c level_0/private-file.c level_0/file/common.c level_0/file/stream.c build_sources_library level_0/fss.c level_0/fss/common.c level_0/fss/item.c level_0/fss/named.c level_0/fss/nest.c level_0/fss/payload.c level_0/fss/quote.c level_0/fss/set.c level_0/fss/set_quote.c level_0/fss/simple_packet.c +build_sources_library level_0/private-fss.c build_sources_library level_0/iki.c level_0/iki/common.c level_0/iki/data.c level_0/iki/eki.c level_0/private-iki.c level_0/iki/private-data.c level_0/iki/private-eki.c build_sources_library level_0/limit.c level_0/limit/set.c level_0/limit/value.c build_sources_library level_0/memory.c level_0/memory/array.c level_0/memory/arrays.c @@ -153,7 +154,7 @@ build_sources_library level_0/string/static.c level_0/string/statics.c level_0/s build_sources_library level_0/string/triple.c level_0/string/triples.c level_0/string/tripless.c build_sources_library level_0/time.c build_sources_library level_0/type_array/cell.c level_0/type_array/date.c level_0/type_array/file.c level_0/type_array/fll_id.c level_0/type_array/int8.c level_0/type_array/int16.c level_0/type_array/int32.c level_0/type_array/int64.c level_0/type_array/int128.c level_0/type_array/number_signed.c level_0/type_array/number_unsigned.c level_0/type_array/pid.c level_0/type_array/poll.c level_0/type_array/quantity.c level_0/type_array/quantitys.c level_0/type_array/quantityss.c level_0/type_array/range.c level_0/type_array/ranges.c level_0/type_array/rangess.c level_0/type_array/range_double.c level_0/type_array/range_doubles.c level_0/type_array/range_doubless.c level_0/type_array/state.c level_0/type_array/status.c level_0/type_array/time.c level_0/type_array/uint8.c level_0/type_array/uint16.c level_0/type_array/uint32.c level_0/type_array/uint64.c level_0/type_array/uint128.c -build_sources_library level_0/utf.c level_0/private-utf.c level_0/private-utf_alphabetic.c level_0/private-utf_combining.c level_0/private-utf_control.c level_0/private-utf_digit.c level_0/private-utf_emoji.c level_0/private-utf_numeric.c level_0/private-utf_phonetic.c level_0/private-utf_private.c level_0/private-utf_punctuation.c level_0/private-utf_subscript.c level_0/private-utf_superscript.c level_0/private-utf_symbol.c level_0/private-utf_unassigned.c level_0/private-utf_valid.c level_0/private-utf_whitespace.c level_0/private-utf_wide.c level_0/private-utf_word.c level_0/private-utf_zero_width.c +build_sources_library level_0/utf.c level_0/private-utf.c level_0/private-utf_alphabetic.c level_0/private-utf_combining.c level_0/private-utf_control.c level_0/private-utf_digit.c level_0/private-utf_emoji.c level_0/private-utf_joiner.c level_0/private-utf_numeric.c level_0/private-utf_phonetic.c level_0/private-utf_private.c level_0/private-utf_punctuation.c level_0/private-utf_subscript.c level_0/private-utf_superscript.c level_0/private-utf_symbol.c level_0/private-utf_unassigned.c level_0/private-utf_valid.c level_0/private-utf_whitespace.c level_0/private-utf_wide.c level_0/private-utf_word.c level_0/private-utf_zero_width.c build_sources_library level_0/utf/common.c level_0/utf/convert.c build_sources_library level_0/utf/dynamic.c level_0/utf/dynamics.c level_0/utf/dynamicss.c build_sources_library level_0/utf/is.c level_0/utf/is_character.c diff --git a/build/stand_alone/byte_dump.config.h b/build/stand_alone/byte_dump.config.h index 2469e5e21..476ebb639 100644 --- a/build/stand_alone/byte_dump.config.h +++ b/build/stand_alone/byte_dump.config.h @@ -1331,6 +1331,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ //#define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner //#define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1339,6 +1340,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1367,6 +1369,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1375,6 +1378,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/byte_dump.settings b/build/stand_alone/byte_dump.settings index 144f596ee..91a944e75 100644 --- a/build/stand_alone/byte_dump.settings +++ b/build/stand_alone/byte_dump.settings @@ -52,7 +52,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/example.config.h b/build/stand_alone/example.config.h index 2f78b4851..cd01de073 100644 --- a/build/stand_alone/example.config.h +++ b/build/stand_alone/example.config.h @@ -1296,6 +1296,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1304,6 +1305,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1332,6 +1334,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1340,6 +1343,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/example.settings b/build/stand_alone/example.settings index acee48eb3..1eeb8ff06 100644 --- a/build/stand_alone/example.settings +++ b/build/stand_alone/example.settings @@ -52,7 +52,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fake.config.h b/build/stand_alone/fake.config.h index cdde1b78b..4664af0e7 100644 --- a/build/stand_alone/fake.config.h +++ b/build/stand_alone/fake.config.h @@ -657,8 +657,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1787,6 +1789,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1795,6 +1798,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1823,6 +1827,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1831,6 +1836,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fake.settings b/build/stand_alone/fake.settings index 62e79ae47..b8f7f9678 100644 --- a/build/stand_alone/fake.settings +++ b/build/stand_alone/fake.settings @@ -54,6 +54,7 @@ build_sources_program fll/level_0/environment.c build_sources_program fll/level_0/execute.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/iki.c fll/level_0/private-iki.c fll/level_0/iki/common.c fll/level_0/iki/private-data.c fll/level_0/iki/private-eki.c fll/level_0/iki/data.c fll/level_0/iki/eki.c build_sources_program fll/level_0/limit.c fll/level_0/limit/set.c fll/level_0/limit/value.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c @@ -68,7 +69,7 @@ build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/leve build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/time.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_identify.config.h b/build/stand_alone/fss_identify.config.h index 88702454b..d67fb67b5 100644 --- a/build/stand_alone/fss_identify.config.h +++ b/build/stand_alone/fss_identify.config.h @@ -416,8 +416,8 @@ #define _di_f_fss_count_lines_range_ //#define _di_f_fss_default_d_ #define _di_f_fss_e_ -#define _di_f_fss_fail_utf_ -#define _di_f_fss_fail_utf_to_false_ +//#define _di_f_fss_fail_utf_ +//#define _di_f_fss_fail_utf_to_false_ #define _di_f_fss_header_s_ #define _di_f_fss_is_combining_ #define _di_f_fss_is_graph_ @@ -469,7 +469,7 @@ #define _di_f_fss_payload_object_total_s_ #define _di_f_fss_payload_object_type_s_ #define _di_f_fss_payload_s_ -#define _di_f_fss_placeholder_s_ +//#define _di_f_fss_placeholder_s_ #define _di_f_fss_quote_double_null_s_ #define _di_f_fss_quote_grave_null_s_ #define _di_f_fss_quote_single_null_s_ @@ -510,9 +510,11 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +#define _di_f_fss_skip_past_combining_ #define _di_f_fss_skip_past_delimit_ #define _di_f_fss_skip_past_space_ -#define _di_f_fss_state_flag_e_ +#define _di_f_fss_skip_past_space_previous_ +//#define _di_f_fss_state_flag_e_ #define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ #define _di_f_gid_t_ @@ -1140,8 +1142,8 @@ #define _di_f_string_mash_nulless_ #define _di_f_string_mish_ #define _di_f_string_mish_nulless_ -#define _di_f_string_null_s_ -#define _di_f_string_placeholder_s_ +//#define _di_f_string_null_s_ +//#define _di_f_string_placeholder_s_ #define _di_f_string_prepend_ #define _di_f_string_prepend_assure_ #define _di_f_string_prepend_assure_nulless_ @@ -1476,6 +1478,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1484,6 +1487,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1512,6 +1516,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1520,6 +1525,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_identify.settings b/build/stand_alone/fss_identify.settings index cad80b56c..750257340 100644 --- a/build/stand_alone/fss_identify.settings +++ b/build/stand_alone/fss_identify.settings @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.config.fss_basic.h b/build/stand_alone/fss_read.config.fss_basic.h index 7b1627bab..a9277040b 100644 --- a/build/stand_alone/fss_read.config.fss_basic.h +++ b/build/stand_alone/fss_read.config.fss_basic.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.fss_basic_list.h b/build/stand_alone/fss_read.config.fss_basic_list.h index 6a2ee483a..bfcb03b39 100644 --- a/build/stand_alone/fss_read.config.fss_basic_list.h +++ b/build/stand_alone/fss_read.config.fss_basic_list.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.fss_embedded_list.h b/build/stand_alone/fss_read.config.fss_embedded_list.h index d4c26f7a9..f961cdebe 100644 --- a/build/stand_alone/fss_read.config.fss_embedded_list.h +++ b/build/stand_alone/fss_read.config.fss_embedded_list.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.fss_extended.h b/build/stand_alone/fss_read.config.fss_extended.h index 86f6837d7..20590656e 100644 --- a/build/stand_alone/fss_read.config.fss_extended.h +++ b/build/stand_alone/fss_read.config.fss_extended.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.fss_extended_list.h b/build/stand_alone/fss_read.config.fss_extended_list.h index d0039047e..9275671f0 100644 --- a/build/stand_alone/fss_read.config.fss_extended_list.h +++ b/build/stand_alone/fss_read.config.fss_extended_list.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.fss_payload.h b/build/stand_alone/fss_read.config.fss_payload.h index 11f175548..7dd5fde10 100644 --- a/build/stand_alone/fss_read.config.fss_payload.h +++ b/build/stand_alone/fss_read.config.fss_payload.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.config.h b/build/stand_alone/fss_read.config.h index f50d6f853..da6dea24e 100644 --- a/build/stand_alone/fss_read.config.h +++ b/build/stand_alone/fss_read.config.h @@ -567,8 +567,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1547,6 +1549,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1555,6 +1558,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1583,6 +1587,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1591,6 +1596,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_read.settings b/build/stand_alone/fss_read.settings index 30dae3c0a..8c46c6e78 100644 --- a/build/stand_alone/fss_read.settings +++ b/build/stand_alone/fss_read.settings @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_basic b/build/stand_alone/fss_read.settings.fss_basic index 024476563..cd72e0405 100644 --- a/build/stand_alone/fss_read.settings.fss_basic +++ b/build/stand_alone/fss_read.settings.fss_basic @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_basic_list b/build/stand_alone/fss_read.settings.fss_basic_list index c68c1b385..c920d4187 100644 --- a/build/stand_alone/fss_read.settings.fss_basic_list +++ b/build/stand_alone/fss_read.settings.fss_basic_list @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_embedded_list b/build/stand_alone/fss_read.settings.fss_embedded_list index 83021d1b9..53d5da50c 100644 --- a/build/stand_alone/fss_read.settings.fss_embedded_list +++ b/build/stand_alone/fss_read.settings.fss_embedded_list @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_extended b/build/stand_alone/fss_read.settings.fss_extended index 453947cd0..a4d0c3645 100644 --- a/build/stand_alone/fss_read.settings.fss_extended +++ b/build/stand_alone/fss_read.settings.fss_extended @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_extended_list b/build/stand_alone/fss_read.settings.fss_extended_list index f57b7c039..d62be4cdb 100644 --- a/build/stand_alone/fss_read.settings.fss_extended_list +++ b/build/stand_alone/fss_read.settings.fss_extended_list @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_read.settings.fss_payload b/build/stand_alone/fss_read.settings.fss_payload index 3779f2e36..c4eeba2ed 100644 --- a/build/stand_alone/fss_read.settings.fss_payload +++ b/build/stand_alone/fss_read.settings.fss_payload @@ -48,6 +48,7 @@ build_sources_program fll/level_0/directory.c fll/level_0/directory/common.c fll build_sources_program fll/level_0/private-directory.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -58,7 +59,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.config.fss_basic.h b/build/stand_alone/fss_write.config.fss_basic.h index 26846825a..f5a8a965d 100644 --- a/build/stand_alone/fss_write.config.fss_basic.h +++ b/build/stand_alone/fss_write.config.fss_basic.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.fss_basic_list.h b/build/stand_alone/fss_write.config.fss_basic_list.h index 40fb24fa9..73b9f5104 100644 --- a/build/stand_alone/fss_write.config.fss_basic_list.h +++ b/build/stand_alone/fss_write.config.fss_basic_list.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.fss_embedded_list.h b/build/stand_alone/fss_write.config.fss_embedded_list.h index 86419a0dd..db29a22a8 100644 --- a/build/stand_alone/fss_write.config.fss_embedded_list.h +++ b/build/stand_alone/fss_write.config.fss_embedded_list.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.fss_extended.h b/build/stand_alone/fss_write.config.fss_extended.h index 5a56a2293..101cb9b12 100644 --- a/build/stand_alone/fss_write.config.fss_extended.h +++ b/build/stand_alone/fss_write.config.fss_extended.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.fss_extended_list.h b/build/stand_alone/fss_write.config.fss_extended_list.h index 41088a876..17e6eea91 100644 --- a/build/stand_alone/fss_write.config.fss_extended_list.h +++ b/build/stand_alone/fss_write.config.fss_extended_list.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.fss_payload.h b/build/stand_alone/fss_write.config.fss_payload.h index 6bdfa83bd..d3499ca28 100644 --- a/build/stand_alone/fss_write.config.fss_payload.h +++ b/build/stand_alone/fss_write.config.fss_payload.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.config.h b/build/stand_alone/fss_write.config.h index ed889622e..8421653ae 100644 --- a/build/stand_alone/fss_write.config.h +++ b/build/stand_alone/fss_write.config.h @@ -419,7 +419,7 @@ //#define _di_f_fss_fail_utf_ //#define _di_f_fss_fail_utf_to_false_ //#define _di_f_fss_header_s_ -#define _di_f_fss_is_combining_ +//#define _di_f_fss_is_combining_ //#define _di_f_fss_is_graph_ //#define _di_f_fss_is_space_ //#define _di_f_fss_is_zero_width_ @@ -510,8 +510,10 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +//#define _di_f_fss_skip_past_combining_ //#define _di_f_fss_skip_past_delimit_ //#define _di_f_fss_skip_past_space_ +//#define _di_f_fss_skip_past_space_previous_ //#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ //#define _di_f_fss_type_s_ @@ -1490,6 +1492,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1498,6 +1501,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1526,6 +1530,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1534,6 +1539,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/fss_write.settings b/build/stand_alone/fss_write.settings index f0a61d8d7..11b94a9e4 100644 --- a/build/stand_alone/fss_write.settings +++ b/build/stand_alone/fss_write.settings @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_basic b/build/stand_alone/fss_write.settings.fss_basic index 7201a0d25..13c650c18 100644 --- a/build/stand_alone/fss_write.settings.fss_basic +++ b/build/stand_alone/fss_write.settings.fss_basic @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_basic_list b/build/stand_alone/fss_write.settings.fss_basic_list index 49f0d91f5..66ecdcf88 100644 --- a/build/stand_alone/fss_write.settings.fss_basic_list +++ b/build/stand_alone/fss_write.settings.fss_basic_list @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_embedded_list b/build/stand_alone/fss_write.settings.fss_embedded_list index 3ca322518..53fb614e5 100644 --- a/build/stand_alone/fss_write.settings.fss_embedded_list +++ b/build/stand_alone/fss_write.settings.fss_embedded_list @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_extended b/build/stand_alone/fss_write.settings.fss_extended index 1e3f12785..eb45958d3 100644 --- a/build/stand_alone/fss_write.settings.fss_extended +++ b/build/stand_alone/fss_write.settings.fss_extended @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_extended_list b/build/stand_alone/fss_write.settings.fss_extended_list index 9174e84a8..160f2e45f 100644 --- a/build/stand_alone/fss_write.settings.fss_extended_list +++ b/build/stand_alone/fss_write.settings.fss_extended_list @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/fss_write.settings.fss_payload b/build/stand_alone/fss_write.settings.fss_payload index aa0b81b9b..28305e052 100644 --- a/build/stand_alone/fss_write.settings.fss_payload +++ b/build/stand_alone/fss_write.settings.fss_payload @@ -46,6 +46,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/parse.c fll/level_0/parse/utf.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/iki_read.config.eki.h b/build/stand_alone/iki_read.config.eki.h index 624a8c9ca..3e207ebb0 100644 --- a/build/stand_alone/iki_read.config.eki.h +++ b/build/stand_alone/iki_read.config.eki.h @@ -1392,6 +1392,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1400,6 +1401,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1428,6 +1430,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1436,6 +1439,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/iki_read.config.h b/build/stand_alone/iki_read.config.h index 5c517afa9..afc332121 100644 --- a/build/stand_alone/iki_read.config.h +++ b/build/stand_alone/iki_read.config.h @@ -1392,6 +1392,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1400,6 +1401,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1428,6 +1430,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1436,6 +1439,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/iki_read.settings b/build/stand_alone/iki_read.settings index 634d4a80f..d31ff4b84 100644 --- a/build/stand_alone/iki_read.settings +++ b/build/stand_alone/iki_read.settings @@ -55,7 +55,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/iki_read.settings.eki b/build/stand_alone/iki_read.settings.eki index afb369c0e..3493673a2 100644 --- a/build/stand_alone/iki_read.settings.eki +++ b/build/stand_alone/iki_read.settings.eki @@ -54,7 +54,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/iki_write.config.eki.h b/build/stand_alone/iki_write.config.eki.h index e9cafeabe..3d03e3957 100644 --- a/build/stand_alone/iki_write.config.eki.h +++ b/build/stand_alone/iki_write.config.eki.h @@ -1343,6 +1343,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1351,6 +1352,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1379,6 +1381,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1387,6 +1390,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/iki_write.config.h b/build/stand_alone/iki_write.config.h index adc7079fc..3832f9a3b 100644 --- a/build/stand_alone/iki_write.config.h +++ b/build/stand_alone/iki_write.config.h @@ -1343,6 +1343,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1351,6 +1352,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1379,6 +1381,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1387,6 +1390,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/iki_write.settings b/build/stand_alone/iki_write.settings index 9e073f850..38b97eb4b 100644 --- a/build/stand_alone/iki_write.settings +++ b/build/stand_alone/iki_write.settings @@ -53,7 +53,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/iki_write.settings.eki b/build/stand_alone/iki_write.settings.eki index e98efb304..d1bd23126 100644 --- a/build/stand_alone/iki_write.settings.eki +++ b/build/stand_alone/iki_write.settings.eki @@ -53,7 +53,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/status_code.config.fss.h b/build/stand_alone/status_code.config.fss.h index 7094cfeab..1c40e66dd 100644 --- a/build/stand_alone/status_code.config.fss.h +++ b/build/stand_alone/status_code.config.fss.h @@ -416,8 +416,8 @@ #define _di_f_fss_count_lines_range_ #define _di_f_fss_default_d_ #define _di_f_fss_e_ -#define _di_f_fss_fail_utf_ -#define _di_f_fss_fail_utf_to_false_ +//#define _di_f_fss_fail_utf_ +//#define _di_f_fss_fail_utf_to_false_ #define _di_f_fss_header_s_ #define _di_f_fss_is_combining_ #define _di_f_fss_is_graph_ @@ -469,7 +469,7 @@ #define _di_f_fss_payload_object_total_s_ #define _di_f_fss_payload_object_type_s_ #define _di_f_fss_payload_s_ -#define _di_f_fss_placeholder_s_ +//#define _di_f_fss_placeholder_s_ #define _di_f_fss_quote_double_null_s_ #define _di_f_fss_quote_grave_null_s_ #define _di_f_fss_quote_single_null_s_ @@ -510,11 +510,13 @@ #define _di_f_fss_simple_packetss_delete_callback_ #define _di_f_fss_simple_packetss_destroy_callback_ #define _di_f_fss_simple_packetss_t_ +#define _di_f_fss_skip_past_combining_ #define _di_f_fss_skip_past_delimit_ #define _di_f_fss_skip_past_space_ -#define _di_f_fss_state_flag_e_ +#define _di_f_fss_skip_past_space_previous_ +//#define _di_f_fss_state_flag_e_ //#define _di_f_fss_status_code_e_ -#define _di_f_fss_type_s_ +//#define _di_f_fss_type_s_ #define _di_f_gid_t_ #define _di_f_gids_t_ #define _di_f_gidss_delete_callback_ @@ -1124,8 +1126,8 @@ #define _di_f_string_mash_nulless_ #define _di_f_string_mish_ #define _di_f_string_mish_nulless_ -#define _di_f_string_null_s_ -#define _di_f_string_placeholder_s_ +//#define _di_f_string_null_s_ +//#define _di_f_string_placeholder_s_ #define _di_f_string_prepend_ #define _di_f_string_prepend_assure_ #define _di_f_string_prepend_assure_nulless_ @@ -1460,6 +1462,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1468,6 +1471,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1496,6 +1500,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +//#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1504,6 +1509,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +//#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/status_code.config.h b/build/stand_alone/status_code.config.h index c9a2e4662..a08336293 100644 --- a/build/stand_alone/status_code.config.h +++ b/build/stand_alone/status_code.config.h @@ -1391,6 +1391,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ #define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner #define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1399,6 +1400,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1427,6 +1429,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1435,6 +1438,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ #define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ #define _di_f_utf_is_private_ diff --git a/build/stand_alone/status_code.settings b/build/stand_alone/status_code.settings index 856c354a9..acafba500 100644 --- a/build/stand_alone/status_code.settings +++ b/build/stand_alone/status_code.settings @@ -55,7 +55,7 @@ build_sources_program fll/level_0/status_string.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/status_code.settings.fss b/build/stand_alone/status_code.settings.fss index f10c93e54..233f90ee1 100644 --- a/build/stand_alone/status_code.settings.fss +++ b/build/stand_alone/status_code.settings.fss @@ -45,6 +45,7 @@ build_sources_program fll/level_0/console.c fll/level_0/console/common.c fll/lev build_sources_program fll/level_0/conversion.c fll/level_0/private-conversion.c fll/level_0/conversion/common.c build_sources_program fll/level_0/file.c fll/level_0/private-file.c fll/level_0/file/common.c fll/level_0/file/stream.c build_sources_program fll/level_0/fss.c fll/level_0/fss/common.c fll/level_0/fss/item.c fll/level_0/fss/named.c fll/level_0/fss/nest.c fll/level_0/fss/payload.c fll/level_0/fss/quote.c fll/level_0/fss/set.c fll/level_0/fss/set_quote.c fll/level_0/fss/simple_packet.c +build_sources_program fll/level_0/private-fss.c build_sources_program fll/level_0/memory.c fll/level_0/memory/array.c fll/level_0/memory/arrays.c build_sources_program fll/level_0/private-memory.c fll/level_0/memory/private-array.c build_sources_program fll/level_0/pipe.c @@ -56,7 +57,7 @@ build_sources_program fll/level_0/status_string.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/build/stand_alone/utf8.config.h b/build/stand_alone/utf8.config.h index 166f35f53..ec0642738 100644 --- a/build/stand_alone/utf8.config.h +++ b/build/stand_alone/utf8.config.h @@ -1349,6 +1349,7 @@ #define _di_f_utf_character_is_alphabetic_numeric_ #define _di_f_utf_character_is_ascii_ //#define _di_f_utf_character_is_combining_ +#define _di_f_utf_character_is_combining_joiner //#define _di_f_utf_character_is_control_ #define _di_f_utf_character_is_control_code_ #define _di_f_utf_character_is_control_format_ @@ -1357,6 +1358,7 @@ #define _di_f_utf_character_is_emoji_ #define _di_f_utf_character_is_fragment_ #define _di_f_utf_character_is_graph_ +#define _di_f_utf_character_is_joiner_ #define _di_f_utf_character_is_numeric_ #define _di_f_utf_character_is_phonetic_ #define _di_f_utf_character_is_private_ @@ -1385,6 +1387,7 @@ #define _di_f_utf_is_alphabetic_numeric_ #define _di_f_utf_is_ascii_ //#define _di_f_utf_is_combining_ +#define _di_f_utf_is_combining_joiner_ //#define _di_f_utf_is_control_ #define _di_f_utf_is_control_code_ #define _di_f_utf_is_control_format_ @@ -1393,6 +1396,7 @@ #define _di_f_utf_is_emoji_ #define _di_f_utf_is_fragment_ //#define _di_f_utf_is_graph_ +#define _di_f_utf_is_joiner_ #define _di_f_utf_is_numeric_ #define _di_f_utf_is_phonetic_ //#define _di_f_utf_is_private_ diff --git a/build/stand_alone/utf8.settings b/build/stand_alone/utf8.settings index 5cd856078..7bcd09d85 100644 --- a/build/stand_alone/utf8.settings +++ b/build/stand_alone/utf8.settings @@ -53,7 +53,7 @@ build_sources_program fll/level_0/signal.c build_sources_program fll/level_0/string.c fll/level_0/private-string.c fll/level_0/string/common.c build_sources_program fll/level_0/string/dynamic.c fll/level_0/string/dynamics.c fll/level_0/string/dynamicss.c fll/level_0/string/map.c fll/level_0/string/maps.c fll/level_0/string/mapss.c fll/level_0/string/map_multi.c fll/level_0/string/map_multis.c fll/level_0/string/map_multiss.c fll/level_0/string/static.c fll/level_0/string/statics.c fll/level_0/string/staticss.c fll/level_0/string/triple.c fll/level_0/string/triples.c fll/level_0/string/tripless.c build_sources_program fll/level_0/type_array/cell.c fll/level_0/type_array/date.c fll/level_0/type_array/file.c fll/level_0/type_array/fll_id.c fll/level_0/type_array/int8.c fll/level_0/type_array/int16.c fll/level_0/type_array/int32.c fll/level_0/type_array/int64.c fll/level_0/type_array/int128.c fll/level_0/type_array/number_signed.c fll/level_0/type_array/number_unsigned.c fll/level_0/type_array/poll.c fll/level_0/type_array/quantity.c fll/level_0/type_array/quantitys.c fll/level_0/type_array/quantityss.c fll/level_0/type_array/range.c fll/level_0/type_array/ranges.c fll/level_0/type_array/rangess.c fll/level_0/type_array/state.c fll/level_0/type_array/status.c fll/level_0/type_array/time.c fll/level_0/type_array/uint8.c fll/level_0/type_array/uint16.c fll/level_0/type_array/uint32.c fll/level_0/type_array/uint64.c fll/level_0/type_array/uint128.c -build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c +build_sources_program fll/level_0/utf.c fll/level_0/private-utf.c fll/level_0/private-utf_alphabetic.c fll/level_0/private-utf_combining.c fll/level_0/private-utf_control.c fll/level_0/private-utf_digit.c fll/level_0/private-utf_emoji.c fll/level_0/private-utf_joiner.c fll/level_0/private-utf_numeric.c fll/level_0/private-utf_phonetic.c fll/level_0/private-utf_private.c fll/level_0/private-utf_punctuation.c fll/level_0/private-utf_subscript.c fll/level_0/private-utf_superscript.c fll/level_0/private-utf_symbol.c fll/level_0/private-utf_unassigned.c fll/level_0/private-utf_valid.c fll/level_0/private-utf_whitespace.c fll/level_0/private-utf_wide.c fll/level_0/private-utf_word.c fll/level_0/private-utf_zero_width.c build_sources_program fll/level_0/utf/common.c fll/level_0/utf/convert.c fll/level_0/utf/dynamic.c fll/level_0/utf/dynamics.c fll/level_0/utf/dynamicss.c fll/level_0/utf/is.c fll/level_0/utf/is_character.c fll/level_0/utf/map.c fll/level_0/utf/maps.c fll/level_0/utf/mapss.c fll/level_0/utf/map_multi.c fll/level_0/utf/map_multis.c fll/level_0/utf/map_multiss.c fll/level_0/utf/static.c fll/level_0/utf/statics.c fll/level_0/utf/staticss.c fll/level_0/utf/string.c fll/level_0/utf/triple.c fll/level_0/utf/triples.c fll/level_0/utf/tripless.c build_sources_program fll/level_0/utf/private-dynamics.c fll/level_0/utf/private-maps.c fll/level_0/utf/private-map_multis.c fll/level_0/utf/private-string.c fll/level_0/utf/private-triples.c diff --git a/level_0/f_console/c/console.h b/level_0/f_console/c/console.h index 4c68ae324..a1f2570b2 100644 --- a/level_0/f_console/c/console.h +++ b/level_0/f_console/c/console.h @@ -162,6 +162,8 @@ extern "C" { * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not if argc is 0, argv is a NULL pointer, or if argv[0] is NULL. diff --git a/level_0/f_fss/c/fss.c b/level_0/f_fss/c/fss.c index 3bf7cda93..e7342db2a 100644 --- a/level_0/f_fss/c/fss.c +++ b/level_0/f_fss/c/fss.c @@ -1,4 +1,5 @@ #include "fss.h" +#include "private-fss.h" #ifdef __cplusplus extern "C" { @@ -24,6 +25,11 @@ extern "C" { for (f_number_unsigned_t i = 0; i < delimits.used; ++i) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (delimits.array[i] < buffer->used) { buffer->string[delimits.array[i]] = f_fss_placeholder_s.string[0]; } @@ -53,6 +59,11 @@ extern "C" { for (f_number_unsigned_t i = 0; i < delimits.used; ++i) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (delimits.array[i] < buffer->used && delimits.array[i] >= range.start && delimits.array[i] <= range.stop) { buffer->string[delimits.array[i]] = f_fss_placeholder_s.string[0]; } @@ -82,6 +93,11 @@ extern "C" { for (f_number_unsigned_t i = 0; i < before && i < buffer.used; i += macro_f_utf_byte_width(buffer.string[i])) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (buffer.string[i] == f_fss_eol_s.string[0]) { ++(*line); } @@ -111,6 +127,11 @@ extern "C" { for (f_number_unsigned_t i = range.start; i <= range.stop && i < buffer.used; i += macro_f_utf_byte_width(buffer.string[i])) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (buffer.string[i] == f_fss_eol_s.string[0]) { ++(*line); } @@ -164,14 +185,18 @@ extern "C" { return F_false; } - f_number_unsigned_t width_max = (range.stop - range.start) + 1; + state->status = f_utf_is_combining( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); - if (width_max > buffer.used - range.start) { - width_max = buffer.used - range.start; - } + if (state->status == F_true) { + state->status = F_combarudoo; - state->status = f_utf_is_combining(buffer.string + range.start, width_max); - if (state->status == F_true) return F_true; + return F_true; + } f_fss_fail_utf_to_false(state); @@ -179,8 +204,8 @@ extern "C" { } #endif // _di_f_fss_is_combining_ -#ifndef _di_f_fss_is_graph_ - f_status_t f_fss_is_graph(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { +#ifndef _di_f_fss_is_combining_joiner_ + f_status_t f_fss_is_combining_joiner(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { #ifndef _di_level_0_parameter_checking_ if (!state) return F_status_set_error(F_parameter); #endif // _di_level_0_parameter_checking_ @@ -191,13 +216,59 @@ extern "C" { return F_false; } - f_number_unsigned_t width_max = (range.stop - range.start) + 1; + state->status = f_utf_is_combining( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); + + if (state->status == F_true) { + state->status = F_combarudoo; + + return F_true; + } + + if (F_status_is_error_not(state->status)) { + state->status = f_utf_is_joiner( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); + + if (state->status == F_true) { + state->status = F_bound; + + return F_true; + } + } + + f_fss_fail_utf_to_false(state); + + return F_false; + } +#endif // _di_f_fss_is_combining_joiner_ + +#ifndef _di_f_fss_is_graph_ + f_status_t f_fss_is_graph(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { + #ifndef _di_level_0_parameter_checking_ + if (!state) return F_status_set_error(F_parameter); + #endif // _di_level_0_parameter_checking_ + + if (!buffer.used || range.start > range.stop || range.start >= buffer.used) { + state->status = F_data_not; - if (width_max > buffer.used - range.start) { - width_max = buffer.used - range.start; + return F_false; } - state->status = f_utf_is_graph(buffer.string + range.start, width_max); + state->status = f_utf_is_graph( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); + if (state->status == F_true) return F_true; f_fss_fail_utf_to_false(state); @@ -206,8 +277,8 @@ extern "C" { } #endif // _di_f_fss_is_graph_ -#ifndef _di_f_fss_is_space_ - f_status_t f_fss_is_space(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { +#ifndef _di_f_fss_is_joiner_ + f_status_t f_fss_is_joiner(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { #ifndef _di_level_0_parameter_checking_ if (!state) return F_status_set_error(F_parameter); #endif // _di_level_0_parameter_checking_ @@ -218,29 +289,87 @@ extern "C" { return F_false; } - f_number_unsigned_t width_max = (range.stop - range.start) + 1; + state->status = f_utf_is_joiner( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); - if (width_max > buffer.used - range.start) { - width_max = buffer.used - range.start; - } + if (state->status == F_true) { + state->status = F_bound; - state->status = f_utf_is_zero_width(buffer.string + range.start, width_max); - if (state->status != F_false) return F_false; + return F_true; + } f_fss_fail_utf_to_false(state); - state->status = f_utf_is_whitespace(buffer.string + range.start, width_max, F_false); - if (state->status == F_true) return F_true; + return F_false; + } +#endif // _di_f_fss_is_joiner_ - f_fss_fail_utf_to_false(state); +#ifndef _di_f_fss_is_space_ + f_status_t f_fss_is_space(const f_string_static_t buffer, const f_range_t range, f_state_t * const state) { + #ifndef _di_level_0_parameter_checking_ + if (!state) return F_status_set_error(F_parameter); + #endif // _di_level_0_parameter_checking_ + + if (!buffer.used || range.start > range.stop || range.start >= buffer.used) { + state->status = F_data_not; + + return F_false; + } + + state->status = f_utf_is_zero_width( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); + + if (state->status == F_true) { + state->status = F_space; + + return F_false; + } if (state->status == F_false) { - state->status = f_utf_is_control(buffer.string + range.start, width_max); - if (state->status == F_true) return F_true; + state->status = f_utf_is_whitespace( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1, + F_false + ); + + if (state->status == F_false) return F_false; + } - f_fss_fail_utf_to_false(state); + if (state->status == F_true) { + state->status = f_utf_is_control( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); + + if (state->status == F_true) { + state->status = F_control; + + // Some ASCII control characters are space characters. + if (!macro_f_utf_byte_width_is(buffer.string[range.start])) { + return F_true; + } + + return F_false; + } + else if (state->status == F_false) { + return F_true; + } } + f_fss_fail_utf_to_false(state); + return F_false; } #endif // _di_f_fss_is_space_ @@ -257,13 +386,13 @@ extern "C" { return F_false; } - f_number_unsigned_t width_max = (range.stop - range.start) + 1; - - if (width_max > buffer.used - range.start) { - width_max = buffer.used - range.start; - } + state->status = f_utf_is_zero_width( + buffer.string + range.start, + (range.stop - range.start) + 1 > buffer.used - range.start + ? buffer.used - range.start + : (range.stop - range.start) + 1 + ); - state->status = f_utf_is_zero_width(buffer.string + range.start, width_max); if (state->status == F_true) return F_true; f_fss_fail_utf_to_false(state); @@ -292,6 +421,11 @@ extern "C" { for (;; ++range->start) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (range->start >= buffer.used) { state->status = F_okay_eos; @@ -311,8 +445,8 @@ extern "C" { } #endif // _di_f_fss_seek_to_eol_ -#ifndef _di_f_fss_skip_past_delimit_ - void f_fss_skip_past_delimit(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state) { +#ifndef _di_f_fss_skip_past_combining_joiner_ + void f_fss_skip_past_combining_joiner(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state) { #ifndef _di_level_0_parameter_checking_ if (!state) return; @@ -329,8 +463,14 @@ extern "C" { return; } + // Skip past initial place holders. for (;; ++range->start) { + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + if (range->start >= buffer.used) { state->status = F_okay_eos; @@ -346,53 +486,44 @@ extern "C" { if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) break; } // for - state->status = F_okay; - } -#endif // _di_f_fss_skip_past_delimit_ - -#ifndef _di_f_fss_skip_past_space_ - void f_fss_skip_past_space(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state) { - #ifndef _di_level_0_parameter_checking_ - if (!state) return; + if (macro_f_utf_byte_width_is(buffer.string[range->start])) { + if (range->start + macro_f_utf_byte_width(buffer.string[range->start]) > buffer.used) { + state->status = F_status_set_error(F_complete_not_utf_eos); - if (!range) { - state->status = F_status_set_error(F_parameter); + f_fss_fail_utf(state); return; } - #endif // _di_level_0_parameter_checking_ - - if (!buffer.used || range->start > range->stop || range->start >= buffer.used) { - state->status = F_data_not; - return; - } + if (range->start + (macro_f_utf_byte_width(buffer.string[range->start]) - 1) > range->stop) { + state->status = F_status_set_error(F_complete_not_utf_stop); - uint8_t width = 0; - f_number_unsigned_t width_max = (range->stop - range->start) + 1; + f_fss_fail_utf(state); - if (width_max > buffer.used - range->start) { - width_max = buffer.used - range->start; + return; + } } - // Check that the first character is not a combining character. - state->status = f_utf_is_combining(buffer.string + range->start, width_max); + // 0x1 = last is combining, 0x2 = last is joiner. + uint8_t flag = 0x0; - f_fss_fail_utf_to_false(state); - if (F_status_is_error(state->status)) return; + for (;;) { - if (state->status == F_true) { - state->status = F_status_set_error(F_complete_not_utf_start); + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } - return; - } + if (range->start >= buffer.used) { + state->status = F_okay_eos; - for (;;) { + return; + } - width_max = (range->stop - range->start) + 1; + if (range->start > range->stop) { + state->status = F_okay_stop; - if (width_max > buffer.used - range->start) { - width_max = buffer.used - range->start; + return; } if (buffer.string[range->start] == f_fss_eol_s.string[0]) { @@ -407,35 +538,20 @@ extern "C" { continue; } - state->status = f_utf_is_whitespace(buffer.string + range->start, width_max, F_false); + state->status = f_utf_is_combining_joiner( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1 + ); f_fss_fail_utf_to_false(state); - if (state->status == F_false) { - state->status = f_utf_is_control(buffer.string + range->start, width_max); - - f_fss_fail_utf_to_false(state); - - if (state->status == F_false) { - state->status = f_utf_is_combining(buffer.string + range->start, width_max); - - f_fss_fail_utf_to_false(state); - - if (state->status == F_false) { - state->status = f_utf_is_zero_width(buffer.string + range->start, width_max); - - f_fss_fail_utf_to_false(state); - if (state->status == F_false) break; - } - } - } - + if (state->status == F_false) break; if (F_status_is_error(state->status)) return; - width = macro_f_utf_byte_width(buffer.string[range->start]); - - if (width > 1) { - if (range->start + width >= buffer.used) { + if (macro_f_utf_byte_width_is(buffer.string[range->start])) { + if (range->start + macro_f_utf_byte_width(buffer.string[range->start]) > buffer.used) { state->status = F_status_set_error(F_complete_not_utf_eos); f_fss_fail_utf(state); @@ -443,7 +559,7 @@ extern "C" { return; } - if (range->start + width > range->stop) { + if (range->start + (macro_f_utf_byte_width(buffer.string[range->start]) - 1) > range->stop) { state->status = F_status_set_error(F_complete_not_utf_stop); f_fss_fail_utf(state); @@ -452,7 +568,48 @@ extern "C" { } } - range->start += width; + range->start += macro_f_utf_byte_width(buffer.string[range->start]); + + if (state->status == F_bound) { + flag = 0x1; + } + else if (state->status == F_combarudoo) { + flag = 0x2; + } + } // for + + state->status = flag & 0x1 + ? F_bound + : flag & 0x2 + ? F_combarudoo + : F_okay; + } +#endif // _di_f_fss_skip_past_combining_joiner_ + +#ifndef _di_f_fss_skip_past_delimit_ + void f_fss_skip_past_delimit(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state) { + #ifndef _di_level_0_parameter_checking_ + if (!state) return; + + if (!range) { + state->status = F_status_set_error(F_parameter); + + return; + } + #endif // _di_level_0_parameter_checking_ + + if (!buffer.used || range->start > range->stop || range->start >= buffer.used) { + state->status = F_data_not; + + return; + } + + for (;; ++range->start) { + + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } if (range->start >= buffer.used) { state->status = F_okay_eos; @@ -465,12 +622,60 @@ extern "C" { return; } + + if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) break; } // for state->status = F_okay; } +#endif // _di_f_fss_skip_past_delimit_ + +#ifndef _di_f_fss_skip_past_space_ + void f_fss_skip_past_space(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state) { + #ifndef _di_level_0_parameter_checking_ + if (!state) return; + + if (!range) { + state->status = F_status_set_error(F_parameter); + + return; + } + #endif // _di_level_0_parameter_checking_ + + if (!buffer.used || range->start > range->stop || range->start >= buffer.used) { + state->status = F_data_not; + + return; + } + + f_number_unsigned_t previous = range->start; + + private_f_fss_skip_past_space_previous(buffer, range, state, &previous); + } #endif // _di_f_fss_skip_past_space_ +#ifndef _di_f_fss_skip_past_space_previous_ + void f_fss_skip_past_space_previous(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state, f_number_unsigned_t * const previous) { + #ifndef _di_level_0_parameter_checking_ + if (!state) return; + + if (!range || !previous) { + state->status = F_status_set_error(F_parameter); + + return; + } + #endif // _di_level_0_parameter_checking_ + + if (!buffer.used || range->start > range->stop || range->start >= buffer.used) { + state->status = F_data_not; + + return; + } + + private_f_fss_skip_past_space_previous(buffer, range, state, previous); + } +#endif // _di_f_fss_skip_past_space_previous_ + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_0/f_fss/c/fss.h b/level_0/f_fss/c/fss.h index 656ea2b9f..3fb6010d3 100644 --- a/level_0/f_fss/c/fss.h +++ b/level_0/f_fss/c/fss.h @@ -45,13 +45,18 @@ extern "C" { * An array of locations containing the delimits to apply within the buffer. * @param buffer * The string to process. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_apply_delimit_ @@ -69,13 +74,18 @@ extern "C" { * The range in which to restrict which delimits to apply. * @param buffer * The string to process. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_apply_delimit_range_ @@ -97,14 +107,19 @@ extern "C" { * The total lines found leading up to but not including before. * This value is not reset and only additions are performed. * When F_data_not is returned, this value is not altered. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0 (line is set to 0). * - * F_parameter (with error bit) if a parameter is invalid. + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_count_lines_ extern void f_fss_count_lines(const f_string_static_t buffer, const f_number_unsigned_t before, f_number_unsigned_t * const line, f_state_t * const state); @@ -125,13 +140,18 @@ extern "C" { * The total lines found leading up to but not including before. * This value is not reset and only additions are performed. * When F_data_not is returned, this value is not altered. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but the range.start is greater than buffer.used or buffer.used is 0 (line is set to 0). * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_count_lines_range_ @@ -149,6 +169,8 @@ extern "C" { * - F_utf_fragment * - F_utf_not * + * Must not be NULL. + * * @param state * A state for providing flags and handling interrupts during long running operations. * @@ -170,6 +192,8 @@ extern "C" { * - F_utf_fragment * - F_utf_not * + * Must not be NULL. + * * @param state * A state for providing flags and handling interrupts during long running operations. * @@ -199,18 +223,22 @@ extern "C" { * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: - * F_true if the character in the buffer is a combining character. + * F_combarudoo character in the buffer is a combining character (future versions will provide a more accurate status, like F_combine). + * F_data_not on success but nothing is processed. * F_false if the character in the buffer is not a combining character. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * * Errors (with error bit) from: f_utf_is_combining(). * * @return - * F_true if the character in the buffer is a combining character. * F_false if the character in the buffer is not a combining character. * F_false is returned on any error. + * F_true if the character in the buffer is a combining character. * * @see f_utf_is_combining() */ @@ -218,9 +246,56 @@ extern "C" { extern f_status_t f_fss_is_combining(const f_string_static_t buffer, const f_range_t range, f_state_t * const state); #endif // _di_f_fss_is_combining_ +/** + * Identify whether or not a character in the buffer is a combining (ASCII or UTF-8) character or a joiner (ASCII or UTF-8) character. + * + * This only checks if the given character is a combining character and does not check what this combines into. + * + * The combining characters combine from right to left. + * It is recommended to use this after testing for other characters, such as f_fss_is_space() or f_fss_is_graph(). + * A combining character can follow any character, even if it is something like a control character. + * This is unclear behavior so a simple strategy is to assume that a combining character results in a graph for anything except a non-combining zero-width character. + * U+0020 followed by U+0301 would result in the combination of the two being considered a graph rather than a space. + * Given that NULL characters are ignored by the general FSS standard, combining characters are not considered to combine into NULL. + * + * @param buffer + * The string to process. + * @param range + * The character at the start position will be checked against the graph. + * @param state + * A state for providing flags and handling interrupts during long running operations. + * + * Must not be NULL. + * + * This alters state.status: + * F_bound character in the buffer is a joiner character (future versions will provide a more accurate status, like F_join). + * F_combarudoo character in the buffer is a combining character (future versions will provide a more accurate status, like F_combine). + * F_data_not on success but nothing is processed. + * F_false if the character in the buffer is not a combining character. + * + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_utf_is_combining(). + * Errors (with error bit) from: f_utf_is_joiner(). + * + * @return + * F_false if the character in the buffer is not a combining character. + * F_false is returned on any error. + * F_true if the character in the buffer is a combining character. + * + * @see f_utf_is_combining() + * @see f_utf_is_joiner() + */ +#ifndef _di_f_fss_is_combining_joiner_ + extern f_status_t f_fss_is_combining_joiner(const f_string_static_t buffer, const f_range_t range, f_state_t * const state); +#endif // _di_f_fss_is_combining_joiner_ + /** * Identify whether or not a character in the buffer is a graph (ASCII or UTF-8) character. * + * This does not check characters beyond the current one to check for combining or joining characters. + * * @param buffer * The string to process. * @param range @@ -228,18 +303,22 @@ extern "C" { * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: - * F_true if the character in the buffer is a graph character. + * F_data_not on success but nothing is processed. * F_false if the character in the buffer is not a graph character. + * F_true on success and is a graph character. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * * Errors (with error bit) from: f_utf_is_graph(). * * @return - * F_true if the character in the buffer is a graph character. * F_false if the character in the buffer is not a graph character. * F_false is returned on any error. + * F_true if the character in the buffer is a graph character. * * @see f_utf_is_graph() */ @@ -248,7 +327,16 @@ extern "C" { #endif // _di_f_fss_is_graph_ /** - * Identify whether or not a character in the buffer is a non-zero-width whitespace or non-zero-width control (ASCII or UTF-8) character. + * Identify whether or not a character in the buffer is a joiner (ASCII or UTF-8) character. + * + * This only checks if the given character is a combining character and does not check what this combines into. + * + * The combining characters combine from right to left. + * It is recommended to use this after testing for other characters, such as f_fss_is_space() or f_fss_is_graph(). + * A combining character can follow any character, even if it is something like a control character. + * This is unclear behavior so a simple strategy is to assume that a combining character results in a graph for anything except a non-combining zero-width character. + * U+0020 followed by U+0301 would result in the combination of the two being considered a graph rather than a space. + * Given that NULL characters are ignored by the general FSS standard, combining characters are not considered to combine into NULL. * * @param buffer * The string to process. @@ -257,9 +345,51 @@ extern "C" { * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: - * F_okay on success. + * F_bound character in the buffer is a joiner character (future versions will provide a more accurate status, like F_join). * F_data_not on success but nothing is processed. + * F_false if the character in the buffer is not a combining character. + * + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_utf_is_joiner(). + * + * @return + * F_false if the character in the buffer is not a combining character. + * F_false is returned on any error. + * F_true if the character in the buffer is a combining character. + * + * @see f_utf_is_joiner() + */ +#ifndef _di_f_fss_is_joiner_ + extern f_status_t f_fss_is_joiner(const f_string_static_t buffer, const f_range_t range, f_state_t * const state); +#endif // _di_f_fss_is_joiner_ + +/** + * Identify whether or not a character in the buffer is a non-zero-width white space or non-zero-width control (ASCII or UTF-8) character. + * + * This does not check characters beyond the current one to check for combining or joining characters. + * + * @param buffer + * The string to process. + * @param range + * The character at the start position will be checked against the graph. + * @param state + * A state for providing flags and handling interrupts during long running operations. + * + * Must not be NULL. + * + * This alters state.status: + * F_data_not on success but nothing is processed. + * F_false if the character in the buffer is not a space character. + * F_space if the character is a zero width space (F_false is returned by this function). + * F_control if the character is a control character (F_false is returned by this function when zero-width or F_true when is both a space and a control character). + * F_true on success and is a space character. + * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * * Errors (with error bit) from: f_utf_is_control(). @@ -267,9 +397,9 @@ extern "C" { * Errors (with error bit) from: f_utf_is_zero_width(). * * @return - * F_true if the character in the buffer is a space character. * F_false if the character in the buffer is not a space character. * F_false is returned on any error. + * F_true if the character in the buffer is a space character. * * @see f_utf_is_control() * @see f_utf_is_whitespace() @@ -282,6 +412,8 @@ extern "C" { /** * Identify whether or not a character in the buffer is a zero-width (ASCII or UTF-8) character. * + * This does not check characters beyond the current one to check for combining or joining characters. + * * The NULL character (U+0000) is a zero-width character. * * @param buffer @@ -293,18 +425,22 @@ extern "C" { * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: - * F_true if the character in the buffer is a zero-width character. + * F_data_not on success but nothing is processed. * F_false if the character in the buffer is not a zero-width character. + * F_true if the character in the buffer is a zero-width character. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * * Errors (with error bit) from: f_utf_is_zero_width(). * * @return - * F_true if the character in the buffer is a zero-width character. * F_false if the character in the buffer is not a zero-width character. * F_false is returned on any error. + * F_true if the character in the buffer is a zero-width character. * * @see f_utf_is_zero_width() */ @@ -325,21 +461,70 @@ extern "C" { * @param range * The start and stop positions in the buffer being processed. * This increments range->start. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. * F_okay_eos on success and EOS was reached. * F_okay_stop on success and stop point was reached. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_seek_to_eol_ extern void f_fss_seek_to_eol(const f_string_dynamic_t buffer, f_range_t * const range, f_state_t * const state); #endif // _di_f_fss_seek_to_eol_ +/** + * Skip past all combining and joiner characters. + * + * This will do nothing if the first character is not a combining character and is not a joiner character. + * + * This skips past place holders. + * This does not stop on a place holder (unless it is the end of string or the stop point). + * This stops on end of line. + * + * @param buffer + * The string to process. + * @param range + * The start and stop positions in the buffer being processed. + * This increments range->start. + * + * Must not be NULL. + * @param state + * A state for providing flags and handling interrupts during long running operations. + * + * Must not be NULL. + * + * This alters state.status: + * F_okay on success and last skipped (if any) character is neither a combining character nor a joiner character. + * F_bound on success and last skipped character is a joiner character (future versions will provide a more accurate status, like F_join). + * F_combarudoo on success and last skipped character is a combining character (future versions will provide a more accurate status, like F_combine). + * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. + * F_okay_eol on success and EOL was reached. + * F_okay_eos on success and EOS was reached. + * F_okay_stop on success and stop point was reached. + * + * F_complete_not_utf_eos (with error bit) if unable to get entire UTF-8 sequence due to EOS. + * F_complete_not_utf_start (with error bit) if the first character is a combining character. + * F_complete_not_utf_stop (with error bit) if unable to get entire UTF-8 sequence due to stop point reached. + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_utf_is_combining_joiner(). + * + * @see f_utf_is_combining_joiner() + */ +#ifndef _di_f_fss_skip_past_combining_joiner_ + extern void f_fss_skip_past_combining_joiner(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state); +#endif // _di_f_fss_skip_past_combining_joiner_ + /** * Skip past all delimit placeholders until a non-delimit placeholder is reached. * @@ -348,15 +533,20 @@ extern "C" { * @param range * The start and stop positions in the buffer being processed. * This increments range->start. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. * F_okay_eos on success and EOS was reached. * F_okay_stop on success and stop point was reached. * + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. */ #ifndef _di_f_fss_skip_past_delimit_ @@ -368,15 +558,27 @@ extern "C" { * * If the first character in the given range is a combining character, then this will not skip past anything. * This is because combining characters apply from right to left. + * The caller should check the return status and handle accordingly in this situation. + * + * White space is checked for combining characters after the white space. + * If there is a combining character after, then the white space is not consider as such. + * + * This skips past place holders. + * + * This stops on end of line. * * @param buffer * The string to process. * @param range * The start and stop positions in the buffer being processed. * This increments range->start. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. @@ -385,16 +587,17 @@ extern "C" { * F_okay_stop on success and stop point was reached. * * F_complete_not_utf_eos (with error bit) if unable to get entire UTF-8 sequence due to EOS. - * F_complete_not_utf_start (with error bit) if the first character is a combining character. + * F_complete_not_utf_start (with error bit) if the first character is a combining or a joiner character. * F_complete_not_utf_stop (with error bit) if unable to get entire UTF-8 sequence due to stop point reached. + * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * - * Errors (with error bit) from: f_utf_is_combining(). + * Errors (with error bit) from: f_utf_is_combining_joiner(). * Errors (with error bit) from: f_utf_is_control(). * Errors (with error bit) from: f_utf_is_whitespace(). * Errors (with error bit) from: f_utf_is_zero_width(). * - * @see f_utf_is_combining() + * @see f_utf_is_combining_joiner() * @see f_utf_is_control() * @see f_utf_is_whitespace() * @see f_utf_is_zero_width() @@ -403,6 +606,65 @@ extern "C" { extern void f_fss_skip_past_space(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state); #endif // _di_f_fss_skip_past_space_ +/** + * Skip past all white space, control characters, and zero-width characters, except newline '\n' (U+000A), exposing the previous position to the caller. + * + * If the first character in the given range is a combining character, then this will not skip past anything. + * This is because combining characters apply from right to left. + * The caller should check the return status and handle accordingly in this situation. + * + * White space is checked for combining characters after the white space. + * If there is a combining character after, then the white space is not consider as such. + * + * This skips past place holders. + * + * This stops on end of line. + * + * This does not initializes previous. + * This is left to the caller. + * + * @param buffer + * The string to process. + * @param range + * The start and stop positions in the buffer being processed. + * This increments range->start. + * @param state + * A state for providing flags and handling interrupts during long running operations. + * + * Must not be NULL. + * + * This alters state.status: + * F_okay on success. + * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. + * F_okay_eol on success and EOL was reached. + * F_okay_eos on success and EOS was reached. + * F_okay_stop on success and stop point was reached. + * + * F_complete_not_utf_eos (with error bit) if unable to get entire UTF-8 sequence due to EOS. + * F_complete_not_utf_start (with error bit) if the first character is a combining or a joiner character. + * F_complete_not_utf_stop (with error bit) if unable to get entire UTF-8 sequence due to stop point reached. + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_utf_is_combining_joiner(). + * Errors (with error bit) from: f_utf_is_control(). + * Errors (with error bit) from: f_utf_is_whitespace(). + * Errors (with error bit) from: f_utf_is_zero_width(). + * + * @param previous + * The previous position before + * + * Must not be NULL. + * + * @see f_utf_is_combining_joiner() + * @see f_utf_is_control() + * @see f_utf_is_whitespace() + * @see f_utf_is_zero_width() + */ +#ifndef _di_f_fss_skip_past_space_previous_ + extern void f_fss_skip_past_space_previous(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state, f_number_unsigned_t * const previous); +#endif // _di_f_fss_skip_past_space_previous_ + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_0/f_fss/c/private-fss.c b/level_0/f_fss/c/private-fss.c new file mode 100644 index 000000000..aec1220d1 --- /dev/null +++ b/level_0/f_fss/c/private-fss.c @@ -0,0 +1,183 @@ +#include "fss.h" +#include "private-fss.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#if !defined(_di_f_fss_skip_past_space_) || !defined(_di_f_fss_skip_past_space_previous_) + void private_f_fss_skip_past_space_previous(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state, f_number_unsigned_t * const previous) { + + // Skip past initial place holders. + for (;; ++range->start) { + + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + + if (range->start >= buffer.used) { + state->status = F_okay_eos; + + return; + } + + if (range->start > range->stop) { + state->status = F_okay_stop; + + return; + } + + if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) break; + } // for + + if (macro_f_utf_byte_width_is(buffer.string[range->start])) { + if (range->start + macro_f_utf_byte_width(buffer.string[range->start]) > buffer.used) { + state->status = F_status_set_error(F_complete_not_utf_eos); + + f_fss_fail_utf(state); + + return; + } + + if (range->start + (macro_f_utf_byte_width(buffer.string[range->start]) - 1) > range->stop) { + state->status = F_status_set_error(F_complete_not_utf_stop); + + f_fss_fail_utf(state); + + return; + } + } + + // Check that the first character is not a combining character or a joiner character. + state->status = f_utf_is_combining_joiner( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1 + ); + + f_fss_fail_utf_to_false(state); + + if (F_status_is_error(state->status)) return; + + if (state->status == F_true) { + state->status = F_status_set_error(F_complete_not_utf_start); + + return; + } + + for (;;) { + + if (state->interrupt) { + state->interrupt((void *) state, 0); + if (F_status_set_fine(state->status) == F_interrupt) return; + } + + if (range->start >= buffer.used) { + state->status = F_okay_eos; + + return; + } + + if (range->start > range->stop) { + state->status = F_okay_stop; + + return; + } + + if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + state->status = F_okay_eol; + + return; + } + + if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) { + ++range->start; + + continue; + } + + state->status = f_utf_is_whitespace( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1, + F_false + ); + + f_fss_fail_utf_to_false(state); + + if (state->status == F_false) { + state->status = f_utf_is_combining_joiner( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1 + ); + + f_fss_fail_utf_to_false(state); + + // This is combining, so any previous white space is no longer considered white space as such. + if (state->status == F_true) { + range->start = *previous; + + break; + } + + if (state->status == F_false) { + state->status = f_utf_is_control( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1 + ); + + f_fss_fail_utf_to_false(state); + + if (state->status == F_false) { + state->status = f_utf_is_zero_width( + buffer.string + range->start, + (range->stop - range->start) + 1 > buffer.used - range->start + ? buffer.used - range->start + : (range->stop - range->start) + 1 + ); + + f_fss_fail_utf_to_false(state); + if (state->status == F_false) break; + } + } + } + + if (F_status_is_error(state->status)) return; + + *previous = range->start; + + if (macro_f_utf_byte_width_is(buffer.string[*previous])) { + if (*previous + macro_f_utf_byte_width(buffer.string[*previous]) >= buffer.used) { + state->status = F_status_set_error(F_complete_not_utf_eos); + + f_fss_fail_utf(state); + + return; + } + + if (*previous + macro_f_utf_byte_width(buffer.string[*previous]) > range->stop) { + state->status = F_status_set_error(F_complete_not_utf_stop); + + f_fss_fail_utf(state); + + return; + } + } + + range->start += macro_f_utf_byte_width(buffer.string[*previous]); + } // for + + state->status = F_okay; + } +#endif // !defined(_di_f_fss_skip_past_space_) || !defined(_di_f_fss_skip_past_space_previous_) + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_fss/c/private-fss.h b/level_0/f_fss/c/private-fss.h new file mode 100644 index 000000000..8a4a9d568 --- /dev/null +++ b/level_0/f_fss/c/private-fss.h @@ -0,0 +1,65 @@ +/** + * FLL - Level 0 + * + * Project: File + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Provides structures and data types for a file I/O. + * Provides operations for opening/closing files. + */ +#ifndef _PRIVATE_F_fss_h +#define _PRIVATE_F_fss_h + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Private implementation of f_fss_skip_past_space_previous(). + * + * Intended to be shared to each of the different implementation variations. + * + * @param buffer + * The string to process. + * @param range + * The start and stop positions in the buffer being processed. + * This increments range->start. + * @param state + * A state for providing flags and handling interrupts during long running operations. + * + * This alters state.status: + * F_okay on success. + * F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. + * F_okay_eol on success and EOL was reached. + * F_okay_eos on success and EOS was reached. + * F_okay_stop on success and stop point was reached. + * + * F_complete_not_utf_eos (with error bit) if unable to get entire UTF-8 sequence due to EOS. + * F_complete_not_utf_start (with error bit) if the first character is a combining or a joiner character. + * F_complete_not_utf_stop (with error bit) if unable to get entire UTF-8 sequence due to stop point reached. + * F_interrupt (with error bit) if stopping due to an interrupt. + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_utf_is_combining_joiner(). + * Errors (with error bit) from: f_utf_is_control(). + * Errors (with error bit) from: f_utf_is_whitespace(). + * Errors (with error bit) from: f_utf_is_zero_width(). + * + * @see f_utf_is_combining_joiner() + * @see f_utf_is_control() + * @see f_utf_is_whitespace() + * @see f_utf_is_zero_width() + * + * @see f_fss_skip_past_space() + * @see f_fss_skip_past_space_previous() + */ +#if !defined(_di_f_fss_skip_past_space_) || !defined(_di_f_fss_skip_past_space_previous_) + extern void private_f_fss_skip_past_space_previous(const f_string_static_t buffer, f_range_t * const range, f_state_t * const state, f_number_unsigned_t * const previous) F_attribute_visibility_internal_d; +#endif // !defined(_di_f_fss_skip_past_space_) || !defined(_di_f_fss_skip_past_space_previous_) + +#ifdef __cplusplus +} // extern "C" +#endif + +#endif // _PRIVATE_F_fss_h diff --git a/level_0/f_fss/data/build/settings b/level_0/f_fss/data/build/settings index ce8230227..9614421a1 100644 --- a/level_0/f_fss/data/build/settings +++ b/level_0/f_fss/data/build/settings @@ -44,6 +44,7 @@ build_libraries_static-individual -l:libf_memory.a -l:libf_string.a -l:libf_type build_sources_headers fss.h fss/common.h fss/item.h fss/named.h fss/nest.h fss/payload.h fss/quote.h fss/set.h fss/set_quote.h fss/simple_packet.h build_sources_library fss.c fss/common.c fss/item.c fss/named.c fss/nest.c fss/payload.c fss/quote.c fss/set.c fss/set_quote.c fss/simple_packet.c +build_sources_library private-fss.c build_sources_library config.c diff --git a/level_0/f_fss/data/build/settings-mocks b/level_0/f_fss/data/build/settings-mocks index d1d865134..0cb90bfc8 100644 --- a/level_0/f_fss/data/build/settings-mocks +++ b/level_0/f_fss/data/build/settings-mocks @@ -33,6 +33,7 @@ build_libraries-individual -lf_memory -lf_string -lf_type_array -lf_utf build_sources_headers fss.h fss/common.h fss/item.h fss/named.h fss/nest.h fss/payload.h fss/quote.h fss/set.h fss/set_quote.h fss/simple_packet.h build_sources_library fss.c fss/common.c fss/item.c fss/named.c fss/nest.c fss/payload.c fss/quote.c fss/set.c fss/set_quote.c fss/simple_packet.c +build_sources_library private-fss.c build_sources_library ../../tests/unit/c/mock-fss.c build_static no diff --git a/level_0/f_fss/data/build/settings-tests b/level_0/f_fss/data/build/settings-tests index 0b6b643b8..fb9145026 100644 --- a/level_0/f_fss/data/build/settings-tests +++ b/level_0/f_fss/data/build/settings-tests @@ -25,7 +25,7 @@ build_language c build_libraries -lc -lcmocka build_libraries-individual -lf_memory -lf_string -lf_type_array -lf_utf -lf_fss -build_sources_program test-fss-apply_delimit.c test-fss-apply_delimit_range.c test-fss-count_lines.c test-fss-count_lines_range.c test-fss-fail_utf.c test-fss-fail_utf_to_false.c test-fss-is_combining.c test-fss-is_graph.c test-fss-is_space.c test-fss-is_zero_width.c test-fss-seek_to_eol.c test-fss-skip_past_delimit.c test-fss-skip_past_space.c +build_sources_program test-fss-apply_delimit.c test-fss-apply_delimit_range.c test-fss-count_lines.c test-fss-count_lines_range.c test-fss-fail_utf.c test-fss-fail_utf_to_false.c test-fss-is_combining.c test-fss-is_graph.c test-fss-is_space.c test-fss-is_zero_width.c test-fss-seek_to_eol.c test-fss-skip_past_combining_joiner.c test-fss-skip_past_delimit.c test-fss-skip_past_space.c test-fss-skip_past_space_previous.c build_sources_program test-fss-item_delete.c test-fss-item_destroy.c build_sources_program test-fss-items_delete_callback.c test-fss-items_destroy_callback.c test-fss-itemss_delete_callback.c test-fss-itemss_destroy_callback.c build_sources_program test-fss-named_delete.c test-fss-named_destroy.c diff --git a/level_0/f_fss/data/documentation/man/man3/f_fss_is_graph.3 b/level_0/f_fss/data/documentation/man/man3/f_fss_is_graph.3 index 92d7765e6..594c63102 100644 --- a/level_0/f_fss/data/documentation/man/man3/f_fss_is_graph.3 +++ b/level_0/f_fss/data/documentation/man/man3/f_fss_is_graph.3 @@ -15,7 +15,9 @@ f_fss_is_graph \- .PP Identify whether or not a character in the buffer is a graph (ASCII or UTF-8) character. .PP -This alters state.status: F_true if the character in the buffer is a graph character. F_false if the character in the buffer is not a graph character. +If the character is not a graph character but is followed by a combining character, then it is considered a graph character due to the combination result. +.PP +This alters state.status: F_true if the character in the buffer is a graph character or the character is followed by a combining character. F_false if the character in the buffer is not a graph character. .PP F_parameter (with error bit) if a parameter is invalid. .PP @@ -81,7 +83,7 @@ struct f_range_t { .RE .SH RETURN VALUE .PP -F_true if the character in the buffer is a graph character. F_false if the character in the buffer is not a graph character. F_false is returned on any error. +F_true if the character in the buffer is a graph character or the character is followed by a combining character. F_false if the character in the buffer is not a graph character. F_false is returned on any error. .SH SEE ALSO .PP .nh diff --git a/level_0/f_fss/data/documentation/man/man3/f_fss_is_space.3 b/level_0/f_fss/data/documentation/man/man3/f_fss_is_space.3 index 9f8a4865a..c5dde9ade 100644 --- a/level_0/f_fss/data/documentation/man/man3/f_fss_is_space.3 +++ b/level_0/f_fss/data/documentation/man/man3/f_fss_is_space.3 @@ -15,6 +15,8 @@ f_fss_is_space \- .PP Identify whether or not a character in the buffer is a non-zero-width whitespace or non-zero-width control (ASCII or UTF-8) character. .PP +A valid white space (or control character) that is followed by a combining character is considered a non-space unless that control character is a new line. The new line is an exception due to the current handling of that character with combining characters as observed in common editors. +.PP This alters state.status: F_okay on success. F_data_not on success but nothing is processed. F_parameter (with error bit) if a parameter is invalid. .PP Errors (with error bit) from: f_utf_is_control(). Errors (with error bit) from: f_utf_is_whitespace(). Errors (with error bit) from: f_utf_is_zero_width(). @@ -79,7 +81,7 @@ struct f_range_t { .RE .SH RETURN VALUE .PP -F_true if the character in the buffer is a space character. F_false if the character in the buffer is not a space character. F_false is returned on any error. +F_true if the character in the buffer is a space character (and the following character, if any, is not a combining character). F_false if the character in the buffer is not a space character. F_false is returned on any error. .SH SEE ALSO .PP .nh diff --git a/level_0/f_fss/data/documentation/man/man3/f_fss_skip_past_space_previous.3 b/level_0/f_fss/data/documentation/man/man3/f_fss_skip_past_space_previous.3 new file mode 100644 index 000000000..82d1687d1 --- /dev/null +++ b/level_0/f_fss/data/documentation/man/man3/f_fss_skip_past_space_previous.3 @@ -0,0 +1,97 @@ +.TH f_fss_skip_past_space_previous "3" "January 2026" "FLL - Featureless Linux Library 0.8.2 - Level 0" "Library Functions" +.SH "NAME" +f_fss_skip_past_space_previous \- +.SH SYNOPSIS +.nf +.B #include +.sp +\fBvoid f_fss_skip_past_space_previous\fP( + \fBconst f_string_static_t \fP\fIbuffer\fP, + \fBf_range_t *const \fP\fIrange\fP, + \fBf_state_t *const \fP\fIstate\fP +); +.fi +.SH DESCRIPTION +.PP +Skip past all white space, control characters, and zero-width characters, except newline ' +.PP +' (U+000A), exposing the previous position to the caller. +.PP +If the first character in the given range is a combining character, then this will not skip past anything. This is because combining characters apply from right to left. +.PP +This alters state.status: F_okay on success. F_data_not on success but buffer.used is 0, initial range.start is greater than range.stop, or initial range.start is greater than or equal to buffer.used. F_okay_eol on success and EOL was reached. F_okay_eos on success and EOS was reached. F_okay_stop on success and stop point was reached. +.PP +F_complete_not_utf_eos (with error bit) if unable to get entire UTF-8 sequence due to EOS. F_complete_not_utf_start (with error bit) if the first character is a combining character. F_complete_not_utf_stop (with error bit) if unable to get entire UTF-8 sequence due to stop point reached. F_parameter (with error bit) if a parameter is invalid. +.PP +Errors (with error bit) from: f_utf_is_combining(). Errors (with error bit) from: f_utf_is_control(). Errors (with error bit) from: f_utf_is_whitespace(). Errors (with error bit) from: f_utf_is_zero_width(). +.SH PARAMETERS +.TP +.B buffer +The string to process. + +.TP +.B range +The start and stop positions in the buffer being processed. This increments range->start. + +.TP +.B state +A state for providing flags and handling interrupts during long running operations. + +.SH STRUCTURES +.SS "" +.PP +.sp +.PP +A string that is analogous to f_string_dynamic_t but intended for static-only uses. +.PP +The f_string_static_t type should always have size set to 0 to represent that it is a static string. This allows for easier integration with f_string_dynamic_t. +.PP +If size is set to some value other than 0 for a static string, then take special care not to intermix it with f_string_dynamic_t. +.PP +A special macro_f_string_static_t_initialize_1() is provided to easily initialize a static string. A special macro_f_string_static_t_initialize_2() is provided to easily initialize a static string with an identical used and size. +.PP +size: Total amount of space available. +.sp +.RS +.nf +\fB +struct f_string_static_t { + f_string_t \fIstring\fP; + f_number_unsigned_t \fIused\fP; + f_number_unsigned_t \fIsize\fP; +}; +\fP +.fi +.RE +.SS "" +.PP +.sp +.PP +A structure designating a start and stop range for some string. +.PP +In general, this project uses the start and stop position inclusively, meaning that a range of 0 to 1 would include positions 0 and position 1. Therefore, a range from 0 to 0 would be include position 0. Set start to some value larger than stop to designate that there is no range (such as start = 1, stop = 0). +.PP +stop: The inclusive stop position. +.sp +.RS +.nf +\fB +struct f_range_t { + f_number_unsigned_t \fIstart\fP; + f_number_unsigned_t \fIstop\fP; +}; +\fP +.fi +.RE +.SH SEE ALSO +.PP +.nh +.ad l +\fIfss.h\fP(3), \fIf_fss_apply_delimit\fP(3), \fIf_fss_apply_delimit_range\fP(3), \fIf_fss_count_lines\fP(3), \fIf_fss_count_lines_range\fP(3), \fIf_fss_fail_utf\fP(3), \fIf_fss_fail_utf_to_false\fP(3), \fIf_fss_is_combining\fP(3), \fIf_fss_is_graph\fP(3), \fIf_fss_is_space\fP(3), \fIf_fss_is_zero_width\fP(3), \fIf_fss_seek_to_eol\fP(3), \fIf_fss_skip_past_delimit\fP(3), \fIf_fss_skip_past_space_previous\fP(3), \fIf_utf_is_combining()\fP, \fIf_utf_is_control()\fP, \fIf_utf_is_whitespace()\fP, \fIf_utf_is_zero_width()\fP, \fIf_utf_is_combining()\fP, \fIf_utf_is_control()\fP, \fIf_utf_is_whitespace()\fP, \fIf_utf_is_zero_width()\fP +.ad +.hy +.SH AUTHOR +Written by Kevin Day. +.SH COPYRIGHT +.PP +Copyright \(co 2007-2026 Kevin Day, GNU LGPL Version 2.1 or later. diff --git a/level_0/f_fss/tests/unit/c/test-fss-is_space.c b/level_0/f_fss/tests/unit/c/test-fss-is_space.c index b8ee291b6..aadd8db3e 100644 --- a/level_0/f_fss/tests/unit/c/test-fss-is_space.c +++ b/level_0/f_fss/tests/unit/c/test-fss-is_space.c @@ -10,18 +10,21 @@ void test__f_fss_is_space__works(void **state) { f_state_t state_data = f_state_t_initialize; f_string_static_t tests[] = { - macro_f_string_static_t_initialize_1("\0", 1, 0), - macro_f_string_static_t_initialize_1("\a", 1, 0), - macro_f_string_static_t_initialize_1("\t", 1, 0), - macro_f_string_static_t_initialize_1("\n", 1, 0), - macro_f_string_static_t_initialize_1(" ", 1, 0), - macro_f_string_static_t_initialize_1("⸙", 3, 0), - macro_f_string_static_t_initialize_1("t", 1, 0), - macro_f_string_static_t_initialize_1("全", 3, 0), - macro_f_string_static_t_initialize_1("$", 1, 0), - macro_f_string_static_t_initialize_1(".", 1, 0), - macro_f_string_static_t_initialize_1(" ́", 3, 0), // Space followed by a combining character (U+0020 U+0301). - macro_f_string_static_t_initialize_1("​", 3, 0), // Zero-width space (U+200B). + macro_f_string_static_t_initialize_1("\0", 1, 0), + macro_f_string_static_t_initialize_1("\a", 1, 0), + macro_f_string_static_t_initialize_1("\t", 1, 0), + macro_f_string_static_t_initialize_1("\n", 1, 0), + macro_f_string_static_t_initialize_1(" ", 1, 0), + macro_f_string_static_t_initialize_1("⸙", 3, 0), + macro_f_string_static_t_initialize_1("t", 1, 0), + macro_f_string_static_t_initialize_1("全", 3, 0), + macro_f_string_static_t_initialize_1("$", 1, 0), + macro_f_string_static_t_initialize_1(".", 1, 0), + macro_f_string_static_t_initialize_1(" ́", 3, 0), // Space followed by a combining character (U+0020 U+0301). + macro_f_string_static_t_initialize_1("​", 3, 0), // Zero-width space (U+200B). + macro_f_string_static_t_initialize_1(" ‍🧔", 8, 0), // Space followed by a zero-width joiner and a joined character (U+0020 U+200D U+1F9D4). + macro_f_string_static_t_initialize_1(" ⁠b", 5, 0), // Space followed by a word joiner and a joined character (U+0020 U+2060 U+0062). + macro_f_string_static_t_initialize_1("test", 4, 0), }; f_status_t expects[] = { @@ -37,14 +40,21 @@ void test__f_fss_is_space__works(void **state) { F_false, F_true, F_false, + F_true, + F_true, + F_false, }; - for (f_number_unsigned_t i = 0; i < 12; ++i) { + for (uint8_t i = 0; i < 15; ++i) { const f_range_t range = macro_f_range_t_initialize_1(0, tests[i].used - 1); const f_status_t status = f_fss_is_space(tests[i], range, &state_data); + if (status != expects[i]) { + printf("[ -------> ] --- [00] Expected %s at index %d for string '%s'.\n", expects[i] ? "true" : "false", i, tests[i].used ? tests[i].string : ""); + } + assert_int_equal(status, expects[i]); } // for @@ -53,7 +63,17 @@ void test__f_fss_is_space__works(void **state) { const f_status_t status = f_fss_is_space(f_string_empty_s, range, &state_data); + if (status) { + printf("[ -------> ] --- [01] Expected true for empty string with range (%lu, %lu).\n", range.start, range.stop); + } + assert_int_equal(status, F_false); + + if (state_data.status != F_data_not) { + printf("[ -------> ] --- [02] Expected F_data_not in state.status for empty string with range (%lu, %lu).\n", range.start, range.stop); + } + + assert_int_equal(state_data.status, F_data_not); } { @@ -61,16 +81,17 @@ void test__f_fss_is_space__works(void **state) { const f_status_t status = f_fss_is_space(f_string_empty_s, range, &state_data); - assert_int_equal(status, F_false); - } + if (status) { + printf("[ -------> ] --- [03] Expected true for empty string with (out of) range (%lu, %lu).\n", range.start, range.stop); + } - { - const f_string_static_t test = macro_f_string_static_t_initialize_1("test", 4, 0); - const f_range_t range = f_range_t_initialize; + assert_int_equal(status, F_false); - const f_status_t status = f_fss_is_space(test, range, &state_data); + if (state_data.status != F_data_not) { + printf("[ -------> ] --- [04] Expected F_data_not in state.status for empty string with (out of) range (%lu, %lu).\n", range.start, range.stop); + } - assert_int_equal(status, F_false); + assert_int_equal(state_data.status, F_data_not); } } diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.c b/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.c new file mode 100644 index 000000000..90c0ffb2f --- /dev/null +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.c @@ -0,0 +1,183 @@ +#include "test-fss.h" +#include "test-fss-skip_past_combining_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_fss_skip_past_combining_joiner__parameter_checking(void **state) { + + const f_string_static_t test = macro_f_string_static_t_initialize_1("test", 4, 0); + f_state_t state_data = f_state_t_initialize; + + { + f_fss_skip_past_combining_joiner(test, 0, &state_data); + + assert_int_equal(state_data.status, F_status_set_error(F_parameter)); + } +} + +void test__f_fss_skip_past_combining_joiner__returns_data_not(void **state) { + + const f_string_static_t test = macro_f_string_static_t_initialize_1("test", 4, 0); + f_state_t state_data = f_state_t_initialize; + + { + f_range_t range = f_range_t_initialize; + + f_fss_skip_past_combining_joiner(f_string_empty_s, &range, &state_data); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = f_range_t_initialize; + + f_fss_skip_past_combining_joiner(test, &range, &state_data); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = macro_f_range_t_initialize_1(0, test.used - 1); + + f_fss_skip_past_combining_joiner(f_string_empty_s, &range, &state_data); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = macro_f_range_t_initialize_1(test.used, test.used + 2); + + f_fss_skip_past_combining_joiner(test, &range, &state_data); + + assert_int_equal(state_data.status, F_data_not); + } +} + +void test__f_fss_skip_past_combining_joiner__works(void **state) { + + f_state_t state_data = f_state_t_initialize; + + f_string_static_t tests[] = { + + // Set 1 + macro_f_string_static_t_initialize_1("́", 2, 0), + macro_f_string_static_t_initialize_1("\0́", 3, 0), + macro_f_string_static_t_initialize_1("\0\0\0́", 5, 0), + macro_f_string_static_t_initialize_1("\0\0\0́\0\0", 7, 0), + + // Set 2 + macro_f_string_static_t_initialize_1("́.", 3, 0), + macro_f_string_static_t_initialize_1("\0̃.", 4, 0), + macro_f_string_static_t_initialize_1("\0\0\0̃.", 6, 0), + macro_f_string_static_t_initialize_1("\0\0\0̃\0\0.", 8, 0), + + // Set 3 + macro_f_string_static_t_initialize_1("̠.", 3, 0), + macro_f_string_static_t_initialize_1("\0̠.", 4, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠.", 6, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠\0\0.", 8, 0), + + // Set 4 + macro_f_string_static_t_initialize_1("̠́.", 5, 0), + macro_f_string_static_t_initialize_1("\0̠́.", 6, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠́.", 8, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠́\0\0.", 10, 0), + + // Set 5 + macro_f_string_static_t_initialize_1("̠́́.", 7, 0), + macro_f_string_static_t_initialize_1("\0̠́́.", 8, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠́́.", 10, 0), + macro_f_string_static_t_initialize_1("\0\0\0̠́́\0\0.", 12, 0), + }; + + uint16_t statuss[] = { + + // Set 1 + F_okay_eos, + F_okay_eos, + F_okay_eos, + F_okay_eos, + + // Set 2 + F_okay, + F_okay, + F_okay, + F_okay, + + // Set 3 + F_okay, + F_okay, + F_okay, + F_okay, + + // Set 4 + F_okay, + F_okay, + F_okay, + F_okay, + + // Set 5 + F_okay, + F_okay, + F_okay, + F_okay, + }; + + f_number_unsigned_t ranges[] = { + + // Set 1 + 2, + 3, + 5, + 7, + + // Set 2 + 2, + 3, + 5, + 7, + + // Set 3 + 2, + 3, + 5, + 7, + + // Set 4 + 4, + 5, + 7, + 9, + + // Set 5 + 6, + 7, + 9, + 11, + }; + + for (uint8_t i = 0; i < 20; ++i) { + + f_range_t range = macro_f_range_t_initialize_1(0, tests[i].used - 1); + + f_fss_skip_past_combining_joiner(tests[i], &range, &state_data); + + if (state_data.status != statuss[i]) { + printf("[ -------> ] --- [00] Expected %u in state.status but got %u at index %d for string '%s'.\n", statuss[i], state_data.status, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(state_data.status, statuss[i]); + + if (range.start != ranges[i]) { + printf("[ -------> ] --- [01] Expected %lu in range.start but got %lu at index %d for string '%s'.\n", ranges[i], range.start, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(range.start, ranges[i]); + } // for +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.h b/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.h new file mode 100644 index 000000000..c4f7adfab --- /dev/null +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_combining_joiner.h @@ -0,0 +1,34 @@ +/** + * FLL - Level 0 + * + * Project: FSS + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test skip past combining joiner in the fss project. + */ +#ifndef _TEST__F_fss__skip_past_combining_joiner +#define _TEST__F_fss__skip_past_combining_joiner + +/** + * Test that the function correctly fails on invalid parameter. + * + * @see f_fss_skip_past_combining_joiner() + */ +extern void test__f_fss_skip_past_combining_joiner__parameter_checking(void **state); + +/** + * Test that the function returns F_data_not. + * + * @see f_fss_skip_past_combining_joiner() + */ +extern void test__f_fss_skip_past_combining_joiner__returns_data_not(void **state); + +/** + * Test that the function works. + * + * @see f_fss_skip_past_combining_joiner() + */ +extern void test__f_fss_skip_past_combining_joiner__works(void **state); + +#endif // _TEST__F_fss__skip_past_combining_joiner diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_delimit.c b/level_0/f_fss/tests/unit/c/test-fss-skip_past_delimit.c index 121189d2e..790aef579 100644 --- a/level_0/f_fss/tests/unit/c/test-fss-skip_past_delimit.c +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_delimit.c @@ -66,7 +66,14 @@ void test__f_fss_skip_past_delimit__works(void **state) { macro_f_string_static_t_initialize_1("\0\0\0test\0\0", 9, 0), }; - f_number_unsigned_t expects[] = { + uint16_t statuss[] = { + F_okay, + F_okay, + F_okay, + F_okay, + }; + + f_number_unsigned_t ranges[] = { 0, 1, 3, @@ -79,8 +86,16 @@ void test__f_fss_skip_past_delimit__works(void **state) { f_fss_skip_past_delimit(tests[i], &range, &state_data); - assert_int_equal(state_data.status, F_okay); - assert_int_equal(range.start, expects[i]); + if (state_data.status != statuss[i]) { + printf("[ -------> ] --- [00] Expected %u in state.status but got %u at index %d for string '%s'.\n", statuss[i], state_data.status, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(state_data.status, statuss[i]); + + if (range.start != ranges[i]) { + printf("[ -------> ] --- [01] Expected %lu in range.start but got %lu at index %d for string '%s'.\n", ranges[i], range.start, i, tests[i].used ? tests[i].string : ""); + } + } // for } diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_space.c b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space.c index e61026ed4..95ddd01cc 100644 --- a/level_0/f_fss/tests/unit/c/test-fss-skip_past_space.c +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space.c @@ -59,34 +59,77 @@ void test__f_fss_skip_past_space__works(void **state) { f_state_t state_data = f_state_t_initialize; + // Space followed by a zero-width joiner and a joined character with NULLs before joiner (U+0020 \0 \0 U+200D U+1F9D4). + char space_null_join[] = { 0x20, 0x0, 0x0, 0xe2, 0x80, 0x8d, 0xf0, 0x9f, 0xa7, 0x94 }; + f_string_static_t tests[] = { - macro_f_string_static_t_initialize_1("test", 4, 0), - macro_f_string_static_t_initialize_1(" test", 5, 0), - macro_f_string_static_t_initialize_1(" test", 7, 0), - macro_f_string_static_t_initialize_1(" \0test ", 9, 0), - macro_f_string_static_t_initialize_1(" \t\atest ", 9, 0), - macro_f_string_static_t_initialize_1("\a\t $test ", 10, 0), - macro_f_string_static_t_initialize_1(" ́test", 7, 0), + macro_f_string_static_t_initialize_1("test", 4, 0), + macro_f_string_static_t_initialize_1(" test", 5, 0), + macro_f_string_static_t_initialize_1(" test", 7, 0), + macro_f_string_static_t_initialize_1(" \0test ", 9, 0), + macro_f_string_static_t_initialize_1(" \t\atest ", 9, 0), + macro_f_string_static_t_initialize_1("\a\t $test ", 10, 0), + macro_f_string_static_t_initialize_1(" ́test", 7, 0), // Space followed by a combining character (U+0020 U+0301). + macro_f_string_static_t_initialize_1("​test", 7, 0), // Starts with zero-width space (U+200B). + macro_f_string_static_t_initialize_1(" ‍🧔test", 12, 0), // Space followed by a zero-width joiner and a joined character (U+0020 U+200D U+1F9D4). + macro_f_string_static_t_initialize_1(" ⁠test", 8, 0), // Space followed by a word joiner and a joined character (U+0020 U+2060 U+0062). + macro_f_string_static_t_initialize_1("́⁠test", 6, 0), // Starts with combining character (U+0301). + macro_f_string_static_t_initialize_1("‍́test", 6, 0), // Starts with joiner character (U+2060). + macro_f_string_static_t_initialize_1(" ‍\0\0🧔test", 14, 0), // Space followed by a zero-width joiner and a joined character with NULLs after joiner (U+0020 U+200D \0 \0 U+1F9D4). + macro_f_string_static_t_initialize_1(space_null_join, 14, 0), + }; + + uint16_t statuss[] = { + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_status_set_error(F_complete_not_utf_start), + F_status_set_error(F_complete_not_utf_start), + F_okay, + F_okay, }; - f_number_unsigned_t expects[] = { + f_number_unsigned_t ranges[] = { 0, 1, 3, 3, 3, 3, + 0, 3, + 0, + 0, + 0, + 0, + 0, + 0, }; - for (uint8_t i = 0; i < 7; ++i) { + for (uint8_t i = 0; i < 14; ++i) { f_range_t range = macro_f_range_t_initialize_1(0, tests[i].used - 1); f_fss_skip_past_space(tests[i], &range, &state_data); - assert_int_equal(state_data.status, F_okay); - assert_int_equal(range.start, expects[i]); + if (state_data.status != statuss[i]) { + printf("[ -------> ] --- [00] Expected %u in state.status but got %u at index %d for string '%s'.\n", statuss[i], state_data.status, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(state_data.status, statuss[i]); + + if (range.start != ranges[i]) { + printf("[ -------> ] --- [01] Expected %lu in range.start but got %lu at index %d for string '%s'.\n", ranges[i], range.start, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(range.start, ranges[i]); } // for } diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.c b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.c new file mode 100644 index 000000000..e62531f79 --- /dev/null +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.c @@ -0,0 +1,156 @@ +#include "test-fss.h" +#include "test-fss-skip_past_space_previous.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_fss_skip_past_space_previous__parameter_checking(void **state) { + + const f_string_static_t test = macro_f_string_static_t_initialize_1("test", 4, 0); + f_state_t state_data = f_state_t_initialize; + + { + f_fss_skip_past_space_previous(test, 0, &state_data, 0); + + assert_int_equal(state_data.status, F_status_set_error(F_parameter)); + } + + { + f_number_unsigned_t previous = 0; + + f_fss_skip_past_space_previous(test, 0, &state_data, &previous); + + assert_int_equal(state_data.status, F_status_set_error(F_parameter)); + } + + { + f_range_t range = f_range_t_initialize; + + f_fss_skip_past_space_previous(test, &range, &state_data, 0); + + assert_int_equal(state_data.status, F_status_set_error(F_parameter)); + } +} + +void test__f_fss_skip_past_space_previous__returns_data_not(void **state) { + + const f_string_static_t test = macro_f_string_static_t_initialize_1("test", 4, 0); + f_state_t state_data = f_state_t_initialize; + f_number_unsigned_t previous = 0; + + { + f_range_t range = f_range_t_initialize; + + f_fss_skip_past_space_previous(f_string_empty_s, &range, &state_data, &previous); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = f_range_t_initialize; + + f_fss_skip_past_space_previous(test, &range, &state_data, &previous); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = macro_f_range_t_initialize_1(0, test.used - 1); + + f_fss_skip_past_space_previous(f_string_empty_s, &range, &state_data, &previous); + + assert_int_equal(state_data.status, F_data_not); + } + + { + f_range_t range = macro_f_range_t_initialize_1(test.used, test.used + 2); + + f_fss_skip_past_space_previous(test, &range, &state_data, &previous); + + assert_int_equal(state_data.status, F_data_not); + } +} + +void test__f_fss_skip_past_space_previous__works(void **state) { + + f_state_t state_data = f_state_t_initialize; + + // Space followed by a zero-width joiner and a joined character with NULLs before joiner (U+0020 \0 \0 U+200D U+1F9D4). + char space_null_join[] = { 0x20, 0x0, 0x0, 0xe2, 0x80, 0x8d, 0xf0, 0x9f, 0xa7, 0x94 }; + + f_string_static_t tests[] = { + macro_f_string_static_t_initialize_1("test", 4, 0), + macro_f_string_static_t_initialize_1(" test", 5, 0), + macro_f_string_static_t_initialize_1(" test", 7, 0), + macro_f_string_static_t_initialize_1(" \0test ", 9, 0), + macro_f_string_static_t_initialize_1(" \t\atest ", 9, 0), + macro_f_string_static_t_initialize_1("\a\t $test ", 10, 0), + macro_f_string_static_t_initialize_1(" ́test", 7, 0), // Space followed by a combining character (U+0020 U+0301). + macro_f_string_static_t_initialize_1("​test", 7, 0), // Starts with zero-width space (U+200B). + macro_f_string_static_t_initialize_1(" ‍🧔test", 12, 0), // Space followed by a zero-width joiner and a joined character (U+0020 U+200D U+1F9D4). + macro_f_string_static_t_initialize_1(" ⁠test", 8, 0), // Space followed by a word joiner and a joined character (U+0020 U+2060 U+0062). + macro_f_string_static_t_initialize_1("́⁠test", 6, 0), // Starts with combining character (U+0301). + macro_f_string_static_t_initialize_1("‍́test", 6, 0), // Starts with joiner character (U+2060). + macro_f_string_static_t_initialize_1(" ‍\0\0🧔test", 14, 0), // Space followed by a zero-width joiner and a joined character with NULLs after joiner (U+0020 U+200D \0 \0 U+1F9D4). + macro_f_string_static_t_initialize_1(space_null_join, 14, 0), + }; + + uint16_t statuss[] = { + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_okay, + F_status_set_error(F_complete_not_utf_start), + F_status_set_error(F_complete_not_utf_start), + F_okay, + F_okay, + }; + + f_number_unsigned_t ranges[] = { + 0, + 1, + 3, + 3, + 3, + 3, + 0, + 3, + 0, + 0, + 0, + 0, + 0, + 0, + }; + + for (uint8_t i = 0; i < 14; ++i) { + + f_range_t range = macro_f_range_t_initialize_1(0, tests[i].used - 1); + f_number_unsigned_t previous = 0; + + f_fss_skip_past_space_previous(tests[i], &range, &state_data, &previous); + + if (state_data.status != statuss[i]) { + printf("[ -------> ] --- [00] Expected %u in state.status but got %u at index %d for string '%s'.\n", statuss[i], state_data.status, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(state_data.status, statuss[i]); + + if (range.start != ranges[i]) { + printf("[ -------> ] --- [01] Expected %lu in range.start but got %lu at index %d for string '%s'.\n", ranges[i], range.start, i, tests[i].used ? tests[i].string : ""); + } + + assert_int_equal(range.start, ranges[i]); + } // for +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.h b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.h new file mode 100644 index 000000000..db76dce6f --- /dev/null +++ b/level_0/f_fss/tests/unit/c/test-fss-skip_past_space_previous.h @@ -0,0 +1,34 @@ +/** + * FLL - Level 0 + * + * Project: FSS + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test skip past space previous in the fss project. + */ +#ifndef _TEST__F_fss__skip_past_space_previous +#define _TEST__F_fss__skip_past_space_previous + +/** + * Test that the function correctly fails on invalid parameter. + * + * @see f_fss_skip_past_space_previous() + */ +extern void test__f_fss_skip_past_space_previous__parameter_checking(void **state); + +/** + * Test that the function returns F_data_not. + * + * @see f_fss_skip_past_space_previous() + */ +extern void test__f_fss_skip_past_space_previous__returns_data_not(void **state); + +/** + * Test that the function works. + * + * @see f_fss_skip_past_space_previous() + */ +extern void test__f_fss_skip_past_space_previous__works(void **state); + +#endif // _TEST__F_fss__skip_past_space_previous diff --git a/level_0/f_fss/tests/unit/c/test-fss.c b/level_0/f_fss/tests/unit/c/test-fss.c index 999dbb08b..823e15ab2 100644 --- a/level_0/f_fss/tests/unit/c/test-fss.c +++ b/level_0/f_fss/tests/unit/c/test-fss.c @@ -24,8 +24,10 @@ int main(void) { cmocka_unit_test(test__f_fss_count_lines__returns_data_not), cmocka_unit_test(test__f_fss_count_lines_range__returns_data_not), cmocka_unit_test(test__f_fss_seek_to_eol__returns_data_not), + cmocka_unit_test(test__f_fss_skip_past_combining_joiner__returns_data_not), cmocka_unit_test(test__f_fss_skip_past_delimit__returns_data_not), cmocka_unit_test(test__f_fss_skip_past_space__returns_data_not), + cmocka_unit_test(test__f_fss_skip_past_space_previous__returns_data_not), cmocka_unit_test(test__f_fss_apply_delimit__works), cmocka_unit_test(test__f_fss_apply_delimit_range__works), @@ -36,8 +38,10 @@ int main(void) { cmocka_unit_test(test__f_fss_is_space__works), cmocka_unit_test(test__f_fss_is_zero_width__works), cmocka_unit_test(test__f_fss_seek_to_eol__works), + cmocka_unit_test(test__f_fss_skip_past_combining_joiner__works), cmocka_unit_test(test__f_fss_skip_past_delimit__works), cmocka_unit_test(test__f_fss_skip_past_space__works), + cmocka_unit_test(test__f_fss_skip_past_space_previous__works), cmocka_unit_test(test__f_fss_fail_utf__works_for_error), cmocka_unit_test(test__f_fss_fail_utf_to_false__works_for_error), @@ -152,8 +156,10 @@ int main(void) { // f_fss_is_space() doesn't use parameter checking. // f_fss_is_zero_width() doesn't use parameter checking. cmocka_unit_test(test__f_fss_seek_to_eol__parameter_checking), + cmocka_unit_test(test__f_fss_skip_past_combining_joiner__parameter_checking), cmocka_unit_test(test__f_fss_skip_past_delimit__parameter_checking), cmocka_unit_test(test__f_fss_skip_past_space__parameter_checking), + cmocka_unit_test(test__f_fss_skip_past_space_previous__parameter_checking), cmocka_unit_test(test__f_fss_item_delete__parameter_checking), cmocka_unit_test(test__f_fss_item_destroy__parameter_checking), diff --git a/level_0/f_fss/tests/unit/c/test-fss.h b/level_0/f_fss/tests/unit/c/test-fss.h index 6a40e1413..451ac6fe5 100644 --- a/level_0/f_fss/tests/unit/c/test-fss.h +++ b/level_0/f_fss/tests/unit/c/test-fss.h @@ -76,8 +76,10 @@ #include "test-fss-simple_packets_destroy_callback.h" #include "test-fss-simple_packetss_delete_callback.h" #include "test-fss-simple_packetss_destroy_callback.h" +#include "test-fss-skip_past_combining_joiner.h" #include "test-fss-skip_past_delimit.h" #include "test-fss-skip_past_space.h" +#include "test-fss-skip_past_space_previous.h" #ifdef __cplusplus extern "C" { diff --git a/level_0/f_parse/c/parse.c b/level_0/f_parse/c/parse.c index 94f9a8cb5..b0ca30e95 100644 --- a/level_0/f_parse/c/parse.c +++ b/level_0/f_parse/c/parse.c @@ -38,7 +38,7 @@ extern "C" { } else { if (range->start + width > range->stop) return F_status_set_error(F_complete_not_utf_stop); - if (range->start + width >= buffer.used) return F_status_set_error(F_complete_not_utf_eos); + if (range->start + width > buffer.used) return F_status_set_error(F_complete_not_utf_eos); if (width == seek_width) { f_utf_char_t character = 0; @@ -91,7 +91,7 @@ extern "C" { } else { if (range->start + width > range->stop) return F_status_set_error(F_complete_not_utf_stop); - if (range->start + width >= buffer.used) return F_status_set_error(F_complete_not_utf_eos); + if (range->start + width > buffer.used) return F_status_set_error(F_complete_not_utf_eos); } range->start += width; @@ -140,7 +140,7 @@ extern "C" { } else { if (range->start + width > range->stop) return F_status_set_error(F_complete_not_utf_stop); - if (range->start + width >= buffer.used) return F_status_set_error(F_complete_not_utf_eos); + if (range->start + width > buffer.used) return F_status_set_error(F_complete_not_utf_eos); } range->start += width; @@ -194,7 +194,7 @@ extern "C" { } else { if (range->start + width > range->stop) return F_status_set_error(F_complete_not_utf_stop); - if (range->start + width >= buffer.used) return F_status_set_error(F_complete_not_utf_eos); + if (range->start + width > buffer.used) return F_status_set_error(F_complete_not_utf_eos); if (width == seek_width) { f_utf_char_t character = 0; @@ -230,105 +230,71 @@ extern "C" { if (buffer.string[range->start]) break; } // for - if (range->start > range->stop) { - return F_data_not; - } + if (range->start > range->stop) return F_data_not; + + f_status_t status = F_okay; // The FLL Identifier is always at least 6 characters ("X-0000"). if (range->stop - range->start < 5) { - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop && range->start < buffer.used; ) { - - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - - break; - } - - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for - - return F_found_not; + status = F_found_not; } - f_status_t status = F_okay; - - for (; range->start <= range->stop && range->start < buffer.used; ) { - - status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); + if (status != F_found_not) { - if (F_status_is_error(status)) { - if (F_status_set_fine(status) == F_maybe) return F_status_set_error(F_complete_not_utf); + // Seek past initial white space. + for (; range->start <= range->stop && range->start < buffer.used; ) { - return status; - } + status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); - if (status == F_false) { - if (!buffer.string[range->start]) { - ++range->start; + if (F_status_is_error(status)) { + if (F_status_set_fine(status) == F_maybe) return F_status_set_error(F_complete_not_utf); - continue; + return status; } - break; - } - - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - - return F_found_not; - } - - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for - - if (range->start > range->stop || range->start > buffer.used) return F_found_not; + if (status == F_false) { + if (!buffer.string[range->start]) { + ++range->start; - if (range->stop - range->start < 5) { + continue; + } - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop && range->start < buffer.used; ) { + break; + } if (buffer.string[range->start] == f_string_eol_s.string[0]) { ++range->start; - break; + return F_found_not; } range->start += macro_f_utf_byte_width(buffer.string[range->start]); } // for - return F_found_not; - } - - f_number_unsigned_t i = range->start; - - for (; range->start <= range->stop && range->start < buffer.used; ) { - - status = f_utf_is_word(buffer.string + range->start, (range->stop - range->start) + 1, F_true); - if (F_status_is_error(status)) return status; - - if (status == F_false) { - if (!buffer.string[range->start]) { - ++range->start; - - continue; - } + if (range->start > range->stop || range->start > buffer.used) return F_found_not; - break; + // Check minimum required length (a difference of 5 represents the 6 minimum characters because start and stop are inclusive). + if (range->stop - range->start < 5) { + status = F_found_not; } + } - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for - - if (range->start > range->stop || range->start >= buffer.used || buffer.string[range->start] != '-') { + if (status != F_found_not) { + f_number_unsigned_t i = range->start; - // Increment until stop, while taking into consideration UTF-8 character widths. + // Only word characters are allowed on the left side of the dash. for (; range->start <= range->stop && range->start < buffer.used; ) { - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; + status = f_utf_is_word(buffer.string + range->start, (range->stop - range->start) + 1, F_true); + if (F_status_is_error(status)) return status; + + if (status == F_false) { + if (!buffer.string[range->start]) { + ++range->start; + + continue; + } break; } @@ -336,131 +302,136 @@ extern "C" { range->start += macro_f_utf_byte_width(buffer.string[range->start]); } // for - return F_found_not; - } + if (range->start > range->stop || range->start >= buffer.used || buffer.string[range->start] != f_string_ascii_minus_s.string[0]) { + status = F_found_not; + } - { - f_number_unsigned_t j = 0; - f_char_t number[5] = { 0, 0, 0, 0, 0 }; + if (status != F_found_not) { + f_number_unsigned_t j = 0; + f_char_t number[5] = { 0, 0, 0, 0, 0 }; - for (++range->start; range->start <= range->stop && range->start < buffer.used && j < 4; ++range->start, ++j) { + // Only 4 hexidecimal ASCII digits are allowed after the dash. + for (++range->start; range->start <= range->stop && range->start < buffer.used && j < 4; ++range->start, ++j) { - // The hexidecimal representing the number may only be ASCII. - if (macro_f_utf_byte_width_is(buffer.string[range->start])) { + // The hexidecimal representing the number may only be ASCII. + if (macro_f_utf_byte_width_is(buffer.string[range->start])) { - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop && range->start < buffer.used; ) { + // Increment until stop, while taking into consideration UTF-8 character widths. + for (; range->start <= range->stop && range->start < buffer.used; ) { - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; + if (buffer.string[range->start] == f_string_eol_s.string[0]) { + ++range->start; - break; - } + break; + } - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for + range->start += macro_f_utf_byte_width(buffer.string[range->start]); + } // for - if (id) { - id->type = 0; + if (id) { + id->type = 0; + } + + return F_found_not; } - return F_found_not; - } + if ((uint16_t) (buffer.string[range->start] - f_string_ascii_a_s.string[0]) < 6 || (uint16_t) (buffer.string[range->start] - f_string_ascii_A_s.string[0]) < 6 || (uint16_t) (buffer.string[range->start] - f_string_ascii_0_s.string[0]) < 10) { + number[j] = buffer.string[range->start]; + } + else { + if (!buffer.string[range->start]) continue; - if ((uint16_t) (buffer.string[range->start] - 'a') < 6 || (uint16_t) (buffer.string[range->start] - 'A') < 6 || (uint16_t) (buffer.string[range->start] - '0') < 10) { - number[j] = buffer.string[range->start]; - } - else { - if (!buffer.string[range->start]) continue; + break; + } + } // for - break; + if (status != F_found_not) { + if (j == 4) { + if (id) { + id->type = strtol(number, 0, 16); + } + } + else { + status = F_found_not; + } } - } // for - if (j == 4) { - if (id) { - id->type = strtol(number, 0, 16); - } - } - else { + if (status != F_found_not) { - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop && range->start < buffer.used; ) { + // Skip past all NULLs. + for (; range->start <= range->stop && range->start < buffer.used; ++range->start) { + if (buffer.string[range->start]) break; + } // for - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; + // The end of line, white space, or range stop point are the only valid stop points. + if (range->start <= range->stop) { + status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); - break; - } + if (F_status_is_error(status)) { + if (id) { + id->type = 0; + } - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for + return status; + } - if (id) { - id->type = 0; + if (status == F_false) { + status = F_found_not; + } + } } - return F_found_not; - } - } + if (status != F_found_not) { + if (range->start <= range->stop && range->start < buffer.used && buffer.string[range->start] == f_string_eol_s.string[0]) { + ++range->start; + } - // Skip past all NULLs. - for (; range->start <= range->stop && range->start < buffer.used; ++range->start) { - if (buffer.string[range->start]) break; - } // for + // Copy the cached characters over. + if (id) { + for (f_number_unsigned_t j = i, i = 0; j <= range->stop && j < buffer.used; ++j) { - // The end of line, whitespace, or range stop point are the only valid stop points. - if (range->start <= range->stop) { - status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); + if (!buffer.string[j]) continue; + if (buffer.string[j] == f_string_ascii_minus_s.string[0]) break; - if (F_status_is_error(status)) { - if (id) { - id->type = 0; - } + id->name[i++] = buffer.string[j]; - return status; - } + if (i > f_fll_id_name_length_d) { + status = F_found_not; - if (status == F_false) { + break; + } + } // for - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop && range->start < buffer.used; ) { + id->used = i + 1; - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; + if (id->used < f_fll_id_name_length_d) { + id->name[id->used] = 0; + } + else { - break; + // The last byte must always be NULL (f_fll_id_name_max_length_d is always 1 byte longer). + id->name[f_fll_id_name_length_d] = 0; + } } - - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for - - if (id) { - id->type = 0; } - - return F_found_not; } } - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - } + if (status == F_found_not) { - if (id) { - for (f_number_unsigned_t j = i, i = 0; j <= range->stop && j < buffer.used; ++j) { + // Increment until stop, while taking into consideration UTF-8 character widths. + for (; range->start <= range->stop && range->start < buffer.used; ) { - if (!buffer.string[j]) continue; - if (buffer.string[j] == '-') break; + if (buffer.string[range->start] == f_string_eol_s.string[0]) { + ++range->start; - id->name[i++] = buffer.string[j]; - } // for + break; + } - id->used = i + 1; + range->start += macro_f_utf_byte_width(buffer.string[range->start]); + } // for - if (id->used < 64) { - id->name[id->used] = 0; - } + return F_found_not; } return F_found; diff --git a/level_0/f_parse/c/parse.h b/level_0/f_parse/c/parse.h index 09fecff8d..99b4bdfd7 100644 --- a/level_0/f_parse/c/parse.h +++ b/level_0/f_parse/c/parse.h @@ -203,11 +203,9 @@ extern "C" { * F_parameter (with error bit) if a parameter is invalid. * F_utf_not (with error bit) if character is an invalid UTF-8 character. * - * F_parameter (with error bit) from: f_utf_is_combining(). * F_parameter (with error bit) from: f_utf_is_whitespace(). * F_parameter (with error bit) from: f_utf_is_word(). * - * @see f_utf_is_combining() * @see f_utf_is_whitespace() * @see f_utf_is_word() */ diff --git a/level_0/f_status/c/status.h b/level_0/f_status/c/status.h index e5f7cf931..d72d509f8 100644 --- a/level_0/f_status/c/status.h +++ b/level_0/f_status/c/status.h @@ -563,7 +563,13 @@ extern "C" { F_endian_little, F_endian_not, F_enter, + //F_enter_abort, + //F_enter_clean, + //F_enter_error, + //F_enter_interrupt, F_enter_not, + //F_enter_success, + //F_enter_warning, //F_environment, //F_environment_not, //F_environment_ready, @@ -585,11 +591,39 @@ extern "C" { F_error, F_error_not, F_execute, + //F_execute_abort, + //F_execute_attach, + //F_execute_child, + //F_execute_clone, + //F_execute_detach, + //F_execute_exit, + //F_execute_failure, + //F_execute_fork, + //F_execute_interrupt, + //F_execute_join, + //F_execute_merge, + //F_execute_next, F_execute_not, + //F_execute_parent, + //F_execute_pause, + //F_execute_previous, + //F_execute_reap, + //F_execute_resume, + //F_execute_spawn, + //F_execute_start, + //F_execute_stop, + //F_execute_success, + //F_execute_thread, F_exist, F_exist_not, F_exit, + //F_exit_abort, + //F_exit_clean, + //F_exit_error, + //F_exit_interrupt, F_exit_not, + //F_exit_success, + //F_exit_warning, //F_explode, //F_explode_not, //F_extend, @@ -607,6 +641,7 @@ extern "C" { F_file_closed, F_file_descriptor, F_file_descriptor_max, + //F_file_descriptor_min, F_file_descriptor_not, F_file_empty, F_file_empty_not, @@ -651,6 +686,10 @@ extern "C" { F_fire_wall_not, F_fire_wall_open, F_fire_wall_pass, + //F_fire_wall_pause, + //F_fire_wall_resume, + //F_fire_wall_start, + //F_fire_wall_stop, F_fine, F_fine_not, F_first, @@ -666,7 +705,13 @@ extern "C" { F_forward_not, F_forward_slow, F_found, + //F_found_complete, + //F_found_first, + //F_found_last, + //F_found_next, F_found_not, + //F_found_partial, + //F_found_previous, F_fresh, F_fresh_not, F_full, diff --git a/level_0/f_utf/c/private-utf.c b/level_0/f_utf/c/private-utf.c index 7c2796672..dddcd4e68 100644 --- a/level_0/f_utf/c/private-utf.c +++ b/level_0/f_utf/c/private-utf.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) +#if !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_joiner_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) f_status_t private_f_utf_char_to_character(const f_string_t sequence, const f_number_unsigned_t width_max, f_utf_char_t *character_utf) { if (!macro_f_utf_byte_width_is(*sequence)) { @@ -34,7 +34,7 @@ extern "C" { return F_okay; } -#endif // !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) +#endif // !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_joiner_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) #if !defined(_di_f_utf_unicode_to_) || !defined(_di_f_utf_character_unicode_to_) f_status_t private_f_utf_character_unicode_to(const f_utf_char_t sequence, uint32_t *codepoint) { diff --git a/level_0/f_utf/c/private-utf.h b/level_0/f_utf/c/private-utf.h index e4c9f26ee..1d58811f0 100644 --- a/level_0/f_utf/c/private-utf.h +++ b/level_0/f_utf/c/private-utf.h @@ -53,6 +53,7 @@ extern "C" { * @see f_utf_is_digit() * @see f_utf_is_emoji() * @see f_utf_is_graph() + * @see f_utf_is_joiner() * @see f_utf_is_numeric() * @see f_utf_is_phonetic() * @see f_utf_is_private() @@ -70,9 +71,9 @@ extern "C" { * @see f_utf_is_zero_width() * @see f_utf_unicode_to() */ -#if !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) +#if !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_joiner_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) extern f_status_t private_f_utf_char_to_character(const f_string_t character, const f_number_unsigned_t width_max, f_utf_char_t *character_utf) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) +#endif // !defined(_di_f_utf_char_to_character_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_ascii_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_control_) || !defined(_di_f_utf_is_control_picture_) || !defined(_di_f_utf_is_digit_) || !defined(_di_f_utf_is_emoji_) || !defined(_di_f_utf_is_graph_) || !defined(_di_f_utf_is_joiner_) || !defined(_di_f_utf_is_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_private_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_unassigned_) || !defined(_di_f_utf_is_valid_) || !defined(_di_f_utf_is_whitespace_) || !defined(_di_f_utf_is_whitespace_modifier_) || !defined(_di_f_utf_is_whitespace_other_) || !defined(_di_f_utf_is_wide_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) || !defined(_di_f_utf_is_zero_width_) || !defined(_di_f_utf_unicode_to_) /** * Private implementation of f_utf_character_is_zero_width(). diff --git a/level_0/f_utf/c/private-utf_alphabetic.c b/level_0/f_utf/c/private-utf_alphabetic.c index 1aa479351..16c46dd3c 100644 --- a/level_0/f_utf/c/private-utf_alphabetic.c +++ b/level_0/f_utf/c/private-utf_alphabetic.c @@ -4,6 +4,7 @@ #include "private-utf_combining.h" #include "private-utf_control.h" #include "private-utf_digit.h" +#include "private-utf_joiner.h" #include "private-utf_numeric.h" #include "private-utf_phonetic.h" #include "private-utf_punctuation.h" @@ -23,25 +24,21 @@ extern "C" { return F_false; } - if (private_f_utf_character_is_zero_width(sequence)) { + // The is_control() handles both is_control_code() and is_control_format(). + if (private_f_utf_character_is_control(sequence)) { return F_false; } - if (private_f_utf_character_is_combining(sequence)) { + if (private_f_utf_character_is_zero_width(sequence)) { return F_false; } - // The is_control() handles both is_control_code() and is_control_format(). - if (private_f_utf_character_is_control(sequence)) { + if (private_f_utf_character_is_combining(sequence)) { return F_false; } // Skipping control pictures check because they are already represented in the is symbols check. - if (private_f_utf_character_is_combining(sequence)) { - return F_false; - } - if (private_f_utf_character_is_whitespace(sequence, F_true)) { return F_false; } @@ -66,6 +63,10 @@ extern "C" { return F_false; } + if (private_f_utf_character_is_joiner(sequence)) { + return F_false; + } + return F_true; } #endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_) @@ -77,6 +78,11 @@ extern "C" { return F_false; } + // The is_control() handles both is_control_code() and is_control_format(). + if (private_f_utf_character_is_control(sequence)) { + return F_false; + } + if (private_f_utf_character_is_digit(sequence, value)) { return F_true; } @@ -89,11 +95,6 @@ extern "C" { return F_false; } - // The is_control() handles both is_control_code() and is_control_format(). - if (private_f_utf_character_is_control(sequence)) { - return F_false; - } - // Skipping control pictures check because they are already represented in the is symbols check. if (private_f_utf_character_is_whitespace(sequence, F_true)) { @@ -120,6 +121,10 @@ extern "C" { return F_false; } + if (private_f_utf_character_is_joiner(sequence)) { + return F_false; + } + return F_true; } #endif // !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) @@ -131,6 +136,11 @@ extern "C" { return F_false; } + // The is_control() handles both is_control_code() and is_control_format(). + if (private_f_utf_character_is_control(sequence)) { + return F_false; + } + if (private_f_utf_character_is_numeric(sequence)) { return F_true; } @@ -143,11 +153,6 @@ extern "C" { return F_false; } - // The is_control() handles both is_control_code() and is_control_format(). - if (private_f_utf_character_is_control(sequence)) { - return F_false; - } - // Skipping control pictures check because they are already represented in the is symbols check. if (private_f_utf_character_is_whitespace(sequence, F_true)) { @@ -170,6 +175,10 @@ extern "C" { return F_false; } + if (private_f_utf_character_is_joiner(sequence)) { + return F_false; + } + return F_true; } #endif // !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_is_alphabetic_numeric_) diff --git a/level_0/f_utf/c/private-utf_combining.c b/level_0/f_utf/c/private-utf_combining.c index 0679e5b6b..e9f73f591 100644 --- a/level_0/f_utf/c/private-utf_combining.c +++ b/level_0/f_utf/c/private-utf_combining.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_is_combining_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_combining_joiner_) f_status_t private_f_utf_character_is_combining(const f_utf_char_t sequence) { if (macro_f_utf_char_t_width_is(sequence) == 2) { @@ -1485,7 +1485,7 @@ extern "C" { return F_false; } -#endif // !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_is_combining_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_combining_joiner_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_combining.h b/level_0/f_utf/c/private-utf_combining.h index 284ddc4eb..2dad4e7b8 100644 --- a/level_0/f_utf/c/private-utf_combining.h +++ b/level_0/f_utf/c/private-utf_combining.h @@ -34,12 +34,15 @@ extern "C" { * F_utf_fragment (with error bit) if character is a UTF-8 fragment. * F_utf_not (with error bit) if unicode is an invalid Unicode character. * + * @see f_utf_character_is_alphabetic() * @see f_utf_character_is_combining() + * @see f_utf_character_is_combining_joiner() * @see f_utf_is_combining() + * @see f_utf_is_combining_joiner() */ -#if !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_is_combining_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_combining_joiner_) extern f_status_t private_f_utf_character_is_combining(const f_utf_char_t sequence) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_is_combining_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_combining_) || !defined(_di_f_utf_character_is_combining_joiner_)|| !defined(_di_f_utf_is_combining_) || !defined(_di_f_utf_is_combining_joiner_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_joiner.c b/level_0/f_utf/c/private-utf_joiner.c new file mode 100644 index 000000000..1c577f682 --- /dev/null +++ b/level_0/f_utf/c/private-utf_joiner.c @@ -0,0 +1,29 @@ +#include "utf.h" +#include "private-utf.h" +#include "private-utf_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_character_is_joiner_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_combining_joiner_) || !defined(_di_f_utf_is_joiner_) || !defined( _di_f_utf_is_zero_width_) + f_status_t private_f_utf_character_is_joiner(const f_utf_char_t sequence) { + + if (macro_f_utf_char_t_width_is(sequence) == 3) { + + if (macro_f_utf_char_t_to_char_1(sequence) == 0xe2) { + + // General Punctuation: U+200D, U+2060. + if (sequence == 0xe2808d00 || sequence == 0xe281a000) { + return F_true; + } + } + } + + return F_false; + } +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_character_is_joiner_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_combining_joiner_) || !defined(_di_f_utf_is_joiner_) || !defined( _di_f_utf_is_zero_width_) + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_utf/c/private-utf_joiner.h b/level_0/f_utf/c/private-utf_joiner.h new file mode 100644 index 000000000..dabca435f --- /dev/null +++ b/level_0/f_utf/c/private-utf_joiner.h @@ -0,0 +1,57 @@ +/** + * FLL - Level 0 + * + * Project: UTF + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Provides UTF-8 capabilities. + * + * These are provided for internal reduction in redundant code. + * These should not be exposed/used outside of this project. + */ +#ifndef _PRIVATE_F_utf_joiner_h +#define _PRIVATE_F_utf_joiner_h + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Private implementation of f_utf_character_is_joiner(). + * + * Intended to be shared to each of the different implementation variations. + * + * This expects the character width to be of at least size 2. + * + * @param sequence + * The byte sequence to validate as a character. + * + * @return + * F_true if a UTF-8 joiner character. + * F_false if not a UTF-8 joiner character. + * + * F_utf_fragment (with error bit) if character is a UTF-8 fragment. + * F_utf_not (with error bit) if unicode is an invalid Unicode character. + * + * @see f_utf_character_is_alphabetic() + * @see f_utf_character_is_alphabetic_digit() + * @see f_utf_character_is_alphabetic_numeric() + * @see f_utf_character_is_combining_joiner() + * @see f_utf_character_is_joiner() + * @see f_utf_is_alphabetic() + * @see f_utf_is_alphabetic_digit() + * @see f_utf_is_alphabetic_numeric() + * @see f_utf_is_combining_joiner() + * @see f_utf_is_joiner() + * @see f_utf_is_zero_width() + */ +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_character_is_joiner_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_joiner_) || !defined( _di_f_utf_is_zero_width_) + extern f_status_t private_f_utf_character_is_joiner(const f_utf_char_t sequence) F_attribute_visibility_internal_d; +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_combining_joiner_) || !defined(_di_f_utf_character_is_joiner_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_joiner_) || !defined( _di_f_utf_is_zero_width_) + +#ifdef __cplusplus +} // extern "C" +#endif + +#endif // _PRIVATE_F_utf_joiner_h diff --git a/level_0/f_utf/c/private-utf_phonetic.c b/level_0/f_utf/c/private-utf_phonetic.c index 2e7239f41..edc9b7b11 100644 --- a/level_0/f_utf/c/private-utf_phonetic.c +++ b/level_0/f_utf/c/private-utf_phonetic.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) f_status_t private_f_utf_character_is_phonetic(const f_utf_char_t sequence) { if (macro_f_utf_char_t_width_is(sequence) == 3) { @@ -24,7 +24,7 @@ extern "C" { return F_false; } -#endif // !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_phonetic.h b/level_0/f_utf/c/private-utf_phonetic.h index cdff480bc..94b9d622d 100644 --- a/level_0/f_utf/c/private-utf_phonetic.h +++ b/level_0/f_utf/c/private-utf_phonetic.h @@ -34,6 +34,7 @@ extern "C" { * F_utf_fragment (with error bit) if character is a UTF-8 fragment. * F_utf_not (with error bit) if unicode is an invalid Unicode character. * + * @see f_utf_character_is_alphabetic() * @see f_utf_character_is_alphabetic_digit() * @see f_utf_character_is_alphabetic_numeric() * @see f_utf_character_is_phonetic() @@ -47,9 +48,9 @@ extern "C" { * @see f_utf_is_word_dash() * @see f_utf_is_word_dash_plus() */ -#if !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) extern f_status_t private_f_utf_character_is_phonetic(const f_utf_char_t sequence) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_phonetic_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_phonetic_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_punctuation.c b/level_0/f_utf/c/private-utf_punctuation.c index b1e1a19f4..c58b8dc5b 100644 --- a/level_0/f_utf/c/private-utf_punctuation.c +++ b/level_0/f_utf/c/private-utf_punctuation.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) f_status_t private_f_utf_character_is_punctuation(const f_utf_char_t sequence) { if (macro_f_utf_char_t_width_is(sequence) == 2) { @@ -905,7 +905,7 @@ extern "C" { return F_false; } -#endif // !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_punctuation.h b/level_0/f_utf/c/private-utf_punctuation.h index bab222231..58130dab4 100644 --- a/level_0/f_utf/c/private-utf_punctuation.h +++ b/level_0/f_utf/c/private-utf_punctuation.h @@ -34,6 +34,7 @@ extern "C" { * F_utf_fragment (with error bit) if character is a UTF-8 fragment. * F_utf_not (with error bit) if unicode is an invalid Unicode character. * + * @see f_utf_character_is_alphabetic() * @see f_utf_character_is_alphabetic_digit() * @see f_utf_character_is_alphabetic_numeric() * @see f_utf_character_is_punctuation() @@ -47,9 +48,9 @@ extern "C" { * @see f_utf_is_word_dash() * @see f_utf_is_word_dash_plus() */ -#if !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) extern f_status_t private_f_utf_character_is_punctuation(const f_utf_char_t sequence) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_punctuation_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_punctuation_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_symbol.c b/level_0/f_utf/c/private-utf_symbol.c index 5ef647506..ead3e7939 100644 --- a/level_0/f_utf/c/private-utf_symbol.c +++ b/level_0/f_utf/c/private-utf_symbol.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) f_status_t private_f_utf_character_is_symbol(const f_utf_char_t sequence) { if (macro_f_utf_char_t_width_is(sequence) == 2) { @@ -1113,7 +1113,7 @@ extern "C" { return F_false; } -#endif // !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_symbol.h b/level_0/f_utf/c/private-utf_symbol.h index c45df4d18..62f01956a 100644 --- a/level_0/f_utf/c/private-utf_symbol.h +++ b/level_0/f_utf/c/private-utf_symbol.h @@ -34,12 +34,14 @@ extern "C" { * F_utf_fragment (with error bit) if character is a UTF-8 fragment. * F_utf_not (with error bit) if unicode is an invalid Unicode character. * - * @see f_utf_character_is_symbol() + * @see f_utf_character_is_alphabetic() * @see f_utf_character_is_alphabetic_digit() * @see f_utf_character_is_alphabetic_numeric() + * @see f_utf_character_is_symbol() * @see f_utf_character_is_word() * @see f_utf_character_is_word_dash() * @see f_utf_character_is_word_dash_plus() + * @see f_utf_is_alphabetic() * @see f_utf_is_alphabetic_digit() * @see f_utf_is_alphabetic_numeric() * @see f_utf_is_symbol() @@ -47,9 +49,9 @@ extern "C" { * @see f_utf_is_word_dash() * @see f_utf_is_word_dash_plus() */ -#if !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) extern f_status_t private_f_utf_character_is_symbol(const f_utf_char_t sequence) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_symbol_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_symbol_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) #ifdef __cplusplus } // extern "C" diff --git a/level_0/f_utf/c/private-utf_whitespace.c b/level_0/f_utf/c/private-utf_whitespace.c index 51904c15e..6ecd377e9 100644 --- a/level_0/f_utf/c/private-utf_whitespace.c +++ b/level_0/f_utf/c/private-utf_whitespace.c @@ -6,7 +6,7 @@ extern "C" { #endif -#if !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) f_status_t private_f_utf_character_is_whitespace(const f_utf_char_t sequence, const uint8_t strict) { if (macro_f_utf_char_t_width_is(sequence) == 2) { @@ -58,7 +58,7 @@ extern "C" { return F_false; } -#endif // !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) #if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_alphabetic_digit_) || !defined(_di_f_utf_character_is_alphabetic_numeric_) || !defined(_di_f_utf_character_is_word_) || !defined(_di_f_utf_character_is_word_dash_) || !defined(_di_f_utf_character_is_word_dash_plus_) || !defined(_di_f_utf_is_alphabetic_) || !defined(_di_f_utf_is_alphabetic_digit_) || !defined(_di_f_utf_is_alphabetic_numeric_) || !defined(_di_f_utf_is_word_) || !defined(_di_f_utf_is_word_dash_) || !defined(_di_f_utf_is_word_dash_plus_) f_status_t private_f_utf_character_is_whitespace_modifier(const f_utf_char_t sequence) { diff --git a/level_0/f_utf/c/private-utf_whitespace.h b/level_0/f_utf/c/private-utf_whitespace.h index 5459368c6..486bcfe57 100644 --- a/level_0/f_utf/c/private-utf_whitespace.h +++ b/level_0/f_utf/c/private-utf_whitespace.h @@ -37,12 +37,13 @@ extern "C" { * F_utf_fragment (with error bit) if character is a UTF-8 fragment. * F_utf_not (with error bit) if unicode is an invalid Unicode character. * + * @see f_utf_character_is_alphabetic() * @see f_utf_character_is_whitespace() * @see f_utf_is_whitespace() */ -#if !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) +#if !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) extern f_status_t private_f_utf_character_is_whitespace(const f_utf_char_t sequence, const uint8_t strict) F_attribute_visibility_internal_d; -#endif // !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) +#endif // !defined(_di_f_utf_character_is_alphabetic_) || !defined(_di_f_utf_character_is_whitespace_) || !defined(_di_f_utf_is_whitespace_) /** * Private implementation of f_utf_character_is_whitespace_modifier(). diff --git a/level_0/f_utf/c/private-utf_zero_width.c b/level_0/f_utf/c/private-utf_zero_width.c index d7f15f388..52e438fb6 100644 --- a/level_0/f_utf/c/private-utf_zero_width.c +++ b/level_0/f_utf/c/private-utf_zero_width.c @@ -12,11 +12,21 @@ extern "C" { if (macro_f_utf_char_t_width_is(sequence) == 3) { if (macro_f_utf_char_t_to_char_1(sequence) == 0xe2) { + // General Punctuation: U+200D. + if (sequence == 0xe2808d00) { + return F_true; + } + // General Punctuation: U+2028 to U+2029. if (sequence >= 0xf090b4a3 && sequence <= 0xf090b4a7) { return F_true; } + // General Punctuation: U+2060. + if (sequence == 0xe281a000) { + return F_true; + } + // General Punctuation: U+2066 to U+2069. if (sequence >= 0xe281a600 && sequence <= 0xe281a900) { return F_true; diff --git a/level_0/f_utf/c/utf.c b/level_0/f_utf/c/utf.c index 5c04b1b8b..558c493a8 100644 --- a/level_0/f_utf/c/utf.c +++ b/level_0/f_utf/c/utf.c @@ -67,7 +67,7 @@ extern "C" { return F_okay_stop; } else if (range->start + width >= buffer.used) { - if (width > 1 && range->start + width >= buffer.used + 1) { + if (width > 1 && range->start + width > buffer.used) { return F_status_set_error(F_complete_not_utf_eos); } diff --git a/level_0/f_utf/c/utf/is.c b/level_0/f_utf/c/utf/is.c index ef3e4d486..9e6305d6a 100644 --- a/level_0/f_utf/c/utf/is.c +++ b/level_0/f_utf/c/utf/is.c @@ -5,6 +5,7 @@ #include "../private-utf_control.h" #include "../private-utf_digit.h" #include "../private-utf_emoji.h" +#include "../private-utf_joiner.h" #include "../private-utf_numeric.h" #include "../private-utf_phonetic.h" #include "../private-utf_private.h" @@ -147,6 +148,32 @@ extern "C" { } #endif // _di_f_utf_is_combining_ +#ifndef _di_f_utf_is_combining_joiner_ + f_status_t f_utf_is_combining_joiner(const f_string_t sequence, const f_number_unsigned_t width_max) { + + if (width_max < 1) return F_data_not; + + if (macro_f_utf_byte_width_is(*sequence)) { + if (macro_f_utf_byte_width_is(*sequence) > width_max) return F_status_set_error(F_complete_not_utf); + if (macro_f_utf_byte_width_is(*sequence) == 1) return F_status_set_error(F_utf_fragment); + + f_utf_char_t utf = 0; + + { + const f_status_t status = private_f_utf_char_to_character(sequence, width_max, &utf); + if (F_status_is_error(status)) return status; + } + + if (private_f_utf_character_is_combining(utf)) return F_true; + + return private_f_utf_character_is_joiner(utf); + } + + // There are no ASCII combining characters. + return F_false; + } +#endif // _di_f_utf_is_combining_joiner_ + #ifndef _di_f_utf_is_control_ f_status_t f_utf_is_control(const f_string_t sequence, const f_number_unsigned_t width_max) { @@ -332,6 +359,30 @@ extern "C" { } #endif // _di_f_utf_is_graph_ +#ifndef _di_f_utf_is_joiner_ + f_status_t f_utf_is_joiner(const f_string_t sequence, const f_number_unsigned_t width_max) { + + if (width_max < 1) return F_data_not; + + if (macro_f_utf_byte_width_is(*sequence)) { + if (macro_f_utf_byte_width_is(*sequence) > width_max) return F_status_set_error(F_complete_not_utf); + if (macro_f_utf_byte_width_is(*sequence) == 1) return F_status_set_error(F_utf_fragment); + + f_utf_char_t utf = 0; + + { + const f_status_t status = private_f_utf_char_to_character(sequence, width_max, &utf); + if (F_status_is_error(status)) return status; + } + + return private_f_utf_character_is_joiner(utf); + } + + // There are no ASCII joiner characters. + return F_false; + } +#endif // _di_f_utf_is_joiner_ + #ifndef _di_f_utf_is_numeric_ f_status_t f_utf_is_numeric(const f_string_t sequence, const f_number_unsigned_t width_max) { @@ -783,6 +834,7 @@ extern "C" { // Control and combining characters are zero-width. if (private_f_utf_character_is_control(utf)) return F_true; if (private_f_utf_character_is_combining(utf)) return F_true; + if (private_f_utf_character_is_joiner(utf)) return F_true; return private_f_utf_character_is_zero_width(utf); } diff --git a/level_0/f_utf/c/utf/is.h b/level_0/f_utf/c/utf/is.h index 81c9d8b76..9a5b08d21 100644 --- a/level_0/f_utf/c/utf/is.h +++ b/level_0/f_utf/c/utf/is.h @@ -163,6 +163,29 @@ extern "C" { extern f_status_t f_utf_is_combining(const f_string_t sequence, const f_number_unsigned_t width_max); #endif // _di_f_utf_is_combining_ +/** + * Check to see if the entire byte block of the character is a UTF-8 combining character or a UTF-8 joiner character. + * + * @param sequence + * The byte sequence to validate as a character. + * There must be enough space allocated to compare against, as limited by width_max. + * @param width_max + * The maximum width available for checking. + * Can be anything greater than 0. + * + * @return + * F_true if a UTF-8 combining or a joiner character. + * F_false if neither a UTF-8 combining nor a joiner character. + * F_data_not if width_max is less than 1. + * + * F_complete_not_utf (with error bit set) if character is an incomplete UTF-8 sequence. + * F_utf_fragment (with error bit) if character is a UTF-8 fragment. + * F_utf_not (with error bit) if Unicode is an invalid Unicode character. + */ +#ifndef _di_f_utf_is_combining_joiner_ + extern f_status_t f_utf_is_combining_joiner(const f_string_t sequence, const f_number_unsigned_t width_max); +#endif // _di_f_utf_is_combining_joiner_ + /** * Check to see if the entire byte block of the character is an ASCII or UTF-8 control character. * @@ -385,6 +408,31 @@ extern "C" { extern f_status_t f_utf_is_graph(const f_string_t sequence, const f_number_unsigned_t width_max); #endif // _di_f_utf_is_graph_ +/** + * Check to see if the entire byte block of the character is an ASCII or UTF-8 joiner character. + * + * @param sequence + * The byte sequence to validate as a character. + * There must be enough space allocated to compare against, as limited by width_max. + * @param width_max + * The maximum width available for checking. + * Can be anything greater than 0. + * + * @return + * F_true if a UTF-8 joiner. + * F_false if not a UTF-8 joiner. + * F_data_not if width_max is less than 1. + * + * F_complete_not_utf (with error bit set) if character is an incomplete UTF-8 sequence. + * F_maybe (with error bit) if this could be a joiner but width is not long enough. + * F_parameter (with error bit) if a parameter is invalid. + * F_utf_fragment (with error bit) if character is a UTF-8 fragment. + * F_utf_not (with error bit) if Unicode is an invalid Unicode character. + */ +#ifndef _di_f_utf_is_joiner_ + extern f_status_t f_utf_is_joiner(const f_string_t sequence, const f_number_unsigned_t width_max); +#endif // _di_f_utf_is_joiner_ + /** * Check to see if the entire byte block of the character is an ASCII or UTF-8 numeric character. * diff --git a/level_0/f_utf/c/utf/is_character.c b/level_0/f_utf/c/utf/is_character.c index ffb57f196..0786ebebc 100644 --- a/level_0/f_utf/c/utf/is_character.c +++ b/level_0/f_utf/c/utf/is_character.c @@ -5,6 +5,7 @@ #include "../private-utf_control.h" #include "../private-utf_digit.h" #include "../private-utf_emoji.h" +#include "../private-utf_joiner.h" #include "../private-utf_numeric.h" #include "../private-utf_phonetic.h" #include "../private-utf_private.h" @@ -106,6 +107,22 @@ extern "C" { } #endif // _di_f_utf_character_is_combining_ +#ifndef _di_f_utf_character_is_combining_joiner_ + f_status_t f_utf_character_is_combining_joiner(const f_utf_char_t sequence) { + + if (macro_f_utf_char_t_width_is(sequence)) { + if (macro_f_utf_char_t_width_is(sequence) == 1) return F_status_set_error(F_utf_fragment); + + if (private_f_utf_character_is_combining(sequence)) return F_true; + + return private_f_utf_character_is_joiner(sequence); + } + + // There are no combining characters in ASCII. + return F_false; + } +#endif // _di_f_utf_character_is_combining_joiner_ + #ifndef _di_f_utf_character_is_control_ f_status_t f_utf_character_is_control(const f_utf_char_t sequence) { @@ -215,6 +232,20 @@ extern "C" { } #endif // _di_f_utf_character_is_graph_ +#ifndef _di_f_utf_character_is_joiner_ + f_status_t f_utf_character_is_joiner(const f_utf_char_t sequence) { + + if (macro_f_utf_char_t_width_is(sequence)) { + if (macro_f_utf_char_t_width_is(sequence) == 1) return F_status_set_error(F_utf_fragment); + + return private_f_utf_character_is_joiner(sequence); + } + + // There are no ASCII joiner characters. + return F_false; + } +#endif // _di_f_utf_character_is_joiner_ + #ifndef _di_f_utf_character_is_numeric_ f_status_t f_utf_character_is_numeric(const f_utf_char_t sequence) { diff --git a/level_0/f_utf/c/utf/is_character.h b/level_0/f_utf/c/utf/is_character.h index 0cb54d566..e2ae0dc1e 100644 --- a/level_0/f_utf/c/utf/is_character.h +++ b/level_0/f_utf/c/utf/is_character.h @@ -135,6 +135,23 @@ extern "C" { extern f_status_t f_utf_character_is_combining(const f_utf_char_t sequence); #endif // _di_f_utf_character_is_combining_ +/** + * Check to see if the entire byte block of the character is a UTF-8 combining character or a UTF-8 joiner character. + * + * @param sequence + * The byte sequence to validate as a character. + * + * @return + * F_true if a UTF-8 combining character. + * F_false if not a UTF-8 combining character. + * + * F_utf_fragment (with error bit) if character is a UTF-8 fragment. + * F_utf_not (with error bit) if unicode is an invalid Unicode character. + */ +#ifndef _di_f_utf_character_is_combining_joiner_ + extern f_status_t f_utf_character_is_combining_joiner(const f_utf_char_t sequence); +#endif // _di_f_utf_character_is_combining_joiner_ + /** * Check to see if the entire byte block of the character is an ASCII or UTF-8 control character. * @@ -322,6 +339,23 @@ extern "C" { extern f_status_t f_utf_character_is_graph(const f_utf_char_t sequence); #endif // _di_f_utf_character_is_graph_ +/** + * Check to see if the entire byte block of the character is an ASCII or UTF-8 joiner character. + * + * @param sequence + * The byte sequence to validate as a character. + * + * @return + * F_true if a UTF-8 joiner character. + * F_false if not a UTF-8 joiner character. + * + * F_utf_fragment (with error bit) if character is a UTF-8 fragment. + * F_utf_not (with error bit) if unicode is an invalid Unicode character. + */ +#ifndef _di_f_utf_character_is_joiner_ + extern f_status_t f_utf_character_is_joiner(const f_utf_char_t sequence); +#endif // _di_f_utf_character_is_joiner_ + /** * Check to see if the entire byte block of the character is an ASCII or UTF-8 numeric character. * diff --git a/level_0/f_utf/data/build/settings b/level_0/f_utf/data/build/settings index 8e2debf52..13644e8ce 100644 --- a/level_0/f_utf/data/build/settings +++ b/level_0/f_utf/data/build/settings @@ -43,7 +43,7 @@ build_libraries_static-individual -l:libf_memory.a -l:libf_string.a build_sources_headers utf.h utf/common.h utf/convert.h utf/dynamic.h utf/dynamics.h utf/dynamicss.h utf/is.h utf/is_character.h utf/map.h utf/maps.h utf/mapss.h utf/map_multi.h utf/map_multis.h utf/map_multiss.h utf/static.h utf/statics.h utf/staticss.h utf/string.h utf/triple.h utf/triples.h utf/tripless.h -build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c +build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_joiner.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c build_sources_library utf/common.c utf/convert.c utf/dynamic.c utf/dynamics.c utf/dynamicss.c utf/is.c utf/is_character.c utf/map.c utf/maps.c utf/mapss.c utf/map_multi.c utf/map_multis.c utf/map_multiss.c utf/static.c utf/statics.c utf/staticss.c utf/string.c utf/triple.c utf/triples.c utf/tripless.c build_sources_library utf/private-dynamics.c utf/private-maps.c utf/private-map_multis.c utf/private-string.c utf/private-triples.c diff --git a/level_0/f_utf/data/build/settings-mocks b/level_0/f_utf/data/build/settings-mocks index 191a621e8..5795a9c63 100644 --- a/level_0/f_utf/data/build/settings-mocks +++ b/level_0/f_utf/data/build/settings-mocks @@ -32,7 +32,7 @@ build_libraries-individual -lf_memory -lf_string build_sources_headers utf.h utf/common.h utf/convert.h utf/dynamic.h utf/dynamics.h utf/dynamicss.h utf/is.h utf/is_character.h utf/map.h utf/maps.h utf/mapss.h utf/map_multi.h utf/map_multis.h utf/map_multiss.h utf/static.h utf/statics.h utf/staticss.h utf/string.h utf/triple.h utf/triples.h utf/tripless.h -build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c +build_sources_library utf.c private-utf.c private-utf_alphabetic.c private-utf_combining.c private-utf_control.c private-utf_digit.c private-utf_emoji.c private-utf_joiner.c private-utf_numeric.c private-utf_phonetic.c private-utf_private.c private-utf_punctuation.c private-utf_subscript.c private-utf_superscript.c private-utf_symbol.c private-utf_unassigned.c private-utf_valid.c private-utf_whitespace.c private-utf_wide.c private-utf_word.c private-utf_zero_width.c build_sources_library utf/common.c utf/convert.c utf/dynamic.c utf/dynamics.c utf/dynamicss.c utf/is.c utf/is_character.c utf/map.c utf/maps.c utf/mapss.c utf/map_multi.c utf/map_multis.c utf/map_multiss.c utf/static.c utf/statics.c utf/staticss.c utf/string.c utf/triple.c utf/triples.c utf/tripless.c build_sources_library utf/private-dynamics.c utf/private-maps.c utf/private-map_multis.c utf/private-string.c utf/private-triples.c build_sources_library ../../tests/unit/c/mock-utf.c diff --git a/level_0/f_utf/data/build/settings-tests b/level_0/f_utf/data/build/settings-tests index 7dd7308d6..2b9f73353 100644 --- a/level_0/f_utf/data/build/settings-tests +++ b/level_0/f_utf/data/build/settings-tests @@ -30,10 +30,12 @@ build_sources_program test-utf-character_is_alphabetic.c test-utf-is_alphabetic. build_sources_program test-utf-character_is_alphabetic_digit.c test-utf-is_alphabetic_digit.c build_sources_program test-utf-character_is_alphabetic_numeric.c test-utf-is_alphabetic_numeric.c build_sources_program test-utf-character_is_combining.c test-utf-is_combining.c +build_sources_program test-utf-character_is_combining_joiner.c test-utf-is_combining_joiner.c build_sources_program test-utf-character_is_control.c test-utf-is_control.c build_sources_program test-utf-character_is_control_picture.c test-utf-is_control_picture.c build_sources_program test-utf-character_is_digit.c test-utf-is_digit.c build_sources_program test-utf-character_is_emoji.c test-utf-is_emoji.c +build_sources_program test-utf-character_is_joiner.c test-utf-is_joiner.c build_sources_program test-utf-character_is_numeric.c test-utf-is_numeric.c build_sources_program test-utf-character_is_phonetic.c test-utf-is_phonetic.c build_sources_program test-utf-character_is_private.c test-utf-is_private.c diff --git a/level_0/f_utf/data/tests/bytesequences/joiner-all.txt b/level_0/f_utf/data/tests/bytesequences/joiner-all.txt new file mode 100644 index 000000000..456f70a49 --- /dev/null +++ b/level_0/f_utf/data/tests/bytesequences/joiner-all.txt @@ -0,0 +1,2 @@ +14844045 +14844320 diff --git a/level_0/f_utf/data/tests/codepoints/joiner-all.txt b/level_0/f_utf/data/tests/codepoints/joiner-all.txt new file mode 100644 index 000000000..81ea2d776 --- /dev/null +++ b/level_0/f_utf/data/tests/codepoints/joiner-all.txt @@ -0,0 +1,2 @@ +U+200D +U+2060 diff --git a/level_0/f_utf/tests/unit/c/data-utf.c b/level_0/f_utf/tests/unit/c/data-utf.c index 795de0518..6299056a4 100644 --- a/level_0/f_utf/tests/unit/c/data-utf.c +++ b/level_0/f_utf/tests/unit/c/data-utf.c @@ -14,6 +14,11 @@ FILE *data__bytesequence_file_open__combining(void) { return fopen("./data/tests/bytesequences/combining-all.txt", "r"); } +FILE *data__bytesequence_file_open__combining_joiner(void) { + + return fopen("./data/tests/bytesequences/combining_joiner-all.txt", "r"); +} + FILE *data__bytesequence_file_open__control(void) { return fopen("./data/tests/bytesequences/control-all.txt", "r"); @@ -34,6 +39,11 @@ FILE *data__bytesequence_file_open__emoji(void) { return fopen("./data/tests/bytesequences/emoji-all.txt", "r"); } +FILE *data__bytesequence_file_open__joiner(void) { + + return fopen("./data/tests/bytesequences/joiner-all.txt", "r"); +} + FILE *data__bytesequence_file_open__numeric(void) { return fopen("./data/tests/bytesequences/numeric-all.txt", "r"); diff --git a/level_0/f_utf/tests/unit/c/data-utf.h b/level_0/f_utf/tests/unit/c/data-utf.h index ab25d0836..f2924d76c 100644 --- a/level_0/f_utf/tests/unit/c/data-utf.h +++ b/level_0/f_utf/tests/unit/c/data-utf.h @@ -52,6 +52,21 @@ extern FILE *data__bytesequence_file_open__alphabetic(void); */ extern FILE *data__bytesequence_file_open__combining(void); +/** + * Open the "combining" or "joiner" bytesequence file. + * + * This assumes the following: + * - The file path is relative to the current working directory (tests are run from project root). + * - The file path is "data/tests/bytesequences/combining_joiner-all.txt". + * + * @return + * Non-zero on success. + * 0 on failure. + * + * @see fopen() + */ +extern FILE *data__bytesequence_file_open__combining_joiner(void); + /** * Open the "control" bytesequence file. * @@ -112,6 +127,21 @@ extern FILE *data__bytesequence_file_open__digit(void); */ extern FILE *data__bytesequence_file_open__emoji(void); +/** + * Open the "joiner" bytesequence file. + * + * This assumes the following: + * - The file path is relative to the current working directory (tests are run from project root). + * - The file path is "data/tests/bytesequences/joiner-all.txt". + * + * @return + * Non-zero on success. + * 0 on failure. + * + * @see fopen() + */ +extern FILE *data__bytesequence_file_open__joiner(void); + /** * Open the "numeric" bytesequence file. * diff --git a/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.c b/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.c new file mode 100644 index 000000000..461886639 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.c @@ -0,0 +1,66 @@ +#include "test-utf.h" +#include "test-utf-character_is_combining_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_utf_character_is_combining_joiner__works(void **state) { + + { + FILE *file_combining = data__bytesequence_file_open__combining(); + FILE *file_joiner = data__bytesequence_file_open__joiner(); + + assert_non_null(file_combining); + assert_non_null(file_joiner); + + f_utf_char_t sequence = 0; + ssize_t bytes = 0; + long long code = 0; + + f_number_unsigned_t line = 0; + + do { + bytes = data__bytesequence_get_line(file_combining, &sequence, &code); + + if (bytes > 0) { + const f_status_t status = f_utf_character_is_combining_joiner(sequence); + + if (status != F_true) { + printf("[ -------> ] --- [0] Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + line = 0; + + do { + bytes = data__bytesequence_get_line(file_joiner, &sequence, &code); + + if (bytes > 0) { + const f_status_t status = f_utf_character_is_combining_joiner(sequence); + + if (status != F_true) { + printf("[ -------> ] --- [1] Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + fclose(file_combining); + fclose(file_joiner); + } +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.h b/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.h new file mode 100644 index 000000000..e18da7134 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-character_is_combining_joiner.h @@ -0,0 +1,20 @@ +/** + * FLL - Level 0 + * + * Project: UTF + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test the function in the utf project. + */ +#ifndef _TEST__F_utf_character_is_combining_joiner_h +#define _TEST__F_utf_character_is_combining_joiner_h + +/** + * Test that the function works. + * + * @see f_utf_character_is_combining_joiner() + */ +extern void test__f_utf_character_is_combining_joiner__works(void **state); + +#endif // _TEST__F_utf_character_is_combining_joiner_h diff --git a/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.c b/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.c new file mode 100644 index 000000000..2d3664294 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.c @@ -0,0 +1,43 @@ +#include "test-utf.h" +#include "test-utf-character_is_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_utf_character_is_joiner__works(void **state) { + + { + FILE *file = data__bytesequence_file_open__joiner(); + + assert_non_null(file); + + f_utf_char_t sequence = 0; + ssize_t bytes = 0; + long long code = 0; + + f_number_unsigned_t line = 0; + + do { + bytes = data__bytesequence_get_line(file, &sequence, &code); + if (bytes > 0) { + const f_status_t status = f_utf_character_is_joiner(sequence); + + if (status != F_true) { + printf("[ -------> ] --- Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + fclose(file); + } +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.h b/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.h new file mode 100644 index 000000000..88dfebb78 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-character_is_joiner.h @@ -0,0 +1,20 @@ +/** + * FLL - Level 0 + * + * Project: UTF + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test the function in the utf project. + */ +#ifndef _TEST__F_utf_character_is_joiner_h +#define _TEST__F_utf_character_is_joiner_h + +/** + * Test that the function works. + * + * @see f_utf_character_is_joiner() + */ +extern void test__f_utf_character_is_joiner__works(void **state); + +#endif // _TEST__F_utf_character_is_joiner_h diff --git a/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.c b/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.c new file mode 100644 index 000000000..e9d10b2dd --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.c @@ -0,0 +1,100 @@ +#include "test-utf.h" +#include "test-utf-is_combining_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_utf_is_combining_joiner__works(void **state) { + + { + FILE *file_combining = data__bytesequence_file_open__combining(); + FILE *file_joiner = data__bytesequence_file_open__joiner(); + + assert_non_null(file_combining); + assert_non_null(file_joiner); + + f_utf_char_t sequence = 0; + ssize_t bytes = 0; + long long code = 0; + + f_number_unsigned_t line = 0; + + do { + bytes = data__bytesequence_get_line(file_combining, &sequence, &code); + + if (bytes > 0) { + const uint8_t width = macro_f_utf_char_t_width(sequence); + char buffer[5] = { 0, 0, 0, 0, 0 }; + + buffer[0] = macro_f_utf_char_t_to_char_1(sequence); + + if (width > 1) { + buffer[1] = macro_f_utf_char_t_to_char_2(sequence); + + if (width > 2) { + buffer[2] = macro_f_utf_char_t_to_char_3(sequence); + + if (width > 3) { + buffer[3] = macro_f_utf_char_t_to_char_4(sequence); + } + } + } + + const f_status_t status = f_utf_is_combining_joiner(buffer, 5); + + if (status != F_true) { + printf("[ -------> ] --- [0] Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + line = 0; + + do { + bytes = data__bytesequence_get_line(file_joiner, &sequence, &code); + + if (bytes > 0) { + const uint8_t width = macro_f_utf_char_t_width(sequence); + char buffer[5] = { 0, 0, 0, 0, 0 }; + + buffer[0] = macro_f_utf_char_t_to_char_1(sequence); + + if (width > 1) { + buffer[1] = macro_f_utf_char_t_to_char_2(sequence); + + if (width > 2) { + buffer[2] = macro_f_utf_char_t_to_char_3(sequence); + + if (width > 3) { + buffer[3] = macro_f_utf_char_t_to_char_4(sequence); + } + } + } + + const f_status_t status = f_utf_is_combining_joiner(buffer, 5); + + if (status != F_true) { + printf("[ -------> ] --- [1] Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + fclose(file_combining); + fclose(file_joiner); + } +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.h b/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.h new file mode 100644 index 000000000..86fa4abda --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-is_combining_joiner.h @@ -0,0 +1,20 @@ +/** + * FLL - Level 0 + * + * Project: UTF + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test the function in the utf project. + */ +#ifndef _TEST__F_utf_is_combining_joiner_h +#define _TEST__F_utf_is_combining_joiner_h + +/** + * Test that the function works. + * + * @see f_utf_is_combining_joiner() + */ +extern void test__f_utf_is_combining_joiner__works(void **state); + +#endif // _TEST__F_utf_is_combining_joiner_h diff --git a/level_0/f_utf/tests/unit/c/test-utf-is_joiner.c b/level_0/f_utf/tests/unit/c/test-utf-is_joiner.c new file mode 100644 index 000000000..739ae6dd7 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-is_joiner.c @@ -0,0 +1,61 @@ +#include "test-utf.h" +#include "test-utf-is_joiner.h" + +#ifdef __cplusplus +extern "C" { +#endif + +void test__f_utf_is_joiner__works(void **state) { + + { + FILE *file = data__bytesequence_file_open__joiner(); + + assert_non_null(file); + + f_utf_char_t sequence = 0; + ssize_t bytes = 0; + long long code = 0; + + f_number_unsigned_t line = 0; + + do { + bytes = data__bytesequence_get_line(file, &sequence, &code); + + if (bytes > 0) { + const uint8_t width = macro_f_utf_char_t_width(sequence); + char buffer[5] = { 0, 0, 0, 0, 0 }; + + buffer[0] = macro_f_utf_char_t_to_char_1(sequence); + + if (width > 1) { + buffer[1] = macro_f_utf_char_t_to_char_2(sequence); + + if (width > 2) { + buffer[2] = macro_f_utf_char_t_to_char_3(sequence); + + if (width > 3) { + buffer[3] = macro_f_utf_char_t_to_char_4(sequence); + } + } + } + + const f_status_t status = f_utf_is_joiner(buffer, 5); + + if (status != F_true) { + printf("[ -------> ] --- Failure at line=%lu sequence='%0x' (%llu).\n", line + 1, sequence, code); + } + + assert_int_equal(status, F_true); + } + + ++line; + + } while (bytes > 0); + + fclose(file); + } +} + +#ifdef __cplusplus +} // extern "C" +#endif diff --git a/level_0/f_utf/tests/unit/c/test-utf-is_joiner.h b/level_0/f_utf/tests/unit/c/test-utf-is_joiner.h new file mode 100644 index 000000000..08e2fdd00 --- /dev/null +++ b/level_0/f_utf/tests/unit/c/test-utf-is_joiner.h @@ -0,0 +1,20 @@ +/** + * FLL - Level 0 + * + * Project: UTF + * API Version: 0.8 + * Licenses: clarified_1_0_or_later-lgpl_2_1_or_later + * + * Test the function in the utf project. + */ +#ifndef _TEST__F_utf_is_joiner_h +#define _TEST__F_utf_is_joiner_h + +/** + * Test that the function works. + * + * @see f_utf_is_joiner() + */ +extern void test__f_utf_is_joiner__works(void **state); + +#endif // _TEST__F_utf_is_joiner_h diff --git a/level_0/f_utf/tests/unit/c/test-utf.c b/level_0/f_utf/tests/unit/c/test-utf.c index c6b4d6112..bf2c58b12 100644 --- a/level_0/f_utf/tests/unit/c/test-utf.c +++ b/level_0/f_utf/tests/unit/c/test-utf.c @@ -28,10 +28,12 @@ int main(void) { cmocka_unit_test(test__f_utf_character_is_alphabetic_digit__works), cmocka_unit_test(test__f_utf_character_is_alphabetic_numeric__works), cmocka_unit_test(test__f_utf_character_is_combining__works), + cmocka_unit_test(test__f_utf_character_is_combining_joiner__works), cmocka_unit_test(test__f_utf_character_is_control__works), cmocka_unit_test(test__f_utf_character_is_control_picture__works), cmocka_unit_test(test__f_utf_character_is_digit__works), cmocka_unit_test(test__f_utf_character_is_emoji__works), + cmocka_unit_test(test__f_utf_character_is_joiner__works), cmocka_unit_test(test__f_utf_character_is_numeric__works), cmocka_unit_test(test__f_utf_character_is_phonetic__works), cmocka_unit_test(test__f_utf_character_is_private__works), @@ -141,10 +143,12 @@ int main(void) { cmocka_unit_test(test__f_utf_is_alphabetic_digit__works), cmocka_unit_test(test__f_utf_is_alphabetic_numeric__works), cmocka_unit_test(test__f_utf_is_combining__works), + cmocka_unit_test(test__f_utf_is_combining_joiner__works), cmocka_unit_test(test__f_utf_is_control__works), cmocka_unit_test(test__f_utf_is_control_picture__works), cmocka_unit_test(test__f_utf_is_digit__works), cmocka_unit_test(test__f_utf_is_emoji__works), + cmocka_unit_test(test__f_utf_is_joiner__works), cmocka_unit_test(test__f_utf_is_numeric__works), cmocka_unit_test(test__f_utf_is_phonetic__works), cmocka_unit_test(test__f_utf_is_private__works), diff --git a/level_0/f_utf/tests/unit/c/test-utf.h b/level_0/f_utf/tests/unit/c/test-utf.h index 4d7882d38..3133a20d2 100644 --- a/level_0/f_utf/tests/unit/c/test-utf.h +++ b/level_0/f_utf/tests/unit/c/test-utf.h @@ -37,11 +37,13 @@ #include "test-utf-character_is_alphabetic_digit.h" #include "test-utf-character_is_alphabetic_numeric.h" #include "test-utf-character_is_combining.h" +#include "test-utf-character_is_combining_joiner.h" #include "test-utf-character_is_control.h" #include "test-utf-character_is_control_picture.h" #include "test-utf-character_is_digit.h" #include "test-utf-character_is_digit.h" #include "test-utf-character_is_emoji.h" +#include "test-utf-character_is_joiner.h" #include "test-utf-character_is_numeric.h" #include "test-utf-character_is_phonetic.h" #include "test-utf-character_is_private.h" @@ -103,11 +105,13 @@ #include "test-utf-is_alphabetic_digit.h" #include "test-utf-is_alphabetic_numeric.h" #include "test-utf-is_combining.h" +#include "test-utf-is_combining_joiner.h" #include "test-utf-is_control.h" #include "test-utf-is_control_picture.h" #include "test-utf-is_digit.h" #include "test-utf-is_digit.h" #include "test-utf-is_emoji.h" +#include "test-utf-is_joiner.h" #include "test-utf-is_numeric.h" #include "test-utf-is_phonetic.h" #include "test-utf-is_private.h" diff --git a/level_1/fl_fss/c/fss/basic.c b/level_1/fl_fss/c/fss/basic.c index fdfa200c7..35bba7b21 100644 --- a/level_1/fl_fss/c/fss/basic.c +++ b/level_1/fl_fss/c/fss/basic.c @@ -18,8 +18,7 @@ extern "C" { #endif // _di_level_1_parameter_checking_ f_fss_skip_past_space(buffer, range, state); - if (F_status_is_error(state->status)) return; - if (state->status == F_data_not) return; + if (F_status_is_error(state->status) || state->status == F_data_not) return; if (state->status == F_okay_eol) { ++range->start; @@ -42,38 +41,46 @@ extern "C" { const f_number_unsigned_t begin = range->start; - for (;; ++range->start) { + f_fss_seek_to_eol(buffer, range, state); - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } + if (F_status_is_error_not(state->status)) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); + } - f_fss_skip_past_delimit(buffer, range, state); - if (F_status_is_error(state->status)) break; + if (F_status_is_error_not(state->status)) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); + if (F_status_is_error(state->status)) return; - if (state->status == F_okay_eos || state->status == F_okay_stop) return; - if (buffer.string[range->start] == f_fss_basic_close_s.string[0]) break; - } // for + // The EOS and stop cases only happen here when EOL is not found, so do not increment found.used. + if (range->start >= buffer.used) { + found->array[found->used].stop = buffer.used - 1; + state->status = F_okay_eos; - if (F_status_is_error(state->status)) return; + return; + } - state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); - if (F_status_is_error(state->status)) return; + if (range->start > range->stop) { + found->array[found->used].stop = range->stop; + state->status = F_okay_stop; - if (range->start > begin) { - found->array[found->used].start = begin; - found->array[found->used++].stop = range->start - 1; - } - else { - found->array[found->used].start = 1; - found->array[found->used++].stop = 0; - } + return; + } - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) return; + if (range->start > begin) { + found->array[found->used].start = begin; + found->array[found->used++].stop = range->start - 1; + } + else { + found->array[found->used].start = 1; + found->array[found->used++].stop = 0; + } + + state->status = f_utf_buffer_increment(buffer, range, 1); - state->status = F_fss_found_content; + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_content; + } + } } #endif // _di_fl_fss_basic_content_read_ diff --git a/level_1/fl_fss/c/fss/basic.h b/level_1/fl_fss/c/fss/basic.h index c93f895e9..34457cff6 100644 --- a/level_1/fl_fss/c/fss/basic.h +++ b/level_1/fl_fss/c/fss/basic.h @@ -34,6 +34,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -99,6 +101,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param content * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -155,6 +159,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -234,6 +240,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). * @param quote diff --git a/level_1/fl_fss/c/fss/basic_list.c b/level_1/fl_fss/c/fss/basic_list.c index a04b3a942..5b7c715d2 100644 --- a/level_1/fl_fss/c/fss/basic_list.c +++ b/level_1/fl_fss/c/fss/basic_list.c @@ -37,21 +37,37 @@ extern "C" { state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_do_escape___ 0x1 + #define ___fl_flag_has_delimit_comment___ 0x2 + #define ___fl_flag_has_graph_first___ 0x4 + #define ___fl_flag_has_graph_first_delimit_comment___ 0x6 + #define ___fl_flag_has_graph_first_do_escape___ 0x5 + #define ___fl_flag_has_line_first___ 0x8 + #define ___fl_flag_has_line_first_graph_first___ 0xc + #define ___fl_flag_has_line_first_delimit_comment___ 0xa + const f_number_unsigned_t delimits_used = delimits->used; const f_number_unsigned_t comments_used = comments->used; const f_number_unsigned_t begin = range->start; found->array[found->used].start = range->start; - f_number_unsigned_t newline_last = range->start; + f_number_unsigned_t comment_delimit = 0; + f_number_unsigned_t newline_last = begin; + f_number_unsigned_t previous = 0; f_number_unsigned_t slash_first = 0; f_number_unsigned_t slash_count = 0; - f_number_unsigned_t start = 0; - f_number_unsigned_t comment_delimit = 0; + f_number_unsigned_t line_start = begin; + + uint8_t flag_fl = ___fl_flag_has_line_first_graph_first___; - uint8_t graph_first = 0x1; // 0x0 = false, 0x1 = true, 0x2 = false, but there is a delimited comment, comment_delimit is set. + // The start position with a combining/joiner must be treated as Content and not as an Object. + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); + } + + if (F_status_is_error(state->status)) return; - // Identify where the content ends. while (range->start <= range->stop && range->start < buffer.used) { if (state->interrupt) { @@ -60,25 +76,36 @@ extern "C" { } if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (graph_first == 0x2) { + if (flag_fl & ___fl_flag_has_delimit_comment___) { state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; delimits->array[delimits->used++] = comment_delimit; } + flag_fl |= ___fl_flag_has_graph_first___; + flag_fl &= ~___fl_flag_has_line_first_delimit_comment___; newline_last = range->start; - graph_first = 0x1; - ++range->start; + line_start = ++range->start; + + // Any line that starts with a combining/joiner must be treated as Content and not as an Object. + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); + } + + if (F_status_is_error(state->status)) break; continue; } if (buffer.string[range->start] == f_fss_slash_s.string[0]) { - slash_first = range->start; + slash_first = range->start++; slash_count = 1; + flag_fl |= ___fl_flag_do_escape___; - for (++range->start; range->start <= range->stop && range->start < buffer.used; ++range->start) { + for (; range->start <= range->stop && range->start < buffer.used; ++range->start) { if (state->interrupt) { state->interrupt((void *) state, 0); @@ -86,40 +113,54 @@ extern "C" { } if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) continue; + + // If any slash has a combining/joiner following it, then there is no escaping at all for that slash and therefore it is not considered an escaping character. + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; + + f_fss_skip_past_combining_joiner(buffer, range, state); + + break; + } + + if (F_status_is_error(state->status)) break; if (buffer.string[range->start] != f_fss_slash_s.string[0]) break; ++slash_count; } // for + if (F_status_is_error(state->status)) break; if (range->start > range->stop || range->start >= buffer.used) break; + if (!(flag_fl & ___fl_flag_do_escape___)) continue; + + flag_fl &= ~___fl_flag_do_escape___; if (buffer.string[range->start] == f_fss_basic_list_open_s.string[0]) { - graph_first = 0x0; + flag_fl &= ~___fl_flag_has_graph_first___; ++range->start; - while (range->start <= range->stop && range->start < buffer.used) { - - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } + f_fss_skip_past_space(buffer, range, state); - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; - if (f_fss_is_space(buffer, *range, state) == F_false) break; + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; - state->status = f_utf_buffer_increment(buffer, range, 1); + f_fss_skip_past_combining_joiner(buffer, range, state); if (F_status_is_error(state->status)) break; - } // while - if (range->start > range->stop || range->start >= buffer.used) break; + continue; + } + + if (state->status == F_data_not) return; + if (F_status_is_error(state->status) || range->start > range->stop || range->start >= buffer.used) break; - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - start = range->start; + if (state->status == F_okay_eol) { + previous = range->start; range->start = slash_first; if (slash_count % 2 == 0) { found->array[found->used++].stop = newline_last; - range->start = newline_last + 1; + range->start = previous + 1; state->status = F_fss_found_content; return; @@ -128,7 +169,7 @@ extern "C" { state->status = f_memory_array_increase_by((slash_count / 2) + 1, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; - while (slash_count > 0) { + while (slash_count) { if (buffer.string[range->start] == f_fss_slash_s.string[0]) { if (slash_count % 2 == 1) { @@ -144,21 +185,27 @@ extern "C" { if (F_status_is_error(state->status)) break; - if (graph_first == 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - - graph_first = 0x1; - range->start = start + 1; + range->start = previous; + state->status = F_okay_eol; } } - else if (graph_first == 0x1 && buffer.string[range->start] == f_fss_comment_s.string[0]) { - graph_first = 0x2; - comment_delimit = slash_first; + else if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_comment_s.string[0]) { ++range->start; + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; + + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + } + else { + if (F_status_is_error(state->status)) break; + + flag_fl |= ___fl_flag_has_delimit_comment___; + comment_delimit = slash_first; + } } continue; @@ -166,91 +213,93 @@ extern "C" { if (buffer.string[range->start] == f_fss_basic_list_open_s.string[0]) { ++range->start; - graph_first = 0x0; + flag_fl &= ~___fl_flag_has_graph_first___; - while (range->start <= range->stop && range->start < buffer.used) { + f_fss_skip_past_space(buffer, range, state); - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; - if (f_fss_is_space(buffer, *range, state) == F_false) break; + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; - state->status = f_utf_buffer_increment(buffer, range, 1); + f_fss_skip_past_combining_joiner(buffer, range, state); if (F_status_is_error(state->status)) break; - } // while + + continue; + } if (F_status_is_error(state->status)) break; if (range->start > range->stop || range->start >= buffer.used) break; - // Found a valid object, set stop point to last newline. - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (newline_last == found->array[found->used].start && buffer.string[found->array[found->used].start] != f_fss_eol_s.string[0]) { - range->start = newline_last; - - state->status = F_fss_found_content_not; - - return; - } - - range->start = newline_last + 1; - found->array[found->used++].stop = newline_last; - state->status = F_fss_found_content; - - return; - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (graph_first == 0x2) { + // Found a valid object, set stop point to last newline (if newline_last is begin, there newline_last is not a new line). + if (state->status == F_okay_eol) { + if (flag_fl & ___fl_flag_has_delimit_comment___) { state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; delimits->array[delimits->used++] = comment_delimit; } - newline_last = range->start; - graph_first = 0x1; + if (flag_fl & ___fl_flag_has_line_first___) { + found->array[found->used].start = 1; + found->array[found->used++].stop = 0; + range->start = newline_last; + state->status = F_fss_found_content_not; + } + else { + range->start = begin == newline_last ? newline_last : newline_last + 1; + found->array[found->used++].stop = newline_last; + state->status = F_fss_found_content; + } + + return; } continue; } - if (graph_first == 0x1 && buffer.string[range->start] == f_fss_comment_s.string[0]) { + if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_comment_s.string[0]) { + previous = range->start++; - // The newline_last is initialized to the range->start, which may not actually be a new line. - if (buffer.string[newline_last] == f_string_eol_s.string[0]) { - start = newline_last + 1; - } - else { - start = newline_last; + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; + + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; } - f_fss_seek_to_eol(buffer, range, state); if (F_status_is_error(state->status)) break; - state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &comments->array, &comments->used, &comments->size); - if (F_status_is_error(state->status)) break; + range->start = previous; - if (range->start > range->stop || range->start >= buffer.used) { - --range->start; - } - else { - newline_last = range->start; + line_start = flag_fl & ___fl_flag_has_line_first___ + ? begin + : newline_last + 1; + + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &comments->array, &comments->used, &comments->size); } - comments->array[comments->used].start = start; - comments->array[comments->used++].stop = range->start++; + if (F_status_is_error(state->status)) break; + + comments->array[comments->used].start = line_start; + comments->array[comments->used++].stop = range->start; + newline_last = range->start; + state->status = F_okay_eol; continue; } - if (graph_first == 0x1) { + if (flag_fl & ___fl_flag_has_graph_first___) { if (f_fss_is_space(buffer, *range, state) == F_false) { if (F_status_is_error(state->status)) break; - graph_first = 0x0; + flag_fl &= ~___fl_flag_has_graph_first___; } } @@ -258,6 +307,14 @@ extern "C" { if (F_status_is_error(state->status)) break; } // while + if (flag_fl & ___fl_flag_has_delimit_comment___) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); + + if (F_status_is_error_not(state->status)) { + delimits->array[delimits->used++] = comment_delimit; + } + } + if (F_status_is_error(state->status)) { delimits->used = delimits_used; comments->used = comments_used; @@ -265,6 +322,7 @@ extern "C" { return; } + // The EOS and stop cases only happen here when the next object is not found, so do not increment found.used and let caller handle. if (range->start >= buffer.used) { found->array[found->used].stop = buffer.used - 1; state->status = F_okay_eos; @@ -279,15 +337,25 @@ extern "C" { return; } - if (range->start > begin) { - found->array[found->used++].stop = range->start - 1; - } - else { + if (flag_fl & ___fl_flag_has_line_first___) { + state->status = F_fss_found_content_not; + found->array[found->used].start = 1; found->array[found->used++].stop = 0; } + else { + state->status = F_fss_found_content; - state->status = F_fss_found_content; + found->array[found->used++].stop = range->start - 1; + } + + #undef ___fl_flag_do_escape___ + #undef ___fl_flag_has_delimit_comment___ + #undef ___fl_flag_has_graph_first___ + #undef ___fl_flag_has_graph_first_delimit_comment___ + #undef ___fl_flag_has_graph_first_do_escape___ + #undef ___fl_flag_has_line_first___ + #undef ___fl_flag_has_line_first_graph_first___ } #endif // _di_fl_fss_basic_list_content_read_ @@ -329,14 +397,23 @@ extern "C" { state->status = f_memory_array_increase_by(3, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_do_prepend___ 0x1 + #define ___fl_flag_has_graph___ 0x2 + #define ___fl_flag_is_comment___ 0x4 + #define ___fl_flag_is_combining___ 0x8 + #define ___fl_flag_is_combining_graph___ 0xa + #define ___fl_flag_is_joined___ 0x10 + #define ___fl_flag_is_joiner___ 0x20 + #define ___fl_flag_is_joiner_graph___ 0x22 + #define ___fl_flag_is_joinerd___ 0x30 + const f_number_unsigned_t destination_used = destination->used; - uint8_t is_comment = F_false; - uint8_t has_graph = F_false; - uint8_t do_prepend = prepend ? F_true : F_false; + uint8_t flag_fl = prepend ? ___fl_flag_do_prepend___ : 0x0; uint8_t width = 0; f_number_unsigned_t i = 0; + f_number_unsigned_t previous = 0; f_number_unsigned_t slash_count = 0; f_number_unsigned_t start = 0; @@ -347,18 +424,50 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } - if (content.string[range->start] == f_fss_slash_s.string[0] && !is_comment) { + if (flag_fl & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(content, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; + + flag_fl &= ~___fl_flag_is_combining___; + } + } + + if (flag_fl & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(content, *range, state) == F_true) { + flag_fl |= ___fl_flag_is_joiner___; + flag_fl &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag_fl & ___fl_flag_is_joiner___) { + flag_fl |= ___fl_flag_is_joined___; + flag_fl &= ~___fl_flag_is_joiner___; + } + else { + flag_fl &= ~___fl_flag_is_joined___; + } + } + } + + if (content.string[range->start] == f_fss_slash_s.string[0] && !(flag_fl & ___fl_flag_is_comment___)) { slash_count = 1; - if (do_prepend) { + if (flag_fl & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag_fl &= ~___fl_flag_do_prepend___; } destination->string[destination->used++] = content.string[range->start]; - has_graph = F_true; + flag_fl |= ___fl_flag_has_graph___; for (++range->start; range->start <= range->stop && range->start < content.used; ++range->start) { @@ -386,9 +495,21 @@ extern "C" { start = range->start++; f_fss_skip_past_space(content, range, state); + + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + /*if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; + slash_count = 0; + + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + }*/ + if (F_status_is_error(state->status)) break; - if (content.string[range->start] == f_fss_eol_s.string[0] || range->start >= content.used || range->start > range->stop) { + if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { // Increase by total slashes + 1, along with the basic list open and possible newline. state->status = f_memory_array_increase_by(slash_count + 3, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); @@ -418,22 +539,36 @@ extern "C" { continue; } } - else if (content.string[range->start] == f_fss_basic_list_open_s.string[0] && !is_comment) { + else if (content.string[range->start] == f_fss_basic_list_open_s.string[0] && !(flag_fl & ___fl_flag_is_comment___)) { start = range->start++; - if (do_prepend) { + if (flag_fl & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag_fl &= ~___fl_flag_do_prepend___; } - has_graph = F_true; + flag_fl |= ___fl_flag_has_graph___; + + f_fss_skip_past_space(content, range, state); // TODO: destination->string[destination->used++] = f_fss_slash_s.string[0]; don't skip entirely, also write! Do this for all before continuing based on the "start" position above. + + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + /* + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + flag_fl &= ~___fl_flag_has_graph_first_do_escape___; + slash_count = 0; + + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + } + */ - f_fss_skip_past_space(content, range, state); if (F_status_is_error(state->status)) break; - if (content.string[range->start] == f_fss_eol_s.string[0] || range->start >= content.used || range->start > range->stop) { + if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { // Increase by slash and basic list open and possible newline. state->status = f_memory_array_increase_by(3, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); @@ -458,32 +593,46 @@ extern "C" { continue; } - else if (content.string[range->start] == f_fss_comment_s.string[0] && !has_graph) { - is_comment = F_true; + else if (content.string[range->start] == f_fss_comment_s.string[0] && !(flag_fl & ___fl_flag_has_graph___)) { + flag_fl |= ___fl_flag_is_comment___; } else if (content.string[range->start] == f_fss_eol_s.string[0]) { - has_graph = F_false; - is_comment = F_false; + flag_fl = 0x0; } - else { - if (f_fss_is_graph(content, *range, state) == F_false) { - if (F_status_is_error(state->status)) break; - } - else { - has_graph = F_true; + else if (f_fss_is_graph(content, *range, state) == F_false) { + if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(content, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(content, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag_fl |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag_fl |= ___fl_flag_is_joiner_graph___; + } } + + if (F_status_is_error(state->status)) break; + + range->start = previous; + } + else { + flag_fl |= ___fl_flag_has_graph___; } if (content.string[range->start] != f_fss_placeholder_s.string[0]) { - if (do_prepend) { + if (flag_fl & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag_fl &= ~___fl_flag_do_prepend___; } if (content.string[range->start] == f_fss_eol_s.string[0] && prepend) { - do_prepend = F_true; + flag_fl |= ___fl_flag_do_prepend___; } width = macro_f_utf_byte_width(content.string[range->start]); @@ -536,6 +685,16 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_do_prepend___ + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_basic_list_content_write_ @@ -595,9 +754,21 @@ extern "C" { return; } + #define ___fl_flag_has_graph___ 0x1 + #define ___fl_flag_is_comment___ 0x2 + #define ___fl_flag_is_combining___ 0x4 + #define ___fl_flag_is_combining_graph___ 0x5 + #define ___fl_flag_is_joined___ 0x8 + #define ___fl_flag_is_joiner___ 0x10 + #define ___fl_flag_is_joiner_graph___ 0x11 + #define ___fl_flag_is_joinerd___ 0x18 + const f_number_unsigned_t destination_used = destination->used; + + f_number_unsigned_t previous = 0; f_number_unsigned_t slash_count = 0; + uint8_t flag_fl = 0; uint8_t width = 0; // Find the first graph character. @@ -619,8 +790,57 @@ extern "C" { break; } + if (flag_fl & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(object, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; + + flag_fl &= ~___fl_flag_is_combining___; + } + } + + if (flag_fl & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(object, *range, state) == F_true) { + flag_fl |= ___fl_flag_is_joiner___; + flag_fl &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag_fl & ___fl_flag_is_joiner___) { + flag_fl |= ___fl_flag_is_joined___; + flag_fl &= ~___fl_flag_is_joiner___; + } + else { + flag_fl &= ~___fl_flag_is_joined___; + } + } + } + if (f_fss_is_graph(object, *range, state) == F_false) { if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(object, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(object, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag_fl |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag_fl |= ___fl_flag_is_joiner_graph___; + } + } + + if (F_status_is_error(state->status)) break; + + range->start = previous; } else { break; @@ -770,6 +990,15 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_basic_list_object_write_ diff --git a/level_1/fl_fss/c/fss/basic_list.h b/level_1/fl_fss/c/fss/basic_list.h index 38bab7d33..b6d7165b6 100644 --- a/level_1/fl_fss/c/fss/basic_list.h +++ b/level_1/fl_fss/c/fss/basic_list.h @@ -34,6 +34,12 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * + * The content is still technically valid on F_okay_eos and F_okay_stop state.status values, but because this function cannot actually know it will not increment found.used. + * However, the start and stop values that are technically valid content are still stored. + * The caller should check for the F_okay_eos and F_okay_stop values and increment found.used as desired. + * * @param buffer * The buffer to read from. * @param range @@ -71,7 +77,7 @@ extern "C" { * * This alters state.status: * F_fss_found_content on success and Content is found (start location is at end of Content). - * F_fss_found_content_not on success and no Content is found (start location is after character designating this is not a Content). + * F_fss_found_content_not on success and no Content is found or a valid Object is found before any Content is found (start location is after the character designating this is not a Content). * F_okay_eos on success after reaching the end of the buffer (a valid Object is not yet confirmed). * F_okay_stop on success after reaching stopping point (a valid Object is not yet confirmed). * F_data_not if buffer is empty (buffer.used is 0). @@ -86,6 +92,7 @@ extern "C" { * Errors (with error bit) from: f_fss_is_space(). * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_fss_skip_past_delimit(). + * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_utf_buffer_increment(). @@ -93,6 +100,7 @@ extern "C" { * @see f_fss_is_space() * @see f_fss_seek_to_eol() * @see f_fss_skip_past_delimit() + * @see f_fss_skip_past_space() * @see f_memory_array_increase() * @see f_memory_array_increase_by() * @see f_utf_buffer_increment() @@ -109,6 +117,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param content * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -154,7 +164,10 @@ extern "C" { * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_memory_array_increase(). @@ -162,7 +175,10 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_append(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_skip_past_delimit() * @see f_fss_skip_past_space() * @see f_memory_array_increase() @@ -179,6 +195,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -253,6 +271,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -293,14 +313,20 @@ extern "C" { * F_okay_eol (with error bit) after reaching an EOL, which is not supported by the standard. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_is_space(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_is_space() * @see f_memory_array_increase() * @see f_memory_array_increase_by() diff --git a/level_1/fl_fss/c/fss/embedded_list.c b/level_1/fl_fss/c/fss/embedded_list.c index 63d18ccb9..49c990a4a 100644 --- a/level_1/fl_fss/c/fss/embedded_list.c +++ b/level_1/fl_fss/c/fss/embedded_list.c @@ -49,22 +49,27 @@ extern "C" { if (F_status_is_error(state->status)) return; + #define ___fl_flag_do_escape___ 0x1 + #define ___fl_flag_has_graph_first___ 0x2 + #define ___fl_flag_has_joiner_combining___ 0x4 + #define ___fl_flag_is_object___ 0x8 + #define ___fl_flag_is_open___ 0x10 + #define ___fl_flag_is_open_object___ 0x18 + const f_number_unsigned_t delimits_used = delimits->used; const f_number_unsigned_t comments_used = comments->used; f_number_unsigned_t depth = 0; f_number_unsigned_t position = 0; - f_number_unsigned_t position_previous = range->start; + f_number_unsigned_t previous = range->start; f_number_unsigned_t line_start = range->start; f_number_unsigned_t newline_last = range->start; - f_number_unsigned_t comment_delimit = 0; f_number_unsigned_t slash_first = 0; f_number_unsigned_t slash_last = 0; - f_number_unsigned_t before_list_open = position_previous; + f_number_unsigned_t before_list_open = previous; f_number_unsigned_t close_location = 0; - // 0x0 = is false for all, 0x1 = is first graph, 0x2 = is delimited comment, 0x4 = is open, 0x8 = is object, 0x10 = is delimited close. - uint8_t is_state = 0x1; + uint8_t flag_fl = ___fl_flag_has_graph_first___; // Initialize depth 1 start position. // Positions_start.used is used as a max depth (such that cache->positions->used == max depth + 1). @@ -74,6 +79,13 @@ extern "C" { cache->slashes->array[0] = 0; cache->slashes->used = 1; + // The start position with a combining/joiner must be treated as Content and not as an Object. + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); + } + + if (F_status_is_error(state->status)) return; + while (range->start <= range->stop && range->start < buffer.used) { if (state->interrupt) { @@ -82,19 +94,36 @@ extern "C" { } if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; + newline_last = previous = range->start++; + line_start = range->start; + flag_fl |= ___fl_flag_has_graph_first___; - delimits->array[delimits->used++] = comment_delimit; + // Any line that starts with a combining/joiner must be treated as Content and not as an Object. + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); } - newline_last = position_previous = range->start++; - line_start = range->start; - is_state = 0x1; + if (F_status_is_error(state->status)) break; } - else if (buffer.string[range->start] == f_fss_slash_s.string[0] && (is_state & 0x1)) { - slash_first = slash_last = position_previous = range->start++; + else if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_slash_s.string[0]) { + previous = range->start++; + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + } + + if (F_status_is_error(state->status)) break; + + slash_first = slash_last = previous; cache->slashes->array[depth] = 1; for (; range->start <= range->stop && range->start < buffer.used && (buffer.string[range->start] == f_fss_placeholder_s.string[0] || buffer.string[range->start] == f_fss_slash_s.string[0]); ++range->start) { @@ -104,15 +133,36 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } - position_previous = range->start; + previous = range->start; + + if (buffer.string[previous] == f_fss_slash_s.string[0]) { + ++range->start; + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + cache->slashes->array[depth] = 0; + flag_fl |= ___fl_flag_has_joiner_combining___; + + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + + break; + } - if (buffer.string[position_previous] == f_fss_slash_s.string[0]) { - slash_last = position_previous; + range->start = previous; + slash_last = previous; ++cache->slashes->array[depth]; } } // for - if (F_status_is_error(state->status) || state->status == F_interrupt || range->start >= buffer.used || range->start > range->stop) break; + if (F_status_is_error(state->status) || range->start >= buffer.used || range->start > range->stop) break; + + if (flag_fl & ___fl_flag_has_joiner_combining___) { + flag_fl &= ~___fl_flag_has_joiner_combining___; + + continue; + } // All slashes for an open are delimited (because it could represent a slash in the object name). // For example 'object {' = valid open, name 'object', 'object \{' represents 'object {', 'object \\{' = valid open, name 'object \', 'object \\\{' represents 'object \{', etc.. @@ -120,59 +170,48 @@ extern "C" { // For example '}' = valid close, '\}' represents '}', '\\}' represents '\}', '\\\}' represents '\\}', '\\\\}' represents '\\\}', and so on.. // When slash is odd and a (delimited) valid open/close is found, then save delimited positions and continue. if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - - newline_last = position_previous = range->start++; - line_start = range->start; - is_state = 0x1; + // Do nothing. } else if (buffer.string[range->start] == f_fss_embedded_list_open_s.string[0] || buffer.string[range->start] == f_fss_embedded_list_close_s.string[0]) { - before_list_open = position_previous; - is_state &= ~0x15; + before_list_open = range->start++; - if (buffer.string[range->start] == f_fss_embedded_list_open_s.string[0]) { - is_state |= 0x4; - } + f_fss_skip_past_delimit(buffer, range, state); - position_previous = range->start++; + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { - while (range->start <= range->stop && range->start < buffer.used) { + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } + continue; + } - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; + if (F_status_is_error(state->status)) break; - if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) { - if (f_fss_is_space(buffer, *range, state) == F_false) break; - } + flag_fl &= ~___fl_flag_has_graph_first___; - position_previous = range->start; + // Cannoy already have an open to be valid. + if (buffer.string[before_list_open] == f_fss_embedded_list_open_s.string[0]) { + flag_fl |= ___fl_flag_is_open___; + } - f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) break; - } // while + range->start = before_list_open; + before_list_open = previous; + previous = range->start++; + + f_fss_skip_past_space_previous(buffer, range, state, &previous); - if (F_status_is_error(state->status) || state->status == F_interrupt || range->start >= buffer.used || range->start > range->stop) break; + if (F_status_is_error(state->status)) break; + if (range->start > range->stop || range->start >= buffer.used) break; - // This is a valid object open/close that has been delimited, save the slash delimit positions. - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + // This is an otherwise valid object open/close that has been delimited, save the slash delimit positions. + if (state->status == F_okay_eol) { newline_last = range->start; line_start = newline_last + 1; - if (is_state & 0x4) { + if (flag_fl & ___fl_flag_is_open___) { if (cache->slashes->array[depth] % 2 == 0) { - is_state |= 0x8; - } - else { - is_state &= ~0x8; + flag_fl |= ___fl_flag_is_object___; } range->start = slash_first; @@ -192,24 +231,24 @@ extern "C" { } // Delimit slashes and placeholders are required to be in the ASCII range. - position_previous = range->start++; + previous = range->start++; } // while // When slashes are even, the object is valid and needs to be processed. - if (is_state & 0x8) { + if (flag_fl & ___fl_flag_is_object___) { if (++depth >= cache->objects->size) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_range_t), (void **) &cache->objects->array, &cache->objects->used, &cache->objects->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_range_t), (void **) &cache->objects->array, &cache->objects->used, &cache->objects->size); } if (depth >= cache->positions->size) { if (F_status_is_error_not(state->status)) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_number_unsigned_t), (void **) &cache->positions->array, &cache->positions->used, &cache->positions->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_number_unsigned_t), (void **) &cache->positions->array, &cache->positions->used, &cache->positions->size); } } if (depth >= cache->slashes->size) { if (F_status_is_error_not(state->status)) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_number_unsigned_t), (void **) &cache->slashes->array, &cache->slashes->used, &cache->slashes->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_number_unsigned_t), (void **) &cache->slashes->array, &cache->slashes->used, &cache->slashes->size); } } @@ -232,74 +271,88 @@ extern "C" { cache->slashes->used = depth; } } + + range->start = newline_last; + flag_fl &= ~___fl_flag_is_open_object___; + + continue; } + // An otherwise valid close must be delimited appropriately and only has the first slash delimited. else { - is_state |= 0x10; - } - - if (is_state & 0x12) { state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; - delimits->array[delimits->used++] = slash_last; - } + delimits->array[delimits->used++] = slash_first; + range->start = newline_last; - range->start = line_start; - is_state = 0x1; + continue; + } } + + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; } - else if (buffer.string[range->start] == f_fss_comment_s.string[0] && (is_state & 0x1)) { - is_state |= 0x2; - is_state &= ~0x1; - comment_delimit = slash_first; - } - else { - is_state &= ~0x1; - } - } - else if (buffer.string[range->start] == f_fss_embedded_list_open_s.string[0]) { - is_state &= ~0x1; - before_list_open = position_previous; - position_previous = range->start; + else if (buffer.string[range->start] == f_fss_comment_s.string[0]) { + previous = range->start++; - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) break; + f_fss_skip_past_delimit(buffer, range, state); - while (range->start <= range->stop && range->start < buffer.used) { + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; + continue; + } - if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) { - if (f_fss_is_space(buffer, *range, state) == F_false) break; + if (F_status_is_error_not(state->status)) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); } - position_previous = range->start; + if (F_status_is_error(state->status)) break; + + delimits->array[delimits->used++] = slash_first; + range->start = previous; + } + + flag_fl &= ~___fl_flag_has_graph_first___; + } + else if (buffer.string[range->start] == f_fss_embedded_list_open_s.string[0]) { + before_list_open = previous; + previous = range->start++; - f_utf_buffer_increment(buffer, range, 1); + f_fss_skip_past_space_previous(buffer, range, state, &previous); + + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); if (F_status_is_error(state->status)) break; - } // while - if (F_status_is_error(state->status) || state->status == F_interrupt || range->start >= buffer.used || range->start > range->stop) break; + continue; + } - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + if (F_status_is_error(state->status)) break; + if (range->start > range->stop || range->start >= buffer.used) break; + + flag_fl &= ~___fl_flag_has_graph_first___; + + if (state->status == F_okay_eol) { if (++depth >= cache->objects->size) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_range_t), (void **) &cache->objects->array, &cache->objects->used, &cache->objects->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_range_t), (void **) &cache->objects->array, &cache->objects->used, &cache->objects->size); } if (depth >= cache->positions->size) { if (F_status_is_error_not(state->status)) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_number_unsigned_t), (void **) &cache->positions->array, &cache->positions->used, &cache->positions->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_number_unsigned_t), (void **) &cache->positions->array, &cache->positions->used, &cache->positions->size); } } if (depth >= cache->slashes->size) { if (F_status_is_error_not(state->status)) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_number_unsigned_t), (void **) &cache->slashes->array, &cache->slashes->used, &cache->slashes->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_number_unsigned_t), (void **) &cache->slashes->array, &cache->slashes->used, &cache->slashes->size); } } @@ -321,61 +374,35 @@ extern "C" { if (cache->slashes->used <= depth) { cache->slashes->used = depth + 1; } - - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - - newline_last = range->start; - line_start = newline_last + 1; - is_state = 0x1; } else { // No valid object close found, seek until EOL. f_fss_seek_to_eol(buffer, range, state); if (F_status_is_error(state->status)) break; - - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - - newline_last = range->start; - line_start = newline_last + 1; - is_state = 0x1; } } - else if (buffer.string[range->start] == f_fss_embedded_list_close_s.string[0] && (is_state & 0x1)) { - close_location = range->start; + else if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_embedded_list_close_s.string[0]) { + close_location = previous = range->start++; - while (range->start <= range->stop && range->start < buffer.used) { - - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } + f_fss_skip_past_space_previous(buffer, range, state, &previous); - position_previous = range->start; + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status) || buffer.string[range->start] == f_fss_eol_s.string[0]) break; + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; - if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) { - if (f_fss_is_space(buffer, *range, state) == F_false || F_status_is_error(state->status)) break; - } - } // while + continue; + } - if (F_status_is_error(state->status) || state->status == F_interrupt || range->start >= buffer.used || range->start > range->stop) break; + if (F_status_is_error(state->status)) break; + if (range->start > range->stop || range->start >= buffer.used) break; - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + if (state->status == F_okay_eol) { if (depth >= found->used) { - state->status = f_memory_array_resize(depth + 2, sizeof(f_fss_nest_t), (void **) &found->depth, &found->used, &found->size); + state->status = f_memory_array_resize(depth + state->step_small, sizeof(f_fss_nest_t), (void **) &found->depth, &found->used, &found->size); } if (found->depth[depth].used >= found->depth[depth].size) { @@ -422,21 +449,16 @@ extern "C" { found->used = depth + 1; } - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - found->depth[depth].array[position].close.start = line_start; found->depth[depth].array[position].close.stop = close_location; newline_last = range->start; line_start = newline_last + 1; - is_state = 0x1; - if (!depth) { + if (depth) { + --depth; + } + else { state->status = f_utf_buffer_increment(buffer, range, 1); if (F_status_is_error(state->status)) { @@ -457,44 +479,31 @@ extern "C" { return; } - - --depth; } else { - // No valid object close found, seek until EOL. - while (range->start <= range->stop && range->start < buffer.used) { - - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; + } + } + else if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_comment_s.string[0]) { + previous = range->start++; - delimits->array[delimits->used++] = comment_delimit; - } + f_fss_skip_past_delimit(buffer, range, state); - newline_last = range->start; - line_start = newline_last + 1; - is_state = 0x1; + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { - break; - } + // This cannot be a valid object close at this point, seek until EOL. + f_fss_seek_to_eol(buffer, range, state); + if (F_status_is_error(state->status)) break; - position_previous = range->start; + continue; + } - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) break; - } // while + if (F_status_is_error(state->status)) break; - if (F_status_is_error(state->status) || state->status == F_interrupt) break; - } - } - else if (buffer.string[range->start] == f_fss_comment_s.string[0] && (is_state & 0x1)) { + range->start = previous; // The newline_last is initialized to the range->start, which may not actually be a new line. position = (buffer.string[newline_last] == f_string_eol_s.string[0]) ? newline_last + 1 : newline_last; @@ -506,16 +515,8 @@ extern "C" { --range->start; } else { - if (is_state & 0x2) { - state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); - if (F_status_is_error(state->status)) break; - - delimits->array[delimits->used++] = comment_delimit; - } - newline_last = range->start; line_start = newline_last + 1; - is_state = 0x1; } state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &comments->array, &comments->used, &comments->size); @@ -524,12 +525,12 @@ extern "C" { comments->array[comments->used].start = position; comments->array[comments->used++].stop = range->start++; } - else if (buffer.string[range->start] != f_fss_eol_s.string[0]) { - position_previous = range->start; + else { + previous = range->start; - if (is_state & 0x1) { + if (flag_fl & ___fl_flag_has_graph_first___) { if (f_fss_is_space(buffer, *range, state) == F_false) { - is_state &= ~0x1; + flag_fl &= ~___fl_flag_has_graph_first___; } } @@ -552,6 +553,13 @@ extern "C" { state->status = F_status_set_error(depth ? F_end_not_nest_eos : F_end_not_eos); } } + + #undef ___fl_flag_do_escape___ + #undef ___fl_flag_has_graph_first___ + #undef ___fl_flag_has_joiner_combining___ + #undef ___fl_flag_is_object___ + #undef ___fl_flag_is_open___ + #undef ___fl_flag_is_open_object___ } #endif // _di_fl_fss_embedded_list_content_read_ @@ -595,15 +603,26 @@ extern "C" { return; } + #define ___fl_flag_do_prepend___ 0x1 + #define ___fl_flag_has_eol___ 0x2 + #define ___fl_flag_has_eol_do_prepend___ 0x3 + #define ___fl_flag_has_graph___ 0x4 + #define ___fl_flag_is_comment___ 0x8 + #define ___fl_flag_is_comment_graph___ 0xc + #define ___fl_flag_is_combining___ 0x10 + #define ___fl_flag_is_combining_graph___ 0x14 + #define ___fl_flag_is_joined___ 0x20 + #define ___fl_flag_is_joiner___ 0x40 + #define ___fl_flag_is_joiner_graph___ 0x44 + #define ___fl_flag_is_joinerd___ 0x60 + const f_number_unsigned_t destination_used = destination->used; - uint8_t is_comment = F_false; - uint8_t ends_on_eol = F_false; - uint8_t has_graph = F_false; - uint8_t do_prepend = prepend ? F_true : F_false; + uint8_t flag = prepend ? ___fl_flag_do_prepend___ : 0x0; uint8_t width = 0; f_number_unsigned_t i = 0; + f_number_unsigned_t previous = 0; f_number_unsigned_t r = 0; f_number_unsigned_t slash_count = 0; f_number_unsigned_t start = 0; @@ -615,14 +634,46 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } - if (content.string[range->start] == f_fss_slash_s.string[0] && !is_comment) { + if (flag & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(content, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; + + flag &= ~___fl_flag_is_combining___; + } + } + + if (flag & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(content, *range, state) == F_true) { + flag |= ___fl_flag_is_joiner___; + flag &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag & ___fl_flag_is_joiner___) { + flag |= ___fl_flag_is_joined___; + flag &= ~___fl_flag_is_joiner___; + } + else { + flag &= ~___fl_flag_is_joined___; + } + } + } + + if (content.string[range->start] == f_fss_slash_s.string[0] && !(flag & ___fl_flag_is_comment___)) { slash_count = 1; - if (do_prepend) { + if (flag & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } state->status = f_memory_array_increase(state->step_large, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); @@ -662,7 +713,7 @@ extern "C" { f_fss_skip_past_space(content, range, state); if (F_status_is_error(state->status)) break; - if (range->start >= content.used || range->start > range->stop || content.string[range->start] == f_fss_eol_s.string[0]) { + if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { // Increase by total slashes + 1 embedded list open/close. state->status = f_memory_array_increase_by(slash_count + 2, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); @@ -683,7 +734,7 @@ extern "C" { if (F_status_is_error(state->status)) break; if (content.string[range->start] != f_fss_eol_s.string[0]) { - has_graph = F_true; + flag |= ___fl_flag_has_graph___; } continue; @@ -697,10 +748,10 @@ extern "C" { if (content.string[range->start] == f_fss_eol_s.string[0]) { destination->string[destination->used++] = f_fss_eol_s.string[0]; - ends_on_eol = F_true; + flag |= ___fl_flag_has_eol___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } range->start = start + 1; @@ -708,29 +759,27 @@ extern "C" { continue; } } - else if ((content.string[range->start] == f_fss_embedded_list_open_s.string[0] || (!has_graph && content.string[range->start] == f_fss_embedded_list_close_s.string[0])) && !is_comment) { + else if ((content.string[range->start] == f_fss_embedded_list_open_s.string[0] || (!(flag & ___fl_flag_has_graph___) && content.string[range->start] == f_fss_embedded_list_close_s.string[0])) && !(flag & ___fl_flag_is_comment___)) { start = range->start++; - if (do_prepend) { + if (flag & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } - has_graph = F_true; + flag |= ___fl_flag_has_graph___; f_fss_skip_past_space(content, range, state); if (F_status_is_error(state->status)) break; - if (range->start >= content.used || range->start > range->stop || content.string[range->start] == f_fss_eol_s.string[0]) { - - if (content.string[range->start] == f_fss_eol_s.string[0] && prepend) { - do_prepend = F_true; - ends_on_eol = F_true; + if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { + if (state->status == F_okay_eol && prepend) { + flag |= ___fl_flag_has_eol_do_prepend___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } if (ignore && ignore->used) { @@ -773,34 +822,49 @@ extern "C" { continue; } - else if (content.string[range->start] == f_fss_comment_s.string[0] && !has_graph) { - is_comment = F_true; + else if (content.string[range->start] == f_fss_comment_s.string[0] && !(flag & ___fl_flag_has_graph___)) { + flag |= ___fl_flag_is_comment___; } else if (content.string[range->start] == f_fss_eol_s.string[0]) { - has_graph = F_false; - is_comment = F_false; + flag &= ~___fl_flag_is_comment_graph___; } else if (f_fss_is_graph(content, *range, state) == F_false) { if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(content, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(content, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag |= ___fl_flag_is_joiner_graph___; + } + } + + if (F_status_is_error(state->status)) break; + + range->start = previous; } else { - has_graph = F_true; + flag |= ___fl_flag_has_graph___; } if (content.string[range->start] != f_fss_placeholder_s.string[0]) { - if (do_prepend) { + if (flag & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } if (content.string[range->start] == f_fss_eol_s.string[0] && prepend) { - do_prepend = F_true; - ends_on_eol = F_true; + flag |= ___fl_flag_has_eol_do_prepend___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } width = macro_f_utf_byte_width(content.string[range->start]); @@ -827,7 +891,7 @@ extern "C" { state->status = f_memory_array_increase_by(3, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) return; - if (!ends_on_eol) { + if (!(flag & ___fl_flag_has_eol___)) { destination->string[destination->used++] = f_fss_eol_s.string[0]; } @@ -844,6 +908,19 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_do_prepend___ + #undef ___fl_flag_has_eol___ + #undef ___fl_flag_has_eol_do_prepend___ + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_comment_graph___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_embedded_list_content_write_ @@ -876,8 +953,7 @@ extern "C" { #endif // _di_level_1_parameter_checking_ f_fss_skip_past_delimit(object, range, state); - if (F_status_is_error(state->status)) return; - if (state->status == F_data_not) return; + if (F_status_is_error(state->status) || state->status == F_data_not) return; if (state->status == F_okay_eos) { state->status = F_data_not_eos; @@ -912,11 +988,22 @@ extern "C" { state->status = f_memory_array_increase_by(4, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_ends_on_space___ 0x1 + #define ___fl_flag_has_graph___ 0x2 + #define ___fl_flag_is_comment___ 0x4 + #define ___fl_flag_is_combining___ 0x8 + #define ___fl_flag_is_combining_graph___ 0xa + #define ___fl_flag_is_joined___ 0x10 + #define ___fl_flag_is_joiner___ 0x20 + #define ___fl_flag_is_joiner_graph___ 0x22 + #define ___fl_flag_is_joinerd___ 0x30 + const f_number_unsigned_t destination_used = destination->used; - f_number_unsigned_t slash_count = 0; - uint8_t ends_on_space = F_false; + f_number_unsigned_t previous = 0; + f_number_unsigned_t slash_count = 0; + uint8_t flag = 0x0; uint8_t width = 0; // Find the first graph character. @@ -927,6 +1014,38 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } + if (flag & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(object, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; + + flag &= ~___fl_flag_is_combining___; + } + } + + if (flag & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(object, *range, state) == F_true) { + flag |= ___fl_flag_is_joiner___; + flag &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag & ___fl_flag_is_joiner___) { + flag |= ___fl_flag_is_joined___; + flag &= ~___fl_flag_is_joiner___; + } + else { + flag &= ~___fl_flag_is_joined___; + } + } + } + if (object.string[range->start] == f_fss_comment_s.string[0]) { // When a comment is found, escape it. @@ -940,6 +1059,23 @@ extern "C" { if (f_fss_is_graph(object, *range, state) == F_false) { if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(object, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(object, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag |= ___fl_flag_is_joiner_graph___; + } + } + + if (F_status_is_error(state->status)) break; + + range->start = previous; } else { break; @@ -1026,7 +1162,7 @@ extern "C" { } // while if (range->start > range->stop || range->start >= object.used) { - ends_on_space = F_false; + flag &= ~___fl_flag_ends_on_space___; break; } @@ -1043,7 +1179,7 @@ extern "C" { if (F_status_is_error(state->status)) break; } else { - ends_on_space = F_true; + flag |= ___fl_flag_ends_on_space___; if (object.string[range->start] == f_fss_eol_s.string[0]) { state->status = F_status_set_error(F_okay_eol); @@ -1083,7 +1219,7 @@ extern "C" { } // Prevent a space from being added post-trimming. - ends_on_space = F_true; + flag |= ___fl_flag_ends_on_space___; } if (complete != f_fss_complete_trim_e) { @@ -1095,7 +1231,7 @@ extern "C" { return; } - if (!ends_on_space) { + if (!(flag & ___fl_flag_ends_on_space___)) { destination->string[destination->used++] = f_fss_space_s.string[0]; } @@ -1116,6 +1252,16 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_ends_on_space___ + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_embedded_list_object_write_ diff --git a/level_1/fl_fss/c/fss/embedded_list.h b/level_1/fl_fss/c/fss/embedded_list.h index eb7142193..58d42d897 100644 --- a/level_1/fl_fss/c/fss/embedded_list.h +++ b/level_1/fl_fss/c/fss/embedded_list.h @@ -74,6 +74,8 @@ extern "C" { * Therefore the object for this content will also need to be found.array[0].array[found.array[0].used].object. * The object at depth 0 will not be populated, but all nested objects will be populated at their respective depth along with the content. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -155,6 +157,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param content * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -205,7 +209,10 @@ extern "C" { * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_memory_array_increase(). @@ -213,7 +220,10 @@ extern "C" { * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_skip_past_delimit() * @see f_fss_skip_past_space() * @see f_memory_array_increase() @@ -230,6 +240,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -304,6 +316,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -344,14 +358,20 @@ extern "C" { * F_okay_eol (with error bit) after reaching an EOL, which is not supported by the standard. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_is_space(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_is_space() * @see f_fss_skip_past_delimit() * @see f_memoy_array_increase() diff --git a/level_1/fl_fss/c/fss/extended.c b/level_1/fl_fss/c/fss/extended.c index 2d64e5402..aa21a4b55 100644 --- a/level_1/fl_fss/c/fss/extended.c +++ b/level_1/fl_fss/c/fss/extended.c @@ -18,8 +18,7 @@ extern "C" { #endif // _di_level_1_parameter_checking_ f_fss_skip_past_space(buffer, range, state); - if (F_status_is_error(state->status)) return; - if (state->status == F_data_not) return; + if (F_status_is_error(state->status) || state->status == F_data_not) return; if (state->status == F_okay_eol) { ++range->start; @@ -46,7 +45,6 @@ extern "C" { uint8_t content_found = 0; uint8_t quote = f_fss_quote_type_none_e; f_range_t content_partial = f_range_t_initialize; - f_status_t status = F_okay; while (range->start <= range->stop && range->start < buffer.used) { @@ -66,13 +64,14 @@ extern "C" { } if (state->status == F_fss_found_object || state->status == F_fss_found_object_content_not) { - status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); // The private function sets the error bit on unterminated quoted Object. if (state->status == F_status_set_error(F_fss_found_object_content_not)) { state->status = F_fss_found_object_content_not; } + status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); + if (F_status_is_error_not(status) && quotes) { status = f_memory_array_increase(state->step_small, sizeof(uint8_t), (void **) "es->array, "es->used, "es->size); } @@ -97,18 +96,26 @@ extern "C" { content_found = 1; if (state->status == F_fss_found_object_content_not) break; + + continue; } - else if (state->status == F_fss_found_object_not) { - break; + + if (F_status_is_error(state->status)) { + delimits->used = delimits_used; + + if (quotes) { + quotes->used = quotes_used; + } + + return; } - else if (state->status == F_data_not_eos) { + + if (state->status == F_data_not_eos) { if (content_found) { state->status = F_okay_eos; } content_found = 2; - - break; } else if (state->status == F_data_not_stop) { if (content_found) { @@ -116,25 +123,14 @@ extern "C" { } content_found = 2; - - break; } else if (state->status == F_end_not_group_eos || state->status == F_end_not_group_eos) { - if (content_found) { + if (content_found == 1) { content_found = 2; } - - break; } - else if (F_status_is_error(state->status)) { - delimits->used = delimits_used; - if (quotes) { - quotes->used = quotes_used; - } - - return; - } + break; } // while if (content_found) { diff --git a/level_1/fl_fss/c/fss/extended.h b/level_1/fl_fss/c/fss/extended.h index 6ae5c15a6..86d5621c9 100644 --- a/level_1/fl_fss/c/fss/extended.h +++ b/level_1/fl_fss/c/fss/extended.h @@ -34,6 +34,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -100,7 +102,6 @@ extern "C" { * @see f_utf_buffer_increment() * @see fl_fss_basic_object_read() * @see fl_fss_extended_object_read() - * @see fl_fss_extended_content_read() */ #ifndef _di_fl_fss_extended_content_read_ extern void fl_fss_extended_content_read(const f_string_static_t buffer, f_range_t * const range, f_ranges_t * const found, f_uint8s_t * const quotes, f_number_unsigneds_t * const delimits, f_state_t * const state); @@ -114,6 +115,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param content * The string to write as (does not stop at NULLS, they are ignored and not written). * @param quote @@ -172,7 +175,6 @@ extern "C" { * @see f_utf_buffer_increment() * @see fl_fss_basic_object_write() * @see fl_fss_extended_object_write() - * @see fl_fss_extended_content_write() */ #ifndef _di_fl_fss_extended_content_write_ extern void fl_fss_extended_content_write(const f_string_static_t content, const uint8_t quote, const uint8_t complete, f_range_t * const range, f_string_dynamic_t * const destination, f_state_t * const state); @@ -183,6 +185,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -247,7 +251,6 @@ extern "C" { * @see f_memory_array_increase() * @see f_utf_buffer_increment() * @see fl_fss_basic_object_read() - * @see fl_fss_extended_object_read() * @see fl_fss_extended_content_read() */ #ifndef _di_fl_fss_extended_object_read_ @@ -263,6 +266,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). * @param quote diff --git a/level_1/fl_fss/c/fss/extended_list.c b/level_1/fl_fss/c/fss/extended_list.c index 4a88b779f..5babafe95 100644 --- a/level_1/fl_fss/c/fss/extended_list.c +++ b/level_1/fl_fss/c/fss/extended_list.c @@ -37,16 +37,30 @@ extern "C" { state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_has_line_first___ 0x1 + #define ___fl_flag_has_slash___ 0x2 + + const f_number_unsigned_t begin = range->start; const f_number_unsigned_t delimits_used = delimits->used; const f_number_unsigned_t comments_used = comments->used; - f_number_unsigned_t line_start = range->start; - f_number_unsigned_t newline_last = range->start; + f_number_unsigned_t line_start = begin; + f_number_unsigned_t newline_last = begin; + f_number_unsigned_t previous = 0; f_number_unsigned_t slash_first = 0; f_number_unsigned_t comment_start = 0; + uint8_t flag_fl = ___fl_flag_has_line_first___; + found->array[found->used].start = range->start; + // The start position with a combining/joiner must be treated as Content and not as an Object. + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); + } + + if (F_status_is_error(state->status)) return; + // Identify where the content ends. while (range->start <= range->stop && range->start < buffer.used) { @@ -55,12 +69,33 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } - f_fss_skip_past_space(buffer, range, state); - if (F_status_is_error(state->status)) break; + if (state->status != F_okay_eol) { + f_fss_skip_past_space(buffer, range, state); + + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + } + + if (F_status_is_error(state->status)) break; + } if (state->status == F_okay_eol) { newline_last = range->start++; line_start = range->start; + flag_fl &= ~___fl_flag_has_line_first___; + + // Any line that starts with a combining/joiner must be treated as Content and not as an Object. + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_seek_to_eol(buffer, range, state); + } + + if (F_status_is_error(state->status)) break; continue; } @@ -68,9 +103,10 @@ extern "C" { if (state->status == F_okay_eos || state->status == F_okay_stop) break; if (buffer.string[range->start] == f_fss_slash_s.string[0]) { - slash_first = range->start; + flag_fl |= ___fl_flag_has_slash___; + slash_first = range->start++; - for (++range->start; range->start <= range->stop && range->start < buffer.used; ++range->start) { + for (; range->start <= range->stop && range->start < buffer.used; ++range->start) { if (state->interrupt) { state->interrupt((void *) state, 0); @@ -78,11 +114,25 @@ extern "C" { } if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) continue; - if (buffer.string[range->start] != f_fss_slash_s.string[0]) break; + // If any slash has a combining/joiner following it, then there is no escaping at all for that slash and therefore it is not considered an escaping character. + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl |= ___fl_flag_has_slash___; + + f_fss_skip_past_combining_joiner(buffer, range, state); + + break; + } + + if (F_status_is_error(state->status)) break; + if (buffer.string[range->start] != f_fss_slash_s.string[0]) break; } // for + if (F_status_is_error(state->status)) break; if (range->start > range->stop || range->start >= buffer.used) break; + if (!(flag_fl & ___fl_flag_has_slash___)) continue; + + flag_fl &= ~___fl_flag_has_slash___; // Only the first slash before a close is delimited, all others are maintained. // For example '}' = valid close, '\}' represents '}', '\\}' represents '\}', '\\\}' represents '\\}', '\\\\}' represents '\\\}', and so on.. @@ -90,46 +140,51 @@ extern "C" { if (buffer.string[range->start] == f_fss_extended_list_close_s.string[0]) { ++range->start; - while (range->start <= range->stop && range->start < buffer.used) { - - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; - - if (f_fss_is_space(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) break; - - break; - } + f_fss_skip_past_space(buffer, range, state); - state->status = f_utf_buffer_increment(buffer, range, 1); + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_skip_past_combining_joiner(buffer, range, state); if (F_status_is_error(state->status)) break; - } // while - if (range->start > range->stop || range->start >= buffer.used) break; + continue; + } - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - newline_last = range->start; - line_start = range->start + 1; + if (F_status_is_error(state->status)) break; + if (range->start > range->stop || range->start >= buffer.used) break; + if (state->status == F_okay_eol) { state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; delimits->array[delimits->used++] = slash_first; + state->status = F_okay_eol; continue; } } else if (buffer.string[range->start] == f_fss_comment_s.string[0]) { + previous = ++range->start; + + f_fss_skip_past_delimit(buffer, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + } + + range->start = previous; + state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) break; delimits->array[delimits->used++] = slash_first; } + // It is safe to seek to EOL because only a single f_fss_extended_list_close_s can exist on a single line and be a valid close (aside from white space). f_fss_seek_to_eol(buffer, range, state); if (F_status_is_error(state->status)) break; @@ -139,47 +194,38 @@ extern "C" { if (buffer.string[range->start] == f_fss_extended_list_close_s.string[0]) { ++range->start; - while (range->start <= range->stop && range->start < buffer.used) { + f_fss_skip_past_space(buffer, range, state); - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) break; - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0]) break; - - if (f_fss_is_space(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) break; - - break; - } - - state->status = f_utf_buffer_increment(buffer, range, 1); + // The skip space returns F_complete_not_utf_start with error bit set on combining/joiner. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_skip_past_combining_joiner(buffer, range, state); if (F_status_is_error(state->status)) break; - } // while + + continue; + } if (F_status_is_error(state->status)) break; if (range->start > range->stop || range->start >= buffer.used) break; // Found a valid content close, set stop point to last newline. - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + if (state->status == F_okay_eol) { + state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &found->array, &found->used, &found->size); + if (F_status_is_error(state->status)) break; + close->start = line_start; close->stop = range->start - 1; ++range->start; - // If the last newline is the entire start, then there is no Content. - if (newline_last == found->array[found->used].start) { + if (flag_fl & ___fl_flag_has_line_first___) { found->array[found->used].start = 1; found->array[found->used++].stop = 0; - state->status = F_fss_found_content_not; - - return; } - - found->array[found->used++].stop = newline_last; - state->status = F_fss_found_content; + else { + found->array[found->used++].stop = newline_last; + state->status = F_fss_found_content; + } return; } @@ -188,6 +234,18 @@ extern "C" { } if (buffer.string[range->start] == f_fss_comment_s.string[0]) { + previous = ++range->start; + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) break; + + continue; + } + + range->start = previous; // The newline_last is initialized to the range->start, which may not actually be a new line. comment_start = (buffer.string[newline_last] == f_string_eol_s.string[0]) ? newline_last + 1 : newline_last; @@ -208,6 +266,7 @@ extern "C" { comments->array[comments->used].start = comment_start; comments->array[comments->used++].stop = range->start++; + flag_fl &= ~___fl_flag_has_line_first___; continue; } @@ -225,6 +284,9 @@ extern "C" { } state->status = F_fss_found_content_not; + + #undef ___fl_flag_has_line_first___ + #undef ___fl_flag_has_slash___ } #endif // _di_fl_fss_extended_list_content_read_ @@ -274,13 +336,24 @@ extern "C" { state->status = f_memory_array_increase_by(4, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_do_prepend___ 0x1 + #define ___fl_flag_has_eol___ 0x2 + #define ___fl_flag_has_eol_do_prepend___ 0x3 + #define ___fl_flag_has_graph___ 0x4 + #define ___fl_flag_is_comment___ 0x8 + #define ___fl_flag_is_comment_graph___ 0xc + #define ___fl_flag_is_combining___ 0x10 + #define ___fl_flag_is_combining_graph___ 0x14 + #define ___fl_flag_is_joined___ 0x20 + #define ___fl_flag_is_joiner___ 0x40 + #define ___fl_flag_is_joiner_graph___ 0x44 + #define ___fl_flag_is_joinerd___ 0x60 + const f_number_unsigned_t destination_used = destination->used; - uint8_t is_comment = F_false; - uint8_t ends_on_eol = F_false; - uint8_t has_graph = F_false; - uint8_t do_prepend = prepend ? F_true : F_false; + uint8_t flag = prepend ? ___fl_flag_do_prepend___ : 0x0; + f_number_unsigned_t previous = 0; f_number_unsigned_t r = 0; f_number_unsigned_t start = 0; @@ -293,16 +366,48 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } - if (content.string[range->start] == f_fss_slash_s.string[0] && !is_comment) { + if (flag & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(content, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; - if (do_prepend) { + flag &= ~___fl_flag_is_combining___; + } + } + + if (flag & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(content, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(content, *range, state) == F_true) { + flag |= ___fl_flag_is_joiner___; + flag &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag & ___fl_flag_is_joiner___) { + flag |= ___fl_flag_is_joined___; + flag &= ~___fl_flag_is_joiner___; + } + else { + flag &= ~___fl_flag_is_joined___; + } + } + } + + if (content.string[range->start] == f_fss_slash_s.string[0] && !(flag & ___fl_flag_is_comment___)) { + + if (flag & ___fl_flag_do_prepend___) { state->status = f_memory_array_increase_by(prepend->used, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) break; state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } else { state->status = f_memory_array_increase(state->step_large, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); @@ -342,10 +447,10 @@ extern "C" { f_fss_skip_past_space(content, range, state); if (F_status_is_error(state->status)) break; - if (has_graph) { + if (flag & ___fl_flag_has_graph___) { // Do nothing. } - else if (content.string[range->start] == f_fss_eol_s.string[0] || range->start >= content.used || range->start > range->stop) { + else if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { state->status = f_memory_array_increase_by(2, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) break; @@ -358,7 +463,7 @@ extern "C" { if (F_status_is_error(state->status)) break; if (content.string[range->start] != f_fss_eol_s.string[0]) { - has_graph = F_true; + flag |= ___fl_flag_has_graph___; } continue; @@ -372,10 +477,10 @@ extern "C" { if (content.string[range->start] == f_fss_eol_s.string[0]) { destination->string[destination->used++] = f_fss_eol_s.string[0]; - ends_on_eol = F_true; + flag |= ___fl_flag_has_eol___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } range->start = start + 1; @@ -383,29 +488,27 @@ extern "C" { continue; } } - else if (!has_graph && content.string[range->start] == f_fss_extended_list_close_s.string[0] && !is_comment) { + else if (!(flag & ___fl_flag_is_comment_graph___) && content.string[range->start] == f_fss_extended_list_close_s.string[0]) { start = range->start++; - if (do_prepend) { + if (flag & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } - has_graph = F_true; + flag |= ___fl_flag_has_graph___; f_fss_skip_past_space(content, range, state); if (F_status_is_error(state->status)) break; - if (content.string[range->start] == f_fss_eol_s.string[0] || range->start >= content.used || range->start > range->stop) { - - if (content.string[range->start] == f_fss_eol_s.string[0] && prepend) { - do_prepend = F_true; - ends_on_eol = F_true; + if (state->status == F_okay_eol || range->start >= content.used || range->start > range->stop) { + if (state->status == F_okay_eol && prepend) { + flag |= ___fl_flag_has_eol_do_prepend___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } if (ignore && ignore->used) { @@ -448,34 +551,52 @@ extern "C" { continue; } - else if (content.string[range->start] == f_fss_comment_s.string[0] && !has_graph) { - is_comment = F_true; + else if (content.string[range->start] == f_fss_comment_s.string[0] && !(flag & ___fl_flag_has_graph___)) { + flag |= ___fl_flag_is_comment___; } else if (content.string[range->start] == f_fss_eol_s.string[0]) { - has_graph = F_false; - is_comment = F_false; + flag &= ~(flag & ___fl_flag_is_comment_graph___); + } + else if (f_fss_is_graph(content, *range, state) == F_true) { + flag |= ___fl_flag_has_graph___; } else if (f_fss_is_graph(content, *range, state) == F_false) { if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(content, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(content, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag |= ___fl_flag_is_joiner_graph___; + } + } + + if (F_status_is_error(state->status)) break; + + range->start = previous; } else { - has_graph = F_true; + flag |= ___fl_flag_has_graph___; } if (content.string[range->start] != f_fss_placeholder_s.string[0]) { - if (do_prepend) { + if (flag & ___fl_flag_do_prepend___) { state->status = f_string_dynamic_append(*prepend, destination); if (F_status_is_error(state->status)) break; - do_prepend = F_false; + flag &= ~___fl_flag_do_prepend___; } if (content.string[range->start] == f_fss_eol_s.string[0] && prepend) { - do_prepend = F_true; - ends_on_eol = F_true; + flag |= ___fl_flag_has_eol_do_prepend___; } else { - ends_on_eol = F_false; + flag &= ~___fl_flag_has_eol___; } width = macro_f_utf_byte_width(content.string[range->start]); @@ -490,6 +611,7 @@ extern "C" { state->status = f_utf_buffer_increment(content, range, 1); if (F_status_is_error(state->status)) break; + } // while if (F_status_is_error(state->status)) { @@ -507,7 +629,7 @@ extern "C" { return; } - if (!ends_on_eol) { + if (!(flag & ___fl_flag_has_eol___)) { destination->string[destination->used++] = f_fss_eol_s.string[0]; } @@ -524,6 +646,18 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_do_prepend___ + #undef ___fl_flag_has_eol___ + #undef ___fl_flag_has_eol_do_prepend___ + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_extended_list_content_write_ @@ -592,12 +726,25 @@ extern "C" { state->status = f_memory_array_increase_by(4, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); if (F_status_is_error(state->status)) return; + #define ___fl_flag_ends_on_space___ 0x1 + #define ___fl_flag_has_eol___ 0x2 + #define ___fl_flag_has_eol_do_prepend___ 0x3 + #define ___fl_flag_has_graph___ 0x4 + #define ___fl_flag_is_comment___ 0x8 + #define ___fl_flag_is_comment_graph___ 0xc + #define ___fl_flag_is_combining___ 0x10 + #define ___fl_flag_is_combining_graph___ 0x14 + #define ___fl_flag_is_joined___ 0x20 + #define ___fl_flag_is_joiner___ 0x40 + #define ___fl_flag_is_joiner_graph___ 0x44 + #define ___fl_flag_is_joinerd___ 0x60 + const f_number_unsigned_t destination_used = destination->used; + f_number_unsigned_t previous = 0; f_number_unsigned_t slash_count = 0; - uint8_t ends_on_space = F_false; - + uint8_t flag = 0x0; uint8_t width = 0; // Find the first graph character. @@ -608,6 +755,38 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) break; } + if (flag & ___fl_flag_is_combining___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_combining(object, *range, state) != F_true) { + if (F_status_is_error(state->status)) break; + + flag &= ~___fl_flag_is_combining___; + } + } + + if (flag & ___fl_flag_is_joinerd___) { + f_fss_skip_past_delimit(object, range, state); + if (F_status_is_error(state->status)) break; + + if (f_fss_is_joiner(object, *range, state) == F_true) { + flag |= ___fl_flag_is_joiner___; + flag &= ~___fl_flag_is_joined___; + } + else { + if (F_status_is_error(state->status)) break; + + if (flag & ___fl_flag_is_joiner___) { + flag |= ___fl_flag_is_joined___; + flag &= ~___fl_flag_is_joiner___; + } + else { + flag &= ~___fl_flag_is_joined___; + } + } + } + if (object.string[range->start] == f_fss_comment_s.string[0]) { // When a comment is found, escape it. @@ -621,6 +800,23 @@ extern "C" { if (f_fss_is_graph(object, *range, state) == F_false) { if (F_status_is_error(state->status)) break; + + previous = range->start; + + f_fss_skip_past_delimit(object, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(object, *range, state) == F_true) { + if (state->status == F_combarudoo) { + flag |= ___fl_flag_is_combining_graph___; + } + else if (state->status == F_bound) { + flag |= ___fl_flag_is_joiner_graph___; + } + } + + if (F_status_is_error(state->status)) break; + + range->start = previous; } else { break; @@ -707,7 +903,7 @@ extern "C" { } // while if (range->start > range->stop || range->start >= object.used) { - ends_on_space = F_false; + flag &= ~___fl_flag_ends_on_space___; break; } @@ -724,7 +920,7 @@ extern "C" { if (F_status_is_error(state->status)) break; } else { - ends_on_space = F_true; + flag |= ___fl_flag_ends_on_space___; if (object.string[range->start] == f_fss_eol_s.string[0]) { state->status = F_status_set_error(F_okay_eol); @@ -764,7 +960,7 @@ extern "C" { } // Prevent a space from being added post-trimming. - ends_on_space = F_true; + flag |= ___fl_flag_ends_on_space___; } if (complete != f_fss_complete_trim_e) { @@ -776,7 +972,7 @@ extern "C" { return; } - if (!ends_on_space) { + if (!(flag & ___fl_flag_ends_on_space___)) { destination->string[destination->used++] = f_fss_space_s.string[0]; } @@ -797,6 +993,18 @@ extern "C" { else { state->status = F_okay; } + + #undef ___fl_flag_ends_on_space___ + #undef ___fl_flag_has_eol___ + #undef ___fl_flag_has_eol_do_prepend___ + #undef ___fl_flag_has_graph___ + #undef ___fl_flag_is_comment___ + #undef ___fl_flag_is_combining___ + #undef ___fl_flag_is_combining_graph___ + #undef ___fl_flag_is_joined___ + #undef ___fl_flag_is_joiner___ + #undef ___fl_flag_is_joiner_graph___ + #undef ___fl_flag_is_joinerd___ } #endif // _di_fl_fss_extended_list_object_write_ diff --git a/level_1/fl_fss/c/fss/extended_list.h b/level_1/fl_fss/c/fss/extended_list.h index b28b8f58f..41017ab67 100644 --- a/level_1/fl_fss/c/fss/extended_list.h +++ b/level_1/fl_fss/c/fss/extended_list.h @@ -36,6 +36,8 @@ extern "C" { * * This does not utilize recursion and has a max content depth of 1, see fss-0008 (Embedded List) for that purpose. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -115,6 +117,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param content * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -165,7 +169,10 @@ extern "C" { * F_interrupt (with error bit) if stopping due to an interrupt. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_memory_array_increase(). @@ -173,7 +180,10 @@ extern "C" { * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_skip_past_delimit() * @see f_fss_skip_past_space() * @see f_memory_array_increase() @@ -190,6 +200,8 @@ extern "C" { * * This will record where delimit placeholders exist but will not apply the delimits. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param buffer * The buffer to read from. * @param range @@ -264,6 +276,8 @@ extern "C" { * * The destination string may have NULLs. * + * Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. + * * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). * @param complete @@ -304,14 +318,20 @@ extern "C" { * F_okay_eol (with error bit) after reaching an EOL, which is not supported by the standard. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_is_combining(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). + * Errors (with error bit) from: f_fss_is_joiner(). * Errors (with error bit) from: f_fss_is_space(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_utf_buffer_increment(). * + * @see f_fss_is_combining() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() + * @see f_fss_is_joiner() * @see f_fss_is_space() * @see f_fss_skip_past_delimit() * @see f_memory_array_increase() diff --git a/level_1/fl_fss/c/fss/private-payload-helper.c b/level_1/fl_fss/c/fss/private-payload-helper.c index fd2938e8a..bc8f262e5 100644 --- a/level_1/fl_fss/c/fss/private-payload-helper.c +++ b/level_1/fl_fss/c/fss/private-payload-helper.c @@ -14,9 +14,11 @@ extern "C" { internal->range.stop = buffer->used - 1; private_fl_fss_basic_write(F_false, *buffer, 0, &internal->range, &destinations->array[destinations->used].value, state, (void * const) internal); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_strip_null(&destinations->array[destinations->used].value); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_strip_null(&destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } @@ -34,9 +36,11 @@ extern "C" { internal->range.stop = buffer->used - 1; private_fl_fss_basic_write(F_false, *buffer, 0, &internal->range, &destinations->array[destinations->used].value, state, (void * const) internal); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_strip_null(&destinations->array[destinations->used].value); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_strip_null(&destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } else if (data->flag & append_on_empty) { @@ -54,21 +58,27 @@ extern "C" { uint8_t private_fl_payload_helper_header_map_destination_write_empty_three(fl_fss_payload_header_state_t * const data, f_state_t * const state, fl_fss_payload_header_internal_t * const internal, f_string_static_t * const destination, const f_string_static_t separator) { state->status = f_memory_array_increase_by(internal->quote_null.used * 3 + separator.used * 2, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append(internal->quote_null, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append(internal->quote_null, destination); + } - state->status = f_string_dynamic_append_assure(separator, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(separator, destination); + } - state->status = f_string_dynamic_append(internal->quote_null, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append(internal->quote_null, destination); + } - state->status = f_string_dynamic_append_assure(separator, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(separator, destination); + } + + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append(internal->quote_null, destination); + } - state->status = f_string_dynamic_append(internal->quote_null, destination); if (F_status_is_error(state->status)) return F_true; state->status = F_okay; @@ -81,15 +91,19 @@ extern "C" { uint8_t private_fl_payload_helper_header_map_destination_write_empty_two(fl_fss_payload_header_state_t * const data, f_state_t * const state, fl_fss_payload_header_internal_t * const internal, f_string_static_t * const destination, const f_string_static_t separator) { state->status = f_memory_array_increase_by(internal->quote_null.used * 2 + separator.used, sizeof(f_char_t), (void **) &destination->string, &destination->used, &destination->size); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append(internal->quote_null, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append(internal->quote_null, destination); + } - state->status = f_string_dynamic_append_assure(separator, destination); - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(separator, destination); + } + + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append(internal->quote_null, destination); + } - state->status = f_string_dynamic_append(internal->quote_null, destination); if (F_status_is_error(state->status)) return F_true; state->status = F_okay; diff --git a/level_1/fl_fss/c/fss/private-payload-helper.h b/level_1/fl_fss/c/fss/private-payload-helper.h index 9c0ded726..2b9509e0a 100644 --- a/level_1/fl_fss/c/fss/private-payload-helper.h +++ b/level_1/fl_fss/c/fss/private-payload-helper.h @@ -25,13 +25,13 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: f_string_dynamic_strip_null(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * The internal.range is modified. @@ -65,14 +65,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: f_string_dynamic_append(). * Errors (with error bit) from: f_string_dynamic_strip_null(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * The internal.range is modified. @@ -109,14 +109,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: f_string_dynamic_append(). * Errors (with error bit) from: f_string_dynamic_append_assure(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * The internal.range is modified. @@ -153,14 +153,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: f_string_dynamic_append(). * Errors (with error bit) from: f_string_dynamic_append_assure(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * The internal.range is modified. diff --git a/level_1/fl_fss/c/fss/private-payload.c b/level_1/fl_fss/c/fss/private-payload.c index 49ca04309..92ecd8edd 100644 --- a/level_1/fl_fss/c/fss/private-payload.c +++ b/level_1/fl_fss/c/fss/private-payload.c @@ -31,9 +31,11 @@ extern "C" { if (data->flag & f_fss_payload_header_map_flag_join_dynamics_e) { state->status = f_string_dynamic_append(buffers->array[internal->j], data->cache); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_string_space_s, data->cache); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_string_space_s, data->cache); + } + if (F_status_is_error(state->status)) return F_true; } else { @@ -42,9 +44,11 @@ extern "C" { internal->range.stop = buffers->array[internal->j].used - 1; private_fl_fss_basic_write(F_false, buffers->array[internal->j], internal->quote, &internal->range, &destinations->array[destinations->used].value, state, (void * const) internal); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } else if (data->flag & f_fss_payload_header_map_flag_null_dynamic_e) { @@ -116,9 +120,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -130,9 +135,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -161,9 +167,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } @@ -178,9 +185,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } @@ -244,9 +252,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -262,9 +271,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } } @@ -303,9 +313,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } @@ -324,9 +335,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } } // for @@ -335,9 +347,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } @@ -420,9 +433,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -438,9 +452,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } } // for @@ -449,9 +464,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } } // for @@ -492,9 +508,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -510,9 +527,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } } // for @@ -521,9 +539,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -537,9 +556,11 @@ extern "C" { internal->range.stop = data->cache->used - 1; private_fl_fss_basic_write(F_false, *data->cache, 0, &internal->range, &destinations->array[destinations->used].value, state, (void * const) internal); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } } // for @@ -721,9 +742,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -735,9 +757,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } } // for @@ -778,9 +801,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -792,9 +816,10 @@ extern "C" { state->status = f_string_dynamic_append(internal->quote_null, data->cache); } - if (F_status_is_error(state->status)) return F_true; + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); + } - state->status = f_string_dynamic_append_assure(f_fss_space_s, data->cache); if (F_status_is_error(state->status)) return F_true; } @@ -808,9 +833,12 @@ extern "C" { internal->range.stop = data->cache->used - 1; private_fl_fss_basic_write(F_false, *data->cache, 0, &internal->range, &destinations->array[destinations->used].value, state, (void * const) internal); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } } // for @@ -1063,12 +1091,14 @@ extern "C" { data->cache->used = 0; state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); - if (F_status_is_error(state->status)) return F_true; - if (private_fl_payload_header_map_number_unsigned(data, state, internal, quantitys.array[internal->l].total) == F_true) return F_true; - if (private_fl_payload_helper_header_map_destination_write_buffer(data, state, internal, data->cache, destinations) == F_true) return F_true; + if (F_status_is_error_not(state->status)) { + if (private_fl_payload_header_map_number_unsigned(data, state, internal, quantitys.array[internal->l].total) == F_true) return F_true; + if (private_fl_payload_helper_header_map_destination_write_buffer(data, state, internal, data->cache, destinations) == F_true) return F_true; + + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } // for @@ -1271,9 +1301,11 @@ extern "C" { no_data = F_false; state->status = f_string_dynamic_append(internal->quote_null, &destinations->array[destinations->used].value); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } + if (F_status_is_error(state->status)) return F_true; } @@ -1289,12 +1321,14 @@ extern "C" { data->cache->used = 0; state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); - if (F_status_is_error(state->status)) return F_true; - if (private_fl_payload_header_map_number_unsigned(data, state, internal, ranges.array[internal->l].stop) == F_true) return F_true; - if (private_fl_payload_helper_header_map_destination_write_buffer(data, state, internal, data->cache, destinations) == F_true) return F_true; + if (F_status_is_error_not(state->status)) { + if (private_fl_payload_header_map_number_unsigned(data, state, internal, ranges.array[internal->l].stop) == F_true) return F_true; + if (private_fl_payload_helper_header_map_destination_write_buffer(data, state, internal, data->cache, destinations) == F_true) return F_true; + + state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); + } - state->status = f_string_dynamic_append_assure(f_fss_extended_next_s, &destinations->array[destinations->used].value); if (F_status_is_error(state->status)) return F_true; } // for @@ -1379,9 +1413,11 @@ extern "C" { if (data->flag & f_fss_payload_header_map_flag_join_strings_e) { state->status = f_string_dynamic_append(string_static, data->cache); - if (F_status_is_error(state->status)) return F_true; - state->status = f_string_dynamic_append_assure(f_string_space_s, data->cache); + if (F_status_is_error_not(state->status)) { + state->status = f_string_dynamic_append_assure(f_string_space_s, data->cache); + } + if (F_status_is_error(state->status)) return F_true; } else { diff --git a/level_1/fl_fss/c/fss/private-payload.h b/level_1/fl_fss/c/fss/private-payload.h index ced906722..ca63323a1 100644 --- a/level_1/fl_fss/c/fss/private-payload.h +++ b/level_1/fl_fss/c/fss/private-payload.h @@ -30,6 +30,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -39,8 +41,6 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_strip_null(). * * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -85,14 +85,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -136,14 +136,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -187,14 +187,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -238,14 +238,14 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_memory_array_increase_by(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -293,13 +293,13 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_conversion_number_signed_to_string(). * Errors (with error bit) from: f_string_dynamic_append_assure(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -337,13 +337,13 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * * Errors (with error bit) from: f_conversion_number_unsigned_to_string(). * Errors (with error bit) from: f_string_dynamic_append_assure(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -381,6 +381,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -389,8 +391,6 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_append_assure(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -436,6 +436,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -444,8 +446,6 @@ extern "C" { * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: private_fl_payload_header_map_number_unsigned(). * Errors (with error bit) from: private_fl_payload_helper_header_map_destination_write_buffer(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -491,6 +491,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -499,8 +501,6 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_append_assure(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -546,6 +546,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -554,8 +556,6 @@ extern "C" { * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: private_fl_payload_header_map_number_unsigned(). * Errors (with error bit) from: private_fl_payload_helper_header_map_destination_write_buffer(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -601,6 +601,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -609,8 +611,6 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_append_assure(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -658,6 +658,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -666,8 +668,6 @@ extern "C" { * Errors (with error bit) from: f_string_dynamic_append_assure(). * Errors (with error bit) from: f_string_dynamic_strip_null(). * Errors (with error bit) from: private_fl_fss_basic_write(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * @@ -713,6 +713,8 @@ extern "C" { * @param state * The state passed directly from the fl_fss_payload_header_map() parameters. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -721,8 +723,6 @@ extern "C" { * Errors (with error bit) from: private_fl_fss_basic_write(). * Errors (with error bit) from: private_fl_payload_header_map_number_unsigned(). * Errors (with error bit) from: private_fl_payload_helper_header_map_destination_write_buffer(). - * - * Must not be NULL. * @param internal * The internal state, fl_fss_payload_header_internal_t, created inside of fl_fss_payload_header_map(). * diff --git a/level_1/fl_fss/c/private-fss-list.c b/level_1/fl_fss/c/private-fss-list.c index a7e62792d..353b2d0bd 100644 --- a/level_1/fl_fss/c/private-fss-list.c +++ b/level_1/fl_fss/c/private-fss-list.c @@ -11,6 +11,7 @@ extern "C" { f_fss_skip_past_space(buffer, range, state); if (F_status_is_error(state->status) || state->status == F_data_not) return; + // Return found nothing if this line only contains white space and delimit placeholders. if (state->status == F_okay_eol) { // Move the start position to after the EOL. @@ -26,76 +27,113 @@ extern "C" { return; } - // Return found nothing if this line only contains white space and delimit placeholders. - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + const f_number_unsigned_t begin = range->start; - // Move the start position to after the EOL. - ++range->start; - state->status = F_fss_found_object_not; + // Ignore all comment lines. + if (buffer.string[begin] == f_fss_comment_s.string[0]) { + state->status = f_utf_buffer_increment(buffer, range, 1); - return; - } + if (F_status_is_error_not(state->status)) { + f_fss_skip_past_delimit(buffer, range, state); + } - // Ignore all comment lines. - if (buffer.string[range->start] == f_fss_comment_s.string[0]) { - f_fss_seek_to_eol(buffer, range, state); if (F_status_is_error(state->status)) return; - if (state->status == F_okay_eos || state->status == F_okay_stop) { - state->status = state->status == F_okay_eos ? F_data_not_eos : F_data_not_stop; + if (range->start >= buffer.used || range->start > range->stop) { + state->status = range->start >= buffer.used ? F_data_not_eos : F_data_not_stop; return; } - // Move the start position to after the EOL. - ++range->start; - state->status = F_fss_found_object_not; + if (f_fss_is_combining_joiner(buffer, *range, state) == F_false) { + if (F_status_is_error_not(state->status)) { + f_fss_seek_to_eol(buffer, range, state); + } - return; + if (F_status_is_error(state->status)) return; + + if (range->start >= buffer.used || range->start > range->stop) { + state->status = range->start >= buffer.used ? F_data_not_eos : F_data_not_stop; + + return; + } + + // Move the start position to after the EOL. + ++range->start; + state->status = F_fss_found_object_not; + + return; + } + + range->start = begin; } + #define ___fl_flag_do_escape___ 0x1 + #define ___fl_flag_has_graph_first___ 0x2 + const f_number_unsigned_t delimits_used = delimits->used; + f_number_unsigned_t location = 0; f_number_unsigned_t slash_first = 0; f_number_unsigned_t slash_count = 0; - f_number_unsigned_t start = 1; - f_number_unsigned_t stop = 0; + f_number_unsigned_t line_start = 1; + f_number_unsigned_t line_stop = 0; - uint8_t graph_first = F_true; + uint8_t flag_fl = ___fl_flag_has_graph_first___; - found->start = range->start; + found->start = begin; // Identify where the object ends. - while (range->start <= range->stop && range->start < buffer.used && buffer.string[range->start] != f_fss_eol_s.string[0]) { + while (F_status_is_error_not(state->status) && range->start <= range->stop && range->start < buffer.used && buffer.string[range->start] != f_fss_eol_s.string[0]) { if (state->interrupt) { state->interrupt((void *) state, 0); - - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; - - return; - } + if (F_status_is_error(state->status)) break; } if (buffer.string[range->start] == f_fss_slash_s.string[0]) { slash_first = range->start++; slash_count = 1; + flag_fl |= ___fl_flag_do_escape___; - for (; range->start <= range->stop && range->start < buffer.used && (buffer.string[range->start] == f_fss_placeholder_s.string[0] || buffer.string[range->start] == f_fss_slash_s.string[0]); ++range->start) { + if (flag_fl & ___fl_flag_has_graph_first___) { + line_start = slash_first; + } + + // If any slash has a combining/joiner following it, then there is no escaping at all but this is not a valid escaping slash. + while (F_status_is_error_not(state->status) && range->start <= range->stop && range->start < buffer.used) { if (state->interrupt) { state->interrupt((void *) state, 0); + if (F_status_is_error(state->status)) break; + } - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) { + // Do nothing. + } + else if (buffer.string[range->start] == f_fss_slash_s.string[0]) { + ++slash_count; + } + else if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl &= ~___fl_flag_do_escape___; - return; - } + f_fss_skip_past_combining_joiner(buffer, range, state); + + break; + } + else { + break; } - if (buffer.string[range->start] == f_fss_slash_s.string[0]) ++slash_count; - } // for + if (F_status_is_error_not(state->status)) { + state->status = f_utf_buffer_increment(buffer, range, 1); + } + } // while + + if (F_status_is_error(state->status)) break; + if (!(flag_fl & ___fl_flag_do_escape___)) continue; + + flag_fl &= ~___fl_flag_do_escape___; if (range->start >= buffer.used || range->start > range->stop) { delimits->used = delimits_used; @@ -113,31 +151,24 @@ extern "C" { } if (buffer.string[range->start] == list_open.string[0]) { - graph_first = F_false; - stop = range->start++; - - while (range->start <= range->stop && range->start < buffer.used) { + flag_fl &= ~___fl_flag_has_graph_first___; + line_stop = range->start++; - if (state->interrupt) { - state->interrupt((void *) state, 0); + f_fss_skip_past_delimit(buffer, range, state); - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + slash_count = 0; - return; - } - } - - if (buffer.string[range->start] == f_fss_eol_s.string[0] || f_fss_is_space(buffer, *range, state) == F_false) break; + f_fss_skip_past_combining_joiner(buffer, range, state); - state->status = f_utf_buffer_increment(buffer, range, 1); + continue; + } - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if (F_status_is_error_not(state->status)) { + f_fss_skip_past_space(buffer, range, state); + } - return; - } - } // while + if (F_status_is_error(state->status)) break; if (range->start >= buffer.used || range->start > range->stop) { delimits->used = delimits_used; @@ -154,7 +185,7 @@ extern "C" { return; } - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { + if (state->status == F_okay_eol) { found->stop = range->start; range->start = slash_first; @@ -168,7 +199,7 @@ extern "C" { } if (slash_count % 2 == 0) { - while (slash_count > 0) { + while (slash_count) { if (buffer.string[range->start] == f_fss_slash_s.string[0]) { if (slash_count % 2 == 1) { @@ -190,9 +221,10 @@ extern "C" { } } // while + // Use the found range to test the buffer decrement. range->start = found->stop + 1; - found->start = stop; - found->stop = stop; + found->start = line_stop; + found->stop = line_stop; state->status = f_utf_buffer_decrement(buffer, found, 1); @@ -204,7 +236,7 @@ extern "C" { } found->stop = found->start; - found->start = start; + found->start = line_start; state->status = F_fss_found_object; } else { @@ -215,9 +247,26 @@ extern "C" { return; } } - else if (graph_first && buffer.string[range->start] == f_fss_comment_s.string[0]) { - graph_first = F_false; - start = slash_first; + else if ((flag_fl & ___fl_flag_has_graph_first___) && buffer.string[range->start] == f_fss_comment_s.string[0]) { + flag_fl &= ~___fl_flag_has_graph_first___; + line_start = slash_first; + line_stop = location = range->start++; + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + slash_count = 0; + + f_fss_skip_past_combining_joiner(buffer, range, state); + + line_stop = range->start; + + continue; + } + + if (F_status_is_error(state->status)) break; + + range->start = location; // Comments may only have white space before the '#', therefore only the first slash needs to be delimited. state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); @@ -230,41 +279,48 @@ extern "C" { } delimits->array[delimits->used++] = slash_first; - stop = range->start++; + line_stop = range->start++; } else { - graph_first = F_false; - stop = range->start; + if (flag_fl & ___fl_flag_has_graph_first___) { + flag_fl &= ~___fl_flag_has_graph_first___; + line_start = slash_first; + } + + line_stop = range->start; } continue; } if (buffer.string[range->start] == list_open.string[0]) { + if (flag_fl & ___fl_flag_has_graph_first___) { + line_start = range->start; + } + ++range->start; - while (range->start <= range->stop && range->start < buffer.used) { + f_fss_skip_past_delimit(buffer, range, state); - if (state->interrupt) { - state->interrupt((void *) state, 0); + if (F_status_is_error_not(state->status)) { + location = range->start; - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); - return; - } - } + line_stop = range->start; - if (buffer.string[range->start] == f_fss_eol_s.string[0] || f_fss_is_space(buffer, *range, state) == F_false) break; + continue; + } + } - state->status = f_utf_buffer_increment(buffer, range, 1); + if (F_status_is_error_not(state->status)) { + range->start = location; - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + f_fss_skip_past_space(buffer, range, state); + } - return; - } - } // while + if (F_status_is_error(state->status)) break; if (range->start >= buffer.used) { found->stop = buffer.used - 1; @@ -280,50 +336,63 @@ extern "C" { return; } - if (buffer.string[range->start] == f_fss_eol_s.string[0]) { - found->start = start; - found->stop = stop + macro_f_utf_byte_width(buffer.string[stop]) - 1; + if (state->status == F_okay_eol) { + if (flag_fl & ___fl_flag_has_graph_first___) { + found->start = 1; + found->stop = 0; + } + else { + found->start = line_start; + found->stop = line_stop + macro_f_utf_byte_width(buffer.string[line_stop]) - 1; + } + ++range->start; state->status = F_fss_found_object; return; } - if (graph_first) { - graph_first = F_false; - start = 1; - stop = 0; - } - else { - stop = range->start; + if (flag_fl & ___fl_flag_has_graph_first___) { + flag_fl &= ~___fl_flag_has_graph_first___; + line_start = range->start; } + line_stop = range->start; + continue; } if (f_fss_is_space(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if ((flag_fl & ___fl_flag_has_graph_first___) && range->start > begin) { + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + location = range->start; - return; + state->status = f_utf_buffer_decrement(buffer, range, 1); + } } - if (graph_first) { - graph_first = F_false; - start = range->start; + if (F_status_is_error_not(state->status)) { + if (flag_fl & ___fl_flag_has_graph_first___) { + flag_fl &= ~___fl_flag_has_graph_first___; + line_start = range->start; + } + + line_stop = range->start; } + } - stop = range->start; + if (F_status_is_error_not(state->status)) { + state->status = f_utf_buffer_increment(buffer, range, 1); } - state->status = f_utf_buffer_increment(buffer, range, 1); + if (F_status_is_error(state->status)) break; + } // while - if (F_status_is_error(state->status)) { - delimits->used = delimits_used; + if (F_status_is_error(state->status)) { + delimits->used = delimits_used; - return; - } - } // while + return; + } if (range->start >= buffer.used) { found->stop = buffer.used - 1; @@ -351,6 +420,9 @@ extern "C" { // Move the start position to after the EOL. ++range->start; } + + #undef ___fl_flag_do_escape___ + #undef ___fl_flag_has_graph_first___ } #endif // !defined(_di_fl_fss_basic_list_object_read_) || !defined(_di_fl_fss_embedded_list_object_read_) || !defined(_di_fl_fss_extended_list_object_read_) diff --git a/level_1/fl_fss/c/private-fss-list.h b/level_1/fl_fss/c/private-fss-list.h index 0077e2677..480e7ea78 100644 --- a/level_1/fl_fss/c/private-fss-list.h +++ b/level_1/fl_fss/c/private-fss-list.h @@ -99,8 +99,12 @@ extern "C" { * The buffer to seek through. * @param range * The start/stop location within the buffer string to process. + * + * Must not be NULL. * @param destination * The buffer where the bytes are written to. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * There is no state.handle(). @@ -111,6 +115,8 @@ extern "C" { * Error bit designates an error but must be passed along with F_interrupt. * All other statuses are ignored. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * @@ -134,6 +140,8 @@ extern "C" { * The destination.used value before any operations were performed. * @param destination * The buffer where the object is written to. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * There is no state.handle(). @@ -144,6 +152,8 @@ extern "C" { * Error bit designates an error but must be passed along with F_interrupt. * All other statuses are ignored. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * diff --git a/level_1/fl_fss/c/private-fss.c b/level_1/fl_fss/c/private-fss.c index 3333da780..072e3ad33 100644 --- a/level_1/fl_fss/c/private-fss.c +++ b/level_1/fl_fss/c/private-fss.c @@ -12,6 +12,7 @@ extern "C" { if (F_status_is_error(state->status) || state->status == F_data_not) return; if (state->status == F_okay_eol) { + // Move the start position to after the EOL. ++range->start; state->status = F_fss_found_object_not; @@ -25,16 +26,18 @@ extern "C" { return; } - // Save the delimits used position in case of unterminated quote. - const f_number_unsigned_t delimits_used = delimits->used; - - // Begin the search. const f_number_unsigned_t begin = range->start; - found->start = range->start; + + found->start = begin; // Ignore all comment lines. - if ((flag & 0x1) && buffer.string[range->start] == f_fss_comment_s.string[0]) { - f_fss_seek_to_eol(buffer, range, state); + if ((flag & 0x1) && buffer.string[begin] == f_fss_comment_s.string[0]) { + state->status = f_utf_buffer_increment(buffer, range, 1); + + if (state->status == F_okay) { + f_fss_skip_past_delimit(buffer, range, state); + } + if (F_status_is_error(state->status)) return; if (range->start >= buffer.used || range->start > range->stop) { @@ -43,74 +46,122 @@ extern "C" { return; } - // Move the start position to after the EOL. - ++range->start; - state->status = F_fss_found_object_not; + if (f_fss_is_combining_joiner(buffer, *range, state) == F_false) { + if (F_status_is_error_not(state->status)) { + f_fss_seek_to_eol(buffer, range, state); + } - return; + if (F_status_is_error(state->status)) return; + + if (range->start >= buffer.used || range->start > range->stop) { + state->status = range->start >= buffer.used ? F_data_not_eos : F_data_not_stop; + + return; + } + + // Move the start position to after the EOL. + ++range->start; + state->status = F_fss_found_object_not; + + return; + } + + range->start = begin; } - // Handle quote support. + #define ___fl_flag_do_escape___ 0x1 + #define ___fl_flag_has_joiner_combining___ 0x2 + + const f_number_unsigned_t delimits_used = delimits->used; + + f_number_unsigned_t location = 0; + f_number_unsigned_t previous = 0; + f_number_unsigned_t slash_count = 0; + f_number_unsigned_t slash_first = 0; + + uint8_t flag_fl = 0x0; f_char_t quote_found = 0; if (quote) *quote = f_fss_quote_type_none_e; // Identify where the object begins. if (buffer.string[range->start] == f_fss_slash_s.string[0]) { - f_number_unsigned_t first_slash = range->start; - - found->start = range->start; + slash_first = found->start = range->start; + flag_fl |= ___fl_flag_do_escape___; state->status = f_utf_buffer_increment(buffer, range, 1); if (F_status_is_error_not(state->status)) { - while (range->start <= range->stop && range->start < buffer.used) { + f_fss_skip_past_delimit(buffer, range, state); + } + + // If first slash has a combining/joiner following it, then there is no escaping at all so slash_first is the start of an object and is not considered an escaping character. + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl &= ~___fl_flag_do_escape___; + + f_fss_skip_past_combining_joiner(buffer, range, state); + } + + if (F_status_is_error(state->status)) return; + + if (flag_fl & ___fl_flag_do_escape___) { + while (F_status_is_error_not(state->status) && range->start <= range->stop && range->start < buffer.used) { if (state->interrupt) { state->interrupt((void *) state, 0); if (F_status_is_error(state->status)) return; } - if (f_fss_is_zero_width(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) return; - } - else { - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) return; + if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) { + ++range->start; continue; } - if (buffer.string[range->start] != f_fss_slash_s.string[0]) { - if (f_fss_is_space(buffer, *range, state) == F_false) { + if (f_fss_is_combining_joiner(buffer, *range, state) == F_false) { + if (F_status_is_error(state->status)) return; + + if (f_fss_is_zero_width(buffer, *range, state) == F_false) { if (F_status_is_error(state->status)) return; - } - // Found the end of the object while processing the slash for potential delimits. - if (state->status == F_true) { - if (range->start > begin) { - found->stop = range->start - 1; - } - else { - found->start = 1; - found->stop = 0; - } + if (buffer.string[range->start] != f_fss_slash_s.string[0]) { + if (f_fss_is_space(buffer, *range, state) == F_false) { + if (F_status_is_error(state->status)) return; + } - state->status = f_utf_buffer_increment(buffer, range, 1); + // Found the end of the object while processing the slash for potential delimits. + if (state->status == F_true) { + if (range->start > begin) { + found->stop = range->start - 1; + } + else { + found->start = 1; + found->stop = 0; + } - if (F_status_is_error_not(state->status)) { - state->status = buffer.string[range->start] == f_fss_eol_s.string[0] ? F_fss_found_object_content_not : F_fss_found_object; - } + state->status = f_utf_buffer_increment(buffer, range, 1); - return; + if (F_status_is_error_not(state->status)) { + state->status = buffer.string[range->start] == f_fss_eol_s.string[0] ? F_fss_found_object_content_not : F_fss_found_object; + } + + return; + } + + break; + } } - break; + state->status = f_utf_buffer_increment(buffer, range, 1); + } + else { + f_fss_skip_past_combining_joiner(buffer, range, state); } - - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) return; } // while + + if (F_status_is_error(state->status)) return; + + flag_fl &= ~___fl_flag_do_escape___; } if (range->start >= buffer.used) { @@ -134,20 +185,39 @@ extern "C" { state->status = f_memory_array_increase(state->step_small, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); if (F_status_is_error(state->status)) return; - delimits->array[delimits->used++] = first_slash; + delimits->array[delimits->used++] = slash_first; ++range->start; } } else if (!(flag & 0x2) && (buffer.string[range->start] == f_fss_quote_single_s.string[0] || buffer.string[range->start] == f_fss_quote_double_s.string[0] || buffer.string[range->start] == f_fss_quote_grave_s.string[0])) { - quote_found = buffer.string[range->start]; - found->start = ++range->start; + previous = range->start++; + + f_fss_skip_past_delimit(buffer, range, state); + if (F_status_is_error(state->status)) return; + + if (range->start <= range->stop && range->start < buffer.used) { + if (f_fss_is_combining_joiner(buffer, *range, state) == F_false) { + if (F_status_is_error(state->status)) return; + + quote_found = buffer.string[previous]; + found->start = range->start; + } + else { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) return; + } + } + else { + quote_found = buffer.string[previous]; + found->start = range->start; + } } // Identify where the object ends. if (quote_found) { - f_number_unsigned_t first_slash = 0; - f_number_unsigned_t slash_count = 0; - f_number_unsigned_t location = 0; + slash_first = 0; + slash_count = 0; + location = 0; while (range->start <= range->stop && range->start < buffer.used) { @@ -157,12 +227,24 @@ extern "C" { } if (buffer.string[range->start] == f_fss_slash_s.string[0]) { - first_slash = range->start; + slash_first = range->start; slash_count = 1; state->status = f_utf_buffer_increment(buffer, range, 1); if (F_status_is_error(state->status)) return; + f_fss_skip_past_delimit(buffer, range, state); + if (F_status_is_error(state->status)) return; + + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error(state->status)) return; + + continue; + } + + flag_fl &= ~___fl_flag_has_joiner_combining___; + while (range->start <= range->stop && range->start < buffer.used) { if (state->interrupt) { @@ -171,21 +253,36 @@ extern "C" { } if (buffer.string[range->start] == f_fss_placeholder_s.string[0]) { - state->status = f_utf_buffer_increment(buffer, range, 1); - if (F_status_is_error(state->status)) return; + ++range->start; continue; } - else if (buffer.string[range->start] != f_fss_slash_s.string[0]) { - break; + + if (buffer.string[range->start] == f_fss_slash_s.string[0]) { + ++slash_count; } + else { + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); - ++slash_count; + flag_fl |= ___fl_flag_has_joiner_combining___; + } + + if (F_status_is_error(state->status)) return; + + break; + } state->status = f_utf_buffer_increment(buffer, range, 1); if (F_status_is_error(state->status)) return; } // while + if (flag_fl & ___fl_flag_has_joiner_combining___) { + flag_fl &= ~___fl_flag_has_joiner_combining___; + + continue; + } + if (range->start >= buffer.used) { found->stop = buffer.used - 1; state->status = F_end_not_group_eos; @@ -201,11 +298,10 @@ extern "C" { } if (buffer.string[range->start] == quote_found) { - location = range->start; + location = range->start++; - // Check to see if there is a whitespace, EOS, or EOL after the quote, if not, then this is not a closing quote and delimits do not apply. - if (range->start + 1 <= range->stop && range->start + 1 < buffer.used) { - ++range->start; + // Check to see if there is a whitespace, EOS, or EOL after the quote, if not, then this is not a closing quote and delimits do not apply (also handles combining/joiner after the quote). + if (range->start <= range->stop && range->start < buffer.used) { f_fss_skip_past_delimit(buffer, range, state); if (F_status_is_error(state->status)) return; @@ -217,18 +313,64 @@ extern "C" { state->status = F_true; } else { - if (f_fss_is_space(buffer, *range, state) == F_false) { + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + + // The found quote is not a valid quote because of the combining/joiner character. + state->status = F_false; + } + else { if (F_status_is_error(state->status)) return; + + // This check sets the state status, which is then used by the condition below. + if (f_fss_is_space(buffer, *range, state) == F_true) { + if (state->status == F_control && buffer.string[range->start] == f_fss_eol_s.string[0]) { + + // The EOL is immediately valid and is not to be checked for combining/joiners. + state->status = F_true; + } + else { + previous = range->start; + flag_fl &= ~___fl_flag_has_joiner_combining___; + + state->status = f_utf_buffer_increment(buffer, range, 1); + + if (F_status_is_error_not(state->status)) { + f_fss_skip_past_delimit(buffer, range, state); + } + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl |= ___fl_flag_has_joiner_combining___; + } + else { + if (F_status_is_error(state->status)) return; + } + + // Designate that this is an appropriate space. + if (flag_fl & ___fl_flag_has_joiner_combining___) { + flag_fl &= ~___fl_flag_has_joiner_combining___; + state->status = F_false; + } + else { + state->status = F_true; + range->start = previous; + } + } + } + else { + if (F_status_is_error(state->status)) return; + } } } } else { + --range->start; // EOS or EOL was reached, so it is a valid closing quote. // (for EOL, this is always TRUE, for EOS this could be false but there is no way to know this, so assume TRUE.) state->status = F_true; } + // At this point if state status is TRUE, then we are dealing with a valid quote after the slash. if (state->status == F_true) { if (quote) { if (quote_found == f_fss_quote_double_s.string[0]) { @@ -245,7 +387,7 @@ extern "C" { } } - range->start = first_slash; + range->start = slash_first; if (slash_count % 2 == 0) { state->status = f_memory_array_increase_by(slash_count / 2, sizeof(f_number_unsigned_t), (void **) &delimits->array, &delimits->used, &delimits->size); @@ -270,25 +412,8 @@ extern "C" { range->start = location + 1; - while (buffer.string[range->start] == f_fss_placeholder_s.string[0]) { - - if (state->interrupt) { - state->interrupt((void *) state, 0); - if (F_status_set_fine(state->status) == F_interrupt) return; - } - - if (++range->start >= buffer.used) { - state->status = F_okay_eos; - - return; - } - - if (range->start > range->stop) { - state->status = F_okay_stop; - - return; - } - } // while + f_fss_skip_past_delimit(buffer, range, state); + if (F_status_is_error(state->status)) return; if (f_fss_is_graph(buffer, *range, state) == F_false) { if (F_status_is_error(state->status)) return; @@ -379,24 +504,42 @@ extern "C" { else if (buffer.string[range->start] == quote_found) { // Check to see if there is a whitespace, EOS, or EOL after the quote, if not, then this is not a closing quote. - location = range->start; + location = range->start++; - if (range->start + 1 <= range->stop && range->start + 1 < buffer.used) { - ++range->start; + f_fss_skip_past_delimit(buffer, range, state); + if (F_status_is_error(state->status)) return; - f_fss_skip_past_delimit(buffer, range, state); - if (F_status_is_error(state->status)) return; + if (range->start <= range->stop && range->start < buffer.used) { + if (f_fss_is_space(buffer, *range, state) == F_true) { + previous = range->start; + flag_fl &= ~___fl_flag_has_joiner_combining___; - if (range->start > range->stop || range->start >= buffer.used) { + state->status = f_utf_buffer_increment(buffer, range, 1); + if (F_status_is_error(state->status)) return; - // EOS or EOL was reached, so it is a valid closing quote. - // (for EOL, this is always TRUE, for EOS this could be false but there is no way to know this, so assume TRUE.) - state->status = F_true; - } - else { - if (f_fss_is_space(buffer, *range, state) == F_false) { + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + flag_fl |= ___fl_flag_has_joiner_combining___; + } + else { if (F_status_is_error(state->status)) return; } + + // Designate that this is an appropriate space. + if (flag_fl & ___fl_flag_has_joiner_combining___) { + flag_fl &= ~___fl_flag_has_joiner_combining___; + state->status = F_false; + } + else { + state->status = F_true; + } + } + else { + if (F_status_is_error(state->status)) return; + + // Designate that this is not a valid quote. + state->status = F_false; } } else { @@ -408,6 +551,7 @@ extern "C" { range->start = location; + // At this point if state status is TRUE, then we are dealing with a valid quote. if (state->status == F_true) { if (quote) { if (quote_found == f_fss_quote_double_s.string[0]) { @@ -451,10 +595,7 @@ extern "C" { return; } - if (f_fss_is_space(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) return; - } - else { + if (f_fss_is_space(buffer, *range, state) == F_true) { if (F_status_is_error_not(state->status)) { state->status = f_utf_buffer_increment(buffer, range, 1); @@ -466,6 +607,8 @@ extern "C" { return; } + if (F_status_is_error(state->status)) return; + if (buffer.string[range->start] != f_fss_placeholder_s.string[0]) { while (range->start <= range->stop && range->start < buffer.used && buffer.string[range->start] != f_fss_eol_s.string[0]) { @@ -549,6 +692,14 @@ extern "C" { return; } + else { + if (f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error_not(state->status)) continue; + } + + if (F_status_is_error(state->status)) return; + } state->status = f_utf_buffer_increment(buffer, range, 1); if (F_status_is_error(state->status)) return; @@ -578,14 +729,48 @@ extern "C" { if (F_status_set_fine(state->status) == F_interrupt) return; } - if (f_fss_is_space(buffer, *range, state) == F_false) { - if (F_status_is_error(state->status)) return; + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error_not(state->status)) continue; } - else { + + if (F_status_is_error(state->status)) return; + if (range->start > range->stop || range->start >= buffer.used) break; + + // Break if a space that is not part of a joiner is found (as in if the space is not followed by a combining or joiner). + if (f_fss_is_space(buffer, *range, state) == F_true) { + previous = range->start; + + state->status = f_utf_buffer_increment(buffer, range, 1); + if (F_status_is_error(state->status)) return; + + // There cannot be a combining or joiner once out of range, so reset and break. + if (range->start > range->stop || range->start >= buffer.used) { + range->start = previous; + + break; + } + + f_fss_skip_past_delimit(buffer, range, state); + + if (F_status_is_error_not(state->status) && f_fss_is_combining_joiner(buffer, *range, state) == F_true) { + f_fss_skip_past_combining_joiner(buffer, range, state); + if (F_status_is_error_not(state->status)) continue; + } + + if (F_status_is_error(state->status)) return; + + range->start = previous; + break; } - state->status = f_utf_buffer_increment(buffer, range, 1); + if (F_status_is_error_not(state->status)) { + state->status = f_utf_buffer_increment(buffer, range, 1); + } + if (F_status_is_error(state->status)) return; } // while @@ -623,12 +808,17 @@ extern "C" { ++range->start; state->status = F_fss_found_object_not; + + #undef ___fl_flag_do_escape___ + #undef ___fl_flag_has_joiner_combining___ } #endif // !defined(_di_fl_fss_basic_object_read_) || !defined(_di_fl_fss_extended_object_read_) #if !defined(_di_fl_fss_basic_content_read_) || !defined(_di_fl_fss_basic_object_write_) || !defined(_di_fl_fss_extended_object_write_) || !defined(_di_fl_fss_extended_content_read_) || !defined(_di_fl_fss_extended_content_write_) || !defined(_di_fl_fss_payload_header_map_) void private_fl_fss_basic_write(const uint8_t flag, const f_string_static_t object, const uint8_t quote, f_range_t * const range, f_string_dynamic_t * const destination, f_state_t * const state, void * const internal) { + // TODO: update this (and all FSS write functions) to handle combining and joiners. + f_fss_skip_past_space(object, range, state); if (F_status_is_error(state->status) || state->status == F_data_not) return; @@ -1142,7 +1332,7 @@ extern "C" { } // for } // for - // If there is no whitespace between the quotes, post-trimming, then remove the quotes. + // If there is no white space between the quotes, post-trimming, then remove the quotes. for (destination_range.start = front; destination_range.start < rear; ++destination_range.start) { if (state->interrupt) { diff --git a/level_1/fl_fss/c/private-fss.h b/level_1/fl_fss/c/private-fss.h index 4ff9979c2..39813228f 100644 --- a/level_1/fl_fss/c/private-fss.h +++ b/level_1/fl_fss/c/private-fss.h @@ -33,8 +33,12 @@ extern "C" { * The start location will be updated as the buffer is being processed. * The start location will represent where the read stopped on return. * A start location past the stop location or buffer used means that the entire range was processed. + * + * Must not be NULL. * @param found * A set of all locations where a valid object was found. + * + * Must not be NULL. * @param quote * (optional) This will store the quote type representing the character to use (from the f_fss_quote_type_*_e). * @@ -42,6 +46,8 @@ extern "C" { * @param delimits * An array of delimits detected during processing. * The caller is expected to decide if and when to process them. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * There is no state.handle(). @@ -70,21 +76,27 @@ extern "C" { * * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_memory_array_increase_by(). + * Errors (with error bit) from: f_fss_is_combining_joiner(). * Errors (with error bit) from: f_fss_is_graph(). * Errors (with error bit) from: f_fss_is_space(). * Errors (with error bit) from: f_fss_is_zero_width(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_utf_buffer_increment(). + * Errors (with error bit) from: f_utf_is_combining_joiner(). + * + * Must not be NULL. * * @see f_memory_array_increase() * @see f_memory_array_increase_by() + * @see f_fss_is_combining_joiner() * @see f_fss_is_graph() * @see f_fss_is_space() * @see f_fss_is_zero_width() * @see f_fss_skip_past_delimit() * @see f_fss_skip_past_space() * @see f_utf_buffer_increment() + * @see f_utf_is_combining_joiner() * @see fl_fss_basic_object_read() * @see fl_fss_extended_object_read() * @see fl_fss_extended_content_read() @@ -109,13 +121,19 @@ extern "C" { * As Content, this does nothing special in regards to a leading '#'. * @param object * The string to write as (does not stop at NULLS, they are ignored and not written). + * + * Must not be NULL. * @param quote - * If 0, then double quotes are auto-inserted, when required. + * If NULL, then double quotes are auto-inserted, when required. * Otherwise, this is quote character to wrap the object in when writing. * @param range * The start/stop location within the object string to write as an object. + * + * Must not be NULL. * @param destination * The buffer where the object is written to. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * There is no state.handle(). @@ -126,6 +144,8 @@ extern "C" { * Error bit designates an error but must be passed along with F_interrupt. * All other statuses are ignored. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * F_okay_eos on success after reaching the end of the buffer. @@ -139,6 +159,7 @@ extern "C" { * F_parameter (with error bit) if a parameter is invalid. * * Errors (with error bit) from: f_fss_is_space(). + * Errors (with error bit) from: f_fss_skip_past_combining_joiner(). * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_fss_skip_past_space(). * Errors (with error bit) from: f_memory_array_increase(). @@ -150,6 +171,7 @@ extern "C" { * Set to NULL to not use. * * @see f_fss_is_space() + * @see f_fss_skip_past_combining_joiner_joiner() * @see f_fss_skip_past_delimit() * @see f_fss_skip_past_space() * @see f_memory_array_increase() @@ -176,6 +198,8 @@ extern "C" { * The destination.used value before any operations were perfomed. * @param destination * The buffer where the object is written to. + * + * Must not be NULL. * @param state * A state for providing flags and handling interrupts during long running operations. * There is no state.handle(). @@ -186,6 +210,8 @@ extern "C" { * Error bit designates an error but must be passed along with F_interrupt. * All other statuses are ignored. * + * Must not be NULL. + * * This alters state.status: * F_okay on success. * diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_read.3 index 4347ad6e9..47e3422b6 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_read.3 @@ -19,6 +19,8 @@ Read an fss-0000 content. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_write.3 index b6273cc20..a55752616 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_content_write.3 @@ -21,6 +21,8 @@ This will write the given string range as a valid content. Anything within this .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_read.3 index 486cfafc9..5ffdbc20f 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_read.3 @@ -20,6 +20,8 @@ Read an fss-0002 Content from a given string. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_write.3 index 05dd18388..9014fe7db 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_content_write.3 @@ -22,6 +22,8 @@ This will write the given string range as a valid Content. Anything within this .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_read.3 index 612a51a52..3dda6ce4b 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_read.3 @@ -19,6 +19,8 @@ Read an fss-0002 Object. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_write.3 index a5fc38f16..ce215d25f 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_list_object_write.3 @@ -21,6 +21,8 @@ This will write the given string range as a valid Object. Anything within this r .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_read.3 index 5ff3261c1..eaa686ac8 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_read.3 @@ -20,6 +20,8 @@ Read an fss-0000 object. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_write.3 index d86e267c6..96e78a19f 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_basic_object_write.3 @@ -22,6 +22,8 @@ This will write the given string range as a valid object. Anything within this r .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_read.3 index 412892192..cfb8f6fe8 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_read.3 @@ -22,6 +22,8 @@ This will record where delimit placeholders exist but will not apply the delimit .PP This operates on the assumption that found.array[0].array[found.array[0].used].content is the current content being processed. Therefore the object for this content will also need to be found.array[0].array[found.array[0].used].object. The object at depth 0 will not be populated, but all nested objects will be populated at their respective depth along with the content. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP The state.data must be of type fl_fss_data_embedded_list_t with all properties as valid. .PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_write.3 index 69f6a081a..c8269e7f9 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_content_write.3 @@ -23,6 +23,8 @@ This will write the given string range as a valid content. Anything within this .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_read.3 index 1f9259762..d25ca6fcb 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_read.3 @@ -19,6 +19,8 @@ Read an fss-0008 object. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_write.3 index e9b32ee73..d70dc4f71 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_embedded_list_object_write.3 @@ -21,6 +21,8 @@ This will write the given string range as a valid object. Anything within this r .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_read.3 index 2a484530e..035bc31bc 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_read.3 @@ -20,6 +20,8 @@ Read an fss-0001 content. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP f_fss_state_quote_not_e: Explicitly disable quotes, treating quotes and their respective delimits as normal characters. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_write.3 index d69128906..8e5bb612b 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_content_write.3 @@ -22,6 +22,8 @@ This will write the given string range as a valid content. Anything within this .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_read.3 index e80f908f4..fc7fed5fb 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_read.3 @@ -23,6 +23,8 @@ This will record where delimit placeholders exist but will not apply the delimit .PP This does not utilize recursion and has a max content depth of 1, see fss-0008 (Embedded List) for that purpose. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_write.3 index a8419ffb1..64ff92455 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_content_write.3 @@ -23,6 +23,8 @@ This will write the given string range as a valid content. Anything within this .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_read.3 index f2380708d..3bd9e4906 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_read.3 @@ -19,6 +19,8 @@ Read an fss-0003 object. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_write.3 index 57ff824d7..90026efd4 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_list_object_write.3 @@ -21,6 +21,8 @@ This will write the given string range as a valid object. Anything within this r .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_read.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_read.3 index 42178395f..7edaa1355 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_read.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_read.3 @@ -20,6 +20,8 @@ Read an fss-0001 object. .PP This will record where delimit placeholders exist but will not apply the delimits. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_write.3 b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_write.3 index a5fb24a50..1e0ed5347 100644 --- a/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_write.3 +++ b/level_1/fl_fss/data/documentation/man/man3/fl_fss_extended_object_write.3 @@ -22,6 +22,8 @@ This will write the given string range as a valid object. Anything within this r .PP The destination string may have NULLs. .PP +Combining spaces at the start must be handled by the caller to avoid F_complete_not_utf_start errors relating to combining characters. +.PP When state.interrupt() returns, only F_interrupt and F_interrupt_not are processed. Error bit designates an error but must be passed along with F_interrupt. All other statuses are ignored. .PP Must not be NULL. diff --git a/level_1/fl_fss/data/tests/contents/basic-combining_read.txt b/level_1/fl_fss/data/tests/contents/basic-combining_read.txt new file mode 100644 index 000000000..bbec7e729 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic-combining_read.txt @@ -0,0 +1,6 @@ +valid. +and does not have embedded new line. + + +is valid. +is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/contents/basic_list-combining_read-0.txt b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-0.txt new file mode 100644 index 000000000..b93e370cf --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-0.txt @@ -0,0 +1 @@ + content 1. diff --git a/level_1/fl_fss/data/tests/contents/basic_list-combining_read-1.txt b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-1.txt new file mode 100644 index 000000000..65adf4f41 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-1.txt @@ -0,0 +1,3 @@ +́: + is_second and content 2. + does not have ́ content 3. diff --git a/level_1/fl_fss/data/tests/contents/basic_list-combining_read-2.txt b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-2.txt new file mode 100644 index 000000000..96487d57a --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-2.txt @@ -0,0 +1 @@ + has ͜no_space content 4. diff --git a/level_1/fl_fss/data/tests/contents/basic_list-combining_read-3.txt b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-3.txt new file mode 100644 index 000000000..78c611949 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-3.txt @@ -0,0 +1 @@ + content 5. diff --git a/level_1/fl_fss/data/tests/contents/basic_list-combining_read-4.txt b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-4.txt new file mode 100644 index 000000000..5e7418a06 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/basic_list-combining_read-4.txt @@ -0,0 +1,2 @@ + sixtẖ̚ is content 6. + not a valid list:́ diff --git a/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-0.txt b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-0.txt new file mode 100644 index 000000000..b93e370cf --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-0.txt @@ -0,0 +1 @@ + content 1. diff --git a/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-1.txt b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-1.txt new file mode 100644 index 000000000..2fed02e6c --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-1.txt @@ -0,0 +1,3 @@ +́ + is_second and content 2. + does have ́ content 3. diff --git a/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-2.txt b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-2.txt new file mode 100644 index 000000000..96487d57a --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-2.txt @@ -0,0 +1 @@ + has ͜no_space content 4. diff --git a/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-3.txt b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-3.txt new file mode 100644 index 000000000..78c611949 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-3.txt @@ -0,0 +1 @@ + content 5. diff --git a/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-4.txt b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-4.txt new file mode 100644 index 000000000..449318919 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/embedded_list-combining_read-4.txt @@ -0,0 +1,2 @@ + sixtẖ̚ is content 6. + not a valid list{́ diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-0.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-0.txt new file mode 100644 index 000000000..fd91deecc --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-0.txt @@ -0,0 +1,6 @@ +valid. +and + + +is +is diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-1.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-1.txt new file mode 100644 index 000000000..e989d7c42 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-1.txt @@ -0,0 +1,6 @@ + +does + + +valid. +valid diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-2.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-2.txt new file mode 100644 index 000000000..68f43c1c0 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-2.txt @@ -0,0 +1,6 @@ + +not + + + +with diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-3.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-3.txt new file mode 100644 index 000000000..551dabd95 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-3.txt @@ -0,0 +1,6 @@ + +have + + + +two diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-4.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-4.txt new file mode 100644 index 000000000..2af451a4b --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-4.txt @@ -0,0 +1,6 @@ + +embedded + + + +combiners. diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-5.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-5.txt new file mode 100644 index 000000000..a99c891eb --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-5.txt @@ -0,0 +1,6 @@ + +new + + + + diff --git a/level_1/fl_fss/data/tests/contents/extended-combining_read-6.txt b/level_1/fl_fss/data/tests/contents/extended-combining_read-6.txt new file mode 100644 index 000000000..74dddee11 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended-combining_read-6.txt @@ -0,0 +1,6 @@ + +line. + + + + diff --git a/level_1/fl_fss/data/tests/contents/extended_list-combining_read-0.txt b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-0.txt new file mode 100644 index 000000000..b93e370cf --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-0.txt @@ -0,0 +1 @@ + content 1. diff --git a/level_1/fl_fss/data/tests/contents/extended_list-combining_read-1.txt b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-1.txt new file mode 100644 index 000000000..2fed02e6c --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-1.txt @@ -0,0 +1,3 @@ +́ + is_second and content 2. + does have ́ content 3. diff --git a/level_1/fl_fss/data/tests/contents/extended_list-combining_read-2.txt b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-2.txt new file mode 100644 index 000000000..96487d57a --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-2.txt @@ -0,0 +1 @@ + has ͜no_space content 4. diff --git a/level_1/fl_fss/data/tests/contents/extended_list-combining_read-3.txt b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-3.txt new file mode 100644 index 000000000..78c611949 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-3.txt @@ -0,0 +1 @@ + content 5. diff --git a/level_1/fl_fss/data/tests/contents/extended_list-combining_read-4.txt b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-4.txt new file mode 100644 index 000000000..449318919 --- /dev/null +++ b/level_1/fl_fss/data/tests/contents/extended_list-combining_read-4.txt @@ -0,0 +1,2 @@ + sixtẖ̚ is content 6. + not a valid list{́ diff --git a/level_1/fl_fss/data/tests/counts/basic_list-combining_read.txt b/level_1/fl_fss/data/tests/counts/basic_list-combining_read.txt new file mode 100644 index 000000000..627e1097c --- /dev/null +++ b/level_1/fl_fss/data/tests/counts/basic_list-combining_read.txt @@ -0,0 +1,5 @@ +1 +1 +1 +1 +1 diff --git a/level_1/fl_fss/data/tests/counts/embedded_list-combining_read.txt b/level_1/fl_fss/data/tests/counts/embedded_list-combining_read.txt new file mode 100644 index 000000000..627e1097c --- /dev/null +++ b/level_1/fl_fss/data/tests/counts/embedded_list-combining_read.txt @@ -0,0 +1,5 @@ +1 +1 +1 +1 +1 diff --git a/level_1/fl_fss/data/tests/counts/extended_list-combining_read.txt b/level_1/fl_fss/data/tests/counts/extended_list-combining_read.txt new file mode 100644 index 000000000..627e1097c --- /dev/null +++ b/level_1/fl_fss/data/tests/counts/extended_list-combining_read.txt @@ -0,0 +1,5 @@ +1 +1 +1 +1 +1 diff --git a/level_1/fl_fss/data/tests/matches/basic-combining_read.txt b/level_1/fl_fss/data/tests/matches/basic-combining_read.txt new file mode 100644 index 000000000..87000e719 --- /dev/null +++ b/level_1/fl_fss/data/tests/matches/basic-combining_read.txt @@ -0,0 +1,6 @@ +2 +2 +1 +1 +2 +2 diff --git a/level_1/fl_fss/data/tests/matches/basic_list-combining_read.txt b/level_1/fl_fss/data/tests/matches/basic_list-combining_read.txt new file mode 100644 index 000000000..3f22f14bf --- /dev/null +++ b/level_1/fl_fss/data/tests/matches/basic_list-combining_read.txt @@ -0,0 +1,5 @@ +2 +2 +2 +2 +2 diff --git a/level_1/fl_fss/data/tests/matches/embedded_list-combining_read.txt b/level_1/fl_fss/data/tests/matches/embedded_list-combining_read.txt new file mode 100644 index 000000000..5ee4e7592 --- /dev/null +++ b/level_1/fl_fss/data/tests/matches/embedded_list-combining_read.txt @@ -0,0 +1,6 @@ +2 +2 +2 +2 +2 +2 diff --git a/level_1/fl_fss/data/tests/matches/extended-combining_read.txt b/level_1/fl_fss/data/tests/matches/extended-combining_read.txt new file mode 100644 index 000000000..87000e719 --- /dev/null +++ b/level_1/fl_fss/data/tests/matches/extended-combining_read.txt @@ -0,0 +1,6 @@ +2 +2 +1 +1 +2 +2 diff --git a/level_1/fl_fss/data/tests/matches/extended_list-combining_read.txt b/level_1/fl_fss/data/tests/matches/extended_list-combining_read.txt new file mode 100644 index 000000000..5ee4e7592 --- /dev/null +++ b/level_1/fl_fss/data/tests/matches/extended_list-combining_read.txt @@ -0,0 +1,6 @@ +2 +2 +2 +2 +2 +2 diff --git a/level_1/fl_fss/data/tests/objects/basic-combining_read.txt b/level_1/fl_fss/data/tests/objects/basic-combining_read.txt new file mode 100644 index 000000000..7ed2a8697 --- /dev/null +++ b/level_1/fl_fss/data/tests/objects/basic-combining_read.txt @@ -0,0 +1,6 @@ +first +is_second + ́ +fourth ͜no_space +fifth +sixtẖ̚ diff --git a/level_1/fl_fss/data/tests/objects/basic_list-combining_read.txt b/level_1/fl_fss/data/tests/objects/basic_list-combining_read.txt new file mode 100644 index 000000000..b540243f9 --- /dev/null +++ b/level_1/fl_fss/data/tests/objects/basic_list-combining_read.txt @@ -0,0 +1,5 @@ +first valid. +is_second and does not have embedded new line. +fourth ͜no_space +fifth is valid. +sixtẖ̚ is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/objects/embedded_list-combining_read.txt b/level_1/fl_fss/data/tests/objects/embedded_list-combining_read.txt new file mode 100644 index 000000000..b540243f9 --- /dev/null +++ b/level_1/fl_fss/data/tests/objects/embedded_list-combining_read.txt @@ -0,0 +1,5 @@ +first valid. +is_second and does not have embedded new line. +fourth ͜no_space +fifth is valid. +sixtẖ̚ is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/objects/extended-combining_read.txt b/level_1/fl_fss/data/tests/objects/extended-combining_read.txt new file mode 100644 index 000000000..7ed2a8697 --- /dev/null +++ b/level_1/fl_fss/data/tests/objects/extended-combining_read.txt @@ -0,0 +1,6 @@ +first +is_second + ́ +fourth ͜no_space +fifth +sixtẖ̚ diff --git a/level_1/fl_fss/data/tests/objects/extended_list-combining_read.txt b/level_1/fl_fss/data/tests/objects/extended_list-combining_read.txt new file mode 100644 index 000000000..b540243f9 --- /dev/null +++ b/level_1/fl_fss/data/tests/objects/extended_list-combining_read.txt @@ -0,0 +1,5 @@ +first valid. +is_second and does not have embedded new line. +fourth ͜no_space +fifth is valid. +sixtẖ̚ is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/strings/basic-combining_read.txt b/level_1/fl_fss/data/tests/strings/basic-combining_read.txt new file mode 100644 index 000000000..d2132bf51 --- /dev/null +++ b/level_1/fl_fss/data/tests/strings/basic-combining_read.txt @@ -0,0 +1,6 @@ +first valid. +is_second and does not have embedded new line. + ́ +fourth ͜no_space +fifth is valid. +sixtẖ̚ is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/strings/basic_list-combining_read.txt b/level_1/fl_fss/data/tests/strings/basic_list-combining_read.txt new file mode 100644 index 000000000..38c40624a --- /dev/null +++ b/level_1/fl_fss/data/tests/strings/basic_list-combining_read.txt @@ -0,0 +1,13 @@ +first valid.: + content 1. +is_second and does not have embedded new line.: +́: + is_second and content 2. + does not have ́ content 3. +fourth ͜no_space: + has ͜no_space content 4. +fifth is valid.: + content 5. +sixtẖ̚ is valid with two combiners.: + sixtẖ̚ is content 6. + not a valid list:́ diff --git a/level_1/fl_fss/data/tests/strings/embedded_list-combining_read.txt b/level_1/fl_fss/data/tests/strings/embedded_list-combining_read.txt new file mode 100644 index 000000000..bb228fea1 --- /dev/null +++ b/level_1/fl_fss/data/tests/strings/embedded_list-combining_read.txt @@ -0,0 +1,18 @@ +first valid.{ + content 1. +} +is_second and does not have embedded new line.{ +́ + is_second and content 2. + does have ́ content 3. +} +fourth ͜no_space{ + has ͜no_space content 4. +} +fifth is valid.{ + content 5. +} +sixtẖ̚ is valid with two combiners.{ + sixtẖ̚ is content 6. + not a valid list{́ +} diff --git a/level_1/fl_fss/data/tests/strings/extended-combining_read.txt b/level_1/fl_fss/data/tests/strings/extended-combining_read.txt new file mode 100644 index 000000000..d2132bf51 --- /dev/null +++ b/level_1/fl_fss/data/tests/strings/extended-combining_read.txt @@ -0,0 +1,6 @@ +first valid. +is_second and does not have embedded new line. + ́ +fourth ͜no_space +fifth is valid. +sixtẖ̚ is valid with two combiners. diff --git a/level_1/fl_fss/data/tests/strings/extended_list-combining_read.txt b/level_1/fl_fss/data/tests/strings/extended_list-combining_read.txt new file mode 100644 index 000000000..bb228fea1 --- /dev/null +++ b/level_1/fl_fss/data/tests/strings/extended_list-combining_read.txt @@ -0,0 +1,18 @@ +first valid.{ + content 1. +} +is_second and does not have embedded new line.{ +́ + is_second and content 2. + does have ́ content 3. +} +fourth ͜no_space{ + has ͜no_space content 4. +} +fifth is valid.{ + content 5. +} +sixtẖ̚ is valid with two combiners.{ + sixtẖ̚ is content 6. + not a valid list{́ +} diff --git a/level_1/fl_fss/tests/unit/c/data-fss.c b/level_1/fl_fss/tests/unit/c/data-fss.c index ea4d35db4..cd91e26d1 100644 --- a/level_1/fl_fss/tests/unit/c/data-fss.c +++ b/level_1/fl_fss/tests/unit/c/data-fss.c @@ -26,6 +26,65 @@ FILE *data__file_open__named_at(const f_string_t set, const f_string_t name, con return fopen(path, "r"); } +f_status_t data__file_load_buffer(FILE * const file, f_string_dynamic_t * const buffer) { + + if (!buffer) { + return F_status_set_error(F_parameter); + } + + f_status_t status = F_okay; + ssize_t result = 0; + size_t max = 0; + char *lines = 0; + f_string_static_t temporary = f_string_static_t_initialize; + + buffer->used = 0; + + for (int i = 0; ; ++i, max = 255) { + + result = getline(&lines, &max, file); + if (result <= 0) break; + + temporary.string = lines; + temporary.used = (f_number_unsigned_t) result; + temporary.size = temporary.used; + + status = f_string_dynamic_append_nulless(temporary, buffer); + if (F_status_is_error(status)) break; + + if (lines) free(lines); + + lines = 0; + } // for + + if (F_status_is_error_not(status)) { + status = f_string_dynamic_terminate_after(buffer); + } + + if (lines) free(lines); + + return status; +} + +f_status_t data__extract_from_source(const f_string_static_t source, const f_range_t range, f_string_dynamic_t * const destination) { + + if (!destination) { + return F_status_set_error(F_parameter); + } + + f_status_t status = F_okay; + + destination->used = 0; + + status = f_string_dynamic_partial_append_nulless(source, range, destination); + + if (F_status_is_error_not(status)) { + status = f_string_dynamic_terminate_after(destination); + } + + return status; +} + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/data-fss.h b/level_1/fl_fss/tests/unit/c/data-fss.h index 15a5ee84f..f3ea807ce 100644 --- a/level_1/fl_fss/tests/unit/c/data-fss.h +++ b/level_1/fl_fss/tests/unit/c/data-fss.h @@ -74,6 +74,51 @@ extern FILE *data__file_open__named(const f_string_t set, const f_string_t name, */ extern FILE *data__file_open__named_at(const f_string_t set, const f_string_t name, const f_string_t context, const uint16_t at); +/** + * Append file into buffer. + * + * @param file + * The file, which must already be open. + * @param buffer + * The buffer to append to. + * The buffer.used is set to 0 on start, unless NULL. + * + * @return + * F_okay on success. + * + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_string_dynamic_append_nulless(). + * Errors (with error bit) from: f_string_dynamic_terminate_after(). + * + * @see f_string_dynamic_append() + * @see f_string_dynamic_terminate_after() + */ +extern f_status_t data__file_load_buffer(FILE * const file, f_string_dynamic_t * const buffer); + +/** + * Extract the given string and range from the source buffer. + * + * The removes NULLs when extracting and ensures the string is NULL terminated. + * + * @param source + * The buffer to read from. + * @param range + * The range within the buffer to extract. + * @param destination + * The extracted string. + * The destination.used is set to 0 on start, unless NULL. + * + * @return + * F_okay on success. + * + * F_parameter (with error bit) if a parameter is invalid. + * + * Errors (with error bit) from: f_string_dynamic_partial_append_nulless(). + * Errors (with error bit) from: f_string_dynamic_terminate_after(). + */ +extern f_status_t data__extract_from_source(const f_string_static_t source, const f_range_t range, f_string_dynamic_t * const destination); + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.c b/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.c index 20bba296d..89bbf6a2f 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.c @@ -111,7 +111,7 @@ void test__fl_fss_basic_content_read__works(void **void_state) { f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; f_status_t status_object = F_okay; - for (;;) { + for (f_number_unsigned_t line = 0; ; ++line) { max = 255; @@ -125,10 +125,15 @@ void test__fl_fss_basic_content_read__works(void **void_state) { max = 255; result = getline(&line_content, &max, file_contents); + + if (!result) { + printf("[ -------> ] --- [00] Failure with getline(), line (%lu) '%s', result = %zd, for loading file='./data/tests/%s/%s-%s.txt'.\n", line, line_content, result, "strings", "basic", "all_read"); + } + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_content[result - 1] = 0; + line_content[result ? result - 1 : 0] = 0; state.status = F_none; range.start = 0; @@ -138,6 +143,10 @@ void test__fl_fss_basic_content_read__works(void **void_state) { fl_fss_basic_object_read(buffer_string, &range, &found_object, "e, &delimits, &state); + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [01] Failure with fl_fss_basic_object_read(), line (%lu) '%s', status = %u, expecting %u or %u.\n", line, line_content, state.status, F_fss_found_object, F_fss_found_object_content_not); + } + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_content_not); status_object = state.status; @@ -146,50 +155,90 @@ void test__fl_fss_basic_content_read__works(void **void_state) { fl_fss_basic_content_read(buffer_string, &range, &found, &delimits, &state); if (status_object == F_fss_found_object) { + if (state.status != F_fss_found_content) { + printf("[ -------> ] --- [02] Failure with fl_fss_basic_content_read(), with line %lu '%s', status = %u, expecting %u.\n", line, line_content, state.status, F_fss_found_content); + } + assert_int_equal(state.status, F_fss_found_content); } else { + if (state.status != F_data_not) { + printf("[ -------> ] --- [03] Failure with fl_fss_basic_content_read(), with line %lu '%s', status = %u, expecting %u.\n", line, line_content, state.status, F_data_not); + } + assert_int_equal(state.status, F_data_not); } if (state.status == F_fss_found_content) { + if (!found.used) { + printf("[ -------> ] --- [04] Failure with fl_fss_basic_content_read(), with line %lu '%s', found.used = %lu, expecting found.used <> 0.\n", line, line_content, found.used); + } + assert_true(found.used); { const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_append(), with line %lu '%s', status = %u, expecting %u.\n", line, line_content, state.status, F_okay); + } + assert_int_equal(status, F_okay); } state.status = F_none; f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [06] Failure with f_fss_apply_delimit(), with line %lu '%s', status = %u, expecting %u.\n", line, line_content, state.status, F_okay); + } + assert_int_equal(state.status, F_okay); { const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found.array[0], &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [07] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', status = %u, expecting %u or %u.\n", line, line_content, state.status, F_okay, F_data_not_eos); + } + assert_true(status == F_okay || status == F_data_not_eos); } { const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [08] Failure with f_string_dynamic_terminate_after(), with line %lu '%s', status = %u, expecting %u.\n", line, line_content, state.status, F_okay); + } + assert_int_equal(status, F_okay); } + if (strcmp(result_string.string, line_content)) { + printf("[ -------> ] --- [09] Failure with line %lu, strings do not match.\n", line); + } + assert_string_equal(result_string.string, line_content); } else { + if (found.used) { + printf("[ -------> ] --- [10] Failure with line %lu '%s', found.used = %lu, expecting found.used == 0.\n", line, line_content, found.used); + } + assert_true(!found.used); } - if (line_string) free(line_string); if (line_content) free(line_content); + if (line_string) free(line_string); if (result_string.string) free(result_string.string); if (delimit_string.string) free(delimit_string.string); if (delimits.array) free(delimits.array); if (found.array) free(found.array); - line_string = 0; line_content = 0; + line_string = 0; result_string.string = 0; result_string.used = 0; result_string.size = 0; @@ -209,8 +258,226 @@ void test__fl_fss_basic_content_read__works(void **void_state) { if (delimits.array) free(delimits.array); if (found.array) free(found.array); + if (line_content) free(line_content); if (line_string) free(line_string); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + +void test__fl_fss_basic_content_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_contents = data__file_open__named("contents", "basic", "combining_read"); + FILE *file_matches = data__file_open__named("matches", "basic", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "basic", "combining_read"); + + assert_non_null(file_contents); + assert_non_null(file_matches); + assert_non_null(file_strings); + + int matches = 0; + size_t max = 0; + char *line_string = 0; + char *line_content = 0; + char *line_matches = 0; + ssize_t result = 0; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found_object = f_range_t_initialize; + f_ranges_t found = f_ranges_t_initialize; + uint8_t quote = 0; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + f_status_t status_object = F_okay; + + for (f_number_unsigned_t line = 0; ; ++line) { + + max = 255; + + result = getline(&line_matches, &max, file_matches); + if (result == -1) break; + + matches = atoi(line_matches); + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + max = 255; + + result = getline(&line_content, &max, file_contents); + + if (result == -1) { + printf("[ -------> ] --- [00] Failure with getline(), line %lu for file './data/tests/strings/extended-combining_read.txt', result = %lu.\n", line, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_content[result ? result - 1 : 0] = 0; + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found_object.start = 1; + found_object.stop = 0; + + fl_fss_basic_object_read(buffer_string, &range, &found_object, "e, &delimits, &state); + + if (matches) { + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [01] Failure with fl_fss_basic_object_read(), with line %lu '%s', matches=%d, status=%u, expected %u.\n", line, line_content, matches, state.status, F_fss_found_object); + } + + assert_true(state.status == F_fss_found_object); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [02] Failure with fl_fss_basic_object_read(), with line %lu '%s', matches=%d, status=%u, expected %u.\n", line, line_content, matches, state.status, F_fss_found_object_content_not); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + + // This is currently not supported. + printf("[ -------> ] --- [03] Failure line %lu matches of 0 is not supported.\n", line); + assert_true(F_false); + } + + status_object = state.status; + state.status = F_none; + + fl_fss_basic_content_read(buffer_string, &range, &found, &delimits, &state); + + if (status_object == F_fss_found_object) { + if (state.status != F_fss_found_content) { + printf("[ -------> ] --- [04] Failure with fl_fss_basic_content_read(), with line %lu '%s', expected %u.\n", line, line_content, F_fss_found_content); + } + + assert_int_equal(state.status, F_fss_found_content); + } + else { + if (state.status != F_data_not) { + printf("[ -------> ] --- [05] Failure with fl_fss_basic_content_read(), with line %lu '%s', expected %u.\n", line, line_content, F_data_not); + } + + assert_int_equal(state.status, F_data_not); + } + + if (matches == 2) { + if (state.status == F_fss_found_content) { + if (!found.used) { + printf("[ -------> ] --- [06] Failure with fl_fss_basic_content_read(), with line %lu '%s', expected found.used <> 0.\n", line, line_content); + } + + assert_true(found.used); + + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [07] Failure with f_string_dynamic_append(), with line %lu '%s', expected %u.\n", line, line_content, F_okay); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [08] Failure with f_fss_apply_delimit(), with line %lu '%s', expected %u.\n", line, line_content, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found.array[0], &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [09] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', status=%u, expected %u or %#0x.\n", line, line_content, status, F_okay, F_data_not_eos); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [10] Failure with f_string_dynamic_terminate_after(), Failure with line %lu '%s', expected %u.\n", line, line_content, F_okay); + } + + assert_int_equal(status, F_okay); + } + + if (strcmp((const char *) result_string.string, (const char *) line_content)) { + printf("[ -------> ] --- [11] Failure with line %lu, strings do not match.\n", line); + } + + assert_string_equal(result_string.string, line_content); + } + else { + if (found.used) { + printf("[ -------> ] --- [12] Failure with fl_fss_basic_content_read(), with line %lu '%s', expected found.used == 0.\n", line, line_content); + } + + assert_true(!found.used); + } + } + + if (line_content) free(line_content); + if (line_matches) free(line_matches); + if (line_string) free(line_string); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + if (found.array) free(found.array); + + line_content = 0; + line_matches = 0; + line_string = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + found.array = 0; + found.used = 0; + found.size = 0; + } // for + + if (file_strings) fclose(file_strings); + if (file_contents) fclose(file_contents); + if (file_matches) fclose(file_matches); + + if (delimits.array) free(delimits.array); + if (found.array) free(found.array); if (line_content) free(line_content); + if (line_matches) free(line_matches); + if (line_string) free(line_string); if (result_string.string) free(result_string.string); if (delimit_string.string) free(delimit_string.string); } diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.h b/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.h index 56dc096fc..03c6ea9a2 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_content_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_basic_content_read__returns_data_not(void **state); */ extern void test__fl_fss_basic_content_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_basic_content_read() + */ +extern void test__fl_fss_basic_content_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_basic_content_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.c b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.c index 5fc5f4dfe..cb5347ab8 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.c @@ -203,12 +203,7 @@ void test__fl_fss_basic_list_content_read__works(void **void_state) { result = getline(&line_content[i], &max, file_contents[i]); // The newline is copied by getline(), and so remove that newline before comparing. - if (result > 0) { - line_content[i][result - 1] = 0; - } - else { - line_content[i][0] = 0; - } + line_content[i][result > 0 ? result - 1 : 0] = 0; } // or state.status = F_none; @@ -330,6 +325,312 @@ void test__fl_fss_basic_list_content_read__works(void **void_state) { } } +void test__fl_fss_basic_list_content_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_counts = data__file_open__named("counts", "basic_list", "combining_read"); + FILE *file_matches = data__file_open__named("matches", "basic_list", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "basic_list", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "basic_list", "combining_read"); + + FILE *file_contents[] = { + data__file_open__named_at("contents", "basic_list", "combining_read", 0), + data__file_open__named_at("contents", "basic_list", "combining_read", 1), + data__file_open__named_at("contents", "basic_list", "combining_read", 2), + data__file_open__named_at("contents", "basic_list", "combining_read", 3), + data__file_open__named_at("contents", "basic_list", "combining_read", 4), + }; + + assert_non_null(file_counts); + assert_non_null(file_matches); + assert_non_null(file_objects); + assert_non_null(file_strings); + assert_non_null(file_contents[0]); + assert_non_null(file_contents[1]); + assert_non_null(file_contents[2]); + assert_non_null(file_contents[3]); + assert_non_null(file_contents[4]); + + const uint8_t sets_total = 5; + + int counts = 0; + int matches = 0; + size_t max = 0; + ssize_t result = 0; + + char *line_counts = 0; + char *line_matches = 0; + char *line_object = 0; + + f_range_t range = f_range_t_initialize; + f_range_t found_object = f_range_t_initialize; + f_ranges_t found_content = f_ranges_t_initialize; + f_ranges_t comments = f_ranges_t_initialize; + + f_state_t state = f_state_t_initialize; + f_status_t status_object = F_okay; + f_uint8s_t quotes = f_uint8s_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + + f_string_dynamic_t set_delimit = f_string_dynamic_t_initialize; + f_string_dynamic_t set_result = f_string_dynamic_t_initialize; + f_string_dynamic_t set_strings = f_string_dynamic_t_initialize; + + f_string_dynamic_t set_contents[] = { + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + }; + + { + const f_status_t status = data__file_load_buffer(file_strings, &set_strings); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [00] Failure with data__file_load_buffer() for loading file strings, status=%u has error bit set.\n", status); + } + + assert_true(F_status_is_error_not(status)); + } + + for (uint8_t i = 0; i < sets_total; ++i) { + + const f_status_t status = data__file_load_buffer(file_contents[i], &set_contents[i]); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [01] Failure with data__file_load_buffer() for loading file contents at %u, status=%u has error bit set.\n", i, status); + } + + assert_true(F_status_is_error_not(state.status)); + } // for + + { + const f_status_t status = f_string_dynamic_append(set_strings, &set_delimit); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append() for preparing delimit string, status=%u, expected=%u.\n", status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + range.start = 0; + range.stop = set_strings.used - 1; + + for (f_number_unsigned_t set = 0; set < sets_total; ++set) { + + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (result < 0) { + printf("[ -------> ] --- [03] Failure with getline() for objects, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + max = 255; + + result = getline(&line_matches, &max, file_matches); + + if (result < 0) { + printf("[ -------> ] --- [04] Failure with getline() for matches, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + matches = atoi(line_matches); + + found_object.start = 1; + found_object.stop = 0; + state.status = F_none; + + fl_fss_basic_list_object_read(set_strings, &range, &found_object, &delimits, &state); + + if (matches) { + + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [05] Failure with fl_fss_basic_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object); + } + + assert_true(state.status == F_fss_found_object); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [06] Failure with fl_fss_basic_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_content_not); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + if (!(state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [07] Failure with with fl_fss_basic_list_object_read(), set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_not); + } + + assert_true(state.status == F_fss_found_object_not); + } + + status_object = state.status; + state.status = F_none; + + if (status_object == F_fss_found_object) { + + // The Object had to be read before getting to the Content, so test it as well. + { + const f_status_t status = data__extract_from_source(set_strings, found_object, &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [14] Failure with data__extract_from_source() for extract object, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, line_object)) { + printf("[ -------> ] --- [09] Failure with set %lu, object strings do not match.\n", set); + } + + assert_string_equal(set_result.string, line_object); + + max = 255; + + result = getline(&line_counts, &max, file_counts); + + if (result < 0) { + printf("[ -------> ] --- [10] Failure with getline() for counts, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + counts = atoi(line_counts); + + fl_fss_basic_list_content_read(set_strings, &range, &found_content, &delimits, &comments, &state); + + if (!(state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop || state.status == F_data_not)) { + printf("[ -------> ] --- [11] Failure with fl_fss_basic_list_content_read(), with set %lu, status=%u, expected %u, %u, %u, or %u.\n", set, state.status, F_fss_found_content, F_okay_eos, F_okay_stop, F_data_not); + } + + assert_true(state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop || state.status == F_data_not); + + if (state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop) { + + // The caller must handle these cases (for basic list only) and the unit test is expecting this. + if (state.status == F_okay_eos || state.status == F_okay_stop) { + ++found_content.used; + } + + if (!found_content.used) { + printf("[ -------> ] --- [12] Failure with fl_fss_basic_list_content_read(), with set %lu, expected found_content.used (%lu) <> 0.\n", set, found_content.used); + } + + assert_true(found_content.used); + + if (found_content.used != (f_number_unsigned_t) counts) { + printf("[ -------> ] --- [13] Failure with fl_fss_basic_list_content_read(), with set %lu, expected found_content.used (%lu) == %lu, found = '%s', expected = '%s'.\n", set, found_content.used, (f_number_unsigned_t) counts, set_result.string ? set_result.string : "", set_contents[set].string ? set_contents[set].string : ""); + } + + assert_true(found_content.used == (f_number_unsigned_t) counts); + + state.status = F_none; + + f_fss_apply_delimit(delimits, &set_delimit, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [14] Failure with f_fss_apply_delimit(), with set %lu, status=%u, expected=%u.\n", set, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = data__extract_from_source(set_delimit, found_content.array[0], &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [15] Failure with data__extract_from_source() for extract set contents, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, set_contents[set].string)) { + printf("[ -------> ] --- [16] Failure with set %lu, content strings do not match.\n", set); + } + + assert_string_equal(set_result.string, set_contents[set].string); + } + else { + if (found_content.used) { + printf("[ -------> ] --- [17] Failure with set %lu, should not have any content, but match %lu.\n", set, found_content.used); + } + + assert_true(!found_content.used); + } + } + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (found_content.array) free(found_content.array); + if (quotes.array) free(quotes.array); + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_result.string) free(set_result.string); + + comments.array = 0; + comments.used = 0; + comments.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + found_content.array = 0; + found_content.used = 0; + found_content.size = 0; + line_counts = 0; + line_matches = 0; + line_object = 0; + quotes.array = 0; + quotes.used = 0; + quotes.size = 0; + set_result.string = 0; + set_result.used = 0; + set_result.size = 0; + } // for + + for (uint8_t i = 0; i < sets_total; ++i) { + + if (file_contents[i]) fclose(file_contents[i]); + if (set_contents[i].string) free(set_contents[i].string); + } // for + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_delimit.string) free(set_delimit.string); + if (set_result.string) free(set_result.string); + if (set_strings.string) free(set_strings.string); + + if (file_counts) fclose(file_counts); + if (file_matches) fclose(file_matches); + if (file_objects) fclose(file_objects); + if (file_strings) fclose(file_strings); + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (found_content.array) free(found_content.array); + if (quotes.array) free(quotes.array); + } +} + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.h b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.h index a80cea19e..c03c5ccd1 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_content_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_basic_list_content_read__returns_data_not(void **state) */ extern void test__fl_fss_basic_list_content_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_basic_list_content_read() + */ +extern void test__fl_fss_basic_list_content_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_basic_list_content_read_list_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_object_read.c b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_object_read.c index fb7b4443a..d23a14dca 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_list_object_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_list_object_read.c @@ -107,7 +107,7 @@ void test__fl_fss_basic_list_object_read__works(void **void_state) { f_string_dynamic_t result_string = f_string_dynamic_t_initialize; f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; - for (int line = 1;; ++line) { + for (f_number_unsigned_t line = 1; ; ++line) { max = 255; @@ -126,41 +126,78 @@ void test__fl_fss_basic_list_object_read__works(void **void_state) { fl_fss_basic_list_object_read(buffer_string, &range, &found, &delimits, &state); + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [00] Failure with fl_fss_basic_list_object_read(), with line %lu '%s', status=%u, expected %u or %u.\n", line, line_string, state.status, F_fss_found_object, F_fss_found_object_not); + } + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_not); if (state.status == F_fss_found_object) { max = 255; result = getline(&line_object, &max, file_objects); + + if (!result) { + printf("[ -------> ] --- [01] Failure with getline(), with line %lu '%s', line_object = '%s', result=%zu.\n", line, line_string, line_object, result); + } + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_object[result - 1] = 0; + line_object[result ? result - 1 : 0] = 0; if (found.start <= found.stop) { { const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } state.status = F_none; f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [03] Failure with f_fss_apply_delimit(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, state.status, F_okay); + } + assert_int_equal(state.status, F_okay); { const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [04] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', line_object = '%s', status=%u, expected %u or %u.\n", line, line_string, line_object, status, F_okay, F_data_not_eos); + } + assert_true(status == F_okay || status == F_data_not_eos); } { const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_terminate_after(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } + if (strncmp(result_string.string, line_object, result_string.used)) { + printf("[ -------> ] --- [06] Failure with line %lu, line_object = '%s, strings do not match.\n", line, line_object); + } + assert_string_equal(result_string.string, line_object); } else { + if (line_object[0]) { + printf("[ -------> ] --- [07] Failure with fl_fss_basic_list_object_read(), with line %lu '%s', line_object[0]=%c.\n", line, line_string, line_object[0]); + } + assert_int_equal(line_object[0], 0); } } diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.c b/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.c index 2fd4e35e4..e97030e37 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.c @@ -173,10 +173,11 @@ void test__fl_fss_basic_object_read__works(void **void_state) { max = 255; result = getline(&line_object, &max, file_objects); + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_object[result - 1] = 0; + line_object[result ? result - 1 : 0] = 0; state.status = F_none; range.start = 0; @@ -241,6 +242,188 @@ void test__fl_fss_basic_object_read__works(void **void_state) { } } +void test__fl_fss_basic_object_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_matches = data__file_open__named("matches", "basic", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "basic", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "basic", "combining_read"); + + assert_non_null(file_matches); + assert_non_null(file_objects); + assert_non_null(file_strings); + + int matches = 0; + size_t max = 0; + char *line_matches = 0; + char *line_object = 0; + char *line_string = 0; + ssize_t result = 0; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found = f_range_t_initialize; + uint8_t quote = 0; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + + for (f_number_unsigned_t line = 0; ; ++line) { + + max = 255; + + result = getline(&line_matches, &max, file_matches); + if (result == -1) break; + + matches = atoi(line_matches); + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (result == -1) { + printf("[ -------> ] --- [00] Failure with getline(), loading line line %lu for file './data/tests/strings/basic-combining_read.txt', result = %lu.\n", line, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found.start = 1; + found.stop = 0; + + fl_fss_basic_object_read(buffer_string, &range, &found, "e, &delimits, &state); + + if (matches) { + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [01] Failure with fl_fss_basic_object_read(), Failure with line '%s', matches=%d, status=%u.\n", line_object, matches, state.status); + } + + assert_true(state.status == F_fss_found_object); + + if (!(found.start <= found.stop)) { + printf("[ -------> ] --- [02] Failure with fl_fss_basic_object_read(), with line '%s', matches=%d, found=(%lu, %lu).\n", line_object, matches, found.start, found.stop); + } + + assert_true(found.start <= found.stop); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [03] Failure with fl_fss_basic_object_read(), with line '%s', matches=%d, status=%u.\n", line_object, matches, state.status); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + + // This is currently not supported. + printf("[ -------> ] --- [04] Failure with fl_fss_basic_object_read(), line %lu matches of 0 is not supported.\n", line); + assert_true(F_false); + } + + + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_append(), Failure with line '%s'.\n", line_object); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [06] Failure with f_fss_apply_delimit(), Failure with line '%s'.\n", line_object); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [07] Failure with f_string_dynamic_partial_append_nulless(), Failure with line '%s', status=%u.\n", line_object, status); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [08] Failure with f_string_dynamic_terminate_after(), with line '%s'.\n", line_object); + } + + assert_int_equal(status, F_okay); + } + + if (strcmp((const char *) result_string.string, (const char *) line_object)) { + printf("[ -------> ] --- [09] Failure with line %lu, line_object = '%s', strings do not match.\n", line, line_object); + } + + assert_string_equal(result_string.string, line_object); + + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (line_string) free(line_string); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + + line_matches = 0; + line_object = 0; + line_string = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + } // for + + if (file_matches) fclose(file_matches); + if (file_objects) fclose(file_objects); + if (file_strings) fclose(file_strings); + + if (delimits.array) free(delimits.array); + if (line_matches) free(line_matches); + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.h b/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.h index 246adc41e..7c27af0ab 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-basic_object_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_basic_object_read__returns_data_not(void **state); */ extern void test__fl_fss_basic_object_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_basic_object_read() + */ +extern void test__fl_fss_basic_object_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_basic_object_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.c b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.c index 57c5030f6..946ff5b76 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.c @@ -248,12 +248,7 @@ void test__fl_fss_embedded_list_content_read__works(void **void_state) { result = getline(&line_content[i], &max, file_contents[i]); // The newline is copied by getline(), and so remove that newline before comparing. - if (result > 0) { - line_content[i][result - 1] = 0; - } - else { - line_content[i][0] = 0; - } + line_content[i][result > 0 ? result - 1 : 0] = 0; } // or state.status = F_none; @@ -344,6 +339,325 @@ void test__fl_fss_embedded_list_content_read__works(void **void_state) { } } +void test__fl_fss_embedded_list_content_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_counts = data__file_open__named("counts", "embedded_list", "combining_read"); + FILE *file_matches = data__file_open__named("matches", "embedded_list", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "embedded_list", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "embedded_list", "combining_read"); + + FILE *file_contents[] = { + data__file_open__named_at("contents", "embedded_list", "combining_read", 0), + data__file_open__named_at("contents", "embedded_list", "combining_read", 1), + data__file_open__named_at("contents", "embedded_list", "combining_read", 2), + data__file_open__named_at("contents", "embedded_list", "combining_read", 3), + data__file_open__named_at("contents", "embedded_list", "combining_read", 4), + }; + + assert_non_null(file_counts); + assert_non_null(file_matches); + assert_non_null(file_objects); + assert_non_null(file_strings); + assert_non_null(file_contents[0]); + assert_non_null(file_contents[1]); + assert_non_null(file_contents[2]); + assert_non_null(file_contents[3]); + assert_non_null(file_contents[4]); + + const uint8_t sets_total = 5; + + int counts = 0; + int matches = 0; + size_t max = 0; + ssize_t result = 0; + + char *line_counts = 0; + char *line_matches = 0; + char *line_object = 0; + + f_range_t range = f_range_t_initialize; + f_range_t found_object = f_range_t_initialize; + f_ranges_t comments = f_ranges_t_initialize; + + f_fss_nest_t found_content = f_fss_nest_t_initialize; + f_state_t state = f_state_t_initialize; + f_status_t status_object = F_okay; + f_uint8s_t quotes = f_uint8s_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + + f_string_dynamic_t set_delimit = f_string_dynamic_t_initialize; + f_string_dynamic_t set_result = f_string_dynamic_t_initialize; + f_string_dynamic_t set_strings = f_string_dynamic_t_initialize; + + f_string_dynamic_t set_contents[] = { + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + }; + + f_ranges_t objects = f_ranges_t_initialize; + f_number_unsigneds_t positions = f_number_unsigneds_t_initialize; + f_number_unsigneds_t slashes = f_number_unsigneds_t_initialize; + fl_fss_data_embedded_list_t cache = macro_fl_fss_data_embedded_list_t_initialize_1(&objects, &positions, &slashes); + + state.data = (void *) &cache; + + { + const f_status_t status = data__file_load_buffer(file_strings, &set_strings); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [00] Failure with data__file_load_buffer() for loading file strings, status=%u has error bit set.\n", status); + } + + assert_true(F_status_is_error_not(status)); + } + + for (uint8_t i = 0; i < sets_total; ++i) { + + const f_status_t status = data__file_load_buffer(file_contents[i], &set_contents[i]); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [01] Failure with data__file_load_buffer() for loading file contents at %u, status=%u has error bit set.\n", i, status); + } + + assert_true(F_status_is_error_not(state.status)); + } // for + + { + const f_status_t status = f_string_dynamic_append(set_strings, &set_delimit); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append() for preparing delimit string, status=%u, expected=%u.\n", status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + range.start = 0; + range.stop = set_strings.used - 1; + + for (f_number_unsigned_t set = 0; set < sets_total; ++set) { + + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (result < 0) { + printf("[ -------> ] --- [03] Failure with getline() for objects, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + max = 255; + + result = getline(&line_matches, &max, file_matches); + + if (result < 0) { + printf("[ -------> ] --- [04] Failure with getline() for matches, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + matches = atoi(line_matches); + + found_object.start = 1; + found_object.stop = 0; + state.status = F_none; + + fl_fss_embedded_list_object_read(set_strings, &range, &found_object, &delimits, &state); + + if (matches) { + + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [05] Failure with fl_fss_embedded_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object); + } + + assert_true(state.status == F_fss_found_object); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [06] Failure with fl_fss_embedded_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_content_not); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + if (!(state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [07] Failure with with fl_fss_embedded_list_object_read(), set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_not); + } + + assert_true(state.status == F_fss_found_object_not); + } + + status_object = state.status; + state.status = F_none; + + if (status_object == F_fss_found_object) { + + // The Object had to be read before getting to the Content, so test it as well. + { + const f_status_t status = data__extract_from_source(set_strings, found_object, &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [14] Failure with data__extract_from_source() for extract object, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, line_object)) { + printf("[ -------> ] --- [09] Failure with set %lu, object strings do not match.\n", set); + } + + assert_string_equal(set_result.string, line_object); + + max = 255; + + result = getline(&line_counts, &max, file_counts); + + if (result < 0) { + printf("[ -------> ] --- [10] Failure with getline() for counts, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + counts = atoi(line_counts); + + fl_fss_embedded_list_content_read(set_strings, &range, &found_content, &delimits, &comments, &state); + + if (!(state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop || state.status == F_data_not)) { + printf("[ -------> ] --- [11] Failure with fl_fss_embedded_list_content_read(), with set %lu, status=%u, expected %u, %u, %u, or %u.\n", set, state.status, F_fss_found_content, F_okay_eos, F_okay_stop, F_data_not); + } + + assert_true(state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop || state.status == F_data_not); + + if (state.status == F_fss_found_content || state.status == F_okay_eos || state.status == F_okay_stop) { + if (!(found_content.used && found_content.depth[0].used && found_content.depth[0].array[0].content.used)) { + printf( + "[ -------> ] --- [12] Failure with fl_fss_embedded_list_content_read(), with set %lu, expected found_content.used (%lu) <> 0 or found_content.depth.used (%lu) <> 0 or found_content.depth[0].array[0].content.used (%lu) <> 0 <> 0.\n", + set, + found_content.used, + found_content.used + ? found_content.depth[0].used + : 0, + found_content.used && found_content.depth[0].used + ? found_content.depth[0].array[0].content.used + : 0 + ); + } + + assert_true(found_content.used && found_content.depth[0].used && found_content.depth[0].array[0].content.used); + + if (found_content.depth[0].used != (f_number_unsigned_t) counts) { + printf("[ -------> ] --- [13] Failure with fl_fss_embedded_list_content_read(), with set %lu, expected found_content.depth[0].used (%lu) == %lu, found = '%s', expected = '%s'.\n", set, found_content.depth[0].used, (f_number_unsigned_t) counts, set_result.string ? set_result.string : "", set_contents[set].string ? set_contents[set].string : ""); + } + + assert_true(found_content.depth[0].used == (f_number_unsigned_t) counts); + + state.status = F_none; + + f_fss_apply_delimit(delimits, &set_delimit, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [14] Failure with f_fss_apply_delimit(), with set %lu, status=%u, expected=%u.\n", set, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = data__extract_from_source(set_delimit, found_content.depth[0].array[0].content.array[0], &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [15] Failure with data__extract_from_source() for extract set contents, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, set_contents[set].string)) { + printf("[ -------> ] --- [16] Failure with set %lu, content strings do not match.\n", set); + } + + assert_string_equal(set_result.string, set_contents[set].string); + } + else { + if (found_content.used) { + printf("[ -------> ] --- [17] Failure with set %lu, should not have any content, but match %lu.\n", set, found_content.used); + } + + assert_true(!found_content.used); + } + } + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (quotes.array) free(quotes.array); + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_result.string) free(set_result.string); + + f_fss_nest_delete(&found_content); + + comments.array = 0; + comments.used = 0; + comments.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + found_content.depth = 0; + found_content.used = 0; + found_content.size = 0; + line_counts = 0; + line_matches = 0; + line_object = 0; + quotes.array = 0; + quotes.used = 0; + quotes.size = 0; + set_result.string = 0; + set_result.used = 0; + set_result.size = 0; + } // for + + for (uint8_t i = 0; i < sets_total; ++i) { + + if (file_contents[i]) fclose(file_contents[i]); + if (set_contents[i].string) free(set_contents[i].string); + } // for + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_delimit.string) free(set_delimit.string); + if (set_result.string) free(set_result.string); + if (set_strings.string) free(set_strings.string); + + if (file_counts) fclose(file_counts); + if (file_matches) fclose(file_matches); + if (file_objects) fclose(file_objects); + if (file_strings) fclose(file_strings); + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (quotes.array) free(quotes.array); + + f_fss_nest_delete(&found_content); + } +} + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.h b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.h index 9b35e5343..371490418 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_content_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_embedded_list_content_read__returns_data_not(void **sta */ extern void test__fl_fss_embedded_list_content_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_embedded_list_content_read() + */ +extern void test__fl_fss_embedded_list_content_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_embedded_list_content_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.c b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.c index c7a791de1..10cef3d75 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.c @@ -107,7 +107,7 @@ void test__fl_fss_embedded_list_object_read__works(void **void_state) { f_string_dynamic_t result_string = f_string_dynamic_t_initialize; f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; - for (int line = 1;; ++line) { + for (f_number_unsigned_t line = 1; ; ++line) { max = 255; @@ -126,41 +126,226 @@ void test__fl_fss_embedded_list_object_read__works(void **void_state) { fl_fss_embedded_list_object_read(buffer_string, &range, &found, &delimits, &state); + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [00] Failure with fl_fss_embedded_list_object_read(), with line %lu '%s', status=%u, expected %u or %u.\n", line, line_string, state.status, F_fss_found_object, F_fss_found_object_not); + } + + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_not); + + if (state.status == F_fss_found_object) { + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (!result) { + printf("[ -------> ] --- [01] Failure with getline(), with line %lu '%s', line_object = '%s', result=%zu.\n", line, line_string, line_object, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + if (found.start <= found.stop) { + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [03] Failure with f_fss_apply_delimit(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [04] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', line_object = '%s', status=%u, expected %u or %u.\n", line, line_string, line_object, status, F_okay, F_data_not_eos); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_terminate_after(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + if (strncmp(result_string.string, line_object, result_string.used)) { + printf("[ -------> ] --- [06] Failure with line %lu, line_object = '%s', strings do not match.\n", line, line_object); + } + + assert_string_equal(result_string.string, line_object); + } + else { + if (line_object[0]) { + printf("[ -------> ] --- [07] Failure with fl_fss_embedded_list_object_read(), with line %lu '%s', line_object[0]=%c.\n", line, line_string, line_object[0]); + } + + assert_int_equal(line_object[0], 0); + } + } + + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + + line_string = 0; + line_object = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + } // for + + if (file_strings) fclose(file_strings); + if (file_objects) fclose(file_objects); + + if (delimits.array) free(delimits.array); + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + +void test__fl_fss_embedded_list_object_read__works_using_combining(void **void_state) { + + { + FILE *file_strings = data__file_open__named("strings", "embedded_list", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "embedded_list", "combining_read"); + + assert_non_null(file_strings); + assert_non_null(file_objects); + + size_t max = 0; + char *line_string = 0; + char *line_object = 0; + ssize_t result = 0; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found = f_range_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + + for (f_number_unsigned_t line = 1; ; ++line) { + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found.start = 1; + found.stop = 0; + + fl_fss_embedded_list_object_read(buffer_string, &range, &found, &delimits, &state); + + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [00] Failure with line %lu '%s', status=%u, expected %u or %u.\n", line, line_string, state.status, F_fss_found_object, F_fss_found_object_not); + } + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_not); if (state.status == F_fss_found_object) { max = 255; result = getline(&line_object, &max, file_objects); + + if (!result) { + printf("[ -------> ] --- [01] Failure with line %lu '%s', line_object = '%s', result=%zu.\n", line, line_string, line_object, result); + } + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_object[result - 1] = 0; + line_object[result ? result - 1 : 0] = 0; if (found.start <= found.stop) { { const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } state.status = F_none; f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [03] Failure with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, state.status, F_okay); + } + assert_int_equal(state.status, F_okay); { const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [04] Failure with line %lu '%s', line_object = '%s', status=%u, expected %u or %u.\n", line, line_string, line_object, status, F_okay, F_data_not_eos); + } + assert_true(status == F_okay || status == F_data_not_eos); } { const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } + if (strncmp(result_string.string, line_object, result_string.used)) { + printf("[ -------> ] --- [06] Failure with line %lu, line_object = '%s', strings do not match.\n", line, line_object); + } + assert_string_equal(result_string.string, line_object); } else { + if (line_object[0]) { + printf("[ -------> ] --- [07] Failure with line %lu '%s', line_object[0]=%c.\n", line, line_string, line_object[0]); + } + assert_int_equal(line_object[0], 0); } } diff --git a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.h b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.h index 1c11751ce..6ae5fed66 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-embedded_list_object_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_embedded_list_object_read__returns_data_not(void **stat */ extern void test__fl_fss_embedded_list_object_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_embedded_list_object_read() + */ +extern void test__fl_fss_embedded_list_object_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_embedded_list_object_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.c b/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.c index af0b52226..693a3dcc6 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.c @@ -205,10 +205,9 @@ void test__fl_fss_extended_content_read__works(void **void_state) { max = 255; result = getline(&line_content[i], &max, file_contents[i]); - assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_content[i][result - 1] = 0; + line_content[i][result > 0 ? result - 1 : 0] = 0; } // or state.status = F_none; @@ -270,13 +269,291 @@ void test__fl_fss_extended_content_read__works(void **void_state) { // All remaining positions in the file need to be a zero length string. for (; i < total_content; ++i) { assert_int_equal(line_content[i][0], 0); + } // for + } + } + else { + assert_true(!found.used); + } + + if (line_string) free(line_string); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + if (found.array) free(found.array); + if (quotes.array) free(quotes.array); + + for (uint8_t i = 0; i < total_content; ++i) { + + if (line_content[i]) free(line_content[i]); + + line_content[i] = 0; + } // for + + line_string = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + found.array = 0; + found.used = 0; + found.size = 0; + quotes.array = 0; + quotes.used = 0; + quotes.size = 0; + } // for + + for (uint8_t i = 0; i < total_content; ++i) { + + if (file_contents[i]) fclose(file_contents[i]); + if (line_content[i]) free(line_content[i]); + + line_content[i] = 0; + } // for + + if (file_strings) fclose(file_strings); + + if (delimits.array) free(delimits.array); + if (found.array) free(found.array); + if (quotes.array) free(quotes.array); + if (line_string) free(line_string); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + +void test__fl_fss_extended_content_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_matches = data__file_open__named("matches", "extended", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "extended", "combining_read"); + + FILE *file_contents[] = { + data__file_open__named_at("contents", "extended", "combining_read", 0), + data__file_open__named_at("contents", "extended", "combining_read", 1), + data__file_open__named_at("contents", "extended", "combining_read", 2), + data__file_open__named_at("contents", "extended", "combining_read", 3), + data__file_open__named_at("contents", "extended", "combining_read", 4), + data__file_open__named_at("contents", "extended", "combining_read", 5), + data__file_open__named_at("contents", "extended", "combining_read", 6), + }; + + assert_non_null(file_matches); + assert_non_null(file_strings); + assert_non_null(file_contents[0]); + assert_non_null(file_contents[1]); + assert_non_null(file_contents[2]); + assert_non_null(file_contents[3]); + assert_non_null(file_contents[4]); + assert_non_null(file_contents[5]); + assert_non_null(file_contents[6]); + + int matches = 0; + size_t max = 0; + char *line_string = 0; + char *line_content[] = { + 0, + 0, + 0, + 0, + 0, + 0, + 0, + }; + char *line_matches = 0; + ssize_t result = 0; + const uint8_t total_content = 7; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found_object = f_range_t_initialize; + f_ranges_t found = f_ranges_t_initialize; + uint8_t quote = 0; + f_uint8s_t quotes = f_uint8s_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + f_status_t status_object = F_okay; + + for (f_number_unsigned_t line = 0; ; ++line) { + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + max = 255; + + result = getline(&line_matches, &max, file_matches); + if (result == -1) break; + + matches = atoi(line_matches); + + for (uint8_t i = 0; i < total_content; ++i) { + + max = 255; + + result = getline(&line_content[i], &max, file_contents[i]); + + if (result == -1) { + printf("[ -------> ] --- [00] Failure loading line line %lu for file './data/tests/strings/extended-combining_read-%d.txt' at %d, result = %lu.\n", line, i, i, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_content[i][result ? result - 1 : 0] = 0; + } // for + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found_object.start = 1; + found_object.stop = 0; + + fl_fss_extended_object_read(buffer_string, &range, &found_object, "e, &delimits, &state); + + if (matches) { + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [01] Failure with line %lu, matches=%d, status=%u, expected %u.\n", line, matches, state.status, F_fss_found_object); + } + + assert_true(state.status == F_fss_found_object); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [02] Failure with line %lu, matches=%d, status=%u, expected %u.\n", line, matches, state.status, F_fss_found_object_content_not); } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + + // This is currently not supported. + printf("[ -------> ] --- [03] Failure line %lu matches of 0 is not supported.\n", line); + assert_true(F_false); + } + + status_object = state.status; + state.status = F_none; + + fl_fss_extended_content_read(buffer_string, &range, &found, "es, &delimits, &state); + + if (status_object == F_fss_found_object) { + if (state.status != F_fss_found_content) { + printf("[ -------> ] --- [04] Failure with fl_fss_extended_content_read() line %lu status=%u, expected %u.\n", line, state.status, F_fss_found_content); + } + + assert_int_equal(state.status, F_fss_found_content); + } + else { + if (state.status != F_data_not) { + printf("[ -------> ] --- [05] Failure with fl_fss_extended_content_read() line %lu status=%u, expected %u.\n", line, state.status, F_data_not); + } + + assert_int_equal(state.status, F_data_not); + } + + if (state.status == F_fss_found_content) { + if (!found.used) { + printf("[ -------> ] --- [06] Failure with fl_fss_extended_content_read() line %lu got F_fss_found_content status=%u, expected found.used to be > 0.\n", line, state.status); + } + + assert_true(found.used); + + if (!(found.used <= (f_number_unsigned_t) total_content)) { + printf("[ -------> ] --- [07] Failure with fl_fss_extended_content_read() line %lu got F_fss_found_content status=%u, expected found.used (%lu) should be <= total_content (%lu).\n", line, state.status, found.used, (f_number_unsigned_t) total_content); + } + + assert_true(found.used <= (f_number_unsigned_t) total_content); + + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [08] Failure with f_string_dynamic_append() line %lu, got status=%u, expected %u.\n", line, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [09] Failure with f_fss_apply_delimit() line %lu, got status=%u, expected %u.\n", line, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + uint8_t i = 0; + + for (; i < found.used; ++i) { + + result_string.used = 0; + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found.array[i], &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [10] Failure with f_string_dynamic_partial_append_nulless() line %lu and found[%u], got status=%u, expected either %u or %u.\n", line, i, status, F_okay, F_data_not_eos); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [11] Failure with f_string_dynamic_terminate_after() line %lu and found[%u], got status=%u, expected %u.\n", line, i, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + if (strcmp(result_string.string, line_content[i])) { + printf("[ -------> ] --- [12] Failure with line %lu and found[%u], strings do not match.\n", line, i); + } + + assert_string_equal(result_string.string, line_content[i]); + } // for + + // All remaining positions in the file need to be a zero length string. + for (; i < total_content; ++i) { + + if (line_content[i][0]) { + printf("[ -------> ] --- [13] Failure line %lu line_content[%u][0] should be 0 but got %u.\n", line, i, line_content[i][0]); + } + + assert_int_equal(line_content[i][0], 0); + } // for } } else { assert_true(!found.used); } + if (line_matches) free(line_matches); if (line_string) free(line_string); if (result_string.string) free(result_string.string); if (delimit_string.string) free(delimit_string.string); @@ -291,6 +568,7 @@ void test__fl_fss_extended_content_read__works(void **void_state) { line_content[i] = 0; } // for + line_matches = 0; line_string = 0; result_string.string = 0; result_string.used = 0; @@ -317,11 +595,13 @@ void test__fl_fss_extended_content_read__works(void **void_state) { line_content[i] = 0; } // for + if (file_matches) fclose(file_matches); if (file_strings) fclose(file_strings); if (delimits.array) free(delimits.array); if (found.array) free(found.array); if (quotes.array) free(quotes.array); + if (line_matches) free(line_matches); if (line_string) free(line_string); if (result_string.string) free(result_string.string); if (delimit_string.string) free(delimit_string.string); diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.h b/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.h index 7b545bc21..54db05d31 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_content_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_extended_content_read__returns_data_not(void **state); */ extern void test__fl_fss_extended_content_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_extended_content_read() + */ +extern void test__fl_fss_extended_content_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_extended_content_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.c b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.c index b7129e981..5b20014ff 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.c @@ -334,12 +334,7 @@ void test__fl_fss_extended_list_content_read__works(void **void_state) { result = getline(&line_content[i], &max, file_contents[i]); // The newline is copied by getline(), and so remove that newline before comparing. - if (result > 0) { - line_content[i][result - 1] = 0; - } - else { - line_content[i][0] = 0; - } + line_content[i][result > 0 ? result - 1 : 0] = 0; } // or state.status = F_none; @@ -461,6 +456,307 @@ void test__fl_fss_extended_list_content_read__works(void **void_state) { } } +void test__fl_fss_extended_list_content_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_counts = data__file_open__named("counts", "extended_list", "combining_read"); + FILE *file_matches = data__file_open__named("matches", "extended_list", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "extended_list", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "extended_list", "combining_read"); + + FILE *file_contents[] = { + data__file_open__named_at("contents", "extended_list", "combining_read", 0), + data__file_open__named_at("contents", "extended_list", "combining_read", 1), + data__file_open__named_at("contents", "extended_list", "combining_read", 2), + data__file_open__named_at("contents", "extended_list", "combining_read", 3), + data__file_open__named_at("contents", "extended_list", "combining_read", 4), + }; + + assert_non_null(file_counts); + assert_non_null(file_matches); + assert_non_null(file_objects); + assert_non_null(file_strings); + assert_non_null(file_contents[0]); + assert_non_null(file_contents[1]); + assert_non_null(file_contents[2]); + assert_non_null(file_contents[3]); + assert_non_null(file_contents[4]); + + const uint8_t sets_total = 5; + + int counts = 0; + int matches = 0; + size_t max = 0; + ssize_t result = 0; + + char *line_counts = 0; + char *line_matches = 0; + char *line_object = 0; + + f_range_t close = f_range_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found_object = f_range_t_initialize; + f_ranges_t found_content = f_ranges_t_initialize; + f_ranges_t comments = f_ranges_t_initialize; + + f_state_t state = f_state_t_initialize; + f_status_t status_object = F_okay; + f_uint8s_t quotes = f_uint8s_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + + f_string_dynamic_t set_delimit = f_string_dynamic_t_initialize; + f_string_dynamic_t set_result = f_string_dynamic_t_initialize; + f_string_dynamic_t set_strings = f_string_dynamic_t_initialize; + + f_string_dynamic_t set_contents[] = { + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + macro_f_string_dynamic_t_initialize_2(0, 0), + }; + + { + const f_status_t status = data__file_load_buffer(file_strings, &set_strings); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [00] Failure with data__file_load_buffer() for loading file strings, status=%u has error bit set.\n", status); + } + + assert_true(F_status_is_error_not(status)); + } + + for (uint8_t i = 0; i < sets_total; ++i) { + + const f_status_t status = data__file_load_buffer(file_contents[i], &set_contents[i]); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [01] Failure with data__file_load_buffer() for loading file contents at %u, status=%u has error bit set.\n", i, status); + } + + assert_true(F_status_is_error_not(state.status)); + } // for + + { + const f_status_t status = f_string_dynamic_append(set_strings, &set_delimit); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append() for preparing delimit string, status=%u, expected=%u.\n", status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + range.start = 0; + range.stop = set_strings.used - 1; + + for (f_number_unsigned_t set = 0; set < sets_total; ++set) { + + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (result < 0) { + printf("[ -------> ] --- [03] Failure with getline() for objects, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + max = 255; + + result = getline(&line_matches, &max, file_matches); + + if (result < 0) { + printf("[ -------> ] --- [04] Failure with getline() for matches, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + matches = atoi(line_matches); + + found_object.start = 1; + found_object.stop = 0; + state.status = F_none; + + fl_fss_extended_list_object_read(set_strings, &range, &found_object, &delimits, &state); + + if (matches) { + + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [05] Failure with fl_fss_extended_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object); + } + + assert_true(state.status == F_fss_found_object); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [06] Failure with fl_fss_extended_list_object_read(), with set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_content_not); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + if (!(state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [07] Failure with with fl_fss_extended_list_object_read(), set %lu, matches=%d, status=%u, expected %u.\n", set, matches, state.status, F_fss_found_object_not); + } + + assert_true(state.status == F_fss_found_object_not); + } + + status_object = state.status; + state.status = F_none; + + if (status_object == F_fss_found_object) { + + // The Object had to be read before getting to the Content, so test it as well. + { + const f_status_t status = data__extract_from_source(set_strings, found_object, &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [14] Failure with data__extract_from_source() for extract object, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, line_object)) { + printf("[ -------> ] --- [09] Failure with set %lu, object strings do not match.\n", set); + } + + assert_string_equal(set_result.string, line_object); + + max = 255; + + result = getline(&line_counts, &max, file_counts); + + if (result < 0) { + printf("[ -------> ] --- [10] Failure with getline() for counts, with set %lu, result is negative.\n", set); + } + + assert_return_code(result, 0); + + counts = atoi(line_counts); + + fl_fss_extended_list_content_read(set_strings, &range, &found_content, &close, &delimits, &comments, &state); + + if (!(state.status == F_fss_found_content || state.status == F_data_not)) { + printf("[ -------> ] --- [11] Failure with fl_fss_extended_list_content_read(), with set %lu, status=%u, expected %u, or %u.\n", set, state.status, F_fss_found_content, F_data_not); + } + + assert_true(state.status == F_fss_found_content || state.status == F_data_not); + + if (state.status == F_fss_found_content) { + if (!found_content.used) { + printf("[ -------> ] --- [12] Failure with fl_fss_extended_list_content_read(), with set %lu, expected found_content.used (%lu) <> 0.\n", set, found_content.used); + } + + assert_true(found_content.used); + + if (found_content.used != (f_number_unsigned_t) counts) { + printf("[ -------> ] --- [13] Failure with fl_fss_extended_list_content_read(), with set %lu, expected found_content.used (%lu) == %lu, found = '%s', expected = '%s'.\n", set, found_content.used, (f_number_unsigned_t) counts, set_result.string ? set_result.string : "", set_contents[set].string ? set_contents[set].string : ""); + } + + assert_true(found_content.used == (f_number_unsigned_t) counts); + + state.status = F_none; + + f_fss_apply_delimit(delimits, &set_delimit, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [14] Failure with f_fss_apply_delimit(), with set %lu, status=%u, expected=%u.\n", set, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = data__extract_from_source(set_delimit, found_content.array[0], &set_result); + + if (F_status_is_error(status)) { + printf("[ -------> ] --- [15] Failure with data__extract_from_source() for extract set contents, with set %lu, status=%u has error bit set.\n", set, status); + } + + assert_true(F_status_is_error_not(status)); + } + + if (strcmp(set_result.string, set_contents[set].string)) { + printf("[ -------> ] --- [16] Failure with set %lu, content strings do not match.\n", set); + } + + assert_string_equal(set_result.string, set_contents[set].string); + } + else { + if (found_content.used) { + printf("[ -------> ] --- [17] Failure with set %lu, should not have any content, but match %lu.\n", set, found_content.used); + } + + assert_true(!found_content.used); + } + } + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (found_content.array) free(found_content.array); + if (quotes.array) free(quotes.array); + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_result.string) free(set_result.string); + + comments.array = 0; + comments.used = 0; + comments.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + found_content.array = 0; + found_content.used = 0; + found_content.size = 0; + line_counts = 0; + line_matches = 0; + line_object = 0; + quotes.array = 0; + quotes.used = 0; + quotes.size = 0; + set_result.string = 0; + set_result.used = 0; + set_result.size = 0; + } // for + + for (uint8_t i = 0; i < sets_total; ++i) { + + if (file_contents[i]) fclose(file_contents[i]); + if (set_contents[i].string) free(set_contents[i].string); + } // for + + if (line_counts) free(line_counts); + if (line_matches) free(line_matches); + if (line_object) free(line_object); + if (set_delimit.string) free(set_delimit.string); + if (set_result.string) free(set_result.string); + if (set_strings.string) free(set_strings.string); + + if (file_counts) fclose(file_counts); + if (file_matches) fclose(file_matches); + if (file_objects) fclose(file_objects); + if (file_strings) fclose(file_strings); + + if (comments.array) free(comments.array); + if (delimits.array) free(delimits.array); + if (found_content.array) free(found_content.array); + if (quotes.array) free(quotes.array); + } +} + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.h b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.h index 869efda4c..a57216b9a 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_content_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_extended_list_content_read__returns_data_not(void **sta */ extern void test__fl_fss_extended_list_content_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_extended_list_content_read() + */ +extern void test__fl_fss_extended_list_content_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_extended_list_content_read_list_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.c b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.c index 941479c00..c18531f75 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.c @@ -107,7 +107,7 @@ void test__fl_fss_extended_list_object_read__works(void **void_state) { f_string_dynamic_t result_string = f_string_dynamic_t_initialize; f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; - for (int line = 1;; ++line) { + for (f_number_unsigned_t line = 1; ; ++line) { max = 255; @@ -126,41 +126,226 @@ void test__fl_fss_extended_list_object_read__works(void **void_state) { fl_fss_extended_list_object_read(buffer_string, &range, &found, &delimits, &state); + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [00] Failure with fl_fss_extended_list_object_read(), with line %lu '%s', status=%u, expected %u or %u.\n", line, line_string, state.status, F_fss_found_object, F_fss_found_object_not); + } + + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_not); + + if (state.status == F_fss_found_object) { + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (!result) { + printf("[ -------> ] --- [01] Failure with getline(), with line %lu '%s', line_object = '%s', result=%zu.\n", line, line_string, line_object, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + if (found.start <= found.stop) { + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [03] Failure with f_fss_apply_delimit(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [04] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', line_object = '%s', status=%u, expected %u or %u.\n", line, line_string, line_object, status, F_okay, F_data_not_eos); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_terminate_after(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + if (strncmp(result_string.string, line_object, result_string.used)) { + printf("[ -------> ] --- [06] Failure with line %lu, line_object = '%s', strings do not match.\n", line, line_object); + } + + assert_string_equal(result_string.string, line_object); + } + else { + if (line_object[0]) { + printf("[ -------> ] --- [07] Failure with fl_fss_extended_list_object_read(), with line %lu '%s', line_object[0]=%c.\n", line, line_string, line_object[0]); + } + + assert_int_equal(line_object[0], 0); + } + } + + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + + line_string = 0; + line_object = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + } // for + + if (file_strings) fclose(file_strings); + if (file_objects) fclose(file_objects); + + if (delimits.array) free(delimits.array); + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + +void test__fl_fss_extended_list_object_read__works_using_combining(void **void_state) { + + { + FILE *file_strings = data__file_open__named("strings", "extended_list", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "extended_list", "combining_read"); + + assert_non_null(file_strings); + assert_non_null(file_objects); + + size_t max = 0; + char *line_string = 0; + char *line_object = 0; + ssize_t result = 0; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found = f_range_t_initialize; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + + for (f_number_unsigned_t line = 1; ; ++line) { + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found.start = 1; + found.stop = 0; + + fl_fss_extended_list_object_read(buffer_string, &range, &found, &delimits, &state); + + if (!(state.status == F_fss_found_object || state.status == F_fss_found_object_not)) { + printf("[ -------> ] --- [00] Failure with fl_fss_extended_list_object_read(), with line %lu '%s', status=%u, expected %u or %u.\n", line, line_string, state.status, F_fss_found_object, F_fss_found_object_not); + } + assert_true(state.status == F_fss_found_object || state.status == F_fss_found_object_not); if (state.status == F_fss_found_object) { max = 255; result = getline(&line_object, &max, file_objects); + + if (!result) { + printf("[ -------> ] --- [01] Failure with fl_fss_extended_list_object_read(), with line %lu '%s', line_object = '%s', result=%zu.\n", line, line_string, line_object, result); + } + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_object[result - 1] = 0; + line_object[result ? result - 1 : 0] = 0; if (found.start <= found.stop) { { const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (status != F_okay) { + printf("[ -------> ] --- [02] Failure with f_string_dynamic_append(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } state.status = F_none; f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [03] Failure with f_fss_apply_delimit(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, state.status, F_okay); + } + assert_int_equal(state.status, F_okay); { const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [04] Failure with f_string_dynamic_partial_append_nulless(), with line %lu '%s', line_object = '%s', status=%u, expected %u or %u.\n", line, line_string, line_object, status, F_okay, F_data_not_eos); + } + assert_true(status == F_okay || status == F_data_not_eos); } { const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [05] Failure with f_string_dynamic_terminate_after(), with line %lu '%s', line_object = '%s', status=%u, expected %u.\n", line, line_string, line_object, status, F_okay); + } + assert_int_equal(status, F_okay); } + if (strncmp(result_string.string, line_object, result_string.used)) { + printf("[ -------> ] --- [06] Failure with line %lu, line_object = '%s', strings do not match.\n", line, line_object); + } + assert_string_equal(result_string.string, line_object); } else { + if (line_object[0]) { + printf("[ -------> ] --- [07] Failure with fl_fss_extended_list_object_read(), with line %lu '%s', line_object[0]=%c.\n", line, line_string, line_object[0]); + } + assert_int_equal(line_object[0], 0); } } diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.h b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.h index 03b72f1a5..f44a29bfd 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_list_object_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_extended_list_object_read__returns_data_not(void **stat */ extern void test__fl_fss_extended_list_object_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_extended_list_object_read() + */ +extern void test__fl_fss_extended_list_object_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_extended_list_object_read_list_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.c b/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.c index b91286a34..028382ae7 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.c +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.c @@ -189,10 +189,11 @@ void test__fl_fss_extended_object_read__works(void **void_state) { max = 255; result = getline(&line_object, &max, file_objects); + assert_return_code(result, 0); // The newline is copied by getline(), and so remove that newline before comparing. - line_object[result - 1] = 0; + line_object[result ? result - 1 : 0] = 0; state.status = F_none; range.start = 0; @@ -257,6 +258,186 @@ void test__fl_fss_extended_object_read__works(void **void_state) { } } +void test__fl_fss_extended_object_read__works_using_combining(void **void_state) { + + { + // Note: These files are required to have the same number of lines and each line should probably be at max 255 characters. + FILE *file_matches = data__file_open__named("matches", "extended", "combining_read"); + FILE *file_strings = data__file_open__named("strings", "extended", "combining_read"); + FILE *file_objects = data__file_open__named("objects", "extended", "combining_read"); + + assert_non_null(file_matches); + assert_non_null(file_strings); + assert_non_null(file_objects); + + int matches = 0; + size_t max = 0; + char *line_matches = 0; + char *line_string = 0; + char *line_object = 0; + ssize_t result = 0; + + f_string_static_t buffer_string = f_string_static_t_initialize; + + f_state_t state = f_state_t_initialize; + f_range_t range = f_range_t_initialize; + f_range_t found = f_range_t_initialize; + uint8_t quote = 0; + f_number_unsigneds_t delimits = f_number_unsigneds_t_initialize; + f_string_dynamic_t result_string = f_string_dynamic_t_initialize; + f_string_dynamic_t delimit_string = f_string_dynamic_t_initialize; + + for (f_number_unsigned_t line = 0; ; ++line) { + + max = 255; + + result = getline(&line_matches, &max, file_matches); + if (result == -1) break; + + matches = atoi(line_matches); + + max = 255; + + result = getline(&line_string, &max, file_strings); + if (result == -1) break; + + buffer_string.string = line_string; + buffer_string.used = (f_number_unsigned_t) result; + buffer_string.size = buffer_string.used; + + max = 255; + + result = getline(&line_object, &max, file_objects); + + if (result == -1) { + printf("[ -------> ] --- [00] Failure with getline(), loading line line %lu for file './data/tests/strings/extended-combining_read.txt', result = %lu.\n", line, result); + } + + assert_return_code(result, 0); + + // The newline is copied by getline(), and so remove that newline before comparing. + line_object[result ? result - 1 : 0] = 0; + + state.status = F_none; + range.start = 0; + range.stop = buffer_string.used - 1; + found.start = 1; + found.stop = 0; + + fl_fss_extended_object_read(buffer_string, &range, &found, "e, &delimits, &state); + + if (matches) { + // When matches is 2, then this matches both object and content. + if (matches == 2) { + if (!(state.status == F_fss_found_object)) { + printf("[ -------> ] --- [01] Failure with fl_fss_extended_object_read(), with line %lu '%s', matches=%d, status=%u.\n", line, line_object, matches, state.status); + } + + assert_true(state.status == F_fss_found_object); + + if (!(found.start <= found.stop)) { + printf("[ -------> ] --- [02] Failure with fl_fss_extended_object_read(), with line %lu '%s', matches=%d, found=(%lu, %lu).\n", line, line_object, matches, found.start, found.stop); + } + + assert_true(found.start <= found.stop); + } + + // When matches is 1, then this matches only object. + else { + if (!(state.status == F_fss_found_object_content_not)) { + printf("[ -------> ] --- [03] Failure with fl_fss_extended_object_read(), with line %lu '%s', matches=%d, status=%u.\n", line, line_object, matches, state.status); + } + + assert_true(state.status == F_fss_found_object_content_not); + } + } + else { + + // This is currently not supported. + printf("[ -------> ] --- [04] Failure with fl_fss_extended_object_read(), line %lu matches of 0 is not supported.\n", line); + assert_true(F_false); + } + + { + const f_status_t status = f_string_dynamic_append(buffer_string, &delimit_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [05] Failure with fl_fss_extended_object_read(), with fl_fss_extended_object_read() line %lu, got status=%u, expected either %u or %u.\n", line, status, F_okay, F_data_not_eos); + } + + assert_int_equal(status, F_okay); + } + + state.status = F_none; + + f_fss_apply_delimit(delimits, &delimit_string, &state); + + if (state.status != F_okay) { + printf("[ -------> ] --- [06] Failure with f_fss_apply_delimit() line %lu, got status=%u, expected %u.\n", line, state.status, F_okay); + } + + assert_int_equal(state.status, F_okay); + + { + const f_status_t status = f_string_dynamic_partial_append_nulless(delimit_string, found, &result_string); + + if (!(status == F_okay || status == F_data_not_eos)) { + printf("[ -------> ] --- [07] Failure with f_string_dynamic_partial_append_nulless() line %lu, got status=%u, expected either %u or %u.\n", line, status, F_okay, F_data_not_eos); + } + + assert_true(status == F_okay || status == F_data_not_eos); + } + + { + const f_status_t status = f_string_dynamic_terminate_after(&result_string); + + if (status != F_okay) { + printf("[ -------> ] --- [08] Failure with f_string_dynamic_terminate_after() line %lu, got status=%u, expected %u.\n", line, status, F_okay); + } + + assert_int_equal(status, F_okay); + } + + if (strcmp(result_string.string, line_object)) { + printf("[ -------> ] --- [09] Failure with f_string_dynamic_terminate_after() with line %lu, strings do not match.\n", line); + } + + assert_string_equal(result_string.string, line_object); + + if (line_matches) free(line_matches); + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + if (delimits.array) free(delimits.array); + + line_matches = 0; + line_string = 0; + line_object = 0; + result_string.string = 0; + result_string.used = 0; + result_string.size = 0; + delimit_string.string = 0; + delimit_string.used = 0; + delimit_string.size = 0; + delimits.array = 0; + delimits.used = 0; + delimits.size = 0; + } // for + + if (file_matches) fclose(file_matches); + if (file_strings) fclose(file_strings); + if (file_objects) fclose(file_objects); + + if (delimits.array) free(delimits.array); + if (line_matches) free(line_matches); + if (line_string) free(line_string); + if (line_object) free(line_object); + if (result_string.string) free(result_string.string); + if (delimit_string.string) free(delimit_string.string); + } +} + #ifdef __cplusplus } // extern "C" #endif diff --git a/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.h b/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.h index e5b08836b..b99676e37 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.h +++ b/level_1/fl_fss/tests/unit/c/test-fss-extended_object_read.h @@ -31,4 +31,11 @@ extern void test__fl_fss_extended_object_read__returns_data_not(void **state); */ extern void test__fl_fss_extended_object_read__works(void **state); +/** + * Test that the function works when using combining characters. + * + * @see fl_fss_extended_object_read() + */ +extern void test__fl_fss_extended_object_read__works_using_combining(void **state); + #endif // _TEST__FL_fss_extended_object_read_h diff --git a/level_1/fl_fss/tests/unit/c/test-fss.c b/level_1/fl_fss/tests/unit/c/test-fss.c index 4302d408d..8921e6d1e 100644 --- a/level_1/fl_fss/tests/unit/c/test-fss.c +++ b/level_1/fl_fss/tests/unit/c/test-fss.c @@ -25,7 +25,9 @@ int main(void) { cmocka_unit_test(test__fl_fss_basic_object_write__returns_data_not), cmocka_unit_test(test__fl_fss_basic_content_read__works), + cmocka_unit_test(test__fl_fss_basic_content_read__works_using_combining), cmocka_unit_test(test__fl_fss_basic_object_read__works), + cmocka_unit_test(test__fl_fss_basic_object_read__works_using_combining), cmocka_unit_test(test__fl_fss_basic_list_content_read__returns_data_not), cmocka_unit_test(test__fl_fss_basic_list_content_write__returns_data_not), @@ -33,7 +35,9 @@ int main(void) { cmocka_unit_test(test__fl_fss_basic_list_object_write__returns_data_not), cmocka_unit_test(test__fl_fss_basic_list_content_read__works), + cmocka_unit_test(test__fl_fss_basic_list_content_read__works_using_combining), cmocka_unit_test(test__fl_fss_basic_list_object_read__works), + // fl_fss_basic_list_object_read() combining check is handled by test__fl_fss_basic_list_content_read__works_using_combining(). cmocka_unit_test(test__fl_fss_embedded_list_content_read__returns_data_not), cmocka_unit_test(test__fl_fss_embedded_list_content_write__returns_data_not), @@ -41,7 +45,9 @@ int main(void) { cmocka_unit_test(test__fl_fss_embedded_list_object_write__returns_data_not), cmocka_unit_test(test__fl_fss_embedded_list_content_read__works), + cmocka_unit_test(test__fl_fss_embedded_list_content_read__works_using_combining), cmocka_unit_test(test__fl_fss_embedded_list_object_read__works), + // fl_fss_embedded_list_object_read() combining check is handled by test__fl_fss_embedded_list_content_read__works_using_combining(). cmocka_unit_test(test__fl_fss_extended_content_read__returns_data_not), cmocka_unit_test(test__fl_fss_extended_content_write__returns_data_not), @@ -49,7 +55,9 @@ int main(void) { cmocka_unit_test(test__fl_fss_extended_object_write__returns_data_not), cmocka_unit_test(test__fl_fss_extended_content_read__works), + cmocka_unit_test(test__fl_fss_extended_content_read__works_using_combining), cmocka_unit_test(test__fl_fss_extended_object_read__works), + cmocka_unit_test(test__fl_fss_extended_object_read__works_using_combining), cmocka_unit_test(test__fl_fss_extended_list_content_read__returns_data_not), cmocka_unit_test(test__fl_fss_extended_list_content_write__returns_data_not), @@ -57,7 +65,9 @@ int main(void) { cmocka_unit_test(test__fl_fss_extended_list_object_write__returns_data_not), cmocka_unit_test(test__fl_fss_extended_list_content_read__works), + cmocka_unit_test(test__fl_fss_extended_list_content_read__works_using_combining), cmocka_unit_test(test__fl_fss_extended_list_object_read__works), + // fl_fss_extended_list_object_read() combining check is handled by test__fl_fss_extended_list_content_read__works_using_combining(). cmocka_unit_test(test__fl_fss_payload_header_map__returns_data_not), cmocka_unit_test(test__fl_fss_payload_header_map__abstruse_dynamic__works_join), diff --git a/level_2/fll_fss/c/fss.c b/level_2/fll_fss/c/fss.c index ad8c01494..3c6a6360b 100644 --- a/level_2/fll_fss/c/fss.c +++ b/level_2/fll_fss/c/fss.c @@ -10,155 +10,141 @@ extern "C" { if (!range) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - // Skip past all NULLs. - for (; range->start <= range->stop; ++range->start) { - if (buffer.string[range->start]) break; - } + // Future version will have this passed as a parameter. + f_state_t state = f_state_t_initialize; - if (range->start > range->stop) return F_data_not; + f_fss_skip_past_delimit(buffer, range, &state); + if (F_status_is_error(state.status) || state.status == F_data_not) return state.status; // The first character must be a '#'. - if (buffer.string[range->start] != f_fss_pound_s.string[0]) { - - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop; ) { - - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - - break; - } - - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for + if (buffer.string[range->start] == f_fss_pound_s.string[0]) { - if (ids) { - ids->used = 0; + // A combining joiner after the '#' invalidates the '#'. + if (f_fss_is_combining_joiner(buffer, *range, &state) == F_true) { + state.status = F_skip; } + else { + if (F_status_is_error(state.status)) return state.status; - return F_found_not; - } - - // Skip past all NULLs after the '#'. - for (++range->start; range->start <= range->stop; ++range->start) { - if (buffer.string[range->start]) break; - } - - if (range->start > range->stop) { - if (ids) { - ids->used = 0; + range->start += f_fss_pound_s.used; } - - return F_found_not; + } + else { + state.status = F_skip; } // The minimum requirement for a valid FLL/FSS Identifier for the entire line is 7 characters after the '#' ("# X-FFFF"). - if (range->stop - range->start < 6) { - - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop; ) { - - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - - break; - } + if (state.status != F_skip) { + if (range->start > range->stop || range->start >= buffer.used || range->stop - range->start < 6) state.status = F_skip; + } - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for + if (state.status != F_skip) { + state.status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); - if (ids) { - ids->used = 0; + if (F_status_is_error(state.status)) { + return F_status_set_fine(state.status) == F_maybe + ? F_status_set_error(F_complete_not_utf) + : state.status; } - return F_found_not; + if (state.status == F_false) state.status = F_skip; } - f_status_t status = f_utf_is_whitespace(buffer.string + range->start, (range->stop - range->start) + 1, F_false); - - if (F_status_is_error(status)) return (F_status_set_fine(status) == F_maybe) ? F_status_set_error(F_complete_not_utf) : status; - - if (status == F_false) { - - // Increment until stop, while taking into consideration UTF-8 character widths. - for (; range->start <= range->stop; ) { - - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; - - break; - } - - range->start += macro_f_utf_byte_width(buffer.string[range->start]); - } // for + const f_number_unsigned_t original = ids ? ids->used : 0; - return F_found_not; - } + if (state.status == F_skip || buffer.string[range->start] == f_string_eol_s.string[0]) { + if (state.status == F_skip) { + f_fss_seek_to_eol(buffer, range, &state); + if (F_status_is_error(state.status)) return state.status; + } - if (buffer.string[range->start] == f_string_eol_s.string[0]) { - ++range->start; + if (state.status != F_data_not && range->start <= range->stop && range->start < buffer.used) { + ++range->start; + } if (ids) { - ids->used = 0; + ids->used = original; } return F_found_not; } - // in order to determine if a valid FLL Identifier represents a valid FSS identifier, the data must be saved. + #define ___fll_flag_has_found___ 0x1 + #define ___fll_flag_has_fss___ 0x2 + + // In order to determine if a valid FLL Identifier represents a valid FSS identifier, the data must be saved. f_fll_id_t id = f_fll_id_t_initialize; - uint8_t found_fss = F_false; + uint8_t fll_flag = 0x0; do { if (ids && ids->used + 1 > ids->size) { - status = f_memory_array_increase(F_fss_default_allocation_step_small_d, sizeof(f_fll_id_t), (void **) &ids->array, &ids->used, &ids->size); + state.status = f_memory_array_increase_by(F_fss_default_allocation_step_small_d, sizeof(f_fll_id_t), (void **) &ids->array, &ids->used, &ids->size); } - if (F_status_is_error_not(status)) { - status = f_parse_dynamic_partial_fll_identify(buffer, range, ids ? &ids->array[ids->used] : &id); + if (F_status_is_error_not(state.status)) { + state.status = f_parse_dynamic_partial_fll_identify(buffer, range, ids ? &ids->array[ids->used] : &id); } - if (F_status_is_error(status) || (status == F_found_not && !found_fss)) { + if (F_status_is_error(state.status)) { if (ids) { - ids->used = 0; + ids->used = original; } - return status; + return state.status; } - if (!found_fss) { - if (ids) { - if (ids->used && ids->array[ids->used - 1].used == 3 && ids->array[ids->used - 1].name[0] == f_fss_f_s.string[0] && ids->array[ids->used - 1].name[0] == f_fss_s_s.string[0] && ids->array[ids->used - 1].name[0] == f_fss_s_s.string[0]) { - found_fss = F_true; - } + if (state.status == F_found_not) { + fll_flag = 0; + + break; + } + + if (state.status == F_found) { + if (!fll_flag) { + fll_flag |= ___fll_flag_has_found___; } - else { - if (id.used == 3 && id.name[0] == f_fss_f_s.string[0] && id.name[0] == f_fss_s_s.string[0] && id.name[0] == f_fss_s_s.string[0]) { - found_fss = F_true; + + // At least one FLL identifier must have "FSS". + if (!(fll_flag & ___fll_flag_has_fss___)) { + if (ids) { + if (ids->used && ids->array[ids->used - 1].used == 3 && ids->array[ids->used - 1].name[0] == f_fss_f_s.string[0] && ids->array[ids->used - 1].name[1] == ids->array[ids->used - 1].name[2] == f_fss_s_s.string[0]) { + fll_flag |= ___fll_flag_has_fss___; + } + } + else { + if (id.used == 3 && id.name[0] == f_fss_f_s.string[0] && id.name[1] == id.name[2] == f_fss_s_s.string[0]) { + fll_flag |= ___fll_flag_has_fss___; + } } } - } - if (ids) { - status = f_memory_array_increase(F_fss_default_allocation_step_small_d, sizeof(f_fll_id_t), (void **) &ids->array, &ids->used, &ids->size); + if (ids) { + state.status = f_memory_array_increase(F_fss_default_allocation_step_small_d, sizeof(f_fll_id_t), (void **) &ids->array, &ids->used, &ids->size); - if (F_status_is_error(status)) { - if (ids) { - ids->used = 0; + if (F_status_is_error(state.status)) { + if (ids) { + ids->used = original; + } + + return state.status; } - return status; + ++ids->used; } - - ++ids->used; } } while (range->start <= range->stop); + if (fll_flag) return fll_flag & ___fll_flag_has_fss___ + ? F_found + : F_maybe; + + #undef ___fll_flag_has_found___ + #undef ___fll_flag_has_fss___ + if (ids) { - if (ids->used) return found_fss ? F_found : F_maybe; + ids->used = original; } - else if (id.used) return found_fss ? F_found : F_maybe; return F_found_not; } @@ -172,9 +158,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; @@ -225,9 +209,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; f_ranges_t *content = 0; @@ -291,9 +273,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; f_string_dynamic_t name = f_string_dynamic_t_initialize; @@ -411,9 +391,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; @@ -485,9 +463,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; @@ -554,9 +530,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; f_string_dynamic_t name = f_string_dynamic_t_initialize; @@ -671,9 +645,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; f_string_dynamic_t name = f_string_dynamic_t_initialize; @@ -785,9 +757,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; @@ -840,9 +810,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; @@ -898,9 +866,7 @@ extern "C" { if (!values) return F_status_set_error(F_parameter); #endif // _di_level_2_parameter_checking_ - if (!buffer.used) return F_data_not; - if (!objects.used) return F_data_not; - if (!contents.used) return F_data_not; + if (!buffer.used || !objects.used || !contents.used) return F_data_not; f_status_t status = F_okay; diff --git a/level_2/fll_fss/c/fss.h b/level_2/fll_fss/c/fss.h index 10acbecf9..53e52edd9 100644 --- a/level_2/fll_fss/c/fss.h +++ b/level_2/fll_fss/c/fss.h @@ -40,6 +40,8 @@ extern "C" { * * See the f_parse_dynamic_partial_fll_identify() function for details on individual FLL/FSS Identifier processing. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The string to process. * @param range @@ -65,10 +67,16 @@ extern "C" { * * F_complete_not_utf (with error bit) if character is an incomplete UTF-8 fragment. * + * Errors (with error bit) from: f_fss_is_combining_joiner(). + * Errors (with error bit) from: f_fss_seek_to_eol(). + * Errors (with error bit) from: f_fss_skip_past_delimit(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: f_utf_is_whitespace(). * Errors (with error bit) from: f_parse_dynamic_partial_fll_identify(). * + * @see f_fss_is_combining_joiner() + * @see f_fss_seek_to_eol() + * @see f_fss_skip_past_delimit() * @see f_memory_array_increase() * @see f_utf_is_whitespace() * @see f_parse_dynamic_partial_fll_identify() @@ -88,6 +96,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -138,6 +148,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -192,6 +204,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -246,6 +260,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -299,6 +315,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -352,6 +370,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -407,6 +427,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -459,6 +481,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -509,6 +533,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects @@ -559,6 +585,8 @@ extern "C" { * * This performs only a simple search algorithm that should be acceptable for small sets where performance is generally not a concern. * + * This function needs f_state_t as a parameter in future implementations. + * * @param buffer * The buffer to read from. * @param objects diff --git a/level_2/fll_fss/c/fss/basic.c b/level_2/fll_fss/c/fss/basic.c index fe3f71768..7a89d79d4 100644 --- a/level_2/fll_fss/c/fss/basic.c +++ b/level_2/fll_fss/c/fss/basic.c @@ -18,9 +18,7 @@ extern "C" { f_status_t status = F_okay; f_number_unsigned_t initial_used = objects->used; - uint8_t found_data = F_false; - uint8_t *quoted_object = 0; do { @@ -43,6 +41,17 @@ extern "C" { } fl_fss_basic_object_read(buffer, range, &objects->array[objects->used], quoted_object, objects_delimits, state); + + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } + } + if (F_status_is_error(state->status)) return; if (range->start >= range->stop || range->start >= buffer.used) { diff --git a/level_2/fll_fss/c/fss/basic.h b/level_2/fll_fss/c/fss/basic.h index 107e3ffd1..53039c782 100644 --- a/level_2/fll_fss/c/fss/basic.h +++ b/level_2/fll_fss/c/fss/basic.h @@ -79,10 +79,12 @@ extern "C" { * F_number_overflow (with error bit) if the maximum buffer size is reached. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_basic_content_read(). * Errors (with error bit) from: fl_fss_basic_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_basic_content_read() * @see fl_fss_basic_object_read() diff --git a/level_2/fll_fss/c/fss/basic_list.c b/level_2/fll_fss/c/fss/basic_list.c index 40097009b..6974c3b08 100644 --- a/level_2/fll_fss/c/fss/basic_list.c +++ b/level_2/fll_fss/c/fss/basic_list.c @@ -32,6 +32,17 @@ extern "C" { do { fl_fss_basic_list_object_read(buffer, range, &objects->array[objects->used], objects_delimits, state); + + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } + } + if (F_status_is_error(state->status)) return; if (range->start >= range->stop || range->start >= buffer.used) { @@ -67,6 +78,8 @@ extern "C" { } while (state->status == F_fss_found_object_not); + // The fl_fss_basic_list_content_read() returns F_okay_eos and F_okay_stop but doesn't increment found.used given that it has no way of knowing. + // This function will treat these states as known and declare that this is found content (and therefore found an object). if (state->status == F_okay_eos || state->status == F_okay_stop) { ++contents->array[contents->used++].used; ++objects->used; diff --git a/level_2/fll_fss/c/fss/basic_list.h b/level_2/fll_fss/c/fss/basic_list.h index bf9555cf9..def398c61 100644 --- a/level_2/fll_fss/c/fss/basic_list.h +++ b/level_2/fll_fss/c/fss/basic_list.h @@ -79,10 +79,12 @@ extern "C" { * F_number_overflow (with error bit) if the maximum buffer size is reached. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_basic_list_content_read(). * Errors (with error bit) from: fl_fss_basic_list_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_basic_list_content_read() * @see fl_fss_basic_list_object_read() diff --git a/level_2/fll_fss/c/fss/embedded_list.c b/level_2/fll_fss/c/fss/embedded_list.c index 947f5b94b..e717421ea 100644 --- a/level_2/fll_fss/c/fss/embedded_list.c +++ b/level_2/fll_fss/c/fss/embedded_list.c @@ -17,7 +17,6 @@ extern "C" { #endif // _di_level_2_parameter_checking_ f_number_unsigned_t initial_used = 0; - uint8_t found_data = F_false; if (!nest->used) { @@ -36,6 +35,17 @@ extern "C" { nest->depth[nest->used].used = 0; fl_fss_embedded_list_object_read(buffer, range, &nest->depth[0].array[nest->depth[0].used].object, objects_delimits, state); + + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } + } + if (F_status_is_error(state->status)) return; if (range->start >= range->stop || range->start >= buffer.used) { diff --git a/level_2/fll_fss/c/fss/embedded_list.h b/level_2/fll_fss/c/fss/embedded_list.h index 600e97bd7..a0ecc8ab2 100644 --- a/level_2/fll_fss/c/fss/embedded_list.h +++ b/level_2/fll_fss/c/fss/embedded_list.h @@ -77,10 +77,12 @@ extern "C" { * F_number_overflow (with error bit) if the maximum buffer size is reached. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_embedded_list_content_read(). * Errors (with error bit) from: fl_fss_embedded_list_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_embedded_list_content_read() * @see fl_fss_embedded_list_object_read() diff --git a/level_2/fll_fss/c/fss/extended.c b/level_2/fll_fss/c/fss/extended.c index a47c0531d..029f557b0 100644 --- a/level_2/fll_fss/c/fss/extended.c +++ b/level_2/fll_fss/c/fss/extended.c @@ -18,9 +18,7 @@ extern "C" { f_status_t status = F_okay; f_number_unsigned_t initial_used = objects->used; - uint8_t found_data = F_false; - uint8_t *quoted_object = 0; f_uint8s_t *quoted_content = 0; @@ -51,6 +49,17 @@ extern "C" { } fl_fss_extended_object_read(buffer, range, &objects->array[objects->used], quoted_object, objects_delimits, state); + + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } + } + if (F_status_is_error(state->status)) return; if (range->start >= range->stop || range->start >= buffer.used) { diff --git a/level_2/fll_fss/c/fss/extended.h b/level_2/fll_fss/c/fss/extended.h index 58236272f..f2bcab20a 100644 --- a/level_2/fll_fss/c/fss/extended.h +++ b/level_2/fll_fss/c/fss/extended.h @@ -83,10 +83,12 @@ extern "C" { * F_number_overflow (with error bit) if the maximum buffer size is reached. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_extended_content_read(). * Errors (with error bit) from: fl_fss_extended_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_extended_content_read() * @see fl_fss_extended_object_read() diff --git a/level_2/fll_fss/c/fss/extended_list.c b/level_2/fll_fss/c/fss/extended_list.c index 7edd7b30a..70932d72e 100644 --- a/level_2/fll_fss/c/fss/extended_list.c +++ b/level_2/fll_fss/c/fss/extended_list.c @@ -18,7 +18,6 @@ extern "C" { f_status_t status = F_okay; f_number_unsigned_t initial_used = objects->used; - uint8_t found_data = F_false; do { @@ -35,6 +34,17 @@ extern "C" { do { fl_fss_extended_list_object_read(buffer, range, &objects->array[objects->used], objects_delimits, state); + + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } + } + if (F_status_is_error(state->status)) return; if (range->start >= range->stop || range->start >= buffer.used) { diff --git a/level_2/fll_fss/c/fss/extended_list.h b/level_2/fll_fss/c/fss/extended_list.h index f75f0b23e..9a21c2f24 100644 --- a/level_2/fll_fss/c/fss/extended_list.h +++ b/level_2/fll_fss/c/fss/extended_list.h @@ -87,10 +87,12 @@ extern "C" { * F_number_overflow (with error bit) if the maximum buffer size is reached. * F_parameter (with error bit) if a parameter is invalid. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_extended_list_content_read(). * Errors (with error bit) from: fl_fss_extended_list_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_extended_list_content_read() * @see fl_fss_extended_list_object_read() diff --git a/level_2/fll_fss/c/fss/payload.c b/level_2/fll_fss/c/fss/payload.c index c2ce66771..e150464df 100644 --- a/level_2/fll_fss/c/fss/payload.c +++ b/level_2/fll_fss/c/fss/payload.c @@ -25,8 +25,10 @@ extern "C" { f_status_t status = F_okay; f_number_unsigned_t initial_used = objects->used; - // 0x1 = found data, 0x2 = found payload object. - uint8_t found_data = 0; + #define ___fll_flag_has_data___ 0x1 + #define ___fll_flag_has_payload___ 0x2 + + uint8_t fll_flag = 0x0; do { state->status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &objects->array, &objects->used, &objects->size); @@ -39,12 +41,19 @@ extern "C" { do { fl_fss_basic_list_object_read(buffer, range, &objects->array[objects->used], objects_delimits, state); - if (F_status_is_error(state->status)) return; - if (f_compare_dynamic_partial_string(f_fss_payload_s.string, buffer, f_fss_payload_s.used, objects->array[objects->used]) == F_equal_to) { - found_data |= 0x2; + // Skip lines with combining/joiners at the start. + if (F_status_set_fine(state->status) == F_complete_not_utf_start) { + f_fss_seek_to_eol(buffer, range, state); + + if (F_status_is_error_not(state->status)) { + state->status = F_fss_found_object_not; + continue; + } } + if (F_status_is_error(state->status)) return; + if (range->start >= range->stop || range->start >= buffer.used) { if (state->status == F_fss_found_object || state->status == F_fss_found_object_content_not) { state->status = F_fss_found_object_content_not; @@ -52,7 +61,7 @@ extern "C" { break; } - if (found_data & 0x1) { + if (fll_flag & ___fll_flag_has_data___) { state->status = F_status_set_error((range->start >= buffer.used) ? F_okay_eos : F_okay_stop); } else { @@ -63,10 +72,12 @@ extern "C" { } if (state->status == F_fss_found_object) { - found_data |= 0x1; + fll_flag |= ___fll_flag_has_data___; contents->array[contents->used].used = 0; if (f_compare_dynamic_partial_string(f_fss_payload_s.string, buffer, f_fss_payload_s.used, objects->array[objects->used]) == F_equal_to) { + fll_flag |= ___fll_flag_has_payload___; + status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &contents->array[contents->used].array, &contents->array[contents->used].used, &contents->array[contents->used].size); if (F_status_is_error(status)) { @@ -102,19 +113,21 @@ extern "C" { } if (state->status == F_fss_found_object_content_not) { - found_data |= 0x1; + fll_flag |= ___fll_flag_has_data___; break; } } while (state->status == F_fss_found_object_not); + // The fl_fss_basic_list_content_read() returns F_okay_eos and F_okay_stop but doesn't increment found.used given that it has no way of knowing. + // This function will treat these states as known and declare that this is found content (and therefore found an object). if (state->status == F_okay_eos || state->status == F_okay_stop) { ++contents->array[contents->used].used; ++objects->used; ++contents->used; // Returning without a "payload" is an error. - if (!(found_data & 0x2)) { + if (!(fll_flag & ___fll_flag_has_payload___)) { state->status = F_status_set_error(state->status); } @@ -127,7 +140,7 @@ extern "C" { if (objects->used > initial_used) { // Returning without a "payload" is an error. - if (found_data & 0x2) { + if (fll_flag & ___fll_flag_has_payload___) { state->status = (state->status == F_data_not_eos) ? F_okay_eos : F_okay_stop; } else { @@ -146,7 +159,7 @@ extern "C" { } if (state->status == F_fss_found_object_content_not) { - found_data |= 0x1; + fll_flag |= ___fll_flag_has_data___; status = f_memory_array_increase(state->step_small, sizeof(f_range_t), (void **) &contents->array[contents->used].array, &contents->array[contents->used].used, &contents->array[contents->used].size); @@ -168,7 +181,7 @@ extern "C" { } // Returning without a "payload" is an error. - if (found_data & 0x2) { + if (fll_flag & ___fll_flag_has_payload___) { state->status = (range->start >= buffer.used) ? F_okay_eos : F_okay_stop; } else { @@ -183,6 +196,9 @@ extern "C" { } while (range->start < F_string_t_size_d); + #undef ___fll_flag_has_data___ + #undef ___fll_flag_has_payload___ + state->status = F_status_is_error(F_number_overflow); } #endif // _di_fll_fss_payload_read_ diff --git a/level_2/fll_fss/c/fss/payload.h b/level_2/fll_fss/c/fss/payload.h index 4bf3e5107..8fa70e1b6 100644 --- a/level_2/fll_fss/c/fss/payload.h +++ b/level_2/fll_fss/c/fss/payload.h @@ -94,10 +94,12 @@ extern "C" { * * F_number_overflow (with error bit) if the maximum buffer size is reached. * + * Errors (with error bit) from: f_fss_seek_to_eol(). * Errors (with error bit) from: f_memory_array_increase(). * Errors (with error bit) from: fl_fss_basic_list_content_read(). * Errors (with error bit) from: fl_fss_basic_list_object_read(). * + * @see f_fss_seek_to_eol() * @see f_memory_array_increase() * @see fl_fss_basic_list_content_read() * @see fl_fss_basic_list_object_read() diff --git a/level_3/fss_read/data/build/testfile b/level_3/fss_read/data/build/testfile index 6373e85b9..ffbc7d7f5 100644 --- a/level_3/fss_read/data/build/testfile +++ b/level_3/fss_read/data/build/testfile @@ -7,7 +7,7 @@ settings: fail exit - modes individual individual_thread level monolithic clang test fanalyzer thread threadless + modes debug individual individual_thread level monolithic clang test fanalyzer thread threadless environment PATH LD_LIBRARY_PATH environment LANG LC_ALL LC_COLLATE LC_CTYPE LC_FASTMSG LC_MESSAGES LC_MONETARY LC_NUMERIC LC_TIME LOCPATH NLSPATH diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-at-5.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-at-5.expect index 379654289..ab756b1e0 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-at-5.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-at-5.expect @@ -1,4 +1,4 @@ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0-empty.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0-empty.expect index 379654289..ab756b1e0 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0-empty.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0-empty.expect @@ -1,4 +1,4 @@ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0.expect index 379654289..ab756b1e0 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi-select-0.expect @@ -1,4 +1,4 @@ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi.expect index 379654289..ab756b1e0 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-name-hi.expect @@ -1,4 +1,4 @@ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-original.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-original.expect index 959ccee90..6fcdc4cb8 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-original.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-original.expect @@ -10,7 +10,7 @@ c d another "A" list. quoted, "yep". But this is not a comment. - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0-original.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0-original.expect index 959ccee90..6fcdc4cb8 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0-original.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0-original.expect @@ -10,7 +10,7 @@ c d another "A" list. quoted, "yep". But this is not a comment. - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0.expect index 9c5c57b3d..a1571472a 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-select-0.expect @@ -10,7 +10,7 @@ c d another "A" list. quoted, "yep". But this is not a comment. - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-trim.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-trim.expect index 9c5c57b3d..a1571472a 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-trim.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content-trim.expect @@ -10,7 +10,7 @@ c d another "A" list. quoted, "yep". But this is not a comment. - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content.expect index 9c5c57b3d..a1571472a 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-content.expect @@ -10,7 +10,7 @@ c d another "A" list. quoted, "yep". But this is not a comment. - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-at-5.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-at-5.expect index 07cd77f6d..b6f7815e4 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-at-5.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-at-5.expect @@ -1,5 +1,5 @@ hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0-empty.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0-empty.expect index 07cd77f6d..b6f7815e4 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0-empty.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0-empty.expect @@ -1,5 +1,5 @@ hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0.expect index 07cd77f6d..b6f7815e4 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi-select-0.expect @@ -1,5 +1,5 @@ hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi.expect index 07cd77f6d..b6f7815e4 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-name-hi.expect @@ -1,5 +1,5 @@ hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-original.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-original.expect index 36317e478..c1e4b2a41 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-original.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-original.expect @@ -21,7 +21,7 @@ a{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0-original.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0-original.expect index 36317e478..c1e4b2a41 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0-original.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0-original.expect @@ -21,7 +21,7 @@ a{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0.expect index 8ac8e0e9e..9dc401ed7 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-select-0.expect @@ -21,7 +21,7 @@ a{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-trim.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-trim.expect index 8ac8e0e9e..9dc401ed7 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-trim.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content-trim.expect @@ -21,7 +21,7 @@ a{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content.expect b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content.expect index 8ac8e0e9e..9dc401ed7 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content.expect +++ b/level_3/fss_read/tests/runtime/fss_0003/expect/test-0002-mixed-object_and_content.expect @@ -21,7 +21,7 @@ a{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/fss_read/tests/runtime/fss_0003/source/test-0002-mixed.fss b/level_3/fss_read/tests/runtime/fss_0003/source/test-0002-mixed.fss index 1052a5402..4378e0dc2 100644 --- a/level_3/fss_read/tests/runtime/fss_0003/source/test-0002-mixed.fss +++ b/level_3/fss_read/tests/runtime/fss_0003/source/test-0002-mixed.fss @@ -26,7 +26,7 @@ hi\{ empty{ } hi{ - not a list\: + not a list\{ a b привет c d привет мир a diff --git a/level_3/iki_read/data/build/testfile b/level_3/iki_read/data/build/testfile index 701a699a1..2acc890a6 100644 --- a/level_3/iki_read/data/build/testfile +++ b/level_3/iki_read/data/build/testfile @@ -5,7 +5,7 @@ settings: fail exit - modes individual individual_thread level monolithic clang test fanalyzer thread threadless + modes debug individual individual_thread level monolithic clang test fanalyzer thread threadless environment PATH LD_LIBRARY_PATH environment LANG LC_ALL LC_COLLATE LC_CTYPE LC_FASTMSG LC_MESSAGES LC_MONETARY LC_NUMERIC LC_TIME LOCPATH NLSPATH diff --git a/specifications/fss.txt b/specifications/fss.txt index 7c1699eff..d155d2106 100644 --- a/specifications/fss.txt +++ b/specifications/fss.txt @@ -1,7 +1,7 @@ # fss-0002 iki-0000 # # license open_standard_license_1_1_or_later -# version 2026/05/31 +# version 2026/07/21 # # This file (assumed to be named fss.txt) can be more easily read using the following iki_read commands: # iki_read fss.txt +Q -w -rrrrrrrr anti-KISS anti-KISS ASCII ASCII BOM BOM FSS FSS KISS KISS UTF-8 UTF-8 URL URL XML XML -WWW character "'" "'" code '"' '"' italic '"' '"' @@ -48,17 +48,21 @@ Featureless Settings Specifications: These specifications refer to characters that have no printable representation as italic:"non-printable". White spaces characters that are printable, such as tabs and spaces, must be considered the same type for the purposes of parsing. Non-printing white spaces characters (zero-width characters) are ignored, are treated as placeholders for processing with the exception of combining characters. - White spaces that use combining characters result in printable characters and the resulting combination is treated as not white space. + Using a combining character automatically causes the preceding character to be considered a graph character. + White space characters that use combining characters are treated as a graph character. + Zero-width characters that use combining characters are treated as a graph character. + The new line character:"\n" (unicode:"U+000A") is an exception to this combining characters rule and is always treated as a new line without regard to any combining character. A specification may override this handling of combining characters through explicit definitions. - Zero-width characters that use combining characters are treated as non-printing characters and are skipped. + For the purposes of processing, the use of a joiner character (unicode:"U+200D" and unicode:"U+2060") is to be treated in the same way as a combining character. + This is considered the case even if the joiner joins two non-graph characters. In terms of processing, it is recommended that the code:"NULL" character is not considered the end of a string, but this is only a suggestion. - Any specification may chose to limit, restrict, or otherwise prohibit special Unicode characters such as combining characters or zero-width characters. + Any specification may chose to limit, restrict, or otherwise prohibit special Unicode characters such as combining characters, zero-width, and joiner characters. - The current behavior of existing tools treat new line character:"\n" (unicode:"U+000A") the same regardless if a combining character follows this. + The current behavior of common tools treat new line character:"\n" (unicode:"U+000A") the same regardless if a combining character follows this. As an exception to the above rules (in order to avoid confusion with how existing software handles this behavior) a new line character:"\n" (unicode:"U+000A") is always treated as a such even if followed by a combining character. In this situation the combining characters at the start can be considered invalid, however the standard behavior is to ignore them and not treat them as part of the data. Should the data be sensitive in nature or require strict structure, then treating this situation as an error invalid is recommended. - Should tools, some time in the future, decide to combine onto character:"\n" (unicode:"U+000A"), then this standard can (and should) be reveiwed and updated to a more proper behavior of actually combining onto the control character. + Should tools, some time in the future, decide to combine onto character:"\n" (unicode:"U+000A"), then this standard can (and should) be reviewed and potentially updated to a more proper behavior of actually combining onto the control character. Unless otherwise specified, new lines designate the potential start (or end) of an Object or Content. @@ -75,6 +79,9 @@ Featureless Settings Specifications: "Object 1\" is an unterminated object due to the escaped closing quote. "Object 1\\" has content starting at the 'has', with an Object named "Object 1\". + Unless otherwise specified, quotes with combining or joining characters are not to be considered a quote for these purposes. + Unless otherwise specified, escaping characters with combining or joining characters are not to be considered a escaping characters for these purposes. + Unless otherwise specified, character/data delimits are performed only when required and not unilaterally. In the case of Objects, delimits would only apply when that Object could be potentially identified as an Object. For example, code:"FSS-0001 (Extended)" needs quotes to group parts that include spaces, if there is no initial quote, then a quote following the data italic:"must not" be delimited. @@ -128,6 +135,7 @@ Featureless Settings Specifications: This delimit only applies to the start of a comment (the pound character:"#" character) as there is no terminating character for a comment (other than a new line character:"\n"). A line containing a valid comment is in its entirety ignored. This means that if there is white space before the designation symbol (the pound character:"#" character) then that white space is ignored. + Comments follow the standard combining and joiner rules. Unless otherwise specified, all designation characters italic:"must" represent ASCII:"American Standard Code for Information Interchange" codes. With designation characters being any character code used to designate how to identify an Object or Content (such as a colon character:":" at the end of a basic list).