Kevin Day [Thu, 30 Nov 2023 03:01:02 +0000 (21:01 -0600)]
Bugfix: f_network_from_ip_string() and f_network_to_ip_string() design problems.
Change the inet_pton() and inet_ntop() function calls to use the explicit .v4 and .v6 union members.
Being explicit is much safer.
The inet_pton() function also returns zero without the errno set when the address is invalid.
Handle this case, returning F_address_not (with error bit) when zero is returned by inet_pton().
The unit tests for these functions are incorrect and are not all called (which is why their incorrectness was not previously discovered).
Make sure to use h_errno where appropriate.
Make sure the mock function for inet_pton() allows for returning zero or non-zero.
Make sure the family type is set so that the expected return codes are returned.
Make sure the ip string is set so that the expected return codes are returned.
Handle the newly added F_address_not case.
Add the missing unit test function calls:
- test__f_network_from_ip_address__fails
- test__f_network_from_ip_name__fails
- test__f_network_from_ip_string__fails
- test__f_network_to_ip_string__fails
This could use more review and testing.
Such extra review will be performed while I continue to write TacocaT.
Kevin Day [Thu, 30 Nov 2023 02:57:43 +0000 (20:57 -0600)]
Bugfix: f_socket_connect() should require non-negative socket.id.
I seem to have this one backwards.
The socket.id must be something other than -1.
If the socket.id is -1, then just return F_file_descriptor without error bit to designate that no descriptor is available to connect with.
Kevin Day [Tue, 14 Nov 2023 23:56:43 +0000 (17:56 -0600)]
Update: Specifications for Payload and Simple Packet.
Add "type" as a recommended Object in the header section for the Payload.
Change the size units being used in Simple Packet.
The numbers used in some areas are completely wrong 2^4 is not 32-bits.
Instead use the word "bytes" or "bits" and update the units.
The Payload Block Structure uses "bytes" rather than bits as that is the general focus of that section.
All others use "bits".
Kevin Day [Wed, 1 Nov 2023 02:58:58 +0000 (21:58 -0500)]
Progress: Resume working on custom payload functions.
I'm getting more back into things and this is a small step towards pickup up where I left off.
There are several todos that I will consider and either implement or not.
This performs the work that I mentioned in the previous commit in regards to redesigning the approach in fl_fss_payload_header_write() (now called fl_fss_payload_header_map()).
I started doing some re-designing to improve the approach and prepare for signatures/checksums.
I realized somewhere during that process that I wanted to eventually control or even randomize the header order for integrity, organization, or security reasons.
The current approach makes that impractical.
I decided to save the current state so that I can then pivot to a better approach that will accommodate the above needs.
Appropriate todo notes are added to this end.
A lot of the existing payload code will be re-designed, again, as a result of this.
I decided to use two caches.
One for the main cache.
The other for a smaller, multi-purpose cache.
The second cache should be smaller and so re-allocations should be cheaper.
Use re-allocations on this as much as possible.
There are some code changes that I was indecisive around and jumped around on implementation.
I think I cleaned that all up but I may have missed something.
Kevin Day [Thu, 5 Oct 2023 04:19:44 +0000 (23:19 -0500)]
Progress: Begin adding FSS Payload processing code.
This will use the f_abstruse project.
This is very incomplete and is mostly drafted out.
Update the Featureless Make stand alone build with the latest changes.
Apply the related cleanups from 4ced1b0ba41334ce94d0a9b511b363a6031da136 (Cleanup: Have enumeration types support "none" and have the "// enum" comment at the end.).
Kevin Day [Thu, 5 Oct 2023 03:11:38 +0000 (22:11 -0500)]
Update: Simplify the f_abstruse types a little.
Being a union, the single valued integer types provide very little gain and introduce a cost in complexity.
Remove the integer types that overlap and just stick with the f_number_signed_t and f_number_unsigned_t for single value number types.
Multiple valued number types remain because there is potential for design differences due to the memory allocation and size differences.
The f_char_t type is removed.
Just stick with f_string_t.
Provide an optional "signature" Object and associated Content rules to the specification.
This better separates defining how the "header" and "payload" can be signed or have checksums applied.
The actual rules are left rather vague and flexible for implementation specific decisions.
A suggested strategy for using the "signature" is provided within the example.
Kevin Day [Wed, 4 Oct 2023 02:00:19 +0000 (21:00 -0500)]
Feature: Add f_abstruse project.
This is the initial design and may be subject to significant change as I begin to use it more.
The idea here is to provide a type that can act as a catch all for any sort of header of any specific type.
I use standard types and provide void pointer types for everything else.
I have some concerns about supporting void pointer types due to the special case of how to handle this allocation actions.
I have decided to withhold the "append" and similar standard functions for now until I get a better grasp on how this should be used.
A union is used to use the same memory space at the cost of the largest size.
A dynamic string is likely the most common case.
The resize and similar functions provide warnings that the caller needs to be careful about allocation and de-allocation.
This poses extra security concerns for when callers of these functions do not properly or safely handle the data.
I intend to use this in the FSS Simple Packet processing code, such as writing (building) the Simple Packet string.
Kevin Day [Tue, 19 Sep 2023 22:20:42 +0000 (17:20 -0500)]
Update: Restructure f_type files and add missing f_number_signed_t.
Break apart the f_type headers into multiple header files by major type groups.
This should make navigating the files a little easier.
Some of the header files are order sensitive.
The f_number_signed_t is now implemented.
Added appropriate unit tests.
I manually added the disable defines in the fake stand alone configuration file.
Kevin Day [Mon, 4 Sep 2023 20:04:37 +0000 (15:04 -0500)]
Update: Replace f_fss_simple_packet_identify() with f_fss_simple_packet_extract() and f_fss_simple_packet_extract_range().
The implementation of f_fss_simple_packet_identify() appears to be some sort of hybrid between actual code and brainstorming.
This appears to be by accident and the resulting logic of the function is nonsense.
Fix the logic in f_fss_simple_packet_identify().
I also see no reason to perform an identify call.
The FSS Packet design is so simple that I could just add 1, 4, or 5 to determine any of the parts.
Therefore, instead implement f_fss_simple_packet_extract() and f_fss_simple_packet_extract_range() to extract the Control bits and the Size integer.
The Payload part is not extracted but for the f_fss_simple_packet_extract_range(), the range for the Payload is determined.
This allows for the caller to determine how they want to manage the memory allocation.
This is a case where object-oriented would shine had I used C++ or some other object-oriented language.
I don't have this option available and so I am going to only have f_fss_simple_packet_t and f_fss_simple_packet_range_t with their respective f_fss_simple_packet_extract() and f_fss_simple_packet_extract_range() functions.
Unit tests are now provided for these two new functions.
Several unit checks did as little as necessary to perform the tests when it comes to mocking.
The additional sanity checks impose additional restrictions.
Kevin Day [Thu, 31 Aug 2023 04:19:28 +0000 (23:19 -0500)]
Update: Take advantage of lucky convenience where F_okay could be bumped from 197 to 200 thereby matching HTTP 200 status code.
I recently noticed that the F_okay is very close to being a 200 code after swapping F_none with F_okay.
This was never planned but it is incredibly convenient.
HTTP Status 200 code now matches FLL Status Code 200.
What luck!
Kevin Day [Thu, 31 Aug 2023 03:21:50 +0000 (22:21 -0500)]
Update: Relocate some of the more common or expected to be common format strings to global statics.
This is a minor resource optimization.
There will hopefully be more optimizations in the future where several of the commonly used strings (within individual projects) are moved into a single location for that given project.
Kevin Day [Mon, 28 Aug 2023 02:59:03 +0000 (21:59 -0500)]
Feature: Add -U/--under parameter to Featureless Make.
I discovered that the data build directory customization is incomplete while working on the Kevux Tools TacocaT program.
A new parameter to specify a sub-directory within the data directory, called "under", represented by -U and --under is now added.
This provides the ability to change the "build" part of the "data/build" directory.
This can even be an empty string.
The fakefile documentation and specification is updated because this parameter must now be reserved and exposed to IKI expansion.
I have not yet done a thorough test of this feature and I will hopefully do this before the next release.
I have performed a basic test and I can now compile the TacocaT program easily using this parameter.
Kevin Day [Sun, 27 Aug 2023 21:52:26 +0000 (16:52 -0500)]
Update: Wrap up re-designing the memory logic.
Update the unit tests and stand alone builds as needed.
Also clear out the stand alone builds that are out of date and require being rebuilt from the ground up.
Kevin Day [Sat, 19 Aug 2023 04:19:37 +0000 (23:19 -0500)]
Progress: Continue re-designing of the memory logic.
I would note that I am leaving the append() and append_all() for the IKI structures because they are complex.
In the long term I may end up adding back the append() and append_all() functions for each type.
At this time it is more advantageous for me to not have that consistency for easier maintainability while the project is evolving and refactoring.
I still need to look over the string and UTF string memory related functionality.
This will be more delicate and given the size of these two projects will be a bit time consuming.
Kevin Day [Thu, 17 Aug 2023 23:44:27 +0000 (18:44 -0500)]
Update: Use F_okay instead of F_none as the first status code.
This change is in two parts.
This part is just changing the status codes.
The next part will be changing all code using the status code F_none as the standard return to instead use F_okay.
I believe that F_okay makes more semantic sense than F_none when providing a standard code.
The original idea of F_none is that there are no errors.
I think "okay" is even more accurate.
Furthermore, the F_okay better matches the return code of the HTTP standard "HTTP 200 Okay".
Kevin Day [Thu, 17 Aug 2023 23:24:14 +0000 (18:24 -0500)]
Update: Simplify disable defines for status codes.
I like the idea of having the pre-created disable defines for each group.
The problem is that this is extra maintenance and I do not see significant use for this functionality.
Remove these to make my life slightly easier.
Kevin Day [Thu, 17 Aug 2023 03:48:37 +0000 (22:48 -0500)]
Progress: Continue re-designing of the memory logic, mostly getting tests back in synchronization.
Get the tests working.
Some of the tests I am not mocking the allocation and performing the real allocation actions.
This allows for the unit test to also test memory if run with valgrind.
I did this after getting much of the work done and so I ended up not consistently doing this on all of the "works" tests.
Generally, the tests for functions with the double 's' have the actual memory allocations.
Kevin Day [Sun, 13 Aug 2023 22:11:13 +0000 (17:11 -0500)]
Progress: Continue re-designing of the memory logic also removing *_append() and *_append_all().
In some cases *_append() and *_append_all() functions are removed.
This is a lot to manage at this time so remove it.
At some future point in time I need to implement a good generic append and append all design.
This is likely to use some sort of callback that performs the individual copy calls.
Many of the existing *_append() and *_append_all() are left alone but may eventually be removed.
The dynamic string based ones, however, will remain as an exception case.
The names I originally chose for the callbacks are misleading.
Rename them to be "delete" and "destroy" rather than "resize" and "adjust", respectively.
Half way through changing this I came across the more complex cases for resizing.
These I need callbacks both both array of arrays and arrays of arrays.
I did not finish the process of converting this.
The f_type_array needs more work.
In particular, the arrays of arrays in f_type_array can be moved into a common f_memory function, such as f_memory_arrayss_resize().
The unit tests have not been updated but I did manage to get the unit test for the f_account done.
The stand alone build config.h files are not update to date either.
These changes have shown a significant improvement in the resulting binary size.
The current size is actually just barely smaller than the 0.6.7 equivalent build (when compiled using the same compiler for both).
The changes are not complete and so I anticipate the 0.7.0 branch to be smaller than the 0.6.7 branch as of this change set despite the 0.7.0 development branch having more functionality.
Kevin Day [Sat, 12 Aug 2023 03:28:09 +0000 (22:28 -0500)]
Update: Ensure pointers passed to the function are constant.
The function should never be allowed to change the pointer itself.
A double pointer is used so that the value of the pointer can be another pointer which can then be changed.
After trying to figure out why the tests were failing, I finally discovered that the wrong file is being used.
The settings-mocks needs to be used to pull in the mocked data.
I felt this fix deserved its own distinct commit rather than being part of another Progress commit.
Kevin Day [Tue, 8 Aug 2023 01:21:26 +0000 (20:21 -0500)]
Cleanup: Add explicit casts in the f_memory memset() calls.
The pointer is cast to uint8_t to perform the arithmetic.
The pointer is not cast back but the cast happens in the function call.
Be explicit and designate that the cast back to a void pointer is intentional.
Kevin Day [Sun, 6 Aug 2023 21:44:56 +0000 (16:44 -0500)]
Progress: Continue re-designing of the memory logic.
This wraps up the simple memory append all functions with added unit tests.
I anticipate that I need to do the more complex ones as well.
This adds the adjust and resize callbacks for use in more complex memory functions.
The more complex memory functions for adjust and resize are implemented with unit tests.
The unit tests for the type array callbacks are incomplete because I did not get time to finish writing them.
Those unit tests are written but are incomplete and do not yet work.
Kevin Day [Sun, 6 Aug 2023 00:43:19 +0000 (19:43 -0500)]
Progress: Begin simplifying memory design, starting with f_number_unsigned_t array structures.
I find that the use of custom per-type functions to simplify the usability of the library.
I also find that this increases the complexity and size of the code base.
I believe that over time this will become more complex and even harder to maintain.
I have decided to start changing the design to go back to simple allocation methods that just require more variables.
This simplifies the code and in theory will make it faster by having fewer function calls on the stack.
This may even reduce the resulting binary size (and I hope it does).
The down side is that the users of the library will once more have longer function call lines.
This commit is the beginning of the redesign process.
I also added the "append" and "append all" methods for f_memory_array calls.
Kevin Day [Sun, 6 Aug 2023 00:38:12 +0000 (19:38 -0500)]
Regression: Invalid type used when allocating string, f_string_t should instead be f_char_t.
I noticed odd differences between the 0.6.x and the 0.7.x in terms of memory usage.
I discovered this to be due to a bug where the f_string_t is being used when the type should be instead f_char_t.
This is likely a bug introduced in some past refactor.
Kevin Day [Sat, 5 Aug 2023 15:40:14 +0000 (10:40 -0500)]
Bugfix: The socket id is not the same descriptor for reading.
When binding and listening on some socket id, the accept call on that socket creates a new descriptor.
That descriptor is not the same as the original socket id.
Add a new id_data to the socket structure to represent the data transfer file descriptor that is created via the accept() call.
Add a new f_file_close_id() to close the file by the id in case the f_file_t is not available or in use.
This adds f_time_spec_millisecond() and f_time_spec_nanosecond() for safely handling overflow and underflow when using standard FLL f_number_unsigned_t numbers.
Unit tests are added.
A comment in the controller prrogram is added to designate that the controller-specific functions should be updated to call these functions.
Kevin Day [Sat, 5 Aug 2023 00:36:26 +0000 (19:36 -0500)]
Update: Wrap the static inline functions private_inline_f_print_to_error in custom disable macros.
As an exception case, wrap the private_inline_f_print_to_error() in _di_private_inline_private_f_print_to_error_ and _di_private_inline_f_print_to_error_ as appropriate.
Using a define condition on all of the possible cases would be difficult to manage.
An explicit define is simpler and easier in this exceptional case.
Kevin Day [Fri, 4 Aug 2023 05:00:15 +0000 (00:00 -0500)]
Cleanup: Remove the FSS Embedded List Program.
This embedded_list in fss_read should be used going forward.
The embedded_list in fss_read is not working yet and is just a copy of extended_list.
I will eventually get around to updating the embedded_list in fss_read to work as intended and expected.
Kevin Day [Fri, 4 Aug 2023 04:54:38 +0000 (23:54 -0500)]
Update: Control program to latest design practices, migrating from 0.6.x to 0.7.x design.
This gets the control program to compile under the latest design practices.
I did not to an in depth review as this program does not currently do anything.
Once the controller program is updated I will eventually come back to this.
Kevin Day [Tue, 1 Aug 2023 01:57:54 +0000 (20:57 -0500)]
Bugfix: The Byte Dump program prints extra new lines.
Migrating from the old way to the new way results in extra new lines because of the first and last new line being printed.
Remove the extra new line being printed but still print in all other cases.
Kevin Day [Wed, 26 Jul 2023 04:16:52 +0000 (23:16 -0500)]
Update: Byte Dump program structure, following newer standards.
I noticed two things while working on this that need to be addressed for all programs.
1) When using threads, the signal for piping data (such as byte_dump /bin/bash | head) does not trigger.
2) The color code modes are not being processed correctly.
These observed bugs will be fixed in some other commit.
Kevin Day [Sun, 23 Jul 2023 19:26:19 +0000 (14:26 -0500)]
Update: Finish memory array related changes.
There are still the append() and append_all() function to change but I am going to leave that for another time.
This finishes the recent set of Progress commits addressing the memory array related code.
I have observed a notable reduction in binary size but the primary goal here is reduction of duplicate code and simplification of code.
I should eventually be able to re-introduce macros to map these helper functions back to the base array type to reduce binary size by a significant amount.
This should only be done once the API has stabilized more.
Kevin Day [Sat, 22 Jul 2023 15:38:57 +0000 (10:38 -0500)]
Progress: Continue memory array related changes.
I noticed that I could also implement f_memory_array_append() and f_memory_array_append_all() methods if I used memcpy() and a multiplication over the type size.
This:
const f_status_t status = f_memory_array_increase_by(source.used, sizeof(f_socket_t), (void **) &destination->array, &destination->used, &destination->size);
if (F_status_is_error(status)) return status;
for (f_number_unsigned_t i = 0; i < source.used; ++i) {
destination->array[destination->used++] = source.array[i];
} // for
return F_none;
Could instead become:
const f_status_t status = f_memory_array_increase_by(source.used, sizeof(f_socket_t), (void **) &destination->array, &destination->used, &destination->size);
if (F_status_is_error(status)) return status;
This could then be logic could then be moved into something like f_memory_array_append_all(), further simplifying the code into:
return f_memory_array_append(source.array, source.used, sizeof(f_socket_t), (void **) &destination->array, &destination->used, &destination->size);
I need to think of the function structure that I want and so I only decided to comment in this commit.
Such a change is not implemented yet but may be done at some later time.
Kevin Day [Wed, 19 Jul 2023 04:11:32 +0000 (23:11 -0500)]
Progress: Continue memory array related changes, replacing the f_memory_structure entirely.
The f_memory_array set of functions now fully replaces the f_memory_structure functions and related macros.
I think those structure macros are nifty but they are not in use enough anymore and the f_memory_array structure is what I am going forward with.
There is still an enormouse amount of changes to do.
Kevin Day [Tue, 18 Jul 2023 04:56:33 +0000 (23:56 -0500)]
Progress: Add memory array allocation functionality and begin to use it where possible.
The pattern for handling the standard array structure is now well established.
Provide a memory array functionality for handling the most basic form.
This should reduce repeated code where possible.
It would be interesting if this reduces the size of the resulting binary but with inline optimizations it may be possible that this actually increases the size of the resulting binary.
Update the comments and tests.
There are a lot of files that will need to be updated to use these new functions.
Most of these are in the f_type_array project.
Kevin Day [Mon, 17 Jul 2023 05:24:37 +0000 (00:24 -0500)]
Update: Add FSS Simple Packet related functionality and remove delimit and comment types.
The FSS Simple Packet functionality is added, to provide for processing of the FSS Simple Packet.
The f_fss_simple_packet_ranges_t represents a set of ranges for designating where/how the packet is defined within some string.
The delimit and comment types are just helper type definitions for existing number types.
Simplify the design by getting rid of those, replacing them with the type that they effectively represent.
A lot of code is updated to reflect this change.
This is an encrypted variation of the FSS-000F "Simple Packet" specification.
The third control bit (from the left) represents that the Simple Packet is encrypted.
The requirements for the format of the actual contents of Payload Block is further relaxed due encrypted data needing to be private (for what should be obvious reasons).