Patch Release (small bug fixes)

Fixed a redirection bug on FIOBJ_T_NUMBER type testing which could have caused an 8 byte read from an unknown address (possibly a segfault) when testing usingFIOBJ_TYPE_IS(obj, FIOBJ_T_NUMBER).

This version also addresses some 32 bit compatibility concerns.

Please submit any issues or pull requests at https://github.com/boazsegev/facil.io.

Security and fixes

v. 0.7.5 (2020-05-18)

Security: backport the 0.8.x HTTP/1.1 parser and it's security updates to the 0.7.x version branch. This fixes a request smuggling attack vector and Transfer Encoding attack vector that were exposed by Sam Sanoop from the Snyk Security team (snyk.io). The parser was updated to deal with these potential issues.

Fix: (http) fixes an issue with date calculation by backporting code from the 0.8.x branch.

Fix: (fio) call less signal handlers during shutdown.

from v. 0.7.4

Fix: (http) fixes an issue and improves support for chunked encoded payloads. Credit to Ian Ker-Seymer ( @ianks ) for exposing this, writing tests (for the Ruby wrapper) and opening both the issue boazsegev/iodine#87 and the PR boazsegev/iodine#88.

Fix: (http) requests will fail when the path contains a dangling ? (empty query). Credit to @adam12 for exposing this and opening issue boazsegev/iodine#86.

Security, compatibility and minor fixes

Fix: (http) fixes a security issue in the static file name resolution logic, where a maliciously encoded request could invoke an arbitrary response.

Fix: (fio, fiobj) improved C++ compatibility. Credit to Joey (@joeyhoek) for PR #76.

Fix: (fio) fixes an issue where timer cleanup wasn't performed after fio_stop (or SIGINT/SIGTERM). No a "clean slate" will be provided if fio_start is called more then once. Note: this may break previous behavior, which should be considered undocumented and unexpected behavior. (this fax may be deferred to version 0.8.x, still undecided). Credit to @fbrausse for opening issue #72.

Fix: (fio) fixes an issue where timer cleanup would be performed after the AT_EXIT state callbacks. Now the timer cleanup callbacks will be performed before the AT_EXIT callback (as they should). (See issue #72).

Fix: (fio) fixes signal handler (re)establishment test to prevent recursive signal calling.

Patch Release (fixes)

Fix: (fio_tls) fixes a memory leak in the trusted certificate chain. Credit to @fbrausse for opening PR #71.

Fix: (fio_tls) fixes compilation / linking flags (including a bug caused by the gcc optimizer -fipa-icf) and improves support for OpenSSL using pkg-config. Credit to @fbrausse for PR #71.

Fix: (http1) fixes a race-condition between the on_ready and on_data events, that could result in the on_data event being called twice instead of once (only possible with some clients). On multi-threaded workers, this could result in the CPU spinning while the task lock remains busy. Credit to Néstor Coppi (@Shelvak) for exposing the issue and providing an example application with detailed reports. Issue #75.

Security Update

v. 0.7.1

Security: a heap-overflow vulnerability was fixed in the WebSocket parser. This attack could have been triggered remotely by a maliciously crafted message-header. Credit to Dane (4cad@silvertoque) for exposing this issue and providing a Python script demonstrating the attack.

Stable API release

Stable API release. Future API updates will be wait for the 0.8.x release.

Fix: (fio, fiobj) fixed some gcc and clang compatibility issues and warnings.

Fix: (http) fixed HTTP date format to force the day of the month to use two digits. Credit to @ianks (Ian Ker-Seymer) for exposing this issue (iodine#64).

Compatibility: (http) updated time-zone compile-time tests with a safer fall-back.

An ocean of details

We've been hard at work authoring this (hopefully last) beta release to the 0.7.x facil.io.

There's just one or two planned API updates for the HTTP module (which is why we're still in beta phase). These changes shouldn't effect any code, but they will effect ABI compatibility, so if you need ABI compatibility, you'll need to wait a bit longer.

Here's what this update includes:

Security: (fio) Slowloris mitigation is now part of the core library, where FIO_SLOWLORIS_LIMIT pending calls to write (currently 1,024 backlogged calls) will flag the connection as an attacker and either close the connection or ignore it. This protocol independent approach improves security.

Security: (http) HTTP/1.1 client throttling - new requests will not be consumed until pending responses were sent. Since HTTP/1.1 is a response-request protocol, this protocol specific approach should protect the HTTP application against slow clients.

Fix: (fio) fixed fallback implementation for fio_atomic_xchange when missing atomic primitives in compiler (older compilers). Credit to @Low-power for identifying and fixing the issue (PR #55).

Fix: (fio) fixed a possible unreleased lock when a memory allocation failed (no memory in the system). Credit to @Low-power for identifying and fixing the issue (PR #54).

Fix: (fio) fixed the fio_sock_sendfile_from_fd fall-back for a missing sendfile. Credit to @Low-power for identifying and fixing the typo (PR #49).

Fix: (fio) fixed fio_pending not decrementing packet count before reaching zero.

Fix: (fio) fixed logging message for overflowing log messages. Credit to @weskerfoot (Wesley Kerfoot) and @adam12 (Adam Daniels) for exposing the issue (issue iodine/#56).

Fix: (fio, fio_risky_hash) Florian Weber (@Florianjw) exposed a byte ordering error (last 7 byte reading order) and took time challenge the algorithm. The exposed errors were fixed and the exposed a possible attack on RiskyHash using a variation on a Meet-In-The-Middle attack, written by Hening Makholm (@hmakholm). This prompted an update and fixes to the function.

Fix: (fio) fixed fio_str_resize where data might be lost if data was written beyond the current size and the requested size is larger then the String's capacity (i.e., when fio_str_resize is (mis)used as an alternative to fio_str_capa_assert).

Fix: (json / redis) fixed JSON formatting error caused by buffer reallocation when multiple (more then 48) escape sequences were detected. This issue also effected the Redis command callback handler (which was using JSON for IPC).

Fix: (redis) fixed a potential double free call.

Fix: (redis) fixed a recursive endless loop when converting nested Hash Tables to Redis objects (which normally wouldn't happen anyway, since they would be processed as JSON).

Fix: (redis) fixed Redis reconnection. Address and port data was mistakingly written at the wrong address, causing it to be overwritten by incoming (non-pub/sub) data.

Fix: (redis) fixed a race condition in the Redis reconnection logic which might have caused more then a single pub/sub connection to be established and the first pending command to be sent again.

Fix: (fio) fix capacity maximization log to accommodate issues where getrlimit would return a rlim_max that's too high for rlim_cur (macOS).

Fix: (fio) fix uninitialized kqueue message in fio_poll_remove_fd.

Fix: (http) possible fix for http_connect, where host header length might have been left uninitialized, resulting in possible errors.

Fix: (fio) fixed logging error message for long error messages.

Update: (fio / makefile) improved detection for polling system call, sendfile, etc'.

Update: (fio) improved signal handling. Signal handling now propagates to pre-existing signal handlers. In addition, the fio_signal_handler_reset function was made public, allowing facil.io signal handlers to be removed immediately following startup (using fio_state_callback_add with FIO_CALL_PRE_START to call fio_signal_handler_reset).

Update: (fio) improved pub/sub memory usage to minimize message copying in cluster mode (same memory is used for IPC and local-process message publishing).

Update: (fio) updated the non-cryptographic PRG algorithm for performance and speed. Now the fio_rand functions are modeled after the xoroshiro128+ algorithm, with an automated re-seeding counter based on RiskyHash. This should improve performance for non cryptographic random requirements.

Compatibility: (fio) mitigate undefined MAP_ANONYMOUS on MacOS <= 10.10. Credit to @xicreative (Evan Pavlica) for iodine/PR#61.

Compatibility: (fio) various Solaris OS compatibility patches, courtesy of @Low-power (PR #52, #53).

API and TLS updates, Hash Map security and more...

I spent a bunch of time learning about Hash Map and Hash function security concerns, realizing Hash Map security should be prioritized over a goof Hash function. I also Implemented a TLS client in iodine (the Ruby wrapper for facil.io) and learned a bunch of stuff that required some API changes.

On the positive side, it appears that the 0.7.x API is fairly finalized and facil.io could probably move to a 0.7.0 release soon.

The main changes in the this beta release are:

BREAK: (fio_tls) breaking API changes to the SSL/TLS API... I know, I'm sorry, especially since there's a small and misleading change in argument ordering for fio_tls_cert_add and fio_tls_new... but if we don't fix the API now, before the 0.7.0 release, bad design might ruin our Wednesday meditation for all eternity.

BREAK: (http) breaking API changes to http_connect were required in order to support Unix Socket connections in client mode.

Deprecation: (http) deprecating the http_url_parse in favor of fio_url_parse (moved the function to the core library and rewrote it in part).

Security: facil.io hash maps now limit the number of full-collisions allowed in a hash map. This mitigates the effects of hash flooding attacks. As a side effect, hash maps that are under attack might return false results for collision objects.

Fix: (websocket) fixed an issue with the WebSocket parser where network byte order for long message lengths wasn't always respected and integer bit size was wrong for larger payloads. Credit to Marouane Elmidaoui (@moxgeek) for exposing the issue.

Fix: (http) fixed udata in WebSocket client callback for failed WebSocket client connections.

Fix: (fio) logging message when listening to a Unix Socket.

Fix: (fio) numerous minor design fixes, such as Big-Endian string memory access, allowing fio.h to be used as a header only library (requires FIO_FORCE_MALLOC) and other adjustments.

Fix: (fio) fixed unaligned memory access in SipHash implementation and added secret randomization for each application restart.

Fix: (redis) fixed an issue where destroying the Redis engine and exiting pre-maturely, before running facio.io (fio_start), will cause a segmentation fault during cleanup.

Update: (fio) added Risky Hash, for fast hashing of safe data. This is a fast hashing function (about twice as fast as the default SipHash1-3 secure function) that wasn't tested for security. For this reason it should be limited to internal / safe data, such as CLI argument names.

Security and TLS

This is a security release with partial support for TLS, using OpenSSL.

TLS support is still under development and the API related to TLS should be considered fragile (I will release 0.7.0 once the API is solid enough and things work).

BREAK: (fio_tls) breaking API changes to the SSL/TLS API, adding support for password protected private key files. Note: The TLS API is still fragile and should only be considered stable once version 0.7.0 is released with SSL/TLS support.

Security / Fix: (http) fixed an issue with the HTTP/1.1 parser, where maliciously crafted white-space data could cause a segmentation fault, resulting a potential DoS.

Fix: (fio) fixed an issue exposed by implementing the TLS layer, where the highet fd for a connection that wasn't assigned a protocol_s object immediately after the connection was opened, might avoid timeout review or avoid cleanup during shutdown (which will be marked as a memory leak).

Update: (fio_tls) added experimental support for OpenSSL. This was only partially tested and should be considered experimental.

Update: (fio) added, the fio_rw_hook_replace_unsafe to allow r/w hook switching from within a r/w hook callback.

Update: (fio_cli) a common user-error is a a missing fio_cli_end, resulting in a memory leak notification. Now facil.io protects against this common error by automatically calling fio_cli_end during the exit stage, if fio_cli_start was called.

Quick fix

This is a quick fix release, fixing a minor issue introduced in the 0.7.0.beta4 version:

Fix: (fio_cli) fixed an issue introduced in version 0.7.0.beta4, where fio_cli_get_i would dereference NULL if the value wasn't set. Now fio_cli_get_i returns zero (0) for missing values, as expected. Note: this related to the new hex and binary base support in command line numerals.