emqx4.3/changes/v5.8.3.en.md

v5.8.3

Make sure to check the breaking changes and known issues before upgrading to EMQX 5.8.3.

Enhancements

Core MQTT Functionalities

• #14219 Enhanced Connection Rate Limiter for Improved System Resilience.

  • Improved system stability and responsiveness under high connection rates: Previously, when the connection rate limit was exceeded, listener acceptors would ignore new connection attempts, potentially resulting in an unrecoverable state if a large number of clients connected or reconnected frequently within a short period. Listeners now accept pending connections but immediately close them if the rate limit is reached. This reduces resource strain and improves system resilience during peak loads.

  • New listener option nolinger introduced: When set to true, a TCP-RST is sent immediately upon socket closure, helping to mitigate SYN flood attacks and further enhancing connection-handling efficiency.

  • max_connection configuration for MQTT listeners now capped by system limits: The max_connection value for MQTT listeners is now constrained by the system's limits (e.g., ulimit from the OS and node.process_limit). If configured to infinity or a value greater than the system limit, it will automatically be adjusted to match the system's maximum limit.

  • SSL listeners' ssl_options now validated before changes: Previously, invalid SSL options (such as unsupported TLS versions) could be accepted, causing client connection failures after a listener reconfiguration. With this update:

  • The node will fail to boot if a listener is configured with invalid SSL options.

  • Requests to apply invalid SSL options via the Dashboard or config API will now fail with a 400 status code.

Configuration

• #14195 Added support for client ID override.

EMQX now provides greater flexibility by allowing custom client ID overrides using the mqtt.clientid_override={Expression} configuration. This introduces a more dynamic approach to client ID management. As part of this update, the use_userid_as_clientid and peer_cert_as_clientid options are deprecated, though they will remain available for compatibility until version 6.0.

MQTT over QUIC

• #14283 Improved QUIC transport, upgrade quicer to 0.1.9.
  • Early release of remote stream resources in the abnormal scenarios.
  • Added more troubleshooting APIs. For more details, see: https://github.com/emqx/quic/compare/0.1.6...0.1.9.

Bug Fixes

Core MQTT Functionalities

• #14201 Prevent check_gc warning from appearing when a WebSocket connection encounters a rate limit.
• #14215 Fixed an issue where calls to the retainer (via REST or CLI) would throw an exception if it was disabled.
• #14223 Ensured the WebSocket close reason is returned as an atom to avoid crashes, specifically preventing the error: error: {{case_clause,#{invalid_property_code => 51}},[{cowboy_websocket...}}.
• #14260 Resolved a rare race condition that could cause the connection process to crash if the CONNECT packet was not fully received before the idle timeout (default 15 seconds) expired.
• #14268 Fixed another rare race condition that could cause the WebSocket connection process to crash when the CONNECT packet was not fully received before the idle timeout expired.
• #14266 Updated emqtt from version 1.13.0 to 1.13.5. For more details, please refer to the emqtt changelog.

Durable Sessions

• #14160 Ensured that topic matching rules for durable session subscriptions are properly applied to topics starting with the $ symbol, in accordance with the MQTT specification.

REST API

• #14117 Fixed an issue in the REST API documentation where the Users endpoint was incorrectly listed as supporting Basic Authentication.

Data Integration

• #14172 Resolved a potential race condition where testing a connector using the HTTP API could leave lingering resources if the HTTP request timed out.

• #14178 Fixed an issue where configuration synchronization could become stuck on a particular node due to simultaneous deletion of rules across different nodes in the cluster.

• #14226 Mitigated a scenario where, under high load, a node could lose track of resource metrics (e.g., action/source) and fail to recover without a restart. Now, when restarting a resource or resetting its metrics, the system attempts to recreate the lost metrics.

Additionally, warning logs related to metric failures, such as those for "hot-path" metrics like matched, are now throttled to prevent excessive log flooding. Example of throttled log:

  2024-11-14T13:56:44.134289+00:00 [warning] tag: RESOURCE, clientid: clientid, msg: handle_resource_metrics_failed, peername: 172.100.239.1:33896, reason: {badkey,matched}, stacktrace: [{erlang,map_get,[matched,#{}],[{error_info,#{module => erl_erts_errors}}]},{emqx_metrics_worker,idx_metric,4,[{file,"src/emqx_metrics_worker.erl"},{line,560}]},...
  
  2024-11-14T13:57:12.490503+00:00 [warning] msg: log_events_throttled_during_last_period, period: 1 minutes, 0 seconds, dropped: #{handle_resource_metrics_failed => 2294}

• #14265 Fixed an issue where a badkey error would occur when stopping a connector if the MQTT Source action failed to subscribe successfully.
• #14296 Prevented ecpool_sup from being blocked by a slow-starting ecpool_worker.

Configuration

• #14180 Fixed an issue with variform expressions returning 'undefined' when a variable is bound to the value undefined or null. Now, an empty string is returned instead.

• #14289 Resolved a log file path issue when importing configurations from a different environment. The EMQX_LOG_DIR environment variable is set to /opt/emqx/log in Docker but /var/log/emqx/ when installed via RPM/DEB packages. Prior to this fix, log file paths (default file handler and audit handler) are environment-variable interpolated when being exported. This could cause crashes when importing configs into a different environment where the directory didn’t exist.

With this fix, log file paths are no longer environment-variable interpolated during export. Additionally, absolute log directory paths from older versions are now converted back to environment variables if the path doesn’t exist in the new environment.

Extension

• #14243 Fixed an issue where the client.connect hook was not being triggered for some gateways.

MQTT over QUIC

• #14258 Reduced the QUIC connection shutdown timeout. Previously, QUIC connections had a 5-second timeout for graceful shutdown. If the client was unresponsive, EMQX would log warnings like:

  [warning] msg: session_stepdown_request_timeout, action: discard,

or potentially cause a timeout on the Dashboard when attempting to disconnect the client. The timeout has now been reduced to 1 second for "kick" actions and 3 seconds for other scenarios.

Breaking Changes

• #14305 Removed support of md4, md5 and ripemd160 in authentication as they are not compliant with NIST Secure Hash Standard.