Unreleased Changes in The Mainline

Breaking Changes

Other Changes and Enhancements

• Log records now include an optional session_id field to correlate messages received or sent (depending on the type of the record) on the same connection/session. #316

• Updated embedded libunbound to 1.22

• Use more compact representation of ResolvedAddress in logs. Instead of showing something like ResolvedAddress { name: "some.host.", addr: 10.0.0.1 } we now display it as some.host./10.0.0.1 which is a bit easier to understand and occupies less space in the logs.

• We will now trigger the requeue_message event in the case of an error resolving the ready queue, such as DNS related errors. This gives an opportunity to rebind or reject messages which are experiencing persistent DNS resolution issues. #319

• New dns_mx_resolve_in_progress, dns_mx_resolve_status_ok, dns_mx_resolve_status_fail, dns_mx_resolve_cache_hit, dns_mx_resolve_cache_miss metrics that reflect the status of MX resolution. These are available via the metrics endpoints.

• maildir can now specify file and directory modes. #109

• maildir now supports template expansion in the maildir_path. #109

• TSA now supports "Bounce", "BounceTenant" and "BounceCampaign" actions which create bounces for scheduled queues which match the domain/tenant/campaign of the triggering event. #272

• Ready Queues now use intrusive lists through the internal Message structure, which keeps memory usage for the ready queues bounded to O(number-of-messages) rather than the previous O(number-of-ready-queues * max_ready).

• Provider match rules now also support exactly matching MX hostnames via the new HostName entry.

• kcli queue-summary will now show connection limit and connection rate throttled status effects as part of the ready queue information, making it easier to determine when a (potentially shared with multiple nodes) limit might be responsible for messages in the ready queue. There is a corresponding ready-q-states API endpoint that can be used to retrieve this same information.

• connection_limit may now be explicitly set to be local to the kumod instance, even if redis-sharing is enabled, by using a string value like "local:100".

• New kafka client:send_batch method for submitting a batch of kafka messages and waiting for the results of the complete batch. Thanks to @cai-n! #324

• The Ready Queue maintainer will now perform blocking acquisition of connection limit leases in order to reduce latency when the limit is contended. For Redis-backed limits the "blocking" is really a periodic poll every 3 seconds.

• The smtp_server_ehlo event now supports editing the list of ESMTP extensions that are reported by the SMTP listener in response to the EHLO command.

• You may now configure an egress path to use_lmtp to talk to an LMTP implementation. The mx_list in the queue configuration can now also be set to a unix domain socket address to facilitate this. #267

Fixes

• When enable_tls is set to Required or RequiredInsecure, ignore the effect of remember_broken_tls. This makes it easier to set a default value of remember_broken_tls without having to remember to special case it for sites where TLS is required, or sites that use MTA-STS or DANE to advertise that it should be required.

• When configuring the unbound resolver, the port number was not passed through for the upstream DNS server, so non-standard ports would not be respected. #314

• Expiration was checked only when incrementing the number of attempts, or when spooling in. There are some situations where a message can be delayed and re-queued without incrementing the number of attempts, which meant that some messages could linger in the queues until they are actually attempted again.

• Running validate-shaping without passing any files to validate would report OK instead of telling you that you should have passed one or more file names.

• Some messages could sometimes get delayed slightly longer than intended when using TimerWheels (the default) when moving from the scheduled queue to the ready queue.

• The get_listener_domain event handler results were incorrectly cached globally which allowed a given source IP which successfully authenticated in one session to appear authenticated for other separate connections made by that same IP to/from the same domain, within the same 60 second period. #320

• TSA daemon would not report the list of scheduled queue suspensions in the initial websocket request made by a (re)connecting client.

• Certain providers configurations with multiple MXSuffix rules and multiple candidate MX hosts might not match in cases where they should.

• Connection establishment rate for a ready queue could be constrained to 1-new-connection-per-10-minute period if that ready queue had no new messages being added to it and if a connection limit had prevented new connections being opened.

• When using the HTTP injection API to construct a mailbox using UTF-8 characters, the resulting From header could wrap in an undesirable location and produce an invalid From header that fails to parse.

• When using the HTTP injection API to construct a mailbox with a name containing the @ sign, the resulting From header did not enclose the name portion in double quotes, producing an invalid From header that failed to parse.

• When using the HTTP injection API to construct a subject header with a non-space UTF-8 sequence containing the byte 0x20, the quoted printable encoder would confuse that sequence with a space and produce an invalid output sequence.

• Changing the egress pool associated with a scheduled queue would not actually take effect until the scheduled queue aged out (eg: was idle for 10 minutes).

• RFC3464 reports that used a transfer encoding for their delivery-status parts were not parsed successfully into OOB log records.

• TSA daemon would not apply multiple SetConfig actions that were present in an automation rule, and would instead only apply the first listed rule.

• Egress pools didn't allow de-serializing definitions that had an empty set of sources.