Forum Discussion
Chrome V 124+ on MacOS - Virtual Server Access Issue
- Apr 22, 2024
Brad_Baker / LiefZimmerman / Eric_Chen / Stan_PIRON_F5 / Stanislas_Piro2
I have patched my copy of the original iRule with what I think is a working fix, but I would love if someone in the community could validate it, because my knowledge of the intricacies of the TLS packet is not strong.
Essentially, I have replaced this if check in the original...
# If valid TLS 1.X CLIENT_HELLO handshake packet if { [binary scan $payload cH4Scx3H4x32c tls_record_content_type tls_version tls_recordlen tls_handshake_action tls_handshake_version tls_handshake_sessidlen] == 6 && \ ($tls_record_content_type == 22) && \ ([string match {030[1-3]} $tls_version]) && \ ($tls_handshake_action == 1) && \ ($payloadlen == $tls_recordlen+5)} {
...with this block, which is intended to determine whether the reported TLS record length is longer than the payload we have and, if so, collect more packets until we have enough:
# Keep collecting if CLIENT_HELLO messages that span more than one packet... if {![info exists payloadscan]} { set payloadscan [binary scan $payload cH4Scx3H4x32c tls_record_content_type tls_version tls_recordlen tls_handshake_action \ tls_handshake_version tls_handshake_sessidlen] } if {($payloadscan == 6)} { if {($tls_recordlen < 0 || $tls_recordlen > 16389)} { # if we are asked to collect more than we will handle, bail... log local0.warn "[IP::remote_addr] : parsed TLS record length ${tls_recordlen} outside of handled length (0..16389)" reject return } elseif {($payloadlen < $tls_recordlen+5)} { # if we have not collected enough yet, collect some more TCP::collect [expr {$tls_recordlen+5 - $payloadlen}] return } } # If valid TLS 1.X CLIENT_HELLO handshake packet if {($payloadscan == 6) && \ ($tls_record_content_type == 22) && \ ([string match {030[1-3]} $tls_version]) && \ ($tls_handshake_action == 1) && \ ($payloadlen == $tls_recordlen+5)} {
This has allowed the rest of the original logic to capture the SNI server name and my services to resume operation.
However, one new thing of note, is that the resulting values in $tls_handshake_preferred_version now include an extra value. If I log this value, I can see...
- Firefox returns 0304 0303 0302 0301
- Chrome/Edge with --ssl-version-max=tls1.2 return 0303 only
- Chrome/Edge v124 return xAxA 0304 0303
The xAxA value changes every time -- so far, I have seen 1A1A, 7A7A, 8A8A, DADA and FAFA. (I do not see a 7Fxy, as indicated in the switch statement).
I am now wondering if this is just an implementation detail of the new Chrome TLS packet, or if I/we are now reading the preferred values from the wrong position in the payload.
Is anyone able to verify the above?
I have not heard from anyone so far. It seems this issue may be greater in scope than I realized. Its not just chrome 124 on Mac apparently its impacting Google Chrome 124, Google Chrome 125 and Google Chrome 126 on Windows, Linux, and Mac, Google Chrome 125/126 are pre-release though.
LiefZimmermanMoving this to a new thread would be great - I would love to get some eyes on this as its impacting us greatly - more and more complaints are pouring in and we've had to turn off SNI on some of our more popular VIPs.
I know SNI can be implemented using F5 LTM policies but frankly we found that kludgy - its slow to work with you have to create drafts and then publish them, and set a lot of drop downs that end up being identical for all our sites.
Maybe its just our LTMs but it took 3-5 minutes to publish each policy it was SO incredibly painful to work with that we abandoned it for this irule approach which up until now has worked great. It takes second to add a new site to the datagroup. But if we can't find a solution to this fast we may have to go back to policies 😰
Unfortunately this rule is quite complex - if it was simpler I might be able to edit it but I'm not sure I'll be able to figure out what's going on without some help.
- Brad_BakerApr 19, 2024Cirrus
I suspect the issue is with this change/enhancement in chrome 124:
https://blog.chromium.org/2023/08/protecting-chrome-traffic-with-hybrid.htmlSpecifically the post above mentions:
Using X25519Kyber768 adds over a kilobyte of extra data to the TLS ClientHello message due to the addition of the Kyber-encapsulated key material.Here's how this is disabled in newer versions of Google Chrome:
https://www.thesslstore.com/blog/google-chrome-adds-support-for-a-hybrid-post-quantum-cryptographic-algorithm/The trouble is we can't ask all our users to disable this flag. In order to be usable the rule needs to be modified to take into account the extra TLS ClientHello data but only for Chrome 124+
- LiefZimmermanApr 19, 2024Admin
ok - I'm going to try a move this to a whole new post in the Forum...starting with the top of this thread.
(my oh my - that worked a treat) - Brad_BakerApr 20, 2024Cirrus
The article above mentions "over a kilobyte of extra datain the TLS ClientHello"
I am assuming with the extra data the TCP::playload needs to be increased from 16389 to something higher.
I tried increasing the payload from 16389 to 17408 and then to 18432 and then to 19456 but that didn't help.
So it may be more than that - maybe the pattern match for the SNI needs adjusting?
- Brad_BakerApr 21, 2024Cirrus
LiefZimmermanis there any way you can edit the original post to reflect that this is not just MacOS? It's all Chrome browsers? Maybe that would help draw interest?
I also found this: https://www.mail-archive.com/nginx@nginx.org/msg25020.html
"They do not correctly handle when the ClientHello comes in in two reads. Before Kyber, this wasn't very common because ClientHellos usually fit in a packet. But Kyber makes ClientHellos larger, so it is possible to get only a partial ClientHello in the first read, and require a second read to try again."
Maybe this is contributing as well? I'm not sure.
I also see references to using an iRule with SSL::extensions as an alternative to the above or profiles but I can't find any examples of how to implement that.
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com