Skip to content

merge changed till curl 8.20#39

Open
fadedreamz wants to merge 5289 commits into
masterfrom
malam/curl-8-20
Open

merge changed till curl 8.20#39
fadedreamz wants to merge 5289 commits into
masterfrom
malam/curl-8-20

Conversation

@fadedreamz

Copy link
Copy Markdown

No description provided.

vszakats and others added 30 commits March 31, 2026 17:17
Replacing `cygwin`.

Closes curl#21179
Avoid unchecked data conversions from size_t to unsigned int.

Reported-by: James Fuller
Closes curl#21174
- fail if all retries failed.
- run `dpkg --configure -a` after an aborted slow attempt.

```
Selecting previously unselected package libsys-hostname-long-perl.
Error: slow server, retry
E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.
[...]
```

Bug: curl#21107 (comment)
Follow-up to 5172ba5 curl#21107

Closes curl#21181
A previous refactor changed the TAB check so that the octet could be
accepted in the 'path', which would cause an invalid line in the saved
cookie file so not possible to read the cookie back. Not terrible
because the path cannot contain a raw tab anyway so it would never match
anyway.

Add test 1685 to verify

Reported-by: Izan on hackerone

Closes curl#21185
The value is not used for anything, no point in keeping it in memory.

Closes curl#21188
Avoid never-ending growth.

When adding more entries, it now deletes the first entry in the list,
which is the oldest added entry still held in memory. I decided to avoid
a Least Recently Used concept as I suspect with a list with this many
entries most entries have not been used, and we don't save the timestamp
of recent use anyway.

The net effect might (no matter what) be that the removed entry might
feel a bit "random" in the eyes of the user.

Verify with test 1669.

Reported-by: Geeknik Labs
Fixes curl#21183
Closes curl#21189
They are special and should not be checked like this.

Closes curl#21191
Avoid never-ending growth.

When adding more entries, it now deletes the first entry in the list,
which is the oldest added entry still held in memory. I decided to avoid
a Least Recently Used concept as I suspect with a list with this many
entries most entries have not been used, and we don't save the timestamp
of recent use anyway.

The net effect might (no matter what) be that the removed entry might
feel a bit "random" in the eyes of the user.

Verify with test 1674

Ref curl#21183
Closes curl#21190
- allow to specify when they are wanted on starting a resolve
- match dns cache entries accordingly. An entry which never
  tried to get HTTPS-RRs is no answer for a resolve that wants
  it.
- fix late arrivals of resolve answers to match the "async"
  records that started them - if it still exists.
- provide for multiple "async" resolves in a transfer at the
  same time. We may need to resolve an IP interface while the
  main connection resolve has not finished yet.
- allow lookup of HTTPS-RR information as soon as it is
  available, even if A/AAAA queries are still ongoing.

For this, the "async" infrastructure is changed:

- Defined bits for DNS queries `CURL_DNSQ_A`, `CURL_DNSQ_AAAA`
  and `CURL_DNSQ_HTTPS`. These replace `ip_version` which says
  nothing about HTTPS.
  Use them in dns cache entries for matching.
- enhance the `async->id` to be a unique `uint32_t` for
  resolves inside one multi. This is weak, as the id may
  wrap around. However it is combined with the `mid` of
  the easy handle, making collisions highly unlikely.
  `data->state.async` is only accessed in few places where
  the mid/async-id match is performed.
- vtls: for ECH supporting TLS backends (openssl, rustls, wolfssl),
  retrieve the HTTPS-RR information from the dns connection filter.
  Delay the connect if the HTTPS-RR is needed, but has not
  been resolved yet.

The implementation of all this is complete for the threaded
resolver. c-ares resolver and DoH do not take advantage of
all new async features yet. To be done in separate PRs.

Details:

c-ares: cleanup settings and initialisation. Any ares channel
is only being created on starting a resolve and propagating
operations in setopt.c to the channel are not helpful.

Changed threaded+ares pollset handling so that they do not
overwrite each others `ASYNC_NAME` timeouts.

Add trace name 'threads' for tracing thread queue and
pool used by threaded resolver.

Closes curl#21175
…e()`

To sync names for the same macro logic between lib and src, and to move
it to the curlx namespace, to match `curlx_free()` that it's calling.

Closes curl#21151
- update `actions/cache` from 5.0.3 to 5.0.4
- update `actions/download-artifact` from 7.0.0 to 8.0.1
- update `actions/upload-artifact` from 6.0.0 to 7.0.0
- update `msys2/setup-msys2` from 2.30.0 to 2.31.0
- update pip `ruff` from 0.15.7 to 0.15.8

Closes curl#21195
Closes curl#21197
Add entries for relevant new files added in the last two years.
Make sure the string is non-zero before indexing it -1. Right now, the
path is always non-zero length so this is more for (future) safety
reasons.

Closes curl#21193
Up from 1K.

Reduces the risk that someone could flush the list by tricking a user to
do many transfers to new hostnames.

Document the limit.

Follow-up to 03a792b

Closes curl#21200
req.no_body was only initialized in Curl_connect, while HTTP/2 server
push adds a duplicated handle via Curl_multi_add_perform and calls
Curl_init_do with conn==NULL, never invoking Curl_connect.

Verify it by amending test 1620

Found by Codex Security

Closes curl#21194
Check state of negotiate to allow proper connection reuse.

Closes curl#21203
Now that we support multiple async operations at the same transfer,
fix the thread queue cleanup to match not only the mid but also the
resolv_id.

Found by Codex Security

Closes curl#21202
Bumping `msys2/setup-msys2` from 2.30.0 to 2.31.0 also brought libssh
0.12.0 (after 0.11.3). libssh has been long known for loading OpenSSH
configuration from disk insecurely on Windows. 0.12.0 brings some fixes
to this (CVE-2025-14821), which in turn may be changing its behavior on
the default GH Windows runner to fail all curl SCP/SFTP tests.

Detecting what is the exact root cause, then defining exact runtime
conditions for it is difficult and fragile. Same for changing these
libssh defaults (if at all possible.)

But, since configuration loading remains insecure and broken on Windows
even after the mitigations made by 0.12.0 [0], this patch removes libssh
from the Windows CI job using it previously.

If someone figures out how to run in CI, it can be readded, though in
general, using libssh on Windows seems to be less than ideal.

[0] https://github.com/curl/curl-for-win/blob/0e4bd38a5b90f6431b4c145ed094940b4d6b6c0d/libssh.sh#L11-L94

Refs:
https://www.libssh.org/2026/02/10/libssh-0-12-0-and-0-11-4-security-releases/
msys2/MINGW-packages@9138250
curl#21199 (unsuccessful fix attempts)

Bug: curl#21195 (comment)
Follow-up to fcf946e curl#21195

Closes curl#21204
The spare one is at the time of this patch inaccessible.

Follow-up to fb5541c curl#20583

Closes curl#21207
Now the logic for handling name duplicates and picking the longest
expiry and strictest subdomain is the same for the callback as for when
reading from file.

Also strip trailing dots from the hostname added by the callback.

A minor side-effect is that the hostname provided by the callback can
now enable subdomains by starting the name with a dot, but we discourage
using such hostnames in documentation.

Amended test 1915 to verify.

Closes curl#21201
Verify that curl's --retry behaviour works

- test 502 in serial, works
- test 502 in parallel, hangs forver, test skipped by default
- test 401, no retries done

Fixes curl#20669
Reported-by: BlackFuffey on github
Closes curl#21206
We have noticed this test as failing when run in parallel in Fedora and
Amazon Linux:
https://src.fedoraproject.org/rpms/curl/c/389f1409549178f639afa49f478c544fdaa87be2?branch=rawhide

Debian did not have issues but given there are other FTP tests flagged
as timing-dependent, it makes sense to flag this one too if we notice
failures in other OSes.

Closes curl#21155
Reported-by: Izan on hackerone
Closes curl#21209
vszakats and others added 14 commits April 27, 2026 11:53
Init dynbuf after global init to bring closer to use, improve
readability and sync test sources.

Closes curl#21451
To avoid difficult to track down crashes when a tested function ends up
outputing a message via `errorf()`, `warnf()` or siblings.

Cherry-picked from curl#21449

Closes curl#21454
Fix to create the top directory `foo` when specified as
`X:foo\bar\filename`, on Windows and MS-DOS. Add test to verify.

Caught by Codex Security

Follow-up to 787ee93 curl#16566

Closes curl#21449
Pointed out by Copilot

Closes curl#21455
Skip installing `wcurl.1` also.

Reported-by: Daniel Schulte
Fixes curl#21458
Follow-up to 23bed34 curl#17035

Closes curl#21459
The corresponding tools are never installed, and both are dev tools.
Refer to their `.md` originals instead.

Also markdownify text in lines nearby.

Ref: curl#21460 (comment)

Closes curl#21461
curl 8.20.0 release

plus VERSIONS.md update

/* replace the XXX with the actual private key */
static const char mykey[] =
"-----BEGIN PRIVATE KEY-----\n"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cycode: Secret of type: 'Private Key' was found.
Severity: High
SHA: 5ddfcc8f0f

Description

A private key is a cryptographic key that is used to authenticate the owner of a particular resource or service, typically used in combination with a public key.

Cycode Remediation Guideline

❗ How to revoke


  • Generate a new key pair (public and private key).
  • Update all systems and services to use the new public key.
  • Revoke the old private key from any key management systems or services.
  • Remove the old private key from all storage locations and backups.

Tell us how you wish to proceed using one of the following commands:

Tag Short Description
#cycode_secret_false_positive <reason> Applies to this secret value for all repos in your organization
#cycode_secret_revoked Applies to this secret value for all repos in your organization

⚠️ When commenting on Github, you may need to refresh the page to see the latest updates.

Comment thread docs/examples/synctime.c
size_t len = nmemb < sizeof(header) ? nmemb : sizeof(header) - 1;
memcpy(header, field, len);
header[len] = 0; /* null-terminate local copy */
RetVal = sscanf(header, "Date: %25s %hu %25s %hu %hu:%hu:%hu",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cycode: SAST violation: 'Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')'.

Severity: High

Description

Format specifiers can take optional field widths, which should be
used to limit how many characters are copied into the target buffer.

Example:

  const char str[20] = "AAAAAAAAAAAAAAAAAAA";
  char buf[11] = {0};
  sscanf(str, "%10s", &buf); // buf = AAAAAAAAAA\0
Cycode Remediation Guideline

The possible violation in C code involving the scanf() family functions is the use of the %s specifier without specifying a limit. This mistake can lead to buffer overflow vulnerabilities, as the function reads input without restricting its length, potentially allowing more characters than the buffer is designed to hold.

Buffer overflows are dangerous because they can allow attackers to overwrite adjacent memory, which might lead to arbitrary code execution, crashes, or data corruption. To understand the vulnerability better, let's consider an example of vulnerable code:

#include <stdio.h>

int main() {
    char buffer[100];

    printf("Enter a string: ");
    scanf("%s", buffer); // Vulnerable: No limit specified for %s

    printf("You entered: %s\n", buffer);
    return 0;
}

In this example, the scanf() function reads user input without restricting its length, potentially leading to a buffer overflow if the input string exceeds the size of the buffer.

To fix this issue, developers can either specify a limit for the %s specifier or use alternative input functions like fgets(). Here's the fixed code using both methods:

  1. Limiting the %s specifier in scanf():
#include <stdio.h>

int main() {
    char buffer[100];

    printf("Enter a string: ");
    scanf("%99s", buffer); // Fixed: Limit specified for %s (99 characters + null terminator)

    printf("You entered: %s\n", buffer);
    return 0;
}
  1. Using fgets() instead of scanf():
#include <stdio.h>

int main() {
    char buffer[100];

    printf("Enter a string: ");
    fgets(buffer, sizeof(buffer), stdin); // Fixed: Using fgets with buffer size

    printf("You entered: %s", buffer); // Note: No need to add \n here as fgets retains the newline character
    return 0;
}

Both fixes prevent the buffer overflow vulnerability by limiting the number of characters being read into the buffer, ensuring safer and more secure code.

Tell us how you wish to proceed using one of the following commands:

Tag Short Description
#cycode_sast_false_positive <reason> Mark as false positive — applies to this violation only
#cycode_ai_remediation Request remediation guidance using Cycode AI
#cycode_sast_ignore_here <reason> Ignore this violation — applies to this violation only

⚠️ When commenting on Github, you may need to refresh the page to see the latest updates.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@Petezah

Petezah commented Jun 23, 2026

Copy link
Copy Markdown

I have a request: Can you squash down your two top commits (a14ca44 and 002903d), and then use the commit message from this commit: a73e8a5
Feel free to add to it if there is anything to add.
It is useful to have the reasoning and historical context if we have to do archeology later.

@Petezah Petezah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates to unitytls.c make sense. I think all the other changes are most likely just from upstream.

@Petezah Petezah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving it on request changes just so the squash/comment is not forgotten. Will approve after that.

* vtls: replace "none"-functions with NULL pointers
 2bf48b4
* lots of function pointers removed from Curl_ssl struct
* adjust function prototypes to match changes in Curl_ssl struct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.