Skip to content

common : retry HTTP requests over IPv4 when IPv6 connect fails#23933

Open
CptTZ wants to merge 1 commit into
ggml-org:masterfrom
CptTZ:master
Open

common : retry HTTP requests over IPv4 when IPv6 connect fails#23933
CptTZ wants to merge 1 commit into
ggml-org:masterfrom
CptTZ:master

Conversation

@CptTZ
Copy link
Copy Markdown

@CptTZ CptTZ commented May 31, 2026

Overview

On a host (e.g. HF) that has an IPv6 (AAAA) DNS record but no working IPv6 route, llama-server -hf hangs at the first HTTP request. With -v it stalls:

tonyz@tonyz-sff-lin ~/s/llama.cpp (master)> ./build/bin/llama-server -hf xxx -c 0 -v
0.00.083.740 I common_params_handle_remote_preset: looking for remote preset at https://huggingface.co/xxx/resolve/main/preset.ini
0.00.084.861 D common_download_file_single_online: no previous model file found xxx

Upon investigation, hang is in cli.Head(). huggingface.co has multiple addresses and cpp-httplib tries them sequentially with IPv6 first, and it never falls back to the working IPv4 address. The default connection timeout at 300s, the request blocks for ~5 minutes before failing.

This PR bounds the connect timeout and adds an IPv4 fallback, so dual-stack hosts with broken IPv6 connectivity work again -

tonyz@tonyz-sff-lin ~/s/llama.cpp (master)> ./build/bin/llama-server -hf xxx -c 0 -v --no-mmap
0.00.092.428 I common_params_handle_remote_preset: looking for remote preset at https://huggingface.co/xxx/resolve/main/preset.ini
0.00.093.583 D common_download_file_single_online: no previous model file found xxx
0.05.295.020 I common_download_file_single_online: HEAD failed, status: 404
0.05.296.016 I common_params_handle_remote_preset: no remote preset found, skipping
0.15.671.972 D common_download_file_single_online: using cached file: /home/tonyz/.cache/huggingface/hub/xxx.gguf
0.15.672.141 D common_download_file_single_online: using cached file: /home/tonyz/.cache/huggingface/hub/xxxx.gguf
0.15.672.533 I common_params_print_info: build 9439 (55f5dd2cf) with GNU 13.3.0 for Linux x86_64

Notes

  • Chose cpp template for easier adaption of different httplib methods/signatures
  • Timeouts: I picked 5 seconds connect timeout without a specific reason (libcurl defaults to 300ms, happy to use the same setup if needed), and it only bounds connection setup, not transfers, so it's safe.

Additional information

In #18828 we removed libcurl and in previous changes replaced libcurl with cpp-httplib - this caused a regression as libcurl supports rfc6555 - this is not supported in cpp-httplib, thus this issue.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: Yes, assisted with root-causing hang llama-server start.

@CptTZ CptTZ requested a review from a team as a code owner May 31, 2026 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant