cURL Proxy Setup and 407 Troubleshooting

The dependable baseline is to keep the proxy address, proxy credentials and destination URL in separate arguments. For an HTTPS destination through Databay's HTTP gateway, cURL opens a TCP connection to the gateway, authenticates that proxy hop, asks it to create a CONNECT tunnel to the destination, and only then performs destination TLS inside the tunnel.

curl --disable \
  --proxy http://gw.databay.co:8888 \
  --proxy-user "$DATABAY_PROXY_USER:$DATABAY_PROXY_PASS" \
  --proxy-basic \
  --connect-timeout 5 \
  --max-time 20 \
  --fail-with-body \
  https://api.ipify.org?format=json

--disable prevents a default .curlrc from silently changing the test. --proxy describes the client-to-proxy protocol; the https:// destination still receives end-to-end TLS. --proxy-user is proxy authentication and is separate from --user, which authenticates to the destination. On Windows PowerShell, call curl.exe explicitly if curl resolves to another command.

The response from an IP endpoint is an observed exit for that request, not proof of anonymity, safety or future routing. Run the first check against a harmless endpoint you control when possible, and never paste production credentials into a shared terminal, ticket or screenshot.

We tested the commands on 2026-07-27 with curl 8.21.0 on Windows using isolated listeners bound to 127.0.0.1. The destination name was the reserved example.test; the harness did not contact that domain or a public proxy. It redacted credential values before recording observations.

This separation is the core diagnostic model: a TCP connection can reach the proxy while proxy authentication fails; authentication can succeed while CONNECT is denied; CONNECT can succeed while destination TLS fails; and TLS can succeed while the destination returns an HTTP error. Treating every failure as a bad proxy hides the layer that produced the evidence.

Embedding user:password@ in a proxy URL is convenient but fragile. Reserved characters require URL encoding, the entire value is easy to copy into logs, and a URL often travels through more tooling than a dedicated credential field. cURL's own manual also warns that a --proxy-user user:password argument can briefly be visible to other local users through process inspection even on systems where cURL later hides it.

For an interactive one-off test, injected environment variables are clearer than literals, but they are not a universal secret store. For automation, fetch the values from the platform's secret manager and stream a cURL configuration over standard input so the expanded credential is not part of cURL's argument vector:

{
  printf 'proxy = "%s"\n' 'http://gw.databay.co:8888'
  printf 'proxy-user = "%s:%s"\n' "$DATABAY_PROXY_USER" "$DATABAY_PROXY_PASS"
  printf '%s\n' 'proxy-basic' 'connect-timeout = 5' 'max-time = 20'
  printf 'url = "%s"\n' 'https://api.ipify.org?format=json'
} | curl --disable --config -

The localhost harness verified that --config - authenticated successfully while the cURL argument vector contained no password. Standard input still contains the credential in memory, so restrict access to the job and avoid shell tracing such as set -x. Do not save a credential-bearing .curlrc in a repository or reusable image. If a protected config file is operationally necessary, lock down its permissions, exclude it from backups and source control as appropriate, and rotate the credential after unintended exposure.

There are two schemes in a proxied cURL command: the destination scheme and the proxy scheme. They answer different questions. An https:// destination through an http:// proxy is normal: HTTP describes the proxy control connection, while cURL tunnels destination TLS with CONNECT. An https:// proxy additionally applies TLS to the client-to-proxy hop and needs a cURL build with HTTPS-proxy support.

# HTTPS destination through an HTTP proxy
curl --proxy http://gw.databay.co:8888 https://service.example/

# Destination name resolved by the SOCKS5 proxy
curl --proxy socks5h://gw.databay.co:8888 https://service.example/

# Destination name resolved locally, then the IP is sent to SOCKS5
curl --proxy socks5://gw.databay.co:8888 https://service.example/

The final h in socks5h:// changes who resolves the destination hostname. Our capture received SOCKS5 address type 3 with the literal reserved domain under socks5h://. The separate protocol lab for HTTP versus SOCKS5 explains why that DNS choice is client-specific and why SOCKS5 alone does not add encryption, anonymity or UDP support to an application.

Use the protocol the actual client and network support. Do not switch protocols or DNS modes to work around a source block, authentication boundary or access policy.

cURL can discover proxies from the environment. This is useful for a controlled worker image, but inherited settings are also a frequent reason a test unexpectedly bypasses or uses the wrong gateway.

Windows environment names are case-insensitive, so the lowercase-only distinction cannot be represented in the same way there. When proving an explicit --proxy setup, add --noproxy "" to neutralize an inherited bypass list. When proving environment behavior, print only variable names and routing decisions, never a credential-bearing value. cURL 8.7.0 and later can report proxy_used=1 through --write-out, which is more reliable than guessing from response time.

A 407 is a proxy authentication challenge. RFC 9110 requires a proxy-generated 407 to include Proxy-Authenticate; a client responds with Proxy-Authorization. That is a different exchange from a destination's 401 and a different decision from a destination's 403.

1. Confirm the response belongs to the proxy. Run one approved URL with --verbose locally and find the connection target plus the 407 and Proxy-Authenticate lines. Do not publish the full trace.
2. Separate the credential fields. Use --proxy-user for the gateway. --user does not answer a proxy challenge.
3. Check the authentication scheme. Basic is cURL's default proxy method. Use --proxy-anyauth only when the proxy advertises another supported method and an extra challenge request is acceptable.
4. Validate the account grammar. Recheck the base username, password, pool and documented country or session flags. Do not assume a destination response proves the gateway rejected them.
5. Test one request without concurrency. A minimal request removes retry loops, redirects and application headers from the diagnosis.

Our lab recorded two proxy requests for --proxy-anyauth: the first had no proxy credential and received the challenge; the second carried redacted Basic authentication. That extra round trip is expected. With no credential and --fail-with-body, the HTTP proxy lab produced exit code 22 plus The requested URL returned error: 407. HTTPS tunnel failures can surface differently by build and proxy behavior, so preserve both cURL's numeric exit code and the visible proxy response instead of hard-coding one universal 407 exit code.

Human-readable verbose output is useful at the terminal but noisy in automation. Use a small --write-out record and keep the response body separate:

curl --disable --silent --show-error --fail-with-body \
  --proxy http://gw.databay.co:8888 \
  --proxy-user "$DATABAY_PROXY_USER:$DATABAY_PROXY_PASS" \
  --connect-timeout 5 --max-time 20 \
  --output response.json \
  --write-out 'status=%{response_code} proxy=%{proxy_used} peer=%{remote_ip} connect=%{time_connect} tls=%{time_appconnect} total=%{time_total}\n' \
  https://api.ipify.org?format=json

In our HTTP lab, the record was status=200, proxy=1, and peer=127.0.0.1. The important interpretation is that remote_ip was the connected proxy socket, not the eventual exit IP. The controlled destination response is where an application can observe the exit address. time_connect covers the connection phase, time_appconnect records completion of destination TLS where applicable, and time_total is end-to-end transfer time.

proxy_used was added in cURL 8.7.0, so check curl --version before adopting that field in a shared job. Treat the metrics as evidence from one request, not a performance benchmark. Record the cURL version, operating system, timestamp, destination you control, proxy mode and exit code alongside them.

Start at the first layer that failed and stop once the evidence leaves your control. Exact text varies by cURL build, TLS backend and operating system, but the numeric exit code is designed for automation.

--insecure is not a general proxy fix. It suppresses certificate verification and can hide interception or a misconfigured endpoint. Use a scoped CA option for a controlled private environment when appropriate, and keep verification enabled for production traffic.

--connect-timeout limits the connection phase, including relevant DNS, TCP and TLS work. --max-time limits one transfer. If retries are enabled, --retry-max-time provides a separate overall retry window. A bounded diagnostic request might use --connect-timeout 5 --max-time 20, but those numbers are examples, not universal service objectives.

cURL retries a defined set of transient failures with --retry and honors Retry-After when present. Do not apply --retry-all-errors by default: cURL documents it as a broad option that can duplicate sent or received data. Limit automatic retries to idempotent, authorized operations whose failure semantics you understand. Do not rotate a proxy session in response to 401, 403, 407, CAPTCHA, quota or account controls.

# Example for an approved idempotent health check only
curl --retry 2 \
  --retry-max-time 30 \
  --connect-timeout 5 \
  --max-time 10 \
  --proxy http://gw.databay.co:8888 \
  --proxy-user "$DATABAY_PROXY_USER:$DATABAY_PROXY_PASS" \
  https://health.example.test/

For production, log the attempt count, original session identifier and the final evidence. A new exit should represent an explicit sampling or resilience policy, not an automatic way around a destination decision.

First prove that the base gateway and credential work. Then add one documented routing control at a time to the username so a failure has one new variable:

# Base residential route
DATABAY_PROXY_USER='USER-zone-residential'

# Request a US exit
DATABAY_PROXY_USER='USER-zone-residential-countryCode-us'

# Request continuity for an authorized short-lived workflow
DATABAY_PROXY_USER='USER-zone-residential-countryCode-us-sessionId-qa-us-01'

Country selection requests an exit estimate from the chosen pool; it does not reproduce every localization signal. A sticky session requests temporary route continuity; it is not a dedicated device or identity. Verify the observed exit and country using a controlled endpoint, then run the authorized job without changing the session mid-transaction.

Choose datacenter proxies when a permitted workload values throughput and stable infrastructure, or residential proxies when the test genuinely needs a consumer-network route or country sample. If the destination provides an official API, export, preview or regional testing tool that solves the job, use that first.

1. Run curl --version and record the exact build and TLS backend.
2. Neutralize an inherited NO_PROXY only for the explicit test.
3. Use a harmless destination you control and one request with no redirects or retries.
4. Keep the proxy address, proxy credentials and destination credentials separate.
5. Capture exit code, response code, proxy_used when supported, socket peer and timing.
6. Classify the first failed layer: resolution, TCP, proxy auth, CONNECT or SOCKS, destination TLS, then destination HTTP.
7. Redact Proxy-Authorization, cookies, tokens, query secrets and response data before sharing a trace.
8. Stop on unclear permission, authentication boundaries, 403, 429, CAPTCHA or account enforcement.

When the command works but you need to confirm what the gateway actually observed, use the proxy checker methodology for point-in-time liveness and protocol evidence, then use the browser-only proxy format converter to translate a credential safely between supported client formats. Neither tool can prove that a proxy is trustworthy or appropriate for a destination.