Ruby Net::HTTP and Faraday Proxy Setup: 407 and Timeouts

Use this setup only for destinations you own or are permitted to access, and follow each destination's terms. Net::HTTP.new takes the proxy as four positional arguments after the destination host and port. Pass them explicitly, because the default third argument, :ENV, picks up any inherited http_proxy, as the Net::HTTP proxy documentation describes.

require "net/http"

uri = URI("https://api.ipify.org?format=json")

http = Net::HTTP.new(
  uri.host, uri.port,
  "gw.databay.co", 8888,
  ENV.fetch("DATABAY_PROXY_USER"),
  ENV.fetch("DATABAY_PROXY_PASS")
)
http.use_ssl = true                           # CONNECT, then TLS to the destination
http.verify_mode = OpenSSL::SSL::VERIFY_PEER  # keep certificate checks on
http.open_timeout = 5
http.read_timeout = 15
http.write_timeout = 15
http.max_retries = 0                          # no silent replay

warn "proxy=#{http.proxy?} env=#{http.proxy_from_env?} via=#{http.proxy_address}:#{http.proxy_port}"

http.start do |session|
  response = session.request(Net::HTTP::Get.new(uri))
  abort "destination answered #{response.code}" unless response.is_a?(Net::HTTPSuccess)
  puts response.body
end

ENV.fetch raises KeyError instead of sending an empty credential, and the diagnostic line prints the route, never the credential. Inject both variables from a secret store, keep them out of screenshots, tickets and logs, and rotate the password after any exposure.

Both libraries take the same four values. The username carries Databay's routing options and is sent unchanged as the Basic proxy credential.

Faraday's proxy options keep user and password as separate keys, so nothing is percent-encoded into the URI. Databay documents HTTP proxying with CONNECT here, not a TLS-wrapped proxy hop.

The destination scheme decides what reaches the gateway. A plain http:// destination gets an absolute-form request, such as GET http://example.test/ HTTP/1.1, with a Basic Proxy-Authorization header. With use_ssl = true, Net::HTTP first sends CONNECT api.ipify.org:443 HTTP/1.1 with that credential, checks the reply with Net::HTTPResponse#value, and starts TLS only after a 2xx answer, as RFC 9110 describes. The gateway then relays end-to-end TLS and never sees the HTTPS path or headers.

Net::HTTP.start takes the same four proxy arguments plus an options hash; with use_ssl in that hash it defaults verify_mode to OpenSSL::SSL::VERIFY_PEER:

Net::HTTP.start(
  "api.ipify.org", 443,
  "gw.databay.co", 8888,
  ENV.fetch("DATABAY_PROXY_USER"), ENV.fetch("DATABAY_PROXY_PASS"),
  use_ssl: true, open_timeout: 5, read_timeout: 15, write_timeout: 15
) do |http|
  2.times { puts http.get("/?format=json").body }  # one connection while it stays open
end

Requests in one start block share the connection, and for HTTPS the tunnel, until it closes or idles past keep_alive_timeout (initially 2 seconds). A new connection may select a different available exit and reuse may retain the same one; no different IP is promised, so request continuity with a sessionId.

Never set verify_mode to OpenSSL::SSL::VERIFY_NONE to make a proxied request pass; the tunnel does not need it, and it would hide interception. Fix certificate verify failed with ca_file, ca_path or cert_store.

An omitted third argument defaults to :ENV, so Net::HTTP.new(host), Net::HTTP.get and Net::HTTP.get_response consult the environment. The documented variable is http_proxy, read for every destination, HTTPS included, so https_proxy alone changes nothing for bare Net::HTTP. URI::Generic#find_proxy prefers the lowercase name, falls back to HTTP_PROXY with a warning, ignores HTTP_PROXY under CGI, where a request header can set it, and skips the proxy for hosts matched by no_proxy and for loopback destinations, which it detects by resolving the name locally.

# Recognition only: this form puts the password inside a URL
export http_proxy="http://USER:PASS@gw.databay.co:8888"
export no_proxy="localhost,.internal.example"

Avoid that form: Net::HTTP percent-decodes the URL's user and password, so reserved characters need encoding, and child processes inherit the value. Where a platform forces it, check http.proxy_from_env? and http.proxy? first, because a no_proxy match turns proxy? false and the request goes direct without an error. Net::HTTP.new also takes a seventh argument, p_no_proxy, with the same entries; start does not. A nil third argument gives a direct connection for a deliberate control request.

Faraday's default adapter is built on Net::HTTP and ships with Faraday, so the proxy given to Faraday.new becomes Net::HTTP's proxy arguments. Use the documented hash:

# Gemfile: gem "faraday", "~> 2.15"
require "faraday"

conn = Faraday.new(
  url: "https://api.ipify.org",
  proxy: {
    uri: "http://gw.databay.co:8888",
    user: ENV.fetch("DATABAY_PROXY_USER"),
    password: ENV.fetch("DATABAY_PROXY_PASS")
  },
  request: { open_timeout: 5, read_timeout: 15, write_timeout: 15 },
  ssl: { verify: true }
) do |f|
  f.response :raise_error
  f.adapter :net_http
end

begin
  puts conn.get("/", { format: "json" }).body
rescue Faraday::ProxyAuthError
  warn "407 as a plain-HTTP response"
rescue Faraday::ConnectionFailed => e
  warn "connect phase: #{e.message}"   # DNS, TCP or a refused CONNECT
rescue Faraday::ClientError, Faraday::ServerError => e
  warn "#{e.class}: #{e.response_status}"
end

Without a proxy option, Faraday infers one with URI#find_proxy from the request URL's scheme, so an https:// request reads https_proxy, unlike bare Net::HTTP. An explicit proxy option wins, and Faraday.ignore_env_proxy = true turns discovery off.

The 407 paths differ. A plain-HTTP 407 is a response, which raise_error turns into Faraday::ProxyAuthError; an HTTPS 407 answers CONNECT before any response exists, so the adapter raises Faraday::ConnectionFailed with a message beginning 407. raise_error attaches request details, headers included, to exceptions by default, so log the class and status, not the object.

All three Net::HTTP timeouts start at 60 seconds, too long for a first diagnostic, and each bounds a different phase of the proxied exchange:

http.open_timeout  = 5   # TCP to gw.databay.co:8888, then TLS inside the tunnel
http.read_timeout  = 15  # each read(2): CONNECT reply, headers, every body chunk
http.write_timeout = 15  # each write(2): CONNECT request and request body
http.max_retries   = 0   # the default 1 replays idempotent requests on a new connection

read_timeout and write_timeout bound single system calls, so neither caps a whole download; enforce the job's total budget in the runner. max_retries starts at 1, and a replay after Net::ReadTimeout, EOFError or Errno::ECONNRESET uses a new connection that may reach another exit, which is why diagnostics use 0. Faraday's request options use the same names; set each rather than one timeout, whose mapping is adapter-specific.

Net::HTTP has no SOCKS5 client. Its arguments and http_proxy describe an HTTP proxy, and environment mode reads only the URL's host and port, so a socks5:// value still sends HTTP proxy traffic. Because gw.databay.co:8888 serves HTTP proxying, HTTPS CONNECT and SOCKS5 on one endpoint, that mistake can pass for working SOCKS5. Faraday's default adapter uses the same HTTP path; faraday-net_http 3.5.0 documents an optional socks:// mode that needs a separately installed third-party library and raises RuntimeError without it, which this guide does not cover. If a workload truly needs SOCKS5, choose a client that documents SOCKS5 authentication and verify it separately; see the HTTP versus SOCKS5 comparison.

Point the first request at https://api.ipify.org?format=json or Databay's https://databay.com/what-is-my-ip/json. Record the IP, UTC time, Ruby and gem versions and username variant, never the password. The result is the exit observed for that connection, not proof of anonymity or future routing; What Is My IP is the browser view. Call the authorized destination only after the observation matches.

Then add routing controls one at a time, copying the complete username from the Databay dashboard instead of composing grammar for other pools:

# Request a US exit
DATABAY_PROXY_USER="USER-zone-residential-countryCode-us"

# Request short-lived continuity for one authorized workflow
DATABAY_PROXY_USER="USER-zone-residential-countryCode-us-sessionId-abc123"

A sessionId requests continuity for up to 1440 minutes on Premium Residential and Residential Flex where available, or up to 120 minutes on Datacenter and Mobile. The IP can change before the window ends. Premium Residential takes one country, state, city, ZIP, coordinate or ASN selector at a time; Residential Flex, Datacenter and Mobile target country or continent. Traffic is billed per GB, so keep checks small.

Classify the first failure before changing anything; Net::HTTP's connection errors name the host and port it was opening.

set_debug_output prints the raw CONNECT request, base64 Proxy-Authorization included, and the documentation warns that it opens a serious security hole; never use it in production or paste its output into tickets. To separate a Ruby fault from a route fault, repeat the request from the same worker with the cURL proxy runbook. Keep the authenticated check in your own client; the public proxy checker does not accept paid gateway credentials.