URL Validator

The URL constructor answers exactly one of them. It throws on example.com because there is no scheme, yet it succeeds on javascript:alert(1), on a data: URL containing a script tag, on http://a, and on http://. — a host consisting of a single dot. URL.canParse() returns true for every one of those. A successful parse tells you the string has a recognisable structure and nothing more.

Resolvable means the host exists in DNS. Reachable means something answered. Safe means the scheme, host and target are ones your application is willing to act on. Each is a distinct test with its own cost and its own failure mode, and collapsing them into one is how a form validator becomes a server-side request forgery vector.

The workable order is to parse first so you are holding components rather than a string, assert on those components, and only then perform any network operation. Anything that inspects the raw text before parsing is inspecting a different string from the one your HTTP client will ultimately use.

RFC 3986 offers a regular expression in Appendix B, and it is routinely mistaken for a validator. The RFC describes it as a pattern for "breaking-down a well-formed URI reference into its components" — a splitter that assumes its input is already valid. Every group in ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? is optional, so it matches every possible string, including the empty one.

A genuinely validating expression has to encode the context-sensitive rules the grammar defers: a host is an IPv4 literal, a bracketed IPv6 literal or a registered name, each with a different permitted character set; every % must be followed by two hexadecimal digits; square brackets are legal only as IPv6 delimiters. Attempts run to several hundred characters and tend to accumulate nested quantifiers, which is where catastrophic backtracking comes from — the validator itself becomes a denial-of-service vector on a long crafted input.

Parse with the platform's URL implementation and assert on the fields afterwards. protocol, hostname and port are exact strings you can compare against a list. A regex over the whole URL is an attempt to reimplement a state machine that already exists and is already tested against the web.

RFC 3986 §3.1 permits any scheme starting with a letter and continuing with letters, digits, plus, period or hyphen, which makes the set effectively unbounded. web+myapp:, git+ssh: and every custom application scheme are well-formed. Blocklisting javascript: and data: leaves blob:, file:, vbscript: and whatever ships next; an allowlist of http: and https: does not have that problem.

The protocol property includes the trailing colon. A comparison against "https" is always false, and the version you want is "https:". This is the most common defect in hand-written URL checks, and it fails open rather than closed whenever the check is written as a rejection test — a condition matching "javascript" against a protocol that is actually "javascript:" simply never fires.

Non-special schemes have no authority component, so a host check silently passes on them: foo:bar parses cleanly and its hostname is the empty string rather than an error. Validate the scheme first, and treat the host as meaningful only once you know the scheme is one that has one.

A check shaped like "does the target start with https://example.com" is satisfied by https://example.com.evil.test/, an entirely different registrable domain that happens to share a prefix. Appending a trailing slash to the comparison narrows the gap without closing it.

WHATWG's parser maps backslashes to forward slashes inside special schemes, so https:/\evil.test/x — one forward slash, one backslash — resolved against https://example.com/ produces https://evil.test/x. String filters that look for // to detect an authority do not see it. A bare //evil.test is scheme-relative and inherits whatever scheme the page was loaded over.

The correct check compares parsed origins: resolve the candidate against your own origin as the base, then require origin to match an allowlist entry exactly. Better still, do not accept URLs from users at all — accept an opaque key that maps to a destination on the server, which eliminates the class rather than filtering it. When you need to confirm where a link genuinely ends up, the redirect checker follows the chain to its final hop.

Loopback has many spellings and the WHATWG IPv4 parser normalises all of them. http://2130706433/, http://0x7f.1/, http://0177.0.0.1/ and http://127.1/ each parse to http://127.0.0.1/, in decimal-integer, hexadecimal, octal and shorthand notation respectively. A blocklist of literal strings catches none of them; comparing the parsed hostname catches all four. IPv6 adds [::1] and IPv4-mapped forms such as [::ffff:127.0.0.1].

The address that matters most in cloud deployments is 169.254.169.254, the link-local instance metadata endpoint on AWS, Azure and Google Cloud. Any feature that fetches a user-supplied URL from inside a VPC needs the entire link-local, loopback and RFC 1918 private ranges blocked, not just that one address.

Even a correct host check is time-of-check-to-time-of-use. A name that resolves to a public address when you validate can resolve to 127.0.0.1 when your client connects a second later, which is DNS rebinding. Resolving the name yourself and connecting to the pinned address, or enforcing the rule at network egress, is the only version that holds.

Two smaller details live in the same area. A trailing dot makes example.com. a fully qualified name and a distinct Host value that equality checks against example.com will miss. And deciding whether two hosts share a registrable domain requires the Public Suffix List rather than counting dots, because co.uk is a suffix and treating example.co.uk as a subdomain of it is how cookie and same-site logic goes wrong.