URL Encoding: The Bug That Wasted My Entire Afternoon

The afternoon I lost to URL encoding

True story: I once spent an entire afternoon debugging why my API calls were failing. The error messages were useless. Turns out I'd used encodeURI() when I should have used encodeURIComponent(). One leaves & and = unencoded, the other doesn't. Four hours of my life, gone.

URL encoding seems simple until it isn't. You've got special characters, Unicode, the whole %20 vs + debate, double encoding nightmares... I've hit all of them. So I built the URL Encoder/Decoder tool and wrote down everything I wish I'd known from day one.

How percent encoding works

Percent encoding converts each byte of a character into a sequence starting with % followed by two hex digits representing the byte value. For multi-byte characters (like Unicode), each byte is encoded separately.

• Space becomes %20 (or + in query strings)
• & becomes %26
• = becomes %3D
• ? becomes %3F
• / becomes %2F
• # becomes %23
• Unicode characters use UTF-8 bytes: é becomes %C3%A9
• Already-encoded sequences should not be double-encoded

Reserved vs unreserved characters

RFC 3986 defines which characters are safe in URLs and which require encoding. Understanding this distinction prevents encoding errors.

• Unreserved (never encode): A-Z, a-z, 0-9, hyphen (-), underscore (_), period (.), tilde (~)
• Reserved (encode when used as data): : / ? # [ ] @ ! $ & ' ( ) * + , ; =
• Reserved characters have special meaning in URLs (path separators, query delimiters)
• When reserved characters appear as data, they must be encoded
• When reserved characters serve their structural purpose, they must not be encoded
• Unreserved characters should never be encoded (though encoding them is valid)

Query string structure

Query strings follow the URL path after a question mark. They consist of key-value pairs separated by ampersands. Both keys and values must be properly encoded.

• Format: ?key1=value1&key2=value2
• The ? marks the start of the query string
• Each pair uses = to separate key from value
• Multiple pairs are joined with &
• Empty values are valid: ?key= or just ?key
• Duplicate keys are allowed: ?tag=red&tag=blue
• Order of parameters typically does not matter

encodeURIComponent: The right choice for parameters

In JavaScript, encodeURIComponent encodes everything except unreserved characters. This is the correct function for encoding query parameter keys and values.

• Encodes: ! ' ( ) * and all reserved characters
• Does not encode: A-Z a-z 0-9 - _ . ~
• Use for: query parameter values, query parameter keys, path segments
• Example: encodeURIComponent("hello world") returns "hello%20world"
• Example: encodeURIComponent("[email protected]") returns "user%40email.com"
• Always use this for user-provided data in URLs

encodeURI: For complete URLs only

encodeURI encodes a complete URL while preserving its structure. It does not encode characters that have special meaning in URLs, making it unsuitable for parameter values.

• Does not encode: : / ? # [ ] @ ! $ & ' ( ) * + , ; =
• Use only for encoding complete URLs with proper structure
• Example: encodeURI("https://example.com/path?q=hello world")
• Preserves the : // ? = structure while encoding the space
• Never use for individual parameter values (will not encode & or =)
• Misusing encodeURI for parameters creates broken or vulnerable URLs

Common mistake: Using the wrong function

The most common URL encoding error is using encodeURI instead of encodeURIComponent for parameter values. This leaves dangerous characters unencoded.

• Wrong: encodeURI("name=John&age=30") preserves & breaking the structure
• Right: encodeURIComponent("name=John&age=30") returns "name%3DJohn%26age%3D30"
• Wrong: "?search=" + encodeURI(userInput) — XSS and injection risk
• Right: "?search=" + encodeURIComponent(userInput) — safe encoding
• If user input contains & or =, encodeURI creates malformed queries
• Always use encodeURIComponent for any user-provided data

Building query strings safely

Modern JavaScript provides URLSearchParams for safe query string construction. It handles encoding automatically and prevents common errors.

• const params = new URLSearchParams() creates a new instance
• params.append("key", "value") adds encoded parameters
• params.toString() returns the properly encoded query string
• params.set("key", "value") replaces existing values
• params.get("key") retrieves decoded values
• Works in browsers and Node.js (global or from url module)
• Automatically handles special characters, spaces, and Unicode

URLSearchParams examples

URLSearchParams simplifies working with query strings in both construction and parsing scenarios.

• Building: new URLSearchParams({name: "John Doe", city: "New York"}).toString()
• Result: "name=John+Doe&city=New+York"
• Parsing: new URLSearchParams("?name=John+Doe").get("name") returns "John Doe"
• From form: new URLSearchParams(new FormData(formElement))
• Iterating: for (const [key, value] of params) { }
• Note: URLSearchParams encodes spaces as + not %20 (both are valid)

The URL API for complete URL manipulation

The URL constructor parses and manipulates complete URLs safely. It works together with URLSearchParams for query string handling.

• const url = new URL("https://example.com/path?existing=param")
• url.searchParams.append("new", "value") adds to existing query
• url.pathname = "/new/path" safely sets the path
• url.href returns the complete, properly encoded URL
• url.origin returns the protocol and host
• Parsing: new URL(userUrl) validates URL format (throws if invalid)
• Available in browsers and Node.js

Encoding in other languages

Every programming language provides URL encoding functions. The concepts are the same; only the function names differ.

• Python: urllib.parse.quote() for path, urllib.parse.urlencode() for query
• PHP: urlencode() for query values, rawurlencode() for path segments
• Java: URLEncoder.encode(value, StandardCharsets.UTF_8)
• Go: url.QueryEscape() for values, url.PathEscape() for paths
• Ruby: URI.encode_www_form_component() for query values
• C#: Uri.EscapeDataString() for values, Uri.EscapeUriString() for URLs

Space encoding: %20 vs +

Spaces can be encoded as either %20 or +, but the context matters. This inconsistency causes confusion and bugs.

• %20 is the standard percent encoding for space (RFC 3986)
• + is valid only in query strings (application/x-www-form-urlencoded)
• Path segments should always use %20, never +
• Form submissions encode spaces as + by default
• Most servers accept both in query strings
• When in doubt, %20 is always safe
• URLSearchParams uses + for spaces; encodeURIComponent uses %20

Common mistake: Double encoding

Double encoding occurs when already-encoded strings are encoded again. This creates malformed URLs that servers cannot decode correctly.

• Original: hello world
• Single encoding: hello%20world (correct)
• Double encoding: hello%2520world (broken)
• The % sign becomes %25, making %20 into %2520
• Happens when encoding user input that was already encoded
• Symptoms: %25 sequences in URLs, unexpected characters after decoding
• Prevention: only encode raw data, never encode already-encoded strings

Common mistake: Forgetting to decode

When receiving encoded data, you must decode it before use. Forgetting to decode leads to displaying encoded strings to users or incorrect data processing.

• decodeURIComponent() reverses encodeURIComponent()
• decodeURI() reverses encodeURI()
• URLSearchParams.get() returns decoded values automatically
• new URL(url).searchParams handles decoding
• Server frameworks typically decode query parameters automatically
• Always verify whether your framework decodes or returns raw strings

Encoding non-ASCII characters

Unicode characters like accented letters, emoji, and non-Latin scripts require multi-byte UTF-8 encoding. Each byte is then percent-encoded.

• é (U+00E9) encodes as %C3%A9 (two UTF-8 bytes)
• 日本 encodes as %E6%97%A5%E6%9C%AC (six UTF-8 bytes)
• 🎉 emoji encodes as %F0%9F%8E%89 (four UTF-8 bytes)
• JavaScript functions use UTF-8 automatically
• Ensure your server expects UTF-8 encoded parameters
• Legacy systems might expect Latin-1, causing mojibake

Debugging encoded URLs

When URLs do not work as expected, encoding issues are often the cause. Here are techniques for debugging.

• Browser DevTools Network tab shows raw and decoded URLs
• console.log(encodeURIComponent(value)) to see what gets encoded
• Use our URL Encoder/Decoder tool to inspect encoding
• Check for %25 sequences indicating double encoding
• Compare working and broken URLs character by character
• Test with simple ASCII first, then add special characters
• Verify server logs show expected decoded values

Security considerations

Improper URL encoding can lead to security vulnerabilities. Encoding is part of your defense against injection attacks.

• Unencoded user input in URLs can enable XSS attacks
• Path traversal: unencoded ../ can access parent directories
• Open redirect: unencoded URLs in redirects enable phishing
• Always encode user data before placing it in URLs
• Validate URL structure after construction
• Use allowlists for redirect destinations, not encoding alone
• Encoding prevents injection but does not validate content

API request encoding

When making API requests, encoding requirements depend on the method and content type.

• GET requests: all data in URL must be encoded
• POST with application/x-www-form-urlencoded: body uses same encoding as query strings
• POST with application/json: no URL encoding needed in JSON body
• Request headers may need encoding for non-ASCII values
• Path parameters must be encoded: /users/${encodeURIComponent(userId)}
• Fetch API and axios handle query parameter encoding with URLSearchParams

Server-side handling

Servers receive and decode URL-encoded data. Understanding how your framework handles encoding prevents bugs.

• Express.js: req.query contains decoded query parameters
• Django: request.GET contains decoded parameters
• Rails: params hash contains decoded values
• PHP: $_GET and $_POST are automatically decoded
• Spring: @RequestParam values are decoded
• Raw query string access is available when needed
• Be aware of your framework's handling of arrays and nested parameters

Best practices summary

Following these practices prevents the vast majority of URL encoding problems in web development.

• Use encodeURIComponent() for parameter keys and values
• Use URLSearchParams for building and parsing query strings
• Use the URL API for complete URL manipulation
• Never use encodeURI() for parameter values
• Encode once, at the point of URL construction
• Let your framework handle decoding on the server
• Test with special characters: spaces, ampersands, Unicode
• Use our URL Encoder/Decoder tool to verify encoding behavior