Header Order

This page explains everything you need to know about headers and header order when making requests based modules.

Why Header Order Matters

Header order is one of the most distinctive fingerprinting characteristics that differentiates real browsers from automated scripts. Both HTTP/1.1 and HTTP/2 preserve the exact order in which headers are sent, making it a powerful detection mechanism for antibot systems.

Different browser implementations send headers in distinctly different orders. Chrome, for example, always sends headers in a deterministic order that's specific to its implementation. HTTP/2 additionally uses pseudo-headers (prefixed with :) that also follow implementation-specific ordering patterns:

• Chrome browsers send pseudo-headers as: :method, :authority, :scheme, :path

• Firefox browsers send them as: :method, :path, :authority, :scheme

• Safari browsers use: :method, :scheme, :path, :authority

Normal HTTP request libraries (like Python's requests, Node.js's axios, or Java's HttpURLConnection) don't provide any control over header order, typically sending them in alphabetical order or the order they were added to the request. This makes header order an excellent way for antibot systems to detect automated scripts, even when they correctly match Chrome's TLS fingerprint.

This implementation-specific ordering makes it trivial for servers to identify the client type, regardless of User-Agent spoofing attempts.

Never Use Browser DevTools for Header Analysis

One of the most critical mistakes when building HTTP/2 scripts is relying on browser Developer Tools to understand header order. DevTools do not show headers in the order they're actually sent to the server.

For example, DevTools might display headers like this:

:authority tls.peet.ws
:method GET
:path /api/all
:scheme https
accept text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
accept-encoding gzip, deflate, br, zstd
accept-language en
priority u=0, i
sec-ch-ua "Google Chrome";v="137", "Chromium";v="137", "Not/A)Brand";v="24"

However, the actual order sent by the browser (as captured by Charles Web Proxy) is:

:method	GET
:authority	tls.peet.ws
:scheme	https
:path	/api/all
sec-ch-ua	"Google Chrome";v="137", "Chromium";v="137", "Not/A)Brand";v="24"
sec-ch-ua-mobile	?0
sec-ch-ua-platform	"Windows"
upgrade-insecure-requests	1
user-agent	Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36
accept	text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
sec-fetch-site	none
sec-fetch-mode	navigate
sec-fetch-user	?1
sec-fetch-dest	document
accept-encoding	gzip, deflate, br, zstd
accept-language	en
priority	u=0, i

The difference is significant and using DevTools order will immediately expose your script as non-browser traffic.

Recommended Tools for Header Analysis

Always use one of these tools to capture real header order:

1. Charles Web Proxy - Industry standard for intercepting HTTP/HTTPS traffic

2. powhttp.com - New project that shows exact header order without modifying TLS fingerprint as much as Charles Web Proxy.

These tools show headers exactly as they're received by the server, giving you the accurate order needed for successful browser mimicry.

Implementing Header Order in TLS Clients

The Go-based TLS clients mentioned previously support explicit header ordering through configuration:

http.HeaderOrderKey: {"sec-ch-ua", "sec-ch-ua-mobile", "sec-ch-ua-platform", "upgrade-insecure-requests", "user-agent", "accept", "sec-fetch-site", "sec-fetch-mode", "sec-fetch-user", "sec-fetch-dest", "accept-encoding", "accept-language", "priority"},
http.PHeaderOrderKey: {":method", ":authority", ":scheme", ":path"},

This allows you to precisely control both pseudo-header order (PHeaderOrderKey) and regular header order (HeaderOrderKey) to match your target browser.

Debugging with Diffchecker

When building scripts, always compare your requests against real browser traffic:

1. Capture browser headers using Charles Web Proxy

2. Capture your script's headers using Charles Web Proxy

3. Paste both into diffchecker.com to easily spot differences

4. Adjust your script's header order to match the browser exactly

This side-by-side comparison makes discrepancies immediately obvious and helps ensure perfect mimicry.

Common Pitfalls to Avoid

1. Header Case Sensitivity

A fundamental difference between HTTP/1.1 and HTTP/2 is header casing. HTTP/1.1 headers use title case (e.g., User-Agent, Content-Type), while HTTP/2 headers are always lowercase (e.g., user-agent, content-type). Using the wrong case for your protocol version will immediately expose your script as automated traffic.

2. Setting Content-Length Manually

Never manually set the Content-Length header in your header list. HTTP clients handle this automatically, and including it manually will cause it to be sent twice, resulting in request failures.

3. Mismatched sec-ch-ua Headers

The sec-ch-ua header is version-specific and must match your User-Agent exactly. For example:

• Chrome 137: "Google Chrome";v="137", "Chromium";v="137", "Not/A)Brand";v="24"

• Chrome 138: "Not)A;Brand";v="8", "Chromium";v="138", "Google Chrome";v="138"

Using the wrong string will immediately flag your request as suspicious.

4. Missing Cookie Header in Order

A critical oversight is omitting "cookie" from your header order configuration. When cookies are present but not explicitly ordered, they get appended at the end of the header list. This is problematic because in recent Chrome versions, the priority header comes after cookies in the proper order. If cookies appear at the end instead, it breaks the expected sequence and exposes the automation.

Always include "cookie" in the appropriate position within your HeaderOrderKey, even if you're not sending cookies initially.

5. Having 'Disable cache' enabled in DevTools

This adds two headers that a normal user will never include. Make sure it is disabled when recording traffic from your Chrome browser.

6. Hardcoding `sec-ch-ua-full-version-list`

Especially on sites protected by DataDome, you will see this header added to your requests after solving DataDome:

sec-ch-ua-full-version-list	"Not)A;Brand";v="8.0.0.0", "Chromium";v="138.0.7204.158", "Google Chrome";v="138.0.7204.158"

You should never hardcode this value. At Hyper Solutions, we return this value from our APIs.

7. Having duplicate cookies

This is an issue we sometimes see with people trying to either manage their own cookies (by not using a cookiejar) or people using a badly implemented cookiejar. You should always verify that you are not sending the same cookie names with different values in the same request. Example shown below.

8. Using Charles's External Proxy Feature

Charles has a feature called External Proxy where you can still route your traffic through charles while also using a (residential) proxy to connect to the site. This works great but there is one issue, it moves the Content-Length header to the bottom of the headers, which will result in wrong header order comparison.

Charles Web Proxy Alternative

While Charles Web Proxy is the gold standard, some websites may block it because Charles modifies the TLS fingerprint when intercepting HTTPS traffic. In these cases, powhttp.com serves as an excellent alternative that provides accurate header order analysis without significantly influencing TLS settings like local proxies sometimes do.

Example using tls-client

Here's a practical example showing how to implement proper header ordering using the tls-client library:

package main

import (
	"fmt"
	"io"
	"log"
	
	http "github.com/bogdanfinn/fhttp"
	"github.com/bogdanfinn/fhttp/cookiejar"
	tls_client "github.com/bogdanfinn/tls-client"
	"github.com/bogdanfinn/tls-client/profiles"
)

func main() {
	jar, _ := cookiejar.New(nil)
	options := []tls_client.HttpClientOption{
		tls_client.WithTimeoutSeconds(30),
		tls_client.WithClientProfile(profiles.Chrome_133),
		tls_client.WithNotFollowRedirects(),
		tls_client.WithCookieJar(jar),
		tls_client.WithRandomTLSExtensionOrder(),
	}
	
	client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)
	if err != nil {
		log.Println(err)
		return
	}
	
	req, err := http.NewRequest(http.MethodGet, "https://tls.peet.ws/api/all", nil)
	if err != nil {
		log.Println(err)
		return
	}
	
	req.Header = http.Header{
		"sec-ch-ua":                 {`"Google Chrome";v="137", "Chromium";v="137", "Not/A)Brand";v="24"`},
		"sec-ch-ua-mobile":          {"?0"},
		"sec-ch-ua-platform":        {`"Windows"`},
		"upgrade-insecure-requests": {"1"},
		"user-agent":                {"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36"},
		"accept":                    {"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"},
		"sec-fetch-site":            {"none"},
		"sec-fetch-mode":            {"navigate"},
		"sec-fetch-user":            {"?1"},
		"sec-fetch-dest":            {"document"},
		"accept-encoding":           {"gzip, deflate, br, zstd"},
		"accept-language":           {"en"},
		"priority":                  {"u=0, i"},
		
		http.HeaderOrderKey: {
			"sec-ch-ua",
			"sec-ch-ua-mobile", 
			"sec-ch-ua-platform",
			"upgrade-insecure-requests",
			"user-agent",
			"accept",
			"sec-fetch-site",
			"sec-fetch-mode",
			"sec-fetch-user",
			"sec-fetch-dest",
			"accept-encoding",
			"accept-language",
			"priority",
		},
		http.PHeaderOrderKey: {":method", ":authority", ":scheme", ":path"},
	}
	
	resp, err := client.Do(req)
	if err != nil {
		log.Println(err)
		return
	}
	defer resp.Body.Close()
	
	log.Println(fmt.Sprintf("status code: %d", resp.StatusCode))
	readBytes, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Println(err)
		return
	}
	log.Println(string(readBytes))
}

This example demonstrates:

• Using Chrome 133 profile with random TLS extension order

• Proper header casing (lowercase for HTTP/2)

• Correct header order matching Chrome's behavior

• Proper pseudo-header order configuration

• Chrome 137 compatible sec-ch-ua headers

Example using Python tls-client

For Python users, there's a wrapper for bogdanfinn's tls-client. The most up-to-date fork is available at: https://github.com/Nintendocustom/Python-Tls-Client (original: https://github.com/FlorianREGAZ/Python-Tls-Client).

import tls_client

# Create session with Chrome profile and random TLS extension order
session = tls_client.Session(
    client_identifier="chrome_133",
    random_tls_extension_order=True
)

# Set pseudo-header order for HTTP/2
session.pseudo_header_order = [":method", ":authority", ":scheme", ":path"]

# Define headers in the exact order Chrome sends them
headers = {
    "sec-ch-ua": '"Google Chrome";v="137", "Chromium";v="137", "Not/A)Brand";v="24"',
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": '"Windows"',
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "sec-fetch-site": "none",
    "sec-fetch-mode": "navigate", 
    "sec-fetch-user": "?1",
    "sec-fetch-dest": "document",
    "accept-encoding": "gzip, deflate, br, zstd",
    "accept-language": "en",
    "priority": "u=0, i"
}

# Configure header order to match Chrome exactly, 
# do this for every request before sending it
session.header_order = [
    "sec-ch-ua",
    "sec-ch-ua-mobile",
    "sec-ch-ua-platform", 
    "upgrade-insecure-requests",
    "user-agent",
    "accept",
    "sec-fetch-site",
    "sec-fetch-mode",
    "sec-fetch-user", 
    "sec-fetch-dest",
    "accept-encoding",
    "accept-language",
    "priority"
]

# Make the request
response = session.get("https://tls.peet.ws/api/all", headers=headers)

print(f"Status Code: {response.status_code}")
print(response.text)

This Python example demonstrates the same principles as the Go version:

• Chrome profile with random TLS extension order

• Exact header order matching browser behavior

• Proper pseudo-header configuration

• Chrome 137 compatible headers Get in touch: discord.gg/akamai