How do I match HTTP headers to look like a real browser?
Matching headers correctly is crucial for avoiding detection.
The problem:
Mixing headers from different browsers is a red flag:
- Safari User-Agent with Chrome Sec-Fetch headers = obvious bot
- Old browser version with new header features = suspicious
Header combinations by browser:
Chrome/Chromium headers:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/120.0.0.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
sec-ch-ua: "Not_A Brand";v="8", "Chromium";v="120"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Firefox headers:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Firefox/121.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
DNT: 1
Upgrade-Insecure-Requests: 1
Note: Firefox doesn't send Sec-Fetch or sec-ch-ua headers.
Safari headers:
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
Safari has the simplest header set.
Common mistakes:
- Using Chrome Sec-Fetch headers with Firefox User-Agent
- Using desktop headers with mobile User-Agent
- Using outdated header formats with new browser versions
- Missing Accept-Encoding (all modern browsers support compression)
Header order matters:
Browsers send headers in consistent order. Python dicts and JavaScript objects don't guarantee order, so use:
Python:
from collections import OrderedDict
headers = OrderedDict([
('User-Agent', '...'),
('Accept', '...'),
('Accept-Language', '...'),
])
Dynamic header generation:
For rotating browsers, use a header generator that provides complete, matched sets:
# Use a library or tool that generates complete header sets
headers = generate_browser_headers('chrome', 'desktop', 'windows')
Testing headers:
Visit sites like browserleaks.com or httpbin.org/headers to see what headers real browsers send, then match them.
Recommendation:
Use a header generator tool to get complete, matched header sets. Manual header construction often leads to inconsistent combinations that trigger detection.