Should I pay for proxy bandwidth or proxy requests?
Proxy providers offer different pricing models, each with tradeoffs.
Bandwidth-based pricing:
Pay per GB transferred:
- Best for: Large pages, media-heavy sites, uncertain page sizes
- Advantage: Predictable cost per GB regardless of request count
- Disadvantage: Can be expensive if pages are large
- Typical cost: $1-15 per GB depending on proxy type
Request-based pricing:
Pay per request made:
- Best for: Small pages, API endpoints, text-heavy sites
- Advantage: Cost is predictable per page scraped
- Disadvantage: Expensive if pages are large
- Typical cost: $0.001-0.01 per request
Which is better:
- If average page size < 100 KB: Request-based usually cheaper
- If average page size > 1 MB: Bandwidth-based usually cheaper
- For mixed sizes: Calculate both to see which is more economical
Calculation example:
Scraping 100,000 pages at 500 KB each:
- Bandwidth: 50 GB × $5/GB = $250
- Requests: 100,000 × $0.003/request = $300
Optimization considerations:
With bandwidth pricing:
- Block resources aggressively to reduce page size
- Worth the effort to optimize since savings are direct
With request pricing:
- Resource blocking doesn't save money
- Focus on reducing failed requests instead
Hybrid pricing:
Some providers offer both:
- Use bandwidth pricing for development/testing (high retry rate)
- Switch to request pricing for production (if pages are small)
Hidden factors:
- Failed requests count differently by provider
- Some providers charge for requests but cap bandwidth
- Check whether redirects count as one request or multiple
Always test with your actual scraping workload before committing to large volumes.