The Rate Limiting Policy section lets you configure automated responses that react when a request amount or user behavior is above normal human rates. It comes in handy for blocking either bots or nefarious users who run automated scripts to quickly evaluate links and scrub your site’s content.
These rate limits are broken out by:
Pages Per Minute – Limits the number of pages visited in one minute.
The Pages Per Minute threshold is constantly calculated and is relative to the amount of minutes elapsed in the session. The average amount of pages opened per minute throughout the session should be lower than the "Pages per minute" rate limit set in the Portal.
As an example, with a rate limit of 20 pages per minute, a user will trigger a violation once they try to load the 21st page in the first minute of the session, however if they load 5 pages in the first minute, and 25 pages in the second minute, a violation will not be triggered as the average rate at that point would be 15 pages per minute (ex. (5ppm+25ppm) / 2min = 15ppm).
Pages Per Session – Limits the number of pages visited during a single session
Session Length – Limits the amount of time spent on the domain during a single session
Our system automatically machine-learns once you’ve onboarded a domain with Distil, creating normal human behavioral models for your site. After we have collected traffic for four to five days, our system makes recommendations as to where you should set rate limiting thresholds.
Alternately, you can manually set thresholds to any limit at any time and set an automated action when a rate limit is surpassed.
The rate limit graphs show data for your entire site, rather than a specific domain. Each rate limit type provides a graph showing the number of visitors to your site, your current rate limit setting, and Distil’s recommended setting. Use this graph to dial the rate limit setting up or down according to your business needs.
NOTE: The graphs shows traffic for the entire domain, rather than a specific path.