Follow

What Counts as a Page Request?

Distil counts all HTTP page requests as GET or POST requests made to a web server's content pages and AJAX endpoints. Essentially, they’re all requests made to your web servers for files that are not an image, JavaScript, CSS, or other static content.

Distil defines page requests differently than most client-side analytics libraries, including Google Analytics and Omniture. Therefore it isn’t uncommon for the total number of page requests to be different from that reported by your analytics tool.

File Types

File types having the following file extensions are included as page requests:

  • .htm, .html, .shtml, .xhtml
  • .asp, .aspx
  • .php
  • .pl
  • .cgi
  • .cfm
  • .do
  • .jsp
  • / (pretty URLs)

Static file types having the following file extensions are not included​ as billable page requests:

  • .bmp
  • .class
  • .css
  • .csv
  • .doc
  • .docx
  • .ejs
  • .eot
  • .eps
  • .gif
  • .ico
  • .jar
  • .jpg
  • .jpeg
  • .js
  • .mid
  • .midi
  • .otf
  • .pdf
  • .pict
  • .pls
  • .png
  • .ppt
  • .pptx
  • .ps
  • .svg
  • .svgz
  • .swf
  • .tif
  • .tiff
  • .ttf
  • .webp
  • .woff
  • .woff2
  • .xls
  • .xlsx

Estimating Your Billable Requests

Using standard macOS and Linux commands, you can also use the following four steps to determine a correct estimate of your billable requests outside of the portal. You can use a similar technique for SIEM tools (e.g., Splunk, ELK stack).

1. Count the number of lines in your log file(s).

BASH example:

​$ wc -l access_log
   23998651 access_log

 

2. Count the number of lines in the log file that are not billable by Distil (refer to the static file types listed above).

BASH example:

​$ awk -F"\"" '{ print $2 }' access_log |  grep -E '\.jpg |\.js |\.css |\.jpeg |\.png |\.gif |\.pdf |\.axd ' | wc -l
   10622213

 

NOTE: This example assumes the access_log file has a LogFormat with the request string in double quotes and no other quoted strings before it. For example:

66.249.66.186 - - [01/Jan/1970:01:01:01 -0500] "GET / HTTP/1.0" 200 96 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

3. Subtract the number of non-billable lines from the line total.

[Total lines in log file (step 1)] - [Non-billable lines (step 2)] = [line total (used in step 4, below)]

23998651 - 10622213 = 13376438

4. Extrapolate based on the time period for the inputs (Distil invoicing is based on billable requests per month).

For example, your access_log was for a one week period and came to 13,376,438 billable page requests (found in step 3). Multiply the weekly billable page requests (13,376,438) by 4.3 to extrapolate the number to one month results.

13,376,438 x 4.3 = 57,518,683

 

Distil Page Counter Tools

Distil has created the following scripts to help estimate the volume of page requests for a subset of your log files. You can use these, along with your known variances, to extrapolate the data to determine likely monthly page request volume. 

NOTE: These tools are provided without warranty and their output does not constitute an agreement between you and Distil on the billable volume of requests for your site or sites.  Use of non-standard file extensions may impact the ability of this tool to accurately count your page requests.

Mac

Mac Page Counter

Windows

Windows Page Counter

Linux

Linux Page Counter

Usage

Download the file to a known location and change the permissions on the file to make it executable.

From the command line, run ./distil_log_analyzer.1.0.darwin.amd64 -h to show the help screen:

Examples

To count the page requests in one file:

./distil_log_analyzer.1.0.darwin.amd64 <filename>

To count the page requests in a directory of files:

./distil_log_analyzer.1.0.darwin.amd64 <directory>

 

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments