Cacti: Using Cacti to monitor web page loading – Part 1

In my current project, I’ve been looking into website performance and measuring how long it takes for webpages to be composed. Although there are plenty of tools both commercial and non-commercial which is available to do just that, they either cost too much or is a one-time data capture tool (which would be quite laborious if we have to constantly trigger it every 5 mins manually).

Instead I investigated on Cacti to see if this commonly used tool by Network/System Admin folks for monitoring network devices and systems can be reworked to let me monitor webpage loading time.

And it did wonderfully…

Cacti is a great application written in PHP, which wraps around the rrdtool. I’m not going to go into what rrdtool is, you can find out more at the rrdtool website.

What I liked about using Cacti and rrdtool is scalability and performance. If I’m going to be polling for readings every 5 mins, I would want a database which will be able to handle the large number of data points. Also, Cacti provides a means to allow me to create my own custom monitoring agents. With this feature, I can pretty much create any kind of monitoring agents I like.

This post will be split into 2 3 parts. Part 1 will walk through the php agent script which you can download from here, and how to test that it’s usable for you. In Part 2, I will cover how to configure Cacti to use the agent script, and to start collecting the data into an rra file. In Part 3, I will cover how to create a graph from the data collected in Part 2 and to draw nice looking graphs like this..

AskAboutPHP Homepage Stats

To begin with…
Needless to say, you need to have a copy of Cacti running in your own environment. In my case, I have Ubuntu 8.04 and I just used the Synaptic Package Manager to install a copy. For more installation options, check out the Cacti website.

Next, go grab my script files from here. Inside you should find 2 php scripts – class.pageload.php and pageload-agent.php. I’ve commented both files so hopefully they should be quite self-explanatory.

pageload-agent.php is the agent file, and it takes in 1 parameter, namely the url to your page you want to monitor. class.pageload.php is where the script logic resides. It uses php CURL library functions to make calls to the url you specified and to return with the timing information.

Install the scripts.
Remember, these scripts are meant to be command line executed, so you don’t need them to be in your apache web directories. Just install them in directories which are accessible by Cacti. In my case, I just placed them into my user home directory, in a directory called ‘pageload’.

~$ mkdir pageload
~$ cd pageload
~/pageload$

Next, you need to make pageload-agent.php executable. To do this, just do:

~$ chmod +x pageload-agent.php

Testing the scripts
To test the script is very easy. Decide on the url you want to monitor, in my case I will use AskAboutPHP’s homepage url, and just execute the script like so:

~/pageload$ ./pageload-agent.php http://www.askaboutphp.com/

You should get something like this as the output:

dns:0.004425 con:0.006053 pre:0.006063 str:0.117481 ttl:0.180139 sze:30507 spd:169352

This is the format that Cacti is expecting as its data source. Cacti will read these numbers in and store them in the rrd archive files. So what do they mean?

dns: is the time in seconds until name resolving was complete
con: is the time in seconds it took to establish the connection
pre: is the time in seconds from start until just before file transfer begins
str: is the time in seconds until the first byte is about to be transferred
ttl: is the total transaction time in seconds for last transfer
sze: is the total number of bytes downloaded
spd: is the average download speed

This concludes Part 1. In Part 2, I will step through how to get Cacti setup to start using these scripts to collect the webpage loading time.

Download: download icon

Part 1 | Part 2 | Part 3
Share and Enjoy:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • Propeller
  • Reddit
  • StumbleUpon
  • Technorati
  • Yahoo! Buzz
Posted on September 17, 2008 at 4:15 pm by Eldee · Permalink
In: PHP Tutorials · Tagged with: , ,

26 Responses

Subscribe to comments via RSS

  1. Written by Wells on September 18, 2008 at 4:05 am
    Permalink

    Hey, thanks- this looks great. Looking forward to part two.

  2. Written by Joe on October 3, 2008 at 12:04 am
    Permalink

    I get this error when testing your script.

    “PHP Notice: Undefined offset: 0 in /root/pageload/pageload-agent.php on line 37″

    Any Ideas? Your work looks excellent and I have a real use for this!

  3. Written by webmaster on October 3, 2008 at 3:30 pm
    Permalink

    Hi Joe

    Regarding your error. It probably means that getPageLoadStats() at line 29 is not returning any valid results.

    Can you check your php environment to see if you are able to use the cURL library has been enabled?

  4. Written by Ljubisa on November 27, 2008 at 5:48 pm
    Permalink

    do you know where I can find php script that loads complete web page, including java scripts and images and measures loading time?

    Best regards

  5. Written by comment on March 29, 2009 at 1:02 am
    Permalink

    this doesnt provide true load time experience indicator

  6. Written by webmaster on March 29, 2009 at 10:37 pm
    Permalink

    No, it doesn’t but this is the best I know under a linux server environment. I have another solution using IE but it only runs on a Windows platform.

    Do share if you have a solution. Would be happy to change mine.

  7. Written by Storm on June 3, 2009 at 1:24 am
    Permalink

    I am getting this error when I try to test a test URL. Any idea what might be causing this. I am a newbie in php so please forgive my ignorance.

    PHP Notice: Undefined index:content_type in /home/user/pageload/class.pageload.php on line 51

    Your tutorial looks interesting and I am hoping to implement this in production.

  8. Written by webmaster on June 3, 2009 at 6:16 am
    Permalink

    hi Storm,
    php is telling you that there’s no such index called ‘content_type’ in the $info array at line 51.

    i suspect that the server you are querying did not have an appropriate ‘Content-Type’ in the header.

    see the last bullet point for curl_getinfo doc at
    http://www.php.net/manual/en/function.curl-getinfo.php

    my script did not check for this. maybe you can just check for whether it is NULL or not.

    thanks

  9. Written by imac on June 11, 2009 at 5:32 pm
    Permalink

    Hello,

    Got the exact same error as above.
    I have no experience with scripting in PHP so little knowledge regarding what to do to remedy the error

    [root@localhost pageload]# ./pageload-agent.php http://www.askaboutphp.com
    dns:0.335653 con:0 pre:0 str:0 ttl:0 sze:0 spd:0[root@localhost pageload]#

    PHP Notice: Undefined index: content_type in /root/pageload/class.pageload.php on line 51

    So is ther any chance of some help regarding this issue as i would like to see this ran in a production enviroment.

    Thanks in advance

  10. Written by webmaster on June 13, 2009 at 11:21 am
    Permalink

    hi guys, the

    $s['content_type'] = $info['content_type'];

    at line 51 is not that important for timing. so it’s pretty safe to just comment it out or do a check to see if the key exists or not.

  11. Written by Daniel on June 24, 2009 at 3:58 pm
    Permalink

    Hi

    I made a template so it is easy to import into cacti.
    If you want it, let me know.

  12. Written by Yuri on July 19, 2009 at 11:26 pm
    Permalink

    Daniel,

    Can you please share your templates ?

  13. Written by Daniel on July 28, 2009 at 4:24 pm
    Permalink

    Hi Daniel,

    I really would like that template to – please upload it somewhere?

  14. Written by Fabiano Cese on September 2, 2009 at 9:07 pm
    Permalink

    Hi All,

    thanks for this guy for develop this plugin for monitoring URL’s, i appreciate.

    So, I did a template, if anyone wanna this plugin, please, send a e-mail.

    Thanks a lot.

  15. Written by chaw on October 2, 2009 at 7:32 pm
    Permalink

    Hi Fabiano,

    Could you please send me this template ?

    Thanks a lot

  16. Written by greg taylor on January 28, 2010 at 8:36 am
    Permalink

    I’d like to see your template as well. I would appreciate it.

  17. Written by Jim Ebright on April 22, 2010 at 5:04 am
    Permalink

    The template for this would be nice, I too would like a copy.

  18. Written by waterjiang on June 24, 2010 at 11:19 am
    Permalink

    是个好东西,谢谢。

  19. Written by Rudiger Wolf on September 3, 2010 at 5:19 pm
    Permalink

    I think I am having some permission problems with the page load agent.

    Any suggestions on how to make sure that the agent script is being run and updating the rra file?

    When I turn on debuging in the chart I get a message as follows:
    RRDTool Says:
    ERROR: No DS called ‘con’ in ‘/var/lib/cacti/rra/con_8.rrd’

    I assume that means the data is not getting into rra file. Yet running script for cli works fine.

    Your thoughts?

  20. Written by Raymond on October 6, 2010 at 5:16 pm
    Permalink

    Daniel,

    Can you please share your templates ?

  21. Written by mike on November 12, 2010 at 12:07 am
    Permalink

    has anybody try to do full page download including executing java scripts, etc with cacti?

  22. Written by dang on October 5, 2011 at 5:13 pm
    Permalink

    Hi all, i have this error when try the agent, anyone can help me ??
    PHP Notice: Undefined index: namelookup_time in /home/dangdh/pageload/pageload-agent.php on line 39
    dns:PHP Notice: Undefined index: connect_time in /home/dangdh/pageload/pageload-agent.php on line 40
    con:PHP Notice: Undefined index: pretransfer_time in /home/dangdh/pageload/pageload-agent.php on line 41
    pre:PHP Notice: Undefined index: starttransfer_time in /home/dangdh/pageload/pageload-agent.php on line 42
    str:PHP Notice: Undefined index: total_time in /home/dangdh/pageload/pageload-agent.php on line 43
    ttl:PHP Notice: Undefined index: size_download in /home/dangdh/pageload/pageload-agent.php on line 44
    sze:PHP Notice: Undefined index: speed_download in /home/dangdh/pageload/pageload-agent.php on line 45

  23. Written by Snis on January 19, 2012 at 4:56 am
    Permalink

    Hi, got an error while running the pageload-agent.php, that told me that “eregi” is deprecated.
    It is used on line 10.
    I replaced it with:
    if (!preg_match(‘/^http\:\/\//’, $url_argv)) {

    And it seems to work.

  24. Written by Oliver NZ on February 21, 2012 at 10:10 am
    Permalink

    Hi,

    Just thought I’d share. As a non PHP guru the proggie worked but didn’t return any results. So if you’re on Ubuntu (Linux in general I suppose) you need to…

    apt-get install php5-curl

    Then all works just swimmingly.

    Cheers
    Oliver

  25. Written by Rherdian on June 8, 2012 at 3:48 pm
    Permalink

    Hi,

    i am using cacti for windows. does this scripts running on windows? because after i test it, it wont show the value..

    C:\>C:\php\php.exe D:\Cacti\pageload\pageload-agent.php http://www.askaboutphp.com
    dns: con: pre: str: ttl: sze: spd:

Subscribe to comments via RSS

Leave a Reply