Changesets can be listed by changeset number.
The Git repository is here.
- Revision:
- 297
- Log:
Updated to AWStats 7.0.
- Author:
- rool
- Date:
- Fri Mar 18 13:33:29 +0000 2011
- Size:
- 109259 Bytes
- Properties:
- Property svn:executable is set
| 1 | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
| 2 | <html> |
| 3 | <head> |
| 4 | <meta name="description" content="AWStats Documentation - FAQ page"> |
| 5 | <meta name="keywords" content="awstats, awstat, faq, error, errors, frequently, asked, questions, support, help, problem, solution, troubleshooting, rotate log"> |
| 6 | <meta name="robots" content="index,follow"> |
| 7 | <meta name="title" content="AWStats Documentation - FAQs"> |
| 8 | <title>AWStats Documentation - FAQs</title> |
| 9 | <link rel="stylesheet" href="styles.css" type="text/css"> |
| 10 | <!-- $Revision: 1.187 $ - $Author: eldy $ - $Date: 2010/06/22 21:35:24 $ --> |
| 11 | </head> |
| 12 | |
| 13 | <body topmargin=10 leftmargin=5> |
| 14 | |
| 15 | |
| 16 | <table style="font: 10pt arial,helvetica,verdana" cellpadding=0 cellspacing=0 border=0 bgcolor=#FFFFFF width=100%> |
| 17 | |
| 18 | <!-- Large --> |
| 19 | <tr style="font: 10pt arial,helvetica,verdana"> |
| 20 | <td bgcolor=#9999cc align=center><a href="/"><img src="images/awstats_logo4.png" border=0></a></td> |
| 21 | <td bgcolor=#9999cc align=center> |
| 22 | <br> |
| 23 | <font style="font: 16pt arial,helvetica,sans-serif" color=#EEEEFF><b>AWStats logfile analyzer 7.0 Documentation</b></font><br> |
| 24 | <br> |
| 25 | </td> |
| 26 | <td bgcolor=#9999cc align=center> |
| 27 | |
| 28 | </td> |
| 29 | </tr> |
| 30 | |
| 31 | </table> |
| 32 | |
| 33 | |
| 34 | <br><br><H1 style="font: 26px arial,helvetica,sans-serif">Frequently Asked Questions + Troubleshooting</H1> |
| 35 | |
| 36 | |
| 37 | <br> |
| 38 | <u>ABOUT QUESTIONS:</u><br> |
| 39 | <ul> |
| 40 | FAQ-ABO100 <a href="#SERVERSOS">Which server log files or operating systems are supported ?</a><br> |
| 41 | FAQ-ABO150 <a href="#LOGFORMAT">Which log format can AWStats analyze ?</a><br> |
| 42 | FAQ-ABO200 <a href="#LANG">Which languages are available ? How to add my own language ?</a><br> |
| 43 | FAQ-ABO250 <a href="#PHPNUKE">Can AWStats be integrated with PHP Nuke ?</a><br> |
| 44 | FAQ-ABO300 <a href="#ABOUTHISTORY">About AWStats history</a><br> |
| 45 | </ul> |
| 46 | <br> |
| 47 | |
| 48 | <u>COMMON SETUP/USAGE QUESTIONS:</u><br> |
| 49 | Here, you can find the most common questions and answers about AWStats setup/usage process.<br> |
| 50 | <ul> |
| 51 | FAQ-COM025 <a href="#NOLOG">How to use AWStats with no server log</a><br> |
| 52 | FAQ-COM050 <a href="#LIMITLOG">What is the log size limit AWStats can analyze ?</a><br> |
| 53 | FAQ-COM090 <a href="#FTP">Setup for FTP server log files (proftpd, vsftpd, ...).</a><br> |
| 54 | FAQ-COM100 <a href="#MAIL">Setup for MAIL log files (Postfix, Sendmail, QMail, MDaemon, Exchange).</a><br> |
| 55 | FAQ-COM110 <a href="#MEDIASERVER">Setup for MEDIA SERVER log files (Realmedia, Windows media, Darwin streaming server).</a><br> |
| 56 | FAQ-COM115 <a href="#PERSONALIZEDLOG">Setup/Examples for LogFormat parameter.</a><br> |
| 57 | FAQ-COM120 <a href="#ROTATE">How to rotate my logs without losing data.</a><br> |
| 58 | FAQ-COM130 <a href="#CRONTAB">How to run AWStats frequently ?</a><br> |
| 59 | FAQ-COM140 <a href="#EXCLUDEHOSTS">How to exclude my IP address (or whole subnet mask) from stats ?</a><br> |
| 60 | FAQ-COM142 <a href="#SCREENSIZE">How to get the screen size and browser capabilities report working ?</a><br> |
| 61 | FAQ-COM145 <a href="#EXTRA">How to use the Extra Sections features ?</a><br> |
| 62 | FAQ-COM150 <a href="#BENCHMARK">Benchmark question.</a><br> |
| 63 | FAQ-COM200 <a href="#DNS">How reverse DNS Lookup works, unresolved IP Addresses ?</a><br> |
| 64 | FAQ-COM250 <a href="#DIFFERENT_RESULTS">Different results than other log analyzers (Analog, Webalizer, WUsage, wwwStats...).</a><br> |
| 65 | FAQ-COM300 <a href="#DIFFERENCE_HOURS">Difference between local hour and AWStats reported hour.</a><br> |
| 66 | FAQ-COM320 <a href="#GEOIPEU">What does mean "eu (European country)" in GeoIP country reports.</a><br> |
| 67 | FAQ-COM350 <a href="#OLDLOG">How can I process old log file ?</a><br> |
| 68 | FAQ-COM360 <a href="#MULTILOG">How can I process several log files in one run ?</a><br> |
| 69 | FAQ-COM400 <a href="#LOADLOG">How can I update my statistics when I use a load balancing system that splits my logs ?</a><br> |
| 70 | FAQ-COM500 <a href="#RESET">How can I reset all my statistics ?</a><br> |
| 71 | FAQ-COM600 <a href="#DAILY">How can I compile and build statistics on a daily basis only ?</a><br> |
| 72 | FAQ-COM700 <a href="#EDITHISTORY">Can I safely remove a line in AWStats history files (awstatsMMYYYY*.txt) ?</a><br> |
| 73 | </ul> |
| 74 | <br> |
| 75 | |
| 76 | <u>ERRORS/TROUBLESHOOTING QUESTIONS:</u><br> |
| 77 | Here, you can find the most common questions and answers about errors or problems using AWStats.<br> |
| 78 | <ul> |
| 79 | FAQ-SET050 <a href="#MISSINGDOLLAR">Error "Missing $ on loop variable ..."</a><br> |
| 80 | FAQ-SET100 <a href="#CGISOURCE">I see Perl script's source instead of its execution in my browser.</a><br> |
| 81 | FAQ-SET150 <a href="#SPAWNERROR">Error "...couldn't create/spawn child process..." with Apache for windows.</a><br> |
| 82 | FAQ-SET200 <a href="#INTERNAL">"Internal Error" or "Error 500" in a browser connecting to Apache.</a><br> |
| 83 | FAQ-SET210 <a href="#SPEED">"Internal Error" after a long time in my browser (See FAQ-COM100 "AWStats speed/timeout problems").</a><br> |
| 84 | FAQ-SET220 <a href="#CRASH">Crash while running awstats.pl or page content only partialy loaded</a><br> |
| 85 | FAQ-SET270 <a href="#CORRUPTEDDROPPED">Only corrupted/dropped records</a><br> |
| 86 | FAQ-SET280 <a href="#NOTSAMENUMBER">Error "Not same number of records of...".</a><br> |
| 87 | FAQ-SET300 <a href="#COULDNOTOPEN">Error "Couldn't open file ..."</a><br> |
| 88 | FAQ-SET320 <a href="#MALFORMEDUTF8">Error "Malformed UTF-8 character (unexpected..."</a><br> |
| 89 | FAQ-SET350 <a href="#EMPTY_STATS">Empty or null statistics reported.</a><br> |
| 90 | FAQ-SET360 <a href="#PARTIAL_STATS">Statistics reported except for os, browsers, robots and keywords/keyphrases.</a><br> |
| 91 | FAQ-SET400 <a href="#REDIRECT">Pipe redirection to a file give me an empty file.</a><br> |
| 92 | FAQ-SET450 <a href="#NO_ICON">No pictures/graphics shown.</a><br> |
| 93 | FAQ-SET700 <a href="#MIGRATEDOUBLED">My visits are doubled for old month I migrated from 3.2 to 5.x</a><br> |
| 94 | FAQ-SET750 <a href="#OUTOFMEMORYCYGWIN">AWStats run out of memory during update process with cygwin Perl.</a><br> |
| 95 | FAQ-SET800 <a href="#SPEED">AWStats speed/timeout problems.</a><br> |
| 96 | </ul> |
| 97 | <br> |
| 98 | |
| 99 | <u>SECURITY QUESTIONS:</u><br> |
| 100 | Here, you can find the common questions about security problems when setting or using AWStats.<br> |
| 101 | <ul> |
| 102 | FAQ-SEC100 <a href="#CSSATTACK">Can AWStats be used to make Cross Site Scripting Attacks ?</a><br> |
| 103 | FAQ-SEC150 <a href="#SECUSER">How can I prevent some users to see statistics of other users ?</a><br> |
| 104 | FAQ-SEC200 <a href="#WORMS">How to manage log files (and statistics) corrupted by worms attacks like 'Code Red Virus like'.</a><br> |
| 105 | </ul> |
| 106 | <br> |
| 107 | |
| 108 | <hr> |
| 109 | <br><br> |
| 110 | |
| 111 | |
| 112 | <a name="SERVERSOS"></a><br> |
| 113 | <b><u>FAQ-ABO100 : WHICH SERVER LOG FILES OR OS ARE SUPPORTED ?</u></b><br> |
| 114 | AWStats can works with :<br> |
| 115 | <li> All web server able to write log file with a <u>combined log format (XLF/ELF)</u> like Apache, |
| 116 | a <u>common log format (CLF)</u> like Apache or Squid, a <u>W3C log format</u> like IIS 5.0 or higher, |
| 117 | or any other log format that contains all information AWStats expect to find.<br> |
| 118 | <li> Most of all others Web/Wap/Proxy/Streaming servers.<br> |
| 119 | <li> Some FTP, Syslog or Mail log files.<br> |
| 120 | Because AWStats is in Perl, it can works on all Operating Systems.<br> |
| 121 | <br> |
| 122 | Examples of used platforms (bold means 'tested by author', others were reported by AWStats users to work correctly) :<br> |
| 123 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 124 | <u>OS:</u><br> |
| 125 | <b>Windows 2000</b>, <b>Windows NT 4.0</b>, Windows Me, <b>Linux (RedHat, Mandrake, Debian, Suse...)</b>, Macintosh, <b>Solaris</b>, <b>Aix</b>, BeOS, FreeBSD, ...<br> |
| 126 | <u>Web/Wap/Proxy/Streaming servers</u><br> |
| 127 | <b>Apache 1.3.x and 2.x</b>, <b>IIS 5.0 or higher</b>, ISA, WebStar, WebLogic, WebSite, <b>Windows Media Server</b>, Tomcat, <b>Squid</b>, |
| 128 | Sambar, Roxen, Resin, RealMedia server, Oracle9iAS, <b>Lotus Notes/Domino</b>, Darwin, IPlanet, IceCast, ZeroBrand, Zeus, Zope, Abyss...<br> |
| 129 | <u>FTP servers</u><br> |
| 130 | <b>ProFTPd</b>, vsFTPd...<br> |
| 131 | <u>Mails servers</u><br> |
| 132 | <b>Postfix</b>, <b>Sendmail</b>, QMail, <b>Mdaemon</b>, www4mail, ...<br> |
| 133 | <u>Perl interpreters (all Perl >= 5.005):</u><br> |
| 134 | <b>ActivePerl 5.6</b>, <b>ActivePerl 5.8</b>, <b>Perl 5.8</b>, <b>Perl 5.6</b>, <b>Perl 5.005</b>, <b>mod_perl</b> and mod_perl2 for Apache, ...<br> |
| 135 | </td></tr></table> |
| 136 | <br> |
| 137 | |
| 138 | <a name="LOGFORMAT"></a><br> |
| 139 | <b><u>FAQ-ABO150 : WHICH LOG FORMATS CAN AWSTATS ANALYZE ?</u></b><br> |
| 140 | AWStats setup knows predefined log formats you can use to make AWStats config easier. However, |
| 141 | you can define your own log format, that's the reason why AWStats can analyze nearly all web, wap |
| 142 | and proxy server log files. Some FTP servers log files, Syslog or mail logs can also be analyzed.<br> |
| 143 | The only requirement is "Your log file must contain required information".<br> |
| 144 | <br> |
| 145 | This is very short examples of possible log format:<br> |
| 146 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 147 | Apache common log format (see Note*),<br> |
| 148 | Apache combined log format (known as NCSA combined log format or XLF or ELF format),<br> |
| 149 | Any other personalized Apache log format,<br> |
| 150 | Any IIS log format (known as W3C format),<br> |
| 151 | Webstar native log format,<br> |
| 152 | Realmedia server, Windows Media Server, Darwin streaming server,<br> |
| 153 | ProFTPd server, vsFTPd server,<br> |
| 154 | Postfix, Sendmail, QMail, Mdaemon<br> |
| 155 | A lot of web/wap/proxy/streaming servers log format<br> |
| 156 | </td></tr></table> |
| 157 | <br> |
| 158 | *Note: Apache common log format (AWStats can now analyze such log files but such log files does not |
| 159 | contain all information AWStats is looking for. The problem is in the content, not in the |
| 160 | format). I think analyzing common log files is not interesting because there is a lot of |
| 161 | missing information: no way to filter robots, find search engines, keywords, os, browser. |
| 162 | But a lot of users asked me for it, so AWStats support it. |
| 163 | However, a lot of interesting advanced features can't work: browsers, os's, keywords, robot detection...). |
| 164 | <br> |
| 165 | See also <a href="#PERSONALIZEDLOG">F.A.Q.: LOG FORMAT SETUP OR ERRORS </a>.<br> |
| 166 | <br> |
| 167 | |
| 168 | <a name="LANG"></a><br> |
| 169 | <b><u>FAQ-ABO200 : WHICH LANGUAGES ARE AVAILABLE ?</u></b><br> |
| 170 | AWStats can make reports in 43 languages. This is a list of all of them, for last version, in |
| 171 | alphabetical order (The code you can use for <a href="awstats_config.html#Lang">Lang</a> |
| 172 | and <a href="awstats_config.html#ShowFlagLinks">ShowFlagLinks</a> parameter are |
| 173 | the ISO-639-1 language codes):<br> |
| 174 | <i> |
| 175 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 176 | Albanian=al, Bosnian=ba, Brezhoneg=bzg, Bulgarian=bg, Catalan=ca, |
| 177 | Chinese (Taiwan)=tw, Chinese (Simpliefied)=cn, Croatian=hr, Czech=cz, |
| 178 | Danish=dk, Dutch=nl, English=en, Estonian=et, Euskara=eu, Finnish=fi, |
| 179 | French=fr, Galician=gl, German=de, Greek=gr, Hebrew=he, Hungarian=hu, |
| 180 | Icelandic=is, Indonesian=id, Italian=it, Japanese=jp, Korean=ko, |
| 181 | Latvian=lv, Norwegian (Nynorsk)=nn, Norwegian (Bokmal)=nb, Polish=pl, |
| 182 | Portuguese=pt, Portuguese (Brazilian)=br, Romanian=ro, Russian=ru, |
| 183 | Serbian=sr, Slovak=sk, Solvenian=si, Spanish=es, Swedish=se, Thai=th, |
| 184 | Turkish=tr, Ukrainian=ua, Welsh=cy. |
| 185 | </td></tr></table> |
| 186 | </i> |
| 187 | However, AWStats documentation is only provided in English.<br> |
| 188 | But, you may find small documentation for other languages made by contributors on |
| 189 | <a href="http://awstats.sourceforge.net/awstats_contrib.html#DOC">Documentation Contrib page</a>.<br> |
| 190 | <br> |
| 191 | If your language is not in this list, you can translate it yourself. For this, find what is your |
| 192 | 2 letter language code: <a href="http://www.oasis-open.org/cover/iso639a.html">here</a>.<br> |
| 193 | Once, you get it, for example "gl" for Galician, copy the file awstats-en.txt into awstats-<i>gl</i>.txt, |
| 194 | in langs directory and translate every sentence inside. You can do same for files inside |
| 195 | tooltips_f, tooltips_m and tooltips_w sub-directories. Then send your translated file(s) to eldy@users.sourceforge.net.<br> |
| 196 | <br> |
| 197 | |
| 198 | <a name="PHPNUKE"></a><br> |
| 199 | <b><u>FAQ-ABO250 : CAN AWSTATS BE INTEGRATED WITH PHP NUKE ?</u></b><br> |
| 200 | The only plugin I know to integrate AWStats inside PHPNuke is here: <a href="http://phpnuke.org/modules.php?name=News&file=article&sid=7041">PhpNuke addon for AWStats</a><br> |
| 201 | <br> |
| 202 | |
| 203 | <a name="ABOUTHISTORY"></a><br> |
| 204 | <b><u>FAQ-ABO300 : ABOUT AWSTATS HISTORY</u></b><br> |
| 205 | AWStats was initialy designed for my own use to track visitors on my personal web sites or other projects i worked on |
| 206 | (<a href="http://www.chiensderace.com" alt="Chiens De Race .com">www.chiensderace.com</a>, |
| 207 | <a href="http://www.chatsderace.com" alt="Chats De Race .com">www.chatsderace.com</a>, |
| 208 | <a href="http://www.lesbonnesannonces.com" alt="Petites annonces LesBonnesAnnonces.com">www.lesbonnesannonces.com</a>, |
| 209 | <a href="http://www.pourmaplanete.com" alt="Pour Ma Planete .com, le site de l'ecologie">www.pourmaplanete.com</a>, |
| 210 | <a href="http://www.dolibarr.org" alt="Dolibarr.org, the ERP/CRM to manage your business">www.dolibarr.org</a>, |
| 211 | <a href="http://www.nltechno.com" alt="NLTechno">www.nltechno.com</a>, |
| 212 | and <a href="http://www.destailleur.fr" alt="Site personnel Laurent Destailleur">www.destailleur.fr</a>)<br> |
| 213 | Then I decided to put it on sourceforge in year 2000. Then a lot of new versions were |
| 214 | developed to add enhancements until today. See changelog for full history of changes.<br> |
| 215 | <br> |
| 216 | |
| 217 | |
| 218 | <hr><br> |
| 219 | |
| 220 | |
| 221 | <a name="NOLOG"></a><br> |
| 222 | <b><u>FAQ-COM025 : HOW TO USE AWSTATS WITH NO SERVER LOG FILE</u></b><br> |
| 223 | <font class=CProblem>PROBLEM:</font><br> |
| 224 | I want to have AWStats statistics but i have no access to my server log file.<br> |
| 225 | <font class=CSolution>SOLUTION:</font><br> |
| 226 | Because AWStats is a log analyzer, if you don't have any way to read your server log file, |
| 227 | you have nothing to analyze and you should not be able to use AWStats. |
| 228 | However, this is a trick that you can use to have a log file be built. You must add |
| 229 | a tag to call a CGI script like pslogger into each of your web pages. This will |
| 230 | allow you to have an artificial log file that can be analyzed by AWStats.<br> |
| 231 | You can find a Perl version of CGI pslogger enhanced by AWStats author <a href="/files/pslogger.pl">here</a> |
| 232 | or a php version of CGI pslogger made by Florent CHANTRET <a href="/files/pslogger.phps">here</a>.<br> |
| 233 | <br> |
| 234 | |
| 235 | <a name="LIMITLOG"></a><br> |
| 236 | <b><u>FAQ-COM050 : WHAT IS THE LOG SIZE LIMIT AWSTATS CAN ANALYZE</u></b><br> |
| 237 | <font class=CProblem>PROBLEM:</font><br> |
| 238 | I know I must run AWStats update process frequently on new log files, this means thoose |
| 239 | files have a regular size, but for my first update, I want/need to run update process |
| 240 | on old log files that are very large. Is there a limit on log file size AWStats can analyze ?<br> |
| 241 | <font class=CSolution>SOLUTION:</font><br> |
| 242 | No. There is no limit in AWStats. This means you can use it on large log files (test were |
| 243 | made on 10GB log files).<br> |
| 244 | However your system (Operating System or Perl version) might have a limit. For example, you can |
| 245 | experience size limit errors on files larger than 2 or 4 GB. |
| 246 | If limit is Perl only, try to use a Perl version compiled with "large file" option.<br> |
| 247 | If you can't find it nor build it, you can try to use a LogFile parameter that looks like this |
| 248 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 249 | LogFile="cat /yourlogfilepath/yourlogfile |" |
| 250 | </td></tr></table> |
| 251 | instead of |
| 252 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 253 | LogFile="/yourlogfilepath/yourlogfile" |
| 254 | </td></tr></table> |
| 255 | <br> |
| 256 | |
| 257 | <a name="FTP"></a><br> |
| 258 | <b><u>FAQ-COM090 : SETUP FOR FTP SERVER LOG FILES (proftpd, vsftpd, ...)</u></b><br> |
| 259 | <font class=CProblem>PROBLEM:</font><br> |
| 260 | What do I have to do to use AWStats to analyze some FTP server log files ?<br> |
| 261 | <font class=CSolution>SOLUTION:</font><br> |
| 262 | AWStats can be used with some FTP server log files.<br> |
| 263 | <br> |
| 264 | <u>With ProFTPd</u>:<br> |
| 265 | <br> |
| 266 | 1- Setup your server log file format:<br> |
| 267 | <br> |
| 268 | Modify the proftpd.conf file to add the following two lines : |
| 269 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 270 | <!--<br>With ProFTPd 1.2.2:--> |
| 271 | LogFormat awstats "%t %h %u %m %f %s %b" # WARNING: You must use a tab char between % tags and not a space ! |
| 272 | <br>ExtendedLog /var/log/xferlog read,write awstats # WARNING: ExtendedLog directive might need to be placed inside a virtual host context if you use them. |
| 273 | <!--<br>With ProFTPd 1.2.6: |
| 274 | <i><br>LogFormat awstats ""${%F %H-%M-%S}t %h %u %m %F %s %b"</i> # WARNING: You must use a tab char between % tags and not a space ! |
| 275 | <i><br>ExtendedLog /var/log/xferlog read,write awstats</i> # WARNING: ExtendedLog directive might need to be placed inside a virtual host context if you use them. |
| 276 | --> |
| 277 | </td></tr></table> |
| 278 | <br> |
| 279 | Then turn off old format Transfer log: |
| 280 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 281 | TransferLog none # WARNING: TransferLog directive might need to be placed inside a virtual host context if you use them. |
| 282 | </td></tr></table> |
| 283 | <br> |
| 284 | To have the change effective, stop your server, remove old log file /var/log/xferlog and restart the server.<br> |
| 285 | Download a file by FTP and check that your new log file looks like this:<br> |
| 286 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 287 | [01/Jan/2001:21:49:57 +0200] ftp.server.com user RETR /home/fileiget.txt 226 1499 |
| 288 | </td></tr></table> |
| 289 | <br> |
| 290 | 2- Then setup AWStats to analyze the FTP log file:<br> |
| 291 | <br> |
| 292 | Copy config file "awstats.model.conf" to "awstats.ftp.conf".<br> |
| 293 | Modify this new config file: |
| 294 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 295 | LogFile="/var/log/xferlog" |
| 296 | <br>LogType=F |
| 297 | <br>LogFormat="%time1 %host %logname %method %url %code %bytesd" |
| 298 | <br>LogSeparator="\t" |
| 299 | <br>NotPageList="" |
| 300 | <br>LevelForBrowsersDetection=0 |
| 301 | <br>LevelForOSDetection=0 |
| 302 | <br>LevelForRefererAnalyze=0 |
| 303 | <br>LevelForRobotsDetection=0 |
| 304 | <br>LevelForWormsDetection=0 |
| 305 | <br>LevelForSearchEnginesDetection=0 |
| 306 | <br>ShowLinksOnUrl=0 |
| 307 | <br>ShowMenu=1 |
| 308 | <br>ShowSummary=UVHB |
| 309 | <br>ShowMonthStats=UVHB |
| 310 | <br>ShowDaysOfMonthStats=HB |
| 311 | <br>ShowDaysOfWeekStats=HB |
| 312 | <br>ShowHoursStats=HB |
| 313 | <br>ShowDomainsStats=HB |
| 314 | <br>ShowHostsStats=HBL |
| 315 | <br>ShowAuthenticatedUsers=HBL |
| 316 | <br>ShowRobotsStats=0 |
| 317 | <br>ShowEMailSenders=0 |
| 318 | <br>ShowEMailReceivers=0 |
| 319 | <br>ShowSessionsStats=1 |
| 320 | <br>ShowPagesStats=PBEX |
| 321 | <br>ShowFileTypesStats=HB |
| 322 | <br>ShowFileSizesStats=0 |
| 323 | <br>ShowBrowsersStats=0 |
| 324 | <br>ShowOSStats=0 |
| 325 | <br>ShowOriginStats=0 |
| 326 | <br>ShowKeyphrasesStats=0 |
| 327 | <br>ShowKeywordsStats=0 |
| 328 | <br>ShowMiscStats=0 |
| 329 | <br>ShowHTTPErrorsStats=0 |
| 330 | <br>ShowSMTPErrorsStats=0 |
| 331 | </td></tr></table> |
| 332 | <br> |
| 333 | Now you can use AWStats as usual (run the update process and read statistics).<br> |
| 334 | <br> |
| 335 | <br> |
| 336 | <u>With vsFTPd, or any FTP server that log with xferlog format</u>:<br> |
| 337 | <br> |
| 338 | 1- Check your server log file format:<br> |
| 339 | <br> |
| 340 | Take a look at your FTP server log file. You must have a format that match the following example to |
| 341 | use this FAQ :<br> |
| 342 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 343 | Wed Jan 01 19:29:35 2001 1 192.168.1.1 102 /home/file1.txt b _ o r username ftp 0 * c |
| 344 | </td></tr></table> |
| 345 | <br> |
| 346 | 2- Then setup AWStats to analyze the FTP log file:<br> |
| 347 | <br> |
| 348 | If your FTP log file format looks good, copy config file "awstats.model.conf" to "awstats.ftp.conf".<br> |
| 349 | Modify this new config file: |
| 350 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 351 | LogFile="/var/log/xferlog" |
| 352 | <br>LogType=F |
| 353 | <br>LogFormat="%time3 %other %host %bytesd %url %other %other %method %other %logname %other %code %other %other" |
| 354 | <br>LogSeparator="\s" |
| 355 | <br>NotPageList="" |
| 356 | <br>LevelForBrowsersDetection=0 |
| 357 | <br>LevelForOSDetection=0 |
| 358 | <br>LevelForRefererAnalyze=0 |
| 359 | <br>LevelForRobotsDetection=0 |
| 360 | <br>LevelForWormsDetection=0 |
| 361 | <br>LevelForSearchEnginesDetection=0 |
| 362 | <br>ShowLinksOnUrl=0 |
| 363 | <br>ShowMenu=1 |
| 364 | <br>ShowSummary=UVHB |
| 365 | <br>ShowMonthStats=UVHB |
| 366 | <br>ShowDaysOfMonthStats=HB |
| 367 | <br>ShowDaysOfWeekStats=HB |
| 368 | <br>ShowHoursStats=HB |
| 369 | <br>ShowDomainsStats=HB |
| 370 | <br>ShowHostsStats=HBL |
| 371 | <br>ShowAuthenticatedUsers=HBL |
| 372 | <br>ShowRobotsStats=0 |
| 373 | <br>ShowEMailSenders=0 |
| 374 | <br>ShowEMailReceivers=0 |
| 375 | <br>ShowSessionsStats=1 |
| 376 | <br>ShowPagesStats=PBEX |
| 377 | <br>ShowFileTypesStats=HB |
| 378 | <br>ShowFileSizesStats=0 |
| 379 | <br>ShowBrowsersStats=0 |
| 380 | <br>ShowOSStats=0 |
| 381 | <br>ShowOriginStats=0 |
| 382 | <br>ShowKeyphrasesStats=0 |
| 383 | <br>ShowKeywordsStats=0 |
| 384 | <br>ShowMiscStats=0 |
| 385 | <br>ShowHTTPErrorsStats=0 |
| 386 | <br>ShowSMTPErrorsStats=0 |
| 387 | </td></tr></table> |
| 388 | <br> |
| 389 | Now you can use AWStats as usual (run the update process and read statistics).<br> |
| 390 | <br> |
| 391 | |
| 392 | |
| 393 | <a name="MAIL"></a><br> |
| 394 | <b><u>FAQ-COM100 : SETUP FOR MAIL LOG FILES (Postfix, Sendmail, Qmail, MDaemon, Exchange...)</u></b><br> |
| 395 | <font class=CProblem>PROBLEM:</font><br> |
| 396 | What do I have to do to use AWStats to analyze my mail log files ?<br> |
| 397 | <font class=CSolution>SOLUTION:</font><br> |
| 398 | <br> |
| 399 | This tip works with AWStats 5.5 or higher.<br> |
| 400 | <br> |
| 401 | <u>For Postfix, Sendmail, QMail or MDaemon log files</u><br> |
| 402 | <br> |
| 403 | You must setup AWStats to use a mail log file preprocessor (<i>maillogconvert.pl</i> is provided |
| 404 | into AWStats <i>tools</i> directory, but you can use the one of your choice):<br> |
| 405 | For this, copy config <i>"awstats.model.conf"</i> file to <i>"awstats.mail.conf"</i>.<br> |
| 406 | Modify this new config file: |
| 407 | For standard Postfix, Sendmail, MDaemon and standard QMail logfiles, set<br> |
| 408 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 409 | LogFile="perl /path/to/maillogconvert.pl standard < /pathtomaillog/maillog |" |
| 410 | </td></tr></table> |
| 411 | If the logfiles are compressed, they can be processed this way<br> |
| 412 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 413 | LogFile="gzip -cd /var/log/maillog.0.gz | /path/to/maillogconvert.pl standard |"<br> |
| 414 | </td></tr></table> |
| 415 | And for VAdmin QMail logfiles (multi-host/virtualhost mail servers running vadmin software), set<br> |
| 416 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 417 | LogFile="perl /path/to/maillogconvert.pl vadmin < /pathtomaillog/maillog |"<br> |
| 418 | </td></tr></table> |
| 419 | <br> |
| 420 | Then, whatever is you mail server, you must also change: |
| 421 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 422 | LogType=M |
| 423 | <br>LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" |
| 424 | <br>LevelForBrowsersDetection=0 |
| 425 | <br>LevelForOSDetection=0 |
| 426 | <br>LevelForRefererAnalyze=0 |
| 427 | <br>LevelForRobotsDetection=0 |
| 428 | <br>LevelForWormsDetection=0 |
| 429 | <br>LevelForSearchEnginesDetection=0 |
| 430 | <br>LevelForFileTypesDetection=0 |
| 431 | <br>ShowMenu=1 |
| 432 | <br>ShowSummary=HB |
| 433 | <br>ShowMonthStats=HB |
| 434 | <br>ShowDaysOfMonthStats=HB |
| 435 | <br>ShowDaysOfWeekStats=HB |
| 436 | <br>ShowHoursStats=HB |
| 437 | <br>ShowDomainsStats=0 |
| 438 | <br>ShowHostsStats=HBL |
| 439 | <br>ShowAuthenticatedUsers=0 |
| 440 | <br>ShowRobotsStats=0 |
| 441 | <br>ShowEMailSenders=HBML |
| 442 | <br>ShowEMailReceivers=HBML |
| 443 | <br>ShowSessionsStats=0 |
| 444 | <br>ShowPagesStats=0 |
| 445 | <br>ShowFileTypesStats=0 |
| 446 | <br>ShowFileSizesStats=0 |
| 447 | <br>ShowBrowsersStats=0 |
| 448 | <br>ShowOSStats=0 |
| 449 | <br>ShowOriginStats=0 |
| 450 | <br>ShowKeyphrasesStats=0 |
| 451 | <br>ShowKeywordsStats=0 |
| 452 | <br>ShowMiscStats=0 |
| 453 | <br>ShowHTTPErrorsStats=0 |
| 454 | <br>ShowSMTPErrorsStats=1 |
| 455 | </td></tr></table> |
| 456 | Warning: For MDaemon mail server, you must use the new MDaemon log file that ends |
| 457 | with "-Statistics.log".<br> |
| 458 | <br> |
| 459 | Now you can use AWStats as usual (run the update process and read statistics).<br> |
| 460 | <br> |
| 461 | |
| 462 | <u>For Exchange log files</u><br> |
| 463 | <br> |
| 464 | Despite the high number of possible log format provided with Exchange, |
| 465 | none of them is built enough seriously to offer an interseting analyze (missing informations, |
| 466 | messy data, no id to join multiple records for same mail, etc...). |
| 467 | For this reason, an "exact" log analysis is a joke with Exchange log files. |
| 468 | However a little support is provided. In order to analyze Exchange traffic, you have to |
| 469 | enable "Message Tracking" (see article http://support.microsoft.com/default.aspx?scid=kb;EN-US;246856).<br> |
| 470 | Then copy config awstats.model.conf file to "awstats.mail.conf".<br> |
| 471 | Modify this new config file: |
| 472 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 473 | LogType=M |
| 474 | <br>LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" |
| 475 | <br>LevelForBrowsersDetection=0 |
| 476 | <br>LevelForOSDetection=0 |
| 477 | <br>LevelForRefererAnalyze=0 |
| 478 | <br>LevelForRobotsDetection=0 |
| 479 | <br>LevelForWormsDetection=0 |
| 480 | <br>LevelForSearchEnginesDetection=0 |
| 481 | <br>LevelForFileTypesDetection=0 |
| 482 | <br>ShowMenu=1 |
| 483 | <br>ShowSummary=HB |
| 484 | <br>ShowMonthStats=HB |
| 485 | <br>ShowDaysOfMonthStats=HB |
| 486 | <br>ShowDaysOfWeekStats=HB |
| 487 | <br>ShowHoursStats=HB |
| 488 | <br>ShowDomainsStats=0 |
| 489 | <br>ShowHostsStats=HBL |
| 490 | <br>ShowAuthenticatedUsers=0 |
| 491 | <br>ShowRobotsStats=0 |
| 492 | <br>ShowEMailSenders=HBML |
| 493 | <br>ShowEMailReceivers=HBML |
| 494 | <br>ShowSessionsStats=0 |
| 495 | <br>ShowPagesStats=0 |
| 496 | <br>ShowFileTypesStats=0 |
| 497 | <br>ShowFileSizesStats=0 |
| 498 | <br>ShowBrowsersStats=0 |
| 499 | <br>ShowOSStats=0 |
| 500 | <br>ShowOriginStats=0 |
| 501 | <br>ShowKeyphrasesStats=0 |
| 502 | <br>ShowKeywordsStats=0 |
| 503 | <br>ShowMiscStats=0 |
| 504 | <br>ShowHTTPErrorsStats=0 |
| 505 | <br>ShowSMTPErrorsStats=1 |
| 506 | </td></tr></table> |
| 507 | <br> |
| 508 | Also don't forget that with Exchange, informations in a log analyses can't be exact. |
| 509 | Do not send any questions or requests for using AWStats with Exchange, this is not a problem in |
| 510 | AWStats and we have no time to support non opened products.<br> |
| 511 | If you want to have complete and accurate information with Exchange, forget using AWStats or use a |
| 512 | more serious mail serveur (Postfix, Sendmail, QMail...)<br> |
| 513 | <br> |
| 514 | |
| 515 | <a name="MEDIASERVER"></a><br> |
| 516 | <b><u>FAQ-COM110 : SETUP FOR A MEDIA SERVER (REALMEDIA, WINDOWS MEDIA SERVER, DARWIN STREAMING SERVER)</u></b><br> |
| 517 | <font class=CProblem>PROBLEM:</font><br> |
| 518 | What do I have to do to use AWStats to analyze my Media Server log files.<br> |
| 519 | <font class=CSolution>SOLUTION:</font><br> |
| 520 | <br> |
| 521 | <u>For Realmedia</u><br> |
| 522 | <br> |
| 523 | Your log file will probably looks like this:<br> |
| 524 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 525 | 216.125.146.50 - - [16/Sep/2002:14:57:21 -0500] "GET cme/rhythmcity/rcitycaddy.rm?cloakport=8080,554,7070 RTSP/1.0" 200 6672 [Win95_4.0_6.0.9.374_play32_NS80_en-US_586] [80d280e1-c9ae-11d6-fa53-d52aaed98681] [UNKNOWN] 281712 141 3 0 0 494<br> |
| 526 | </td></tr></table> |
| 527 | <br> |
| 528 | Copy config awstats.model.conf file to "awstats.mediaserver.conf". |
| 529 | Modify this new config file: |
| 530 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 531 | LogFile="/pathtomediaserverlog/mediaserverlog" |
| 532 | <br>LogType=S |
| 533 | <br>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %uabracket %other %other %other %other %other %other %other %other" |
| 534 | <br>LogSeparator="\s+" |
| 535 | <br>ShowMenu=1 |
| 536 | <br>ShowSummary=UHB |
| 537 | <br>ShowMonthStats=UHB |
| 538 | <br>ShowDaysOfMonthStats=HB |
| 539 | <br>ShowDaysOfWeekStats=HB |
| 540 | <br>ShowHoursStats=HB |
| 541 | <br>ShowDomainsStats=HB |
| 542 | <br>ShowHostsStats=HBL |
| 543 | <br>ShowAuthenticatedUsers=0 |
| 544 | <br>ShowRobotsStats=0 |
| 545 | <br>ShowEMailSenders=0 |
| 546 | <br>ShowEMailReceivers=0 |
| 547 | <br>ShowSessionsStats=0 |
| 548 | <br>ShowPagesStats=PB |
| 549 | <br>ShowFileTypesStats=HB |
| 550 | <br>ShowFileSizesStats=0 |
| 551 | <br>ShowBrowsersStats=1 |
| 552 | <br>ShowOSStats=1 |
| 553 | <br>ShowOriginStats=H |
| 554 | <br>ShowKeyphrasesStats=0 |
| 555 | <br>ShowKeywordsStats=0 |
| 556 | <br>ShowMiscStats=0 |
| 557 | <br>ShowHTTPErrorsStats=1 |
| 558 | <br>ShowSMTPErrorsStats=0 |
| 559 | </td></tr></table> |
| 560 | <br> |
| 561 | Now you can use AWStats as usual (run the update process and read statistics).<br> |
| 562 | <br> |
| 563 | <br> |
| 564 | <u>For Windows Media Server / Darwin Streaming Server</u><br> |
| 565 | <br> |
| 566 | 1- If your Windows Media / Darwin streaming Server version allows it, setup your log format to write the following fields: |
| 567 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 568 | c-ip |
| 569 | <br>date |
| 570 | <br>time |
| 571 | <br>cs-uri-stem |
| 572 | <br>c-starttime |
| 573 | <br>x-duration |
| 574 | <br>c-rate |
| 575 | <br>c-status |
| 576 | <br>c-playerid |
| 577 | <br>c-playerversion |
| 578 | <br>c-playerlanguage |
| 579 | <br>cs(User-Agent) |
| 580 | <br>cs(Referer) |
| 581 | <br>c-hostexe |
| 582 | <br>c-hostexever |
| 583 | <br>c-os |
| 584 | <br>c-osversion |
| 585 | <br>c-cpu |
| 586 | <br>filelength |
| 587 | <br>filesize |
| 588 | <br>avgbandwidth |
| 589 | <br>protocol |
| 590 | <br>transport |
| 591 | <br>audiocodec |
| 592 | <br>videocodec |
| 593 | <br>channelURL |
| 594 | <br>sc-bytes |
| 595 | </td></tr></table> |
| 596 | <br> |
| 597 | To make the change effective, stop your server, remove old log files and restart the server.<br> |
| 598 | Listen to streaming files and check that your new log file looks like this:<br> |
| 599 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 600 | 80.223.91.37 2002-10-08 14:18:58 mmst://mydomain.com/mystream 0 106 1 200 {F4A826EE-FA46-480F-A49B-76786320FC6B} 8.0.0.4477 fi-FI - - wmplayer.exe 8.0.0.4477 Windows_2000 5.1.0.2600 Pentium 0 0 20702 mms TCP Windows_Media_Audio_9 - - 277721 |
| 601 | </td></tr></table> |
| 602 | <br> |
| 603 | <br> |
| 604 | If your Windows Media/Darwin Streaming Server version does not allow to define your log format:<br> |
| 605 | Just follow instructions in step 2 directly but use the log format string found in first lines |
| 606 | of your log files (Just after the "<i>#Fields:</i>" string) as value for AWStats LogFormat |
| 607 | parameter. For example, you could have a LogFormat defined like this:<br> |
| 608 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 609 | LogFormat="c-ip date time c-dns cs-uri-stem c-starttime x-duration c-rate |
| 610 | c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) |
| 611 | cs(Referer) c-hostexe c-hostexever c-os c-osversion c-cpu filelength |
| 612 | filesize avgbandwidth protocol transport audiocodec videocodec channelURL |
| 613 | sc-bytes c-bytes s-pkts-sent c-pkts-received c-pkts-lost-client |
| 614 | c-pkts-lost-net c-pkts-lost-cont-net c-resendreqs c-pkts-recovered-ECC |
| 615 | c-pkts-recovered-resent c-buffercount c-totalbuffertime c-quality s-ip s-dns |
| 616 | s-totalclients s-cpu-util" |
| 617 | </td></tr></table> |
| 618 | <br>This means you don't use the AWStats tags but AWStats can often also understand all the IIS and/or |
| 619 | Windows Media Server tags.<br> |
| 620 | |
| 621 | <br> |
| 622 | 2- Then setup AWStats to analyze your Media Server log:<br> |
| 623 | Copy config awstats.model.conf file to "awstats.mediaserver.conf".<br> |
| 624 | Modify this new config file: |
| 625 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 626 | LogFile="/pathtomediaserver/mediaserverlog" |
| 627 | <br>LogType=S |
| 628 | <br>LogFormat="c-ip date time cs-uri-stem c-starttime x-duration c-rate c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) cs(Referer) c-hostexe c-hostexever c-os c-osversion c-cpu filelength filesize avgbandwidth protocol transport audiocodec videocodec channelURL sc-bytes" |
| 629 | <br>DecodeUA=1 |
| 630 | <br>ShowMenu=1 |
| 631 | <br>ShowSummary=UHB |
| 632 | <br>ShowMonthStats=UHB |
| 633 | <br>ShowDaysOfMonthStats=HB |
| 634 | <br>ShowDaysOfWeekStats=HB |
| 635 | <br>ShowHoursStats=HB |
| 636 | <br>ShowDomainsStats=HB |
| 637 | <br>ShowHostsStats=HBL |
| 638 | <br>ShowAuthenticatedUsers=0 |
| 639 | <br>ShowRobotsStats=0 |
| 640 | <br>ShowEMailSenders=0 |
| 641 | <br>ShowEMailReceivers=0 |
| 642 | <br>ShowSessionsStats=0 |
| 643 | <br>ShowPagesStats=PB |
| 644 | <br>ShowFileTypesStats=HB |
| 645 | <br>ShowFileSizesStats=0 |
| 646 | <br>ShowBrowsersStats=1 |
| 647 | <br>ShowOSStats=1 |
| 648 | <br>ShowOriginStats=H |
| 649 | <br>ShowKeyphrasesStats=0 |
| 650 | <br>ShowKeywordsStats=0 |
| 651 | <br>ShowMiscStats=0 |
| 652 | <br>ShowHTTPErrorsStats=1 |
| 653 | <br>ShowSMTPErrorsStats=0 |
| 654 | </td></tr></table> |
| 655 | <br> |
| 656 | Now you can use AWStats as usual (run the update process and read statistics).<br> |
| 657 | <br> |
| 658 | |
| 659 | <a name="PERSONALIZEDLOG"></a><br> |
| 660 | <b><u>FAQ-COM115 : SETUP/EXAMPLES FOR LOGFORMAT PARAMETER</u></b><br> |
| 661 | <font class=CProblem>PROBLEM:</font><br> |
| 662 | Which value do I have to put in the LogFormat parameter to make AWStats working with my log |
| 663 | file format ?<br> |
| 664 | <font class=CSolution>SOLUTION:</font><br> |
| 665 | The AWStats config file gives you all the possible values for LogFormat parameter. |
| 666 | To help you, this is some common cases of log file format, and |
| 667 | the corresponding value of LogFormat you must use in your AWStats config file:<br> |
| 668 | <hr> |
| 669 | <u>If your log records are EXACTLY like this (<b>NCSA combined/XLF/ELF</b> log format):</u><br> |
| 670 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 1234 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"</i><br> |
| 671 | |
| 672 | You must use : <i>LogFormat=1</i><br> |
| 673 | This is same than: <i>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"</i><br> |
| 674 | |
| 675 | <hr> |
| 676 | <u>If your log records are EXACTLY like this (<b>NCSA combined with several virtualhostname</b> sharing same log file).</u><br> |
| 677 | <i>virtualserver1 62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 1234 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"</i><br> |
| 678 | You must use : <i>LogFormat="%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"</i><br> |
| 679 | <hr> |
| 680 | <u>If your log records are EXACTLY like this (<b>NCSA combined and mod_gzip format 1</b> with <b>Apache 1.x</b>):</u><br> |
| 681 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 3904 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" mod_gzip: 66pct.</i><br> |
| 682 | You must use : <i>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %other %gzipratio"</i><br> |
| 683 | |
| 684 | <hr> |
| 685 | <u>If your log records are EXACTLY like this (<b>NCSA combined and mod_gzip format 2</b> with <b>Apache 1.x</b>):</u><br> |
| 686 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 3904 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" mod_gzip: DECHUNK:OK In:11393 Out:3904:66pct.</i><br> |
| 687 | You must use : <i>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %other %other %gzipin %gzipout"</i><br> |
| 688 | <hr> |
| 689 | <u>If your log records are EXACTLY like this (<b>NCSA combined and mod_deflate</b> with <b>Apache 2</b>):</u><br> |
| 690 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 3904 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" (45)</i><br> |
| 691 | You must use : <i>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot %deflateratio"</i><br> |
| 692 | |
| 693 | <hr> |
| 694 | <u>If your log records are EXACTLY like this (<b>NCSA combined with 2 spaces between some fields</b> with <b>Zope</b>):</u><br> |
| 695 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 3904 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" (45)</i><br> |
| 696 | You must use : <i><br> |
| 697 | LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"<br> |
| 698 | LogSeparator=" *"<br> |
| 699 | </i> |
| 700 | <hr> |
| 701 | <u>If your log records are EXACTLY like this (<b>NCSA common CLF</b> log format):</u><br> |
| 702 | <i>62.161.78.73 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 1234</i><br> |
| 703 | You must use : <i>LogFormat=4</i><br> |
| 704 | Note: Browsers, OS's, Keywords and Referers features are not available with a such format.<br> |
| 705 | |
| 706 | <hr> |
| 707 | <u>If your log records are EXACTLY like this (With some <b>Squid</b> versions, after setting <i>emulate_http_log</i> to on):</u><br> |
| 708 | <i>200.135.30.181 - - [dd/mmm/yyyy:hh:mm:ss +0x00] "GET http://www.mydomain.com/page.html HTTP/1.0" 200 456 TCP_CLIENT_REFRESH_MISS:DIRECT</i><br> |
| 709 | You must use : <i>LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %other"</i><br> |
| 710 | <hr> |
| 711 | <u>If your log records are EXACTLY like this (Some old <b>IIS</b> W3C log format):</u><br> |
| 712 | <i>yyyy-mm-dd hh:mm:ss 62.161.78.73 - GET /page.html 200 1234 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.html</i><br> |
| 713 | You must use : <i>LogFormat=2</i><br> |
| 714 | |
| 715 | <hr> |
| 716 | <u>If your log records are EXACTLY like this (Some <b>IIS</b> W3C log format with some <b>.net</b> servers):</u><br> |
| 717 | <i>yyyy-mm-dd hh:mm:ss GET /page.html - 62.161.78.73 - Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.html 200 1234 HTTP/1.1</i><br> |
| 718 | You must use : <i>LogFormat=2 (or LogFormat="%time2 %method %url %logname %host %other %ua %referer %code %bytesd %other")</i><br> |
| 719 | <hr> |
| 720 | <u>If your log records are EXACTLY like this (Some <b>IIS 6+</b> W3C log format):</u><br> |
| 721 | <i>yyyy-mm-dd hh:mm:ss GET /page.html - 62.161.78.73 - Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.html 200 1234</i><br> |
| 722 | You must use : <i>LogFormat=2 (or LogFormat="date time cs-method cs-uri-stem cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes")</i><br> |
| 723 | |
| 724 | <hr> |
| 725 | <u>If your log records are EXACTLY like this (Some <b>ISA</b> W3C log format):</u><br> |
| 726 | <i>62.161.78.73, anonymous, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1), N, 1/1/2001, 0:00:16, W3ReverseProxy, HCSERV2, -, www.host.be, 192.168.141.101, 80, 266, 406, 10042, http, TCP, GET, http://192.168.141.101/, text/html, Inet, 200, 0x42330010, -, -</i><br> |
| 727 | You must use :<br> |
| 728 | <i>LogFile="sed -e 's/, /\t/g' "/yourlogpath/yourlogfile.log" |"</i><br> |
| 729 | <i>LogFormat=2</i><br> |
| 730 | <i>LogSeparator=" "</i><br> |
| 731 | <hr> |
| 732 | <u>If your log records are EXACTLY like this (With some <b>WebSite</b> versions):</u><br> |
| 733 | <i>yyyy-mm-dd hh:mm:ss 62.161.78.73 - 192.168.1.1 80 GET /page.html - 200 11205 0 0 HTTP/1.1 mydomain.com Mozilla/4.0+(compatible;+MSIE+5.5;+Windows+98) - http://www.from.com/from.html</i><br> |
| 734 | You must use : <i>LogFormat="%time2 %host %logname %other %other %method %url %other %code %bytesd %other %other %other %other %ua %other %referer"</i><br> |
| 735 | |
| 736 | <hr> |
| 737 | <u>If your log records are EXACTLY like this (<b>Webstar</b> native log format):</u><br> |
| 738 | <i>05/21/00 00:17:31 OK 200 212.242.30.6 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt) http://www.cover.dk/ "www.cover.dk" :Documentation:graphics:starninelogo.white.gif 1133</i><br> |
| 739 | You must use : <i>LogFormat=3</i><br> |
| 740 | <hr> |
| 741 | <u>If your log records are EXACTLY like this (With some <b>Lotus Notes/Domino</b> versions):</u><br> |
| 742 | <i>62.161.78.73 - Name Surname Service [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 1234 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"</i><br> |
| 743 | You must use : <i>LogFormat=6</i><br> |
| 744 | |
| 745 | <hr> |
| 746 | <u>If your log records are EXACTLY like this (<b>Lotus Notes/Domino 6.x</b> log format):</u><br> |
| 747 | <i>62.161.78.73 - "Name Surname" Service [dd/mmm/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" 200 1234 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"</i><br> |
| 748 | You must use : <i>LogFormat="%host %other %lognamequot %time1 %methodurl %code %bytesd %refererquot %uaquot"</i><br> |
| 749 | <hr> |
| 750 | <u>If your log records are EXACTLY like this (With <b>Oracle9iAS</b>):</u><br> |
| 751 | <i>62.161.78.73 - [dd/mmm/yyyy:hh:mm:ss +0x00] GET /page.html HTTP/1.1 200 1234 - "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"</i><br> |
| 752 | Where separators are "tab" characters or several "spaces", |
| 753 | You must use : <i>LogFormat="%host %logname %time1 %method %url %other %code %bytesd %referer %uaquot"</i> |
| 754 | and <i>LogSeparator="\s+"</i><br> |
| 755 | |
| 756 | <hr> |
| 757 | <u>If you use a FTP server like <b>ProFTPd</b>:</u><br> |
| 758 | See <a href="#FTP">FAQ-COM090</a>.<br> |
| 759 | <hr> |
| 760 | <u>If you want to analyze a mail log file (<b>Postfix</b>, <b>Sendmail</b>, <b>QMail</b>, <b>MDaemon</b>, <b>Exchange</b>):</u><br> |
| 761 | See <a href="#MAIL">FAQ-COM100</a>.<br> |
| 762 | |
| 763 | <hr> |
| 764 | <u>If you use a Media Server (<b>Realmedia</b>, <b>Windows Media Server</b>):</u><br> |
| 765 | See <a href="#MEDIASERVER">FAQ-COM110</a>.<br> |
| 766 | <hr> |
| 767 | <u>If your log records are EXACTLY like this (With some providers):</u><br> |
| 768 | <i>62.161.78.73 - - [dd/Month/yyyy:hh:mm:ss +0x00] "GET /page.html HTTP/1.1" "-" 200 1234</i><br> |
| 769 | You must use : <i>LogFormat="%host %other %logname %time1 %methodurl %other %code %bytesd"</i><br> |
| 770 | Note: Browsers, OS's, Keywords and Referers features are not available with a such format.<br> |
| 771 | |
| 772 | <hr> |
| 773 | <u>There are a lot of other possible log formats.</u><br> |
| 774 | You must use a personalized log format LogFormat ="..." as described in config file to |
| 775 | support other various log formats.<br> |
| 776 | <br><br> |
| 777 | |
| 778 | <a name="ROTATE"></a><br> |
| 779 | <b><u>FAQ-COM120 : HOW TO ROTATE MY LOGS WITHOUT LOSING DATA</u></b><br> |
| 780 | <font class=CProblem>PROBLEM:</font><br> |
| 781 | I want to archive/rotate my logs using my server system (for example logrotate) options or a third |
| 782 | software (rotatelog, cronolog) but I don't want to lose any visits information during the rotate |
| 783 | process.<br> |
| 784 | <font class=CSolution>SOLUTION:</font><br> |
| 785 | <li> If your config file is setup with a <a href="awstats_config.html#LogFile">LogFile</a> parameter |
| 786 | that point to your current running log file (required if you want to use the |
| 787 | <a href="awstats_config.html#AllowToUpdateStatsFromBrowser">AllowToUpdateStatsFromBrowser</a> |
| 788 | option to have "real-time" statistics), to avoid losing too much records during the rotate |
| 789 | process, you must run the AWStats update JUST BEFORE the rotate process is done.<br> |
| 790 | The best way to do that on 'Linux like' OS is to use the linux built-in logrotate feature. You must |
| 791 | edit the logrotate config file used for your web server log file (usually stored in /etc/logrotate.d |
| 792 | directory) by adding the AWStats update process as a preprocessor command, like this example (bold |
| 793 | lines are lines to add for having a prerotate process):<br> |
| 794 | <table width="95%" border=1 cellpadding=0 cellspacing=0 bgcolor=#F4F4F4 class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 795 | /usr/local/apache/logs/*log<br> |
| 796 | {<br> |
| 797 | notifempty<br> |
| 798 | daily<br> |
| 799 | rotate 7<br> |
| 800 | compress<br> |
| 801 | <b> |
| 802 | sharedscripts<br> |
| 803 | prerotate<br> |
| 804 | /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=mydomainconfig<br> |
| 805 | endscript<br> |
| 806 | </b> |
| 807 | postrotate<br> |
| 808 | /usr/bin/killall -HUP httpd<br> |
| 809 | endscript<br> |
| 810 | }<br> |
| 811 | </td></tr></table> |
| 812 | <br> |
| 813 | If using a such solution, this is sequential steps that happens:<br> |
| 814 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ> |
| 815 | <tr class=CFAQ><td class=CFAQ><b>Step</b></td><td class=CFAQ><b>Description</b></td><td class=CFAQ><b>Step name</b></td><td class=CFAQ><b>Date/Time example</b></td></tr> |
| 816 | <tr class=CFAQ valign=top><td class=CFAQ>A</td><td class=CFAQ>logrotate is started (by cron)</td><td class=CFAQ>Start of logrotate</td><td class=CFAQ>04:02:00</td></tr> |
| 817 | <tr valign=top><td class=CFAQ>B</td><td class=CFAQ><ul>awstats -update is launched by logrotate</td><td class=CFAQ>Start of awstats</td><td class=CFAQ>04:02:01</td><tr> |
| 818 | <tr valign=top><td class=CFAQ>C</td><td class=CFAQ><ul><ul>awstats start to read the log file <i>file.log</i></td><td class=CFAQ> </td><td class=CFAQ>04:02:02</td><tr> |
| 819 | <tr valign=top><td class=CFAQ>D</td><td class=CFAQ><ul>awstats has reached the end of log file so now it starts to save its database on disk.</td><td class=CFAQ> </td><td class=CFAQ>04:05:00</td><tr> |
| 820 | <tr valign=top><td class=CFAQ>E</td><td class=CFAQ><ul>awstats has finished to save its new database, so it stops</td><td class=CFAQ>End of awstats</td><td class=CFAQ>04:06:00</td><tr> |
| 821 | <tr valign=top><td class=CFAQ>F</td><td class=CFAQ>logrotate moves old log file <i>file.log</i> to a new name <i>file.log.sav</i>. Apache now logs in this file <i>file.log.sav</i> since log file handle has not been changed (only log file name has been renamed).</td><td class=CFAQ>Log move</td><td class=CFAQ>04:06:01</td><tr> |
| 822 | <tr valign=top><td class=CFAQ>G</td><td class=CFAQ>logrotate sends the -HUP or -USR1 signal to Apache.<br>With -HUP, Apache immediatly kills all its child process/thread, close log file <i>file.log.sav</i>, and reopen file <i>file.log</i>. So now, ALL hits are written to new file.<br>With -USR1, Apache only ask its child process/thread to stop only when HTTP request will be completely served. However it closes immediatly log file <i>file.log.sav</i>, and reopen file <i>file.log</i>. So only NEW hits are written to new log file. HTTP requests that are still running will write in old one.</td><td class=CFAQ>Apache restart</td><td class=CFAQ>04:06:02</td><tr> |
| 823 | <tr valign=top><td class=CFAQ>H</td><td class=CFAQ>logrotate starts compress the old log file <i>file.log.sav</i> into <i>file.log.gz</i></td><td class=CFAQ>Start compress</td><td class=CFAQ>04:06:03</td><tr> |
| 824 | <tr valign=top><td class=CFAQ>I</td><td class=CFAQ><ul>If some apache threads/processes are still running (because the kill sent was -USR1, so child process are waiting end of request before to stop), then those threads/processes are still writing to <i>file.log.sav</i>.<br>If kill -HUP was used, all process are already restarted so all writes in new <i>file.log</i>.</td><td class=CFAQ> </td><td class=CFAQ> </td><tr> |
| 825 | <tr valign=top><td class=CFAQ>J</td><td class=CFAQ>logrotate has finished to compress log file into <i>file.log.gz</i>. File <i>file.log.sav</i> is deleted.</td><td class=CFAQ>End of compress<br>End of logrotate</td><td class=CFAQ>04:07:03</td><tr> |
| 826 | <tr valign=top><td class=CFAQ>K</td><td class=CFAQ><ul>If signal was -USR1, some old childs can still run (when serving a very long request for example). So the log writing, still done in same file handle are going to a file that has been removed. So log writing are lost nowhere (this is only if -USR1 was used and if request was very long).</td><td class=CFAQ> </td><td class=CFAQ> </td><tr> |
| 827 | </table> |
| 828 | <br> |
| 829 | The advantage of this solution is that it is a very common way of working, used by a lot of |
| 830 | products, and easy to setup. You will notice that you can "lose" some hits:<br> |
| 831 | If you use the -HUP signal, you will only lose all hits that were written during D and E. |
| 832 | Note that you will also break all requests still running at G. In the example, it's a |
| 833 | 1 minute lost (for small or medium web sites, it will be less than few seconds), so this |
| 834 | give you an error lower than 0.07% (less for small web sites). This is not significant, |
| 835 | above all for a "statistics" progam.<br> |
| 836 | If you use the -USR1 signal, you will not kill any request. But you will lose all hits that |
| 837 | were wrote during D and E (like with -HUP) but also all hits that are still running after H |
| 838 | (all very long request that requires several minutes to be served). If hit ends during I, it is |
| 839 | wrote in a log file already analyzed, if hit ends at K, it is wrote nowhere. In the example, |
| 840 | it's also a 0.07% error plus error for other not visible hits that were finished during I or K, |
| 841 | but number of such hits should be very low since only hits that started before G and not |
| 842 | finished after H are concerned. In most cases a hits needs only few milliseconds to be served |
| 843 | so lost hits could be ignored.<br> |
| 844 | <br> |
| 845 | Note also that if you have x logrotate config files, with each of them a postrotate with a |
| 846 | kill -HUP, you send a kill x times to your server process. So try to include several log files |
| 847 | in same logrotate config file. You can have several awstats update command in the same |
| 848 | prerotate section and you will send the -HUP only once, after all updates are finished. |
| 849 | However, doing this, you will have a lap time between D and F (were some hits are lost) that will |
| 850 | be higher.<br> |
| 851 | <br> |
| 852 | |
| 853 | <li> Another common way of working is to choose to run the AWStats update process only once the log file has been |
| 854 | archived.<br> |
| 855 | This is required for example if you use the <a href="http://cronolog.org" target=awstatsbis>cronolog</a> |
| 856 | or rotatelog tools to rotate your log files. For example, Apache users can setup their Apache |
| 857 | httpd config file to write log file through a pipe to cronolog or rotatelog using |
| 858 | Apache <i>CustomLog</i> directive:<br> |
| 859 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 860 | CustomLog "|/usr/sbin/cronolog [cronolog_options] /var/logs/access.%Y%m%d.log" combined |
| 861 | </td></tr></table> |
| 862 | If you use a such feature, you can't trigger AWStats update process to be ran just BEFORE the |
| 863 | rotate is done, so you must run it AFTER the rotate process, so on the archived log file.<br> |
| 864 | To setup awstats to always point to last archive log file, you can use the 'tags' available for |
| 865 | <a href="awstats_config.html#LogFile">LogFile</a>.<br> |
| 866 | The problem with that is that your data are refreshed only after a rotate has done. However, |
| 867 | you will miss absolutely nothing (no hits) and your server processes are never killed.<br> |
| 868 | <br> |
| 869 | <li> So, if you really want to not lose absolutely no hit and want to have updates more |
| 870 | frequently than the rotate frequency, the best way is still an hybrid solution (i am not sure |
| 871 | that it worth the pain, and remember that statistics are only statistics):<br> |
| 872 | You run the awstats update process from you crontab frequently, every hour for example, and half and hour |
| 873 | before the rotate has done. See next FAQ to know how to setup a scheduled job.<br> |
| 874 | Then, once the rotate has been done (by the logrotate or by a piped cronolog log file), and |
| 875 | before the next scheduled awstats update process start, |
| 876 | you run another update process on the archived log file using the -logfile option to force |
| 877 | update on the archived log file and not the current log file defined in awstats |
| 878 | config file. This will allow you to update the half hour missing, |
| 879 | until the log rotate (AWStats will find the new lines). However don't forget that this |
| 880 | particular update MUST be finished before the next croned update.<br> |
| 881 | <br> |
| 882 | |
| 883 | <a name="CRONTAB"></a><br> |
| 884 | <b><u>FAQ-COM130 : HOW TO RUN AWSTATS UPDATE PROCESS FREQUENTLY</u></b><br> |
| 885 | <font class=CProblem>PROBLEM:</font><br> |
| 886 | AWStats must be ran frequently to update statistics. How can I do this ?<br> |
| 887 | <font class=CSolution>SOLUTION:</font><br> |
| 888 | A good way of working is to run the AWStats update process as a preprocessor of your log rotate |
| 889 | process. See previous FAQ (<a href="#ROTATE">FAQ-COM120</a>) for this.<br> |
| 890 | But you can also run AWStats update process regularly by a scheduler:<br> |
| 891 | <br> |
| 892 | <u>With Windows</u>, you can use the internal task scheduler. |
| 893 | The use of this tool is not an AWStats related problem, so please take a look at your Windows manual. |
| 894 | Warning, if you use <i>"awstats.pl -config=mysite -update"</i> in your scheduled task, you might |
| 895 | experience problem of failing task. Try this instead<br> |
| 896 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 897 | "C:\WINNT\system32\CMD.EXE /C C:\[awstats_path]\awstats.pl -config=mysite -update" |
| 898 | </td></tr></table> |
| 899 | or<br> |
| 900 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 901 | "C:\[perl_path]\perl.exe C:\[awstats_path]\awstats.pl -config=mysite -update" |
| 902 | </td></tr></table> |
| 903 | A lot of other open source schedulers are often better (otherwise there is also good sharewares or freewares).<br> |
| 904 | <br> |
| 905 | <u>With unix-like operating systems</u>, you can use the "<b>crontab</b>".<br> |
| 906 | This is examples of lines you can add in the cron file (see your unix reference manual for cron) :<br> |
| 907 | To run update every day at 03:50, use :<br> |
| 908 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 909 | 50 3 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -config=mysite -update >/dev/null<br> |
| 910 | </td></tr></table> |
| 911 | To run update every hour, use :<br> |
| 912 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 913 | 0 * * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -config=mysite -update >/dev/null<br> |
| 914 | </td></tr></table> |
| 915 | <br> |
| 916 | |
| 917 | <a name="EXCLUDEHOSTS"></a><br> |
| 918 | <b><u>FAQ-COM140 : HOW CAN I EXCLUDE MY IP ADDRESS (OR WHOLE SUBNET MASK) FROM STATS ?</u></b><br> |
| 919 | <font class=CProblem>PROBLEM:</font><br> |
| 920 | I don't want to see my own IP address in the stats or I want to exclude counting visits from a whole subnet.<br> |
| 921 | <font class=CSolution>SOLUTION:</font><br> |
| 922 | You must edit the config file to change the <a href="awstats_config.html#SkipHosts">SkipHosts</a> parameter.<br> |
| 923 | For example, to exclude:<br> |
| 924 | <li> your own IP address 123.123.123.123, use <a href="awstats_config.html#SkipHosts">SkipHosts</a>="123.123.123.123"<br> |
| 925 | <li> the whole subnet 123.123.123.xxx, use <a href="awstats_config.html#SkipHosts">SkipHosts</a>="REGEX[^123\.123\.123\.]"<br> |
| 926 | <li> all sub hosts xxx.myintranet.com, use <a href="awstats_config.html#SkipHosts">SkipHosts</a>="REGEX[\.myintranet\.com$]" (This one works only if DNS lookup is already done in your |
| 927 | log file).<br> |
| 928 | <br> |
| 929 | |
| 930 | <a name="SCREENSIZE"></a><br> |
| 931 | <b><u>FAQ-COM142 : HOW TO GET THE SCREEN SIZE AND BROWSER CAPABILITIES REPORT WORKING ?</u></b><br> |
| 932 | <font class=CProblem>PROBLEM:</font><br> |
| 933 | I see in the AWStats features list that it can report the screen size used by visitors and other browsers' informations, |
| 934 | (like if browser support Flash, Java, Javascript, PDF, MAcromedia, Audio plugins, etc...). How can I do that ?<br> |
| 935 | <font class=CSolution>SOLUTION:</font><br> |
| 936 | AWStats is a log analyzer, so to report the screen size of your visitor, we need to have information inside the log file itself. |
| 937 | For this, the only way to do that, is to add some HTML tags inside some of your pages (the home page is enough to |
| 938 | get use ratios). This tag will add call to a javascript that ask your browser to get a virtual URL that |
| 939 | includes, in its parameters, the screen size resolution and all other informations about browser capabilities (Flash, Java, Javascript, PDF, Macromedia, Audio plgins...).<br> |
| 940 | <br> |
| 941 | This is the code you must add (at bottom of your home page for example) :<br> |
| 942 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 943 | <script language="javascript" type="text/javascript" src="/js/awstats_misc_tracker.js" ></script><br> |
| 944 | <noscript><img src="/js/awstats_misc_tracker.js?nojs=y" height="0" width="0" border="0" style="display: none"></noscript><br> |
| 945 | </td></tr></table> |
| 946 | <br> |
| 947 | Note that you must also place the <i>awstats_misc_tracker.js</i> script (provided in /js directory with AWStats) inside |
| 948 | a js directory stored in your web root.<br> |
| 949 | Once this is done, load your home page with your browser and go to check that inside your log file |
| 950 | if you can see a line that looks like that:<br> |
| 951 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 952 | 123.123.123.123 - - [24/Apr/2005:16:09:38 +0200] "GET /js/awstats_misc_tracker.js?screen=800x600&win=724x517&...&sid=awssession_id123 HTTP/1.1" 200 6237 "http://therefererwebsite.com/index.php" "Mozilla/5.0 (Linux) Gecko/20050414 Firefox/1.0.3" |
| 953 | </td></tr></table> |
| 954 | <br> |
| 955 | If yes, you can then run the AWStats update process. Screen sizes information will be analyzed. All you have to do |
| 956 | now is to edit your config file to tell AWStats to add the report on html output. For this, change |
| 957 | the <a href="awstats_config.html#Show">ShowMiscStats</a> parameter.<br> |
| 958 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 959 | ShowMiscStats=anjdfrqwp |
| 960 | </td></tr></table> |
| 961 | <br> |
| 962 | Note: If you change directory where <i>awstats_misc_tracker.js</i> is stored (somewhere else than the /js directory), |
| 963 | you must modify, according to your change:<br> |
| 964 | - the html tags added<br> |
| 965 | - the line: <i>var awstatsmisctrackerurl="/js/awstats_misc_tracker.js";</i> inside the awstats_misc_tracker.js script<br> |
| 966 | - the parameter <a href="awstats_config.html#MiscTrackerUrl">MiscTrackerUrl</a> inside AWStats configuration file.<br> |
| 967 | <br> |
| 968 | |
| 969 | <a name="EXTRA"></a><br> |
| 970 | <b><u>FAQ-COM145 : HOW TO USE THE EXTRA SECTIONS FEATURES ?</u></b><br> |
| 971 | <font class=CProblem>PROBLEM:</font><br> |
| 972 | I want to build personalized reports not provided in default AWStats reports. How can I setup |
| 973 | the Extra Sections parameters in my AWStats config file to do so ?<br> |
| 974 | <font class=CSolution>SOLUTION:</font><br> |
| 975 | Take a look at the <a href="awstats_extra.html">Using AWStats Extra Sections features</a><br> |
| 976 | <br> |
| 977 | |
| 978 | <a name="BENCHMARK"></a><br> |
| 979 | <b><u>FAQ-COM150 : BENCHMARK / FREQUENCY TO LAUNCH AWSTATS TO UPDATE STATISTICS</u></b><br> |
| 980 | <font class=CProblem>PROBLEM:</font><br> |
| 981 | What is AWStats speed ?<br> |
| 982 | What is the frequency to launch AWStats process to update my statistics ?<br> |
| 983 | <font class=CSolution>SOLUTION:</font><br> |
| 984 | All benchmarks information and advice on frequency for update process are related into |
| 985 | the <a href="awstats_benchmark.html">Benchmark page</a>.<br> |
| 986 | <br> |
| 987 | |
| 988 | <a name="DNS"></a><br> |
| 989 | <b><u>FAQ-COM200 : HOW REVERSE DNS LOOKUP WORKS, UNRESOLVED IP ADDRESSES</u></b><br> |
| 990 | <font class=CProblem>PROBLEM:</font><br> |
| 991 | The reported page AWStats shows me has no hostnames, only IP addresses, countries reported are all "unknown".<br> |
| 992 | <font class=CSolution>SOLUTION:</font><br> |
| 993 | When AWStats find an IP address in your log file, it tries a reverse DNS lookup to find |
| 994 | the hostname and domain if the <a href="awstats_config.html#DNSLookup">DNSLookup</a> parameter, in your AWStats config file, is <a href="awstats_config.html#DNSLookup">DNSLookup</a>=1 |
| 995 | (Default value). So, first, check if you have the good value. The <a href="awstats_config.html#DNSLookup">DNSLookup</a>=0 |
| 996 | must be used only if your log file contains already resolved IP address. For example, |
| 997 | when you set up Apache with the <i>HostNameLookups=on</i> directive. When you |
| 998 | ask your web server to make itself the reverse DNS lookup to log hostname instead |
| 999 | of IP address, you will still find some IP addresses in your log file because |
| 1000 | the reverse DNS lookup is not always possible. But if your web server fails in |
| 1001 | it, AWStats will also fails (All reverse DNS lookups use the same system API). |
| 1002 | So to avoid AWStats to make an already done lookup (with success or not), you |
| 1003 | can set <a href="awstats_config.html#DNSLookup">DNSLookup</a>=0 in AWStats config file. |
| 1004 | If you prefer, you can make the reverse DNS lookup on a log file before running |
| 1005 | your log analyzer (If you only need to convert a logfile with IP Addresses into a |
| 1006 | logfile with resolved hostnames). You can use for this <a href="awstats_tools.html#logresolvemerge">logresolvemerge</a> tool |
| 1007 | provided with AWStats distribution (This tools is an improved version of <i>logresolve</i> provided with Apache).<br> |
| 1008 | <br> |
| 1009 | |
| 1010 | <a name="DIFFERENT_RESULTS"></a><br> |
| 1011 | <b><u>FAQ-COM250 : DIFFERENT RESULTS THAN OTHER ANALYZER</u></b><br> |
| 1012 | <font class=CProblem>PROBLEM:</font><br> |
| 1013 | I also use Webalizer, Analog (or another log analyzer) and it doesn't report the same results than AWStats. Why ?<br> |
| 1014 | <font class=CSolution>SOLUTION:</font><br> |
| 1015 | If you compare AWStats results with an other log file analyzer, you will found some differences, |
| 1016 | sometimes very important. In fact, all analyzer (even AWStats) make "over reporting" because of the |
| 1017 | problem of proxy-servers and robots. However AWStats is one of the most accurate and its "over reporting" |
| 1018 | is very low where all other analyzers, even the most famous, have a VERY HIGH error rate (10% to 200% more than reality !).<br> |
| 1019 | <b>This is the most important reasons why you can find important differences:</b><br> |
| 1020 | <li> Some dynamic pages generated by CGI programs are not counted by some analyzer (ie Webalizer) like |
| 1021 | a "Page" (but only like a "Hit") if CGI prog does not end with a defined extension (.cgi, ...), so they are not included |
| 1022 | correctly in their statistics. AWStats use on oposite policy, assuming a file is a page except if |
| 1023 | type is in a list (See <a href="awstats_config.html#NotPageList">NotPageList</a> parameter). Error rate |
| 1024 | with a such policy is lower.<br> |
| 1025 | <li> AWStats is able to detect robots visits. Most analyzers think robots visits are human visitors. |
| 1026 | This error make them to report more visits and visitors than reality. |
| 1027 | When AWStats reports a "1 visitor", it means "1 human visitor" (even if it's not posible to detect |
| 1028 | all robots, most of them are detected). "Robots visitors" are reported separately in the "Robots/Spiders visitors" chart.<br> |
| 1029 | <li> Some log analyzers use the "Hits" to count visitors. This is a very bad way of working : |
| 1030 | Some visitors use a lot of proxy servers to surf (ie: AOL users), this means it's possible that several |
| 1031 | hosts (with several IP addresses) are used to reach your site for only one visitor (ie: one proxy server download |
| 1032 | the page and 2 other servers download all images). Because of this, if stats of unique visitors are made on "hits", |
| 1033 | 3 users are reported but it's wrong. So AWStats considers only HTML "Pages" to count unique visitors. |
| 1034 | This decrease the error, not completely, because it's always possible that a proxy server download one HTML frame and |
| 1035 | another one download another frame, but this make the over-reporting of unique visitors less important.<br> |
| 1036 | <li> Another important reason to have difference is that an error log files is not always completely sorted |
| 1037 | but only "nearly" sorted because of cache and writing log engines used by server. Nearly all log |
| 1038 | analyzers (commercial and not) assumes that log file is "exactly" sorted by hit date to calculate |
| 1039 | visits, entry and exit pages. But there is nothing that guaranties this and some log files |
| 1040 | are only "nearly" sorted, above all log files on highly loaded servers. |
| 1041 | AWStats has an advanced parsing algorithm that is able to count |
| 1042 | correctly visits, entry and exit pages even if log file is only "nearly" sorted.<br> |
| 1043 | <li> Then, there is internal bugs in log analyzers that make reports wrong. |
| 1044 | For example, a lot of users have reported that Webalizer "doubles" the number of visits or visitors |
| 1045 | in some circumstances.<br> |
| 1046 | <b>There is also other reasons, however those points explains only small differences:</b><br> |
| 1047 | <li> To differenciate new visits of a same visitor, log analyers uses a visit time-out. If value differs, |
| 1048 | then results differ (on visit count and entry and exit pages). |
| 1049 | A such time-out is a fixed value (For example 60 minutes) meaning if a visitor make a hit |
| 1050 | 59 minutes after downloading the previous page, it's the same visits, if he make it 61 minutes after, it's a new visit. |
| 1051 | Of course, there is no realy difference between 59 and 61, but couting visits without |
| 1052 | time-out is not possible. And because the most important is to have a time-out (and not |
| 1053 | really it's value), AWStats time-out is not an "exact" value but is "around" 60 minutes. |
| 1054 | This allows AWStats to have better speed processing time, so you also might experience |
| 1055 | little differences, in visit count, between AWStats and another log analyzer even if |
| 1056 | their time-out are both defined to same value (because AWStats time-out is not exactly |
| 1057 | but nearly value defined).<br> |
| 1058 | <li> There is also differences in log analyzers databases and algorithms that make details of results less or more accurate:<br> |
| 1059 | AWStats has a larger browsers, os', search engines and robots database, so reports concerning this are more accurate.<br> |
| 1060 | AWStats has url syntax rules to find keywords or keyphrases used to find your site, but AWStats has also |
| 1061 | an algorithm to detect keywords of unknown search engines with unknown url syntax rule.<br> |
| 1062 | AWStats does not count twice (by default) redirects made by rewrite rules that makes two hits into |
| 1063 | log files but that are only one page "viewed".<br> |
| 1064 | Etc...<br> |
| 1065 | <br> |
| 1066 | If you want to check how serious your log analyzer is, try to parse the following log file. |
| 1067 | It's a very common log file but results will show you how bad most log analyzers are (above |
| 1068 | all commercial products):<br> |
| 1069 | <table width="95%" border=1 cellpadding=0 cellspacing=0><tr class=CFAQ><td class=CFAQ> |
| 1070 | <i><font style="font: 8px verdana,arial,helvetica"><pre> |
| 1071 | # This is a sample of log file that contains a lot of various data we can find |
| 1072 | # in a log file. Great sample to test reliability and accuracy of any log |
| 1073 | # analyzer. |
| 1074 | # ---------------------------------------------------------------------------- |
| 1075 | # This sample log file contains 10 differents IPs that are : |
| 1076 | # Viewed traffic: 5 different true human visitors making 6 human visits |
| 1077 | # 21 hits on pages and 15 hits not pages (36 hits) |
| 1078 | # Not viewed traffic: 13 pages, 15 hits |
| 1079 | # ---------------------------------------------------------------------------- |
| 1080 | # 80.8.55.1 2 visits (start at 00:00:00 and at 12:00:00 with both entry page on /) |
| 1081 | # And 2 hits to add favourites but first is non root hit with error meaning it's same "add" |
| 1082 | # 80.8.55.2 Not a visit, only an image included into a page of an other site |
| 1083 | # 80.8.55.3 1 visit (and add home page to favourites) |
| 1084 | # 80.8.55.4 same visitor than 80.8.55.3 using aol proxy |
| 1085 | # 80.8.55.5 Not a visit (but bot indexing) |
| 1086 | # 80.8.55.6 1 visit (authenticated visitor) |
| 1087 | # 80.8.55.7 1 visit (authenticated visitor with space in name) |
| 1088 | # 80.8.55.8 Not a visit (try but failed twice with 404 and 405 error) |
| 1089 | # 80.8.55.9 Not a visit (but a worm attack) |
| 1090 | # 80.8.55.10 1 visit that come from a web page that is not a search engine |
| 1091 | |
| 1092 | 80.8.55.1 - - [01/Jan/2001:00:00:10 +0100] "GET /page1.html HTTP/1.0" 200 7009 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1093 | 80.8.55.1 - - [01/Jan/2001:00:00:00 +0100] "GET / HTTP/1.0" 200 7009 "http://www.sitereferer/cgi-bin/search.pl?q=a" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1094 | 80.8.55.1 - - [01/Jan/2001:00:00:20 +0100] "GET /page2.cgi HTTP/1.0" 200 7009 "http://localhost/page1.html" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1095 | 80.8.55.1 - - [01/Jan/2001:00:00:25 +0100] "GET /page3 HTTP/1.0" 200 7009 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1096 | 80.8.55.1 - - [01/Jan/2001:00:00:30 +0100] "GET /image.gif HTTP/1.0" 200 7009 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1097 | 80.8.55.1 - - [01/Jan/2001:00:00:35 +0100] "GET /image2.png HTTP/1.0" 200 7009 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1098 | 80.8.55.1 - - [01/Jan/2001:00:00:40 +0100] "GET /dir/favicon.ico HTTP/1.0" 404 299 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1099 | 80.8.55.1 - - [01/Jan/2001:00:00:40 +0100] "GET /favicon.ico HTTP/1.0" 200 299 "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1100 | |
| 1101 | 80.8.55.1 - - [01/Jan/2001:12:00:00 +0100] "GET / HTTP/1.0" 200 7009 "http://WWW.SiteRefereR:80/cgi-bin/azerty.pl?q=a" "Mozilla/4.7 [fr] (Win95; I)" |
| 1102 | 80.8.55.1 - - [01/Jan/2001:12:00:10 +0100] "GET /page1.html HTTP/1.0" 200 7009 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1103 | 80.8.55.1 - - [01/Jan/2001:12:00:20 +0100] "GET /page2.cgi HTTP/1.0" 200 7009 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1104 | 80.8.55.1 - - [01/Jan/2001:12:00:25 +0100] "GET /page3 HTTP/1.0" 200 7009 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1105 | 80.8.55.1 - - [01/Jan/2001:12:00:30 +0100] "GET /image.gif HTTP/1.0" 200 7009 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1106 | 80.8.55.1 - - [01/Jan/2001:12:00:35 +0100] "GET /image2.png HTTP/1.0" 200 7009 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1107 | 80.8.55.1 - - [01/Jan/2001:12:00:40 +0100] "GET /js/awstats_misc_tracker.js HTTP/1.1" 200 4998 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1108 | 80.8.55.1 - - [01/Jan/2001:12:00:45 +0100] "GET /js/awstats_misc_tracker.js?screen=1024x768&cdi=32&java=true&shk=n&fla=y&rp=y&mov=n&wma=y&pdf=y&uid=awsuser_id1073036758306r9417&sid=awssession_id1073036758306r9417 HTTP/1.1" 200 4998 "-" "Mozilla/4.7 [fr] (Win95; I)" |
| 1109 | |
| 1110 | 80.8.55.2 - - [01/Jan/2001:12:01:00 +0100] "GET /hitfromothersitetoimage.gif HTTP/1.0" 200 7009 "-" "Mozilla/5.0+(Macintosh;+U;+PPC+Mac+OS+X+Mach-O;+en-US;+rv:1.4)+Gecko/20030624+Netscape/7.1" |
| 1111 | |
| 1112 | 80.8.55.3 - - [01/Jan/2001:12:01:10 +0100] "GET / HTTP/1.0" 200 7009 "http://www.sitereferer:81/cgi-bin/azerty.pl" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1113 | 80.8.55.3 - - [01/Jan/2001:12:01:15 +0100] "GET /page1.html HTTP/1.0" 200 7009 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1114 | 80.8.55.3 - - [01/Jan/2001:12:01:20 +0100] "GET /page2.cgi?x=a&family=a&y=b&familx=x HTTP/1.0" 200 7009 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1115 | 80.8.55.3 - - [01/Jan/2001:12:01:25 +0100] "GET /page3 HTTP/1.0" 200 7009 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1116 | 80.8.55.3 - - [01/Jan/2001:12:01:30 +0100] "GET /image.gif HTTP/1.0" 200 7009 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1117 | 80.8.55.3 - - [01/Jan/2001:12:01:35 +0100] "GET /image2.png HTTP/1.0" 200 7009 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1" |
| 1118 | |
| 1119 | 80.8.55.4 - - [01/Jan/2001:12:01:45 +0100] "GET /samevisitorthan80.8.55.3usingaolproxy.gif HTTP/1.0" 200 7009 "-" "Mozilla/3.0 (Windows 98; U) Opera 6.03" |
| 1120 | |
| 1121 | 80.8.55.5 - - [01/Jan/2001:12:02:00 +0200] "GET /robots.txt HTTP/1.0" 200 299 "-" "This is an unkown user agent" |
| 1122 | 80.8.55.5 - - [01/Jan/2001:12:02:00 +0200] "GET /mydir/robots.txt HTTP/1.0" 200 299 "-" "This is an unkown user agent" |
| 1123 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot1.html HTTP/1.0" 200 7009 "-" "GoogleBot" |
| 1124 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot2a.html HTTP/1.0" 200 7009 "-" "This is bot xxx" |
| 1125 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot2b.html HTTP/1.0" 200 7009 "-" "This_is_bot_xxx" |
| 1126 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot3.html HTTP/1.0" 200 7009 "-" "This is sucker xxx" |
| 1127 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot4.html HTTP/1.0" 200 7009 "-" "woozweb-monitoring" |
| 1128 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot5.html HTTP/1.0" 200 7009 "-" "wget" |
| 1129 | 80.8.55.5 - - [01/Jan/2001:12:02:05 +0200] "GET /pagefromabot6.html HTTP/1.0" 200 7009 "-" "libwww" |
| 1130 | |
| 1131 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /cgi-bin/order.cgi?x=a&family=a&productId=998&titi=i&y=b&y=b HTTP/1.0" 200 7009 "http://www.google.com/search?sourceid=navclient&ie=utf-8&oe=utf-8&q=ma%C3%AEtre+élève" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1132 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /images/image1.gif HTTP/1.0" 200 364 "http://www.google.fr/search?q=cache:dccTQ_Zn4isJ:www.chiensderace.com/cgi-bin/liste_annonces.pl%3FTYPE%3D5%26ORIGINE%3Dchiensderace+labrador+chiensderace&hl=en&lr=lang_en|lang_fr&ie=UTF-8" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1133 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /images/image2.gif HTTP/1.0" 200 364 "http://www.google.fr/search?q=cache:dccTQ_Zn4isJ:www.chiensderace.com/cgi-bin/liste_annonces.pl%3FTYPE%3D5%26ORIGINE%3Dchiensderace+labrador+chiensderace&hl=en&lr=lang_en|lang_fr&ie=UTF-8" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1134 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /images/image3.gif HTTP/1.0" 200 364 "http://www.google.fr/search?q=cache:dccTQ_Zn4isJ:www.chiensderace.com/cgi-bin/liste_annonces.pl%3FTYPE%3D5%26ORIGINE%3Dchiensderace+labrador+chiensderace&hl=en&lr=lang_en|lang_fr&ie=UTF-8" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1135 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /images/image4.gif HTTP/1.0" 200 364 "http://www.google.fr/search?q=cache:dccTQ_Zn4isJ:www.chiensderace.com/cgi-bin/liste_annonces.pl%3FTYPE%3D5%26ORIGINE%3Dchiensderace+labrador+chiensderace&hl=en&lr=lang_en|lang_fr&ie=UTF-8" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1136 | 80.8.55.6 - john [01/Jan/2001:13:00:00 +0100] "GET /images/image5.gif HTTP/1.0" 200 364 "http://www.google.fr/search?q=cache:dccTQ_Zn4isJ:www.chiensderace.com/cgi-bin/liste_annonces.pl%3FTYPE%3D5%26ORIGINE%3Dchiensderace+labrador+chiensderace&hl=en&lr=lang_en|lang_fr&ie=UTF-8" "SAGEM-myX-5m/1.0_UP.Browser/6.1.0.6.1.103_(GUI)_MMP/1.0_(Google_WAP_Proxy/1.0)" |
| 1137 | |
| 1138 | 80.8.55.7 - John Begood [01/Jan/2001:13:01:00 +0100] "GET /cgi-bin/order.cgi;family=f&type=t&productId=999&titi=i#BIS HTTP/1.0" 200 7009 "http://www.a9.com/searchkeyfroma9" "Mozilla/3.01 (compatible;)" |
| 1139 | 80.8.55.7 - John Begood [01/Jan/2001:13:01:00 +0100] "GET /do/Show;jsessionid=6BEF030AB1677BEC333FFCC7BF4DF564?param=1477 HTTP/1.0" 200 7009 "-" "Mozilla/3.01 (compatible;)" |
| 1140 | |
| 1141 | 80.8.55.8 - - [01/Jan/2001:14:01:20 +0100] "GET /404notfoundpage.html?paramnotpagefound=valparamnotpagefound HTTP/1.0" 404 0 "http://refererto404nofoundpage/pageswithbadlink.html?paramrefnotpagefound=valparamrefnotpagefound" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1142 | 80.8.55.8 - - [01/Jan/2001:14:01:20 +0100] "GET /405error.html HTTP/1.0" 405 0 "http://refererto405error/pagesfrom405.html" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" |
| 1143 | |
| 1144 | 80.8.55.9 - - [01/Jan/2001:15:00:00 +0200] "GET /default.ida?XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX HTTP/1.0" 404 299 "-" "-" |
| 1145 | 80.8.55.9 - - [01/Jan/2001:15:00:00 +0200] "SEARCH / -" 411 - "-" "-" |
| 1146 | |
| 1147 | 80.8.55.10 - - [01/Jan/2001:16:00:00 -0300] "GET / HTTP/1.1" 200 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1148 | 80.8.55.10 - - [01/Jan/2001:16:00:00 -0300] "GET /index.html HTTP/1.1" 200 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1149 | 80.8.55.10 - - [01/Jan/2001:16:00:00 -0300] "GET /index.php HTTP/1.1" 200 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1150 | 80.8.55.10 - - [01/Jan/2001:16:00:00 -0300] "GET / HTTP/1.1" 200 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1151 | |
| 1152 | 80.8.55.10 - - [01/Jan/2001:16:00:00 -0300] "GET / HTTP/1.1" 200 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1153 | 80.8.55.10 - - [01/Jan/2001:16:30:00 -0300] "GET /page1.html HTTP/1.1" 200 70476 "http://www.freeweb.hu/icecat/filmek/film04.html" "Mozilla/5.0 (Windows; U; en-US) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5" |
| 1154 | 80.8.55.10 - - [01/Jan/2001:17:00:00 -0300] "GET /cgi-bin/awredir.pl?url=http://xxx.com/aa.html HTTP/1.1" 302 70476 "http://us.f109.mail.yahoo.com/ym/ShowLetter?box=Inbox&MoreYahooParams..." "Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7.5) Gecko/20041108 Firefox/1.0" |
| 1155 | </pre></i> |
| 1156 | </td></tr></table><br> |
| 1157 | <u>This is what you should find:</u><br> |
| 1158 | 6 true human visits<br> |
| 1159 | 5 different true visitors<br> |
| 1160 | 1 bot visit<br> |
| 1161 | 1 worm attack<br> |
| 1162 | The entry pages for true visits should be "/" (even for 80.8.55.1) or "/cgi-bin/order.cgi" but nothing else.<br> |
| 1163 | Note: I did not find any commercial log analyzer that can deal such a common log file correctly, so if you find, let me know !<br> |
| 1164 | <br> |
| 1165 | |
| 1166 | <a name="DIFFERENCE_HOURS"></a><br> |
| 1167 | <b><u>FAQ-COM300 : DIFFERENCE BETWEEN LOCAL HOURS AND AWSTATS REPORTED HOURS</u></b><br> |
| 1168 | <font class=CProblem>PROBLEM:</font><br> |
| 1169 | I use IIS and there's a difference between local hour and AWStats reported hour. For example I made a hit on a page |
| 1170 | at 4:00 and AWStats report I hit it at 2:00.<br> |
| 1171 | <font class=CSolution>SOLUTION:</font><br> |
| 1172 | This is not a problem of time in your local client host. |
| 1173 | AWStats use only time reported in logs by your server |
| 1174 | and all time are related to server hour. The problem is that IIS in some foreign |
| 1175 | versions puts GMT time in its log file (and not local time). So, you have also |
| 1176 | GMT time in your statistics.<br> |
| 1177 | You can wait that Microsoft change this in next IIS versions. |
| 1178 | However, Microsoft sheet Q271196 "IIS Log File Entries Have the Incorrect Date and Time Stamp" says: <br> |
| 1179 | <i>The selected log file format is the W3C Extended Log File |
| 1180 | Format. The extended log file format is defined in the W3C |
| 1181 | Working Draft WD-logfile-960323 specification by Phillip |
| 1182 | M. Hallam-Baker and Brian Behlendorf. This document defines |
| 1183 | the Date and Time files to always be in GMT. This behavior |
| 1184 | is by design.</i><br> |
| 1185 | So this means this way of working might never be changed, so another chance is to use the AWStats plugin 'timezone'. |
| 1186 | Warning, this plugin need the perl module Time::Local and it reduces seriously AWStats speed.<br> |
| 1187 | To enable the plugin, uncomment the following line in your config file.<br> |
| 1188 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1189 | LoadPlugin="timezone TZ"<br> |
| 1190 | </td></tr></table> |
| 1191 | where TZ is value of your signed timezone (+2 for Paris, -8 for ...)<br> |
| 1192 | <br> |
| 1193 | |
| 1194 | <a name="GEOIPEU"></a><br> |
| 1195 | <b><u>FAQ-COM320 : WHAT DOES "EU" (EUROPEAN COUNTRY)" MEAN IN GEOIP COUNTRY REPORTS ?</u></b><br> |
| 1196 | <font class=CProblem>PROBLEM:</font><br> |
| 1197 | I use the <a href="awstats_contrib.html#geoip">AWStats GeoIp country plugins</a> to report countries according |
| 1198 | to geolocalisation of IP address. |
| 1199 | In country report, I have some visitors said to come from "eu (European country)". If this visitor |
| 1200 | come from France, does this means it is reported twice ?<br> |
| 1201 | <font class=CSolution>SOLUTION:</font><br> |
| 1202 | No.<br> |
| 1203 | "eu (European country)" means we are sure that visitor come from an european country but we can't tell |
| 1204 | wich one. It might be a major country like France, Great Britain, Spain, Germany... like a very small one.<br> |
| 1205 | So, in the country report, each visitor is in one and only one group. If reported in "eu (European country)", |
| 1206 | it is not counted in another country, and if reported in a particular country, it is not counted in "eu (European country)", |
| 1207 | even if this country is in Europe.<br> |
| 1208 | <br> |
| 1209 | |
| 1210 | <a name="OLDLOG"></a><br> |
| 1211 | <b><u>FAQ-COM350 : HOW CAN I PROCESS AN OLD LOG FILE ?</u></b><br> |
| 1212 | <font class=CProblem>PROBLEM:</font><br> |
| 1213 | I want to process an old log file to include its data in my AWStats reports.<br> |
| 1214 | <font class=CSolution>SOLUTION:</font><br> |
| 1215 | You must change your <a href="awstats_config.html#LogFile">LogFile</a> parameter to point to the old log file and run |
| 1216 | the update (or use the -LogFile option on command line to overwrite <a href="awstats_config.html#LogFile">LogFile</a> parameter). |
| 1217 | The update process can only accept files in chronological order for a particular month, so if you |
| 1218 | have already processed a recent file and forgot to run update |
| 1219 | on a log file that contains older data, you must reset all of |
| 1220 | your statistics (see <a href="#FAQ-COM500">FAQ-COM500</a>) and restart all of the update processes |
| 1221 | for all past log files and in chronological order.<br> |
| 1222 | However, there is a "tip" that allows you to rebuild only the month were you missed data:<br> |
| 1223 | Imagine we are on 5th of July 2003, all your statistics are up to date except for the 10th of |
| 1224 | April 2003 (you forgot to run the update process for this day, so there is no visit for this |
| 1225 | day). You can :<br> |
| 1226 | - Reset the statistics for April only (this means remove the file <i>awstats042003.[config.]txt</i> |
| 1227 | as explained in <a href="#FAQ-COM500">FAQ-COM500</a>),<br> |
| 1228 | - Move the statistics history files for the month after April (file <i>awstats052003.[config.]txt</i>, |
| 1229 | <i>awstats062003.[config.]txt</i>,...) into a temp directory (so that it is no longer in the DirData |
| 1230 | directory; as if they were deleted).<br> |
| 1231 | - Run the update process on all log files for April (in chronological order). AWStats does not |
| 1232 | complain about "too old record" because there is no history files in DirData directory |
| 1233 | that contains compiled data more recent than records into log you process.<br> |
| 1234 | - Moved back the month history files you saved into your DirData directory.<br> |
| 1235 | Your statistics are up to date and the missing days are no longer missing.<br> |
| 1236 | <br> |
| 1237 | |
| 1238 | <a name="MULTILOG"></a><br> |
| 1239 | <b><u>FAQ-COM360 : HOW CAN I PROCESS SEVERAL LOG FILES IN ONE RUN ?</u></b><br> |
| 1240 | <font class=CProblem>PROBLEM:</font><br> |
| 1241 | How can I update my statistics for several log file, in one run ?<br> |
| 1242 | <font class=CSolution>SOLUTION:</font><br> |
| 1243 | A solution should be to setup your config file with something like:<br> |
| 1244 | <i>LogFile=mylog*.log</i><br> |
| 1245 | However, with such a syntax, AWStats can't know in wich order processing log files (wich log file is the first, next or last). So |
| 1246 | to work like this you must use the following syntax:<br> |
| 1247 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1248 | LogFile="/pathto/logresolvemerge.pl mylog*.log |"<br> |
| 1249 | </td></tr></table> |
| 1250 | <br> |
| 1251 | <a href="awstats_tools.html#logresolvemerge">Logresolvemerge</a> is a tool provided with |
| 1252 | AWStats (in tools directory) that merges several log files on the fly. It opens a pointer on each file |
| 1253 | and sends, line by line, the oldest record from this. Using such a tool as a pipe source for AWStats <a href="awstats_config.html#LogFile">LogFile</a> |
| 1254 | parameter is a very good solution because, it allows you to merge log files whatever their size |
| 1255 | with no memory use, no hard disk use (no temporary files built), it is fast, it prevents |
| 1256 | you from a bad order if your log files are not correctly ordered, etc...<br> |
| 1257 | This tool can also be used to process log files from load balanced systems (see <a href="awstats_faq.html#LOADLOG">FAQ-COM400</a>)<br> |
| 1258 | <br> |
| 1259 | |
| 1260 | <a name="LOADLOG"></a><br> |
| 1261 | <b><u>FAQ-COM400 : HOW CAN I UPDATE MY STATISTICS WHEN I USE A LOAD BALANCING SYSTEM THAT SPLITS MY LOGS ?</u></b><br> |
| 1262 | <font class=CProblem>PROBLEM:</font><br> |
| 1263 | How can I update my statistics when i use a load balancing system that split my logs ?<br> |
| 1264 | <font class=CSolution>SOLUTION:</font><br> |
| 1265 | First solution is to merge all split log files resulted from all your load balanced servers into one. For this, you can use |
| 1266 | the <a href="awstats_tools.html#logresolvemerge">logresolvemerge</a> tool provided with AWStats :<br> |
| 1267 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1268 | logresolvemerge.pl file1.log file2.log ... filen.log > newfiletoprocess.log |
| 1269 | </td></tr></table> |
| 1270 | And setup the <a href="awstats_config.html#LogFile">LogFile</a> parameter in your config file to process the <i>newfiletoprocess.log</i> file or use |
| 1271 | the <i>-LogFile</i> command line option to overwrite <a href="awstats_config.html#LogFile">LogFile</a> value.<br> |
| 1272 | <br> |
| 1273 | As an other solution, if you miss disk space, or to save time, you can ask <a href="awstats_tools.html#logresolvemerge">logresolvemerge</a> |
| 1274 | to merge log files on the fly during the AWStats update process. For this, you can use the following syntax in your AWStats config file:<br> |
| 1275 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1276 | LogFile="/pathto/logresolvemerge.pl file*.log |"<br> |
| 1277 | </td></tr></table> |
| 1278 | <br> |
| 1279 | See also <a href="awstats_faq.html#MULTILOG">FAQ-COM360</a> for explanation on logresolvemerge use.<br> |
| 1280 | <br> |
| 1281 | |
| 1282 | <a name="RESET"></a><br> |
| 1283 | <b><u>FAQ-COM500 : HOW CAN I RESET ALL MY STATISTICS ?</u></b><br> |
| 1284 | <font class=CProblem>PROBLEM:</font><br> |
| 1285 | I want to reset all my statistics to restart the update process from the beginning.<br> |
| 1286 | <font class=CSolution>SOLUTION:</font><br> |
| 1287 | All analyzed data are stored by AWStats in history files called <i>awstatsMMYYYY.[config.]txt</i> (one file each month). |
| 1288 | You will find those files in directory defined by <a href="awstats_config.html#DirData">DirData</a> parameter (same directory than awstats.pl by default).<br> |
| 1289 | To reset all your statistics, just delete all files <i>awstatsMMYYYY.txt</i><br> |
| 1290 | To reset all your statistics built for a particular config file, just delete all files <i>awstatsMMYYYY.myconfig.txt</i><br> |
| 1291 | Warning, if you delete those data files, you won't be able to recover your stats back, unless you |
| 1292 | kept old log files somewhere. You will have to process all past log files (in chronological order) |
| 1293 | to get your statistics back.<br> |
| 1294 | <br> |
| 1295 | |
| 1296 | <a name="DAILY"></a><br> |
| 1297 | <b><u>FAQ-COM600 : HOW CAN I COMPILE AND BUILD STATISTICS ON A DAILY BASIS ONLY ?</u></b><br> |
| 1298 | <font class=CProblem>PROBLEM:</font><br> |
| 1299 | How can I compile and build statistics on a daily basis. I mean i want to have a full report |
| 1300 | with all charts with data for a particular day only and want one report for each day of month.<br> |
| 1301 | <font class=CSolution>SOLUTION:</font><br> |
| 1302 | <b>If you use version 6.5 or higher:</b><br> |
| 1303 | <u>To build statistics:</u><br> |
| 1304 | What you can do is rerun the update process by adding the parameter <i>-databasebreak=hour</i> or <i>-databasebreak=day</i>. |
| 1305 | Providing no option is similar than using <i>-databasebreak=month</i>, the default and old behaviour of AWStats.<br> |
| 1306 | Using this hidden option will ask AWStats to build a different database file for each break entity, |
| 1307 | this means that several reports are done for each hour or day, depending on option used.<br> |
| 1308 | <u>To read a report:</u><br> |
| 1309 | Add same option <i>-databasebreak=hour</i> or <i>-databasebreak=day</i> with <i>-output</i> option |
| 1310 | when AWStats report is staticaly built from command line, or add <i>&databasebreak=hour</i> or |
| 1311 | <i>&databasebreak=day</i> if AWStats is called as a CGI. |
| 1312 | Also, complete options month and year used to choose month and year of report with other option |
| 1313 | <i>day</i> (when databasebreak option is 'day' or 'hour') and <i>hour</i> (only when databasebreak is 'hour')<br> |
| 1314 | So use <i>-day=XX</i> and/or <i>-hour=XX</i> when AWStats is run from command line. Use <i>&day=XX</i> and/or <i>&hour=XX</i> |
| 1315 | if AWStats is called as a CGI.<br> |
| 1316 | This feature is recent so may have results not completely reliable, that's why it is not yet |
| 1317 | fully documented.<br> |
| 1318 | <b>If you use version 6.4 or older:</b><br> |
| 1319 | This is an non documented and not supported trick, as this is not the standard way of working:<br> |
| 1320 | First, run the update process at midnight (or on a log file that was rotated at midnight so that |
| 1321 | it contains only data for this particular day (you can choose another hour in night if you want |
| 1322 | to have days that "start" at an different hour).<br> |
| 1323 | Once the update process has been ran, MOVE (and not copy) the history file built by AWStats. For |
| 1324 | example on Unix like systems:<br> |
| 1325 | <i>mv mydirdata/awstatsMMYYYY.mydomain.txt mydirdate/awstatsDDMMYYYY.mydomain.txt</i><br> |
| 1326 | Note that the name has been changed by adding the day. Repeat this each day after the update process.<br> |
| 1327 | With this you will have one history file for each day. You can then see full stats for |
| 1328 | a particular day by adding the non documented parameter -day=DD on command line (with others |
| 1329 | like -month=MM and -year=YYYY). If ran from a browser you can also add &day=DD on URL.<br> |
| 1330 | However, if you have full day by day statistics, you don't have anymore statistics for full month, |
| 1331 | except if you create a second config file that whose history files would not be moved.<br> |
| 1332 | <br> |
| 1333 | |
| 1334 | <a name="EDITHISTORY"></a><br> |
| 1335 | <b><u>FAQ-COM700 : CAN I SAFELY REMOVE A LINE IN HISTORY FILES (awstatsMMYYYY*.txt) ?</u></b><br> |
| 1336 | <font class=CProblem>PROBLEM:</font><br> |
| 1337 | After processing a log file I want to change my statistics |
| 1338 | without running AWStats update process but changing directly data in AWStats historical database files.<br> |
| 1339 | <font class=CSolution>SOLUTION:</font><br> |
| 1340 | If you remove a lines starting with "BEGIN_" or "END_", AWStats will find your file "corrupted" so you must not change those |
| 1341 | two kinds of lines.<br> |
| 1342 | You can change, add or remove any line that is in any sections but if you do this, you must |
| 1343 | also update the MAP section (lines between BEGIN_MAP and END_MAP) because this section contains the offset in |
| 1344 | file of each other sections for direct I/O access. |
| 1345 | If history file is the last one, you can easily do that by removing completely the MAP section and run an update process. |
| 1346 | Like that AWStats will rewrite the history file and the MAP section will be rewritten (MAP section is not read by |
| 1347 | update process, only written). You do this at your own risk. The main risk is that some charts will report wrong values |
| 1348 | or be unavailable.<br> |
| 1349 | <br> |
| 1350 | |
| 1351 | |
| 1352 | <hr><br> |
| 1353 | |
| 1354 | |
| 1355 | <a name="MISSINGDOLLAR"></a><br> |
| 1356 | <b><u>FAQ-SET050 : ERROR "MISSING $ ON LOOP VARIABLE ..."</u></b><br> |
| 1357 | <font class=CProblem>PROBLEM:</font><br> |
| 1358 | When I run awstats.pl from command line, I get:<br> |
| 1359 | <i>"Missing $ on loop variable at awstats.pl line xxx"</i><br> |
| 1360 | <font class=CSolution>SOLUTION:</font><br> |
| 1361 | Problem is in your Perl interpreter. Try to install or reinstall a more recent/stable Perl interpreter.<br> |
| 1362 | You can get new Perl version at <a href="http://www.activestate.com/ActivePerl/">ActivePerl</a> (<font color=#221188>Win32</font>) |
| 1363 | or <a href="http://www.perl.com/pub/language/info/software.html">Perl.com</a> (<font color=#221188>Unix/Linux/Other</font>).<br> |
| 1364 | <br> |
| 1365 | |
| 1366 | <a name="CGISOURCE"></a><br> |
| 1367 | <b><u>FAQ-SET100 : I SEE PERL SCRIPT'S SOURCE INSTEAD OF ITS EXECUTION</u></b><br> |
| 1368 | <font class=CProblem>PROBLEM:</font><br> |
| 1369 | When I try to execute the Perl script through the web server, |
| 1370 | I see the Perl script's source instead of the HTML result page of its execution !<br> |
| 1371 | <font class=CSolution>SOLUTION:</font><br> |
| 1372 | This is not a problem of AWStats but a problem in your web server setup. |
| 1373 | awstats.pl file must be in a directory defined in your web server to be a "cgi" directory, |
| 1374 | this means, a directory configured in your web server to contain "executable" files and |
| 1375 | not to documents files. |
| 1376 | You have to read your web server manual to know how to setup a directory to be an |
| 1377 | "executable cgi" directory (With IIS, you have some checkbox to check in directory |
| 1378 | properties, with Apache you have to use the "ExecCGI" option in the directory "Directive").<br> |
| 1379 | <br> |
| 1380 | |
| 1381 | <a name="SPAWNERROR"></a><a name="INTERNAL"></a><br> |
| 1382 | <b><u>FAQ-SET150 : INTERNAL ERROR 500 IN MY BROWSER</u></b><br> |
| 1383 | <b><u>FAQ-SET200 : ERROR "... COULDN'T CREATE/SPAWN CHILD PROCESS..."</u></b><br> |
| 1384 | <font class=CProblem>PROBLEM:</font><br> |
| 1385 | AWStats seems to run fine at the command prompt but when ran as a CGI from a browser, I get an <i>"Internal Error 500"</i>.<br> |
| 1386 | I also also might have the following message in my Apache error log file (or in browser with Apache 2.0+):<br> |
| 1387 | <i>...couldn't create/spawn child process: c:/mywebroot/cgi-bin/awstats.pl</i><br> |
| 1388 | <font class=CSolution>SOLUTION:</font><br> |
| 1389 | First, try to run awstats.pl from command line to see if file is correct. If you get some |
| 1390 | syntax errors and use a Unix like OS, check if your file is a Unix like text file (This |
| 1391 | means each line end with a LF char and not a CR+LF char).<br> |
| 1392 | If awstats.pl file runs correctly from command line, this is probably because your |
| 1393 | web server is not able to known how to run perl scripts. This problem can occur with Apache web |
| 1394 | servers with no internal Perl interpreter (mod_perl not active). To solve this, you must |
| 1395 | tell Apache where is your external Perl interpreter.<br> |
| 1396 | <u>For this, you have 2 solutions:</u><br> |
| 1397 | 1) Add the following directive in your Apache <b>httpd.conf</b> config (or remove the # to uncomment it if line is already available)<br> |
| 1398 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1399 | ScriptInterpreterSource registry |
| 1400 | </td></tr></table> |
| 1401 | Then restart Apache. This will tell Apache to look into the registry to find the program associated to .pl extension.<br> |
| 1402 | 2) Other solution (not necessary if first solution works): Change the first line of awstats.pl file with the full path of your Perl interpreter.<br> |
| 1403 | Example with Windows OS and ActivePerl Perl interpreter (installed in C:\Program Files\ActiveState\ActivePerl), |
| 1404 | you must change the first line of awstats.pl file with:<br> |
| 1405 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1406 | #!c:/program files/activestate/activeperl/bin/perl |
| 1407 | </td></tr></table> |
| 1408 | <br> |
| 1409 | |
| 1410 | <a name="CRASH"></a><a name="CRASH"></a><br> |
| 1411 | <b><u>FAQ-SET220 : CRASH WHILE RUNNING AWSTATS.PL OR PAGE CONTENT ONLY PARTIALY LOADED ON WINDOWS XP</u></b><br> |
| 1412 | <font class=CProblem>PROBLEM:</font><br> |
| 1413 | Sometimes my browser (Most often IE6) crash while running awstats.pl with some AWStats configuration. |
| 1414 | With some other versions or browsers, page content is partialy loaded.<br> |
| 1415 | <font class=CSolution>SOLUTION:</font><br> |
| 1416 | Problem was with WinXP and WinXPpro as documented at MS site Q317949;<br> |
| 1417 | "Socket Sharing Creates Data Loss When Listen and Accept Occur on Different Processes"<br> |
| 1418 | Result was that MSIE would crash or display nothing. Netscape and Opera handled the socket better but displayed the pages partially.<br> |
| 1419 | The effect of the bug was more prononced as the page contents increased (above 30k).<br> |
| 1420 | <a href="http://support.microsoft.com/default.aspx?scid=kb;EN-US;q317949">http://support.microsoft.com/default.aspx?scid=kb;EN-US;q317949</a><br> |
| 1421 | And also at Apache.org<br> |
| 1422 | <a href="http://www.apache.org/dist/httpd/binaries/win32/">http://www.apache.org/dist/httpd/binaries/win32/</a><br> |
| 1423 | MS produced a Hotfix which is now included in SP1.<br> |
| 1424 | But the best solution is to use a better web browser. Take a look at <a href="http://www.mozilla.org/products/firefox/">Firefox</a>, |
| 1425 | one of the best and most popular web browser.<br> |
| 1426 | <br> |
| 1427 | |
| 1428 | <a name="CORRUPTEDDROPPED"></a><br> |
| 1429 | <b><u>FAQ-SET270 : ONLY CORRUPTED OR DROPPED RECORDS</u></b><br> |
| 1430 | <font class=CProblem>PROBLEM:</font><br> |
| 1431 | After running an AWStats update process, all my records are reported to be corrupted or dropped<br> |
| 1432 | <font class=CSolution>SOLUTION:</font><br> |
| 1433 | First, if you have only a small percent of corrupted or dropped records, don't worry. This |
| 1434 | is a normal behaviour. Few corrupted or dropped records can appear in a log file because of |
| 1435 | internal web server bug, virus attack, error writing, log purge or rotate during a writing, etc...<br> |
| 1436 | However, if ALL your records are reported to be corrupted or dropped, check the following things:<br> |
| 1437 | If they are all dropped, run the update process from command line adding the option -showdropped<br> |
| 1438 | -> You will be able to know why a dropped record is discarded. In most cases, this is because you use a |
| 1439 | too large or bad filter parameter (SkipFiles, SkipHosts, OnlyFiles ...).<br> |
| 1440 | If they are all corrupted, run the update process from command line adding the option -showcorrupted<br> |
| 1441 | -> You will be able to know why a corrupted record is discarded.<br> |
| 1442 | If this is because of the log format, check the <a href="#EMPTY_STATS">FAQ-SET350</a> about log format errors.<br> |
| 1443 | If this is because the date of a record is said to be lower than date of previous, this means |
| 1444 | that you ran update processes on different log files without keeping the chronological order |
| 1445 | of log files.<br> |
| 1446 | If this is because the date is invalid, you might have a problem of date not computed correctly |
| 1447 | this it happens in some Pentium4/Xeon4 processors:<br> |
| 1448 | On some (few) Intel Pentium4 (also Xeon4) based host systems, log file time can not be computed |
| 1449 | correctly. This is not an issue of AWStats itself. This error usually occurs on source-based linux |
| 1450 | distributions (gentoo, slackware etc.), where all system libraries are compiled with CPU optimization. |
| 1451 | AWStats is a highly developed PERL application. PERL itself relies on some system libraries, |
| 1452 | for example GLIBC. The GLIBC library usually is buggy in this case. There is an easy way to figure |
| 1453 | out whether the problem described here is responsible for AWStats problems on your system:<br> |
| 1454 | If you have shell access to your machine, simply type the following command:<br> |
| 1455 | <i>perl -e "print int ('541234567891011165415658')"</i><br> |
| 1456 | (NOTE: any 25-digit number works, there is no need to type this exact number)<br> |
| 1457 | If everything goes fine, you should see a floating point number as output:<br> |
| 1458 | <i>5.41234567891011e+23</i><br> |
| 1459 | In this case, please do more research on your log file formats. Your host system itself is not |
| 1460 | responsible for the error.<br> |
| 1461 | But if simply a "0" returns or some other error, this is an indication of your glibc beeing corrupt.<br> |
| 1462 | ATTENTION: The only solution in this case might be to recompile your GLIBC. This can be a quite tricky |
| 1463 | task. Please consult the documentation and F.A.Q.s of your linux distribution first!! (experts: first |
| 1464 | check your global compile flags, eg. march=Pentium4. Trying with other compile flags can solve problem |
| 1465 | quickly in some cases.<br> |
| 1466 | NOTE: In some cases, this error might occur "suddenly", even though AWStats was already running |
| 1467 | perfect already.<br> |
| 1468 | <br> |
| 1469 | |
| 1470 | <a name="NOTSAMENUMBER"></a><br> |
| 1471 | <b><u>FAQ-SET280 : ERROR "NOT SAME NUMBER OF RECORDS OF..."</u></b><br> |
| 1472 | <font class=CProblem>PROBLEM:</font><br> |
| 1473 | When I run AWStats from command line (or as a cgi from a browser), I get |
| 1474 | a message "Not same number of records of ...".<br> |
| 1475 | <font class=CSolution>SOLUTION:</font><br> |
| 1476 | This means your AWStats reference database files (operating systems, browsers, robots...) are not correct. |
| 1477 | First try to update to last version. Then check in your disk that you have only ONE of those files. They should be |
| 1478 | in '<b>lib</b>' directory ('db' with 4.0) where awstats.pl is installed:<br> |
| 1479 | <i>browsers.pm</i><br> |
| 1480 | <i>browsers_phone.pm</i><br> |
| 1481 | <i>domains.pm</i><br> |
| 1482 | <i>operating_systems.pm</i><br> |
| 1483 | <i>robots.pm</i><br> |
| 1484 | <i>search_engines.pm</i><br> |
| 1485 | <i>worms.pm</i><br> |
| 1486 | <i>status_http.pm</i><br> |
| 1487 | <i>status_smtp.pm</i><br> |
| 1488 | <br> |
| 1489 | |
| 1490 | <a name="COULDNOTOPEN"></a><br> |
| 1491 | <b><u>FAQ-SET300 : ERROR "COULDN'T OPEN FILE ..."</u></b><br> |
| 1492 | <font class=CProblem>PROBLEM:</font><br> |
| 1493 | I have the following error:<br> |
| 1494 | <i>"Couldn't open file /workingpath/awstatsmmyyyy.tmp.9999: Permission denied."</i><br> |
| 1495 | <font class=CSolution>SOLUTION:</font><br> |
| 1496 | This error means that the web server didn't succeed in writing the working temporary file (file ended by .tmp.9999 |
| 1497 | where 9999 is a number) because of permissions problems.<br> |
| 1498 | First check that the directory <i>/workingpath</i> has "Write" permission for<br> |
| 1499 | user <u>nobody</u> (default user used by Apache on Linux systems)<br> |
| 1500 | or user <u>IUSR_<i>SERVERNAME</i></u> (default used user by IIS on NT).<br> |
| 1501 | With Unix, try with a path with no links.<br> |
| 1502 | With NT, you must check NTFS permissions ("Read/Write/Modify"), if your directory is on a NTFS partition.<br> |
| 1503 | With IIS, there is also a "Write" permission attribute, defined in directory properties |
| 1504 | in your IIS setup, that you must check.<br> |
| 1505 | With IIS, if a default cgi-bin directory was created during IIS install, try to |
| 1506 | put AWStats directly into this directory.<br> |
| 1507 | If this still fails, you can change the DirData parameter to say AWStats |
| 1508 | that you want to use another directory (A directory you are sure that the default |
| 1509 | user, used by web server process, can write into).<br> |
| 1510 | <br> |
| 1511 | |
| 1512 | <a name="MALFORMEDUTF8"></a><br> |
| 1513 | <b><u>FAQ-SET320 : ERROR "MALFORMED UTF-8 CHARACTER (UNEXPECTED ..."</u></b><br> |
| 1514 | <font class=CProblem>PROBLEM:</font><br> |
| 1515 | When running AWStats from command line, I get one or several lines like this |
| 1516 | on my output:<br> |
| 1517 | <i>Malformed UTF-8 character (unexpected non-continuation byte 0x6d, immediately after start byte 0xe4) at /www/cgi-bin/lib/xxx.pm line 999.</i> |
| 1518 | <br> |
| 1519 | <font class=CSolution>SOLUTION:</font><br> |
| 1520 | This problem appeared with RedHat 8 and Perl 5.8.<br> |
| 1521 | I don't know if RedHat provides a fix for this, but some users had reported that you can |
| 1522 | remove thoose warmless messages by changing your LANG environment variable, removing |
| 1523 | the ".UTF-8" at the end. For example, set <i>LANG="en_US"</i> instead of <i>LANG="en_US.UTF8"</i><br> |
| 1524 | <br> |
| 1525 | |
| 1526 | <a name="EMPTY_STATS"></a><br> |
| 1527 | <b><u>FAQ-SET350 : EMPTY OR NULL STATISTICS REPORTED</u></b><br> |
| 1528 | <font class=CProblem>PROBLEM:</font><br> |
| 1529 | AWStats seems to work but I'm not getting any results. i get a statistics page that looks like i have no hits.<br> |
| 1530 | <font class=CSolution>SOLUTION:</font><br> |
| 1531 | That's one of the most common problem you can get and there is 3 possible reasons :<br> |
| 1532 | <br> |
| 1533 | 1) Your log file format setup might be wrong.<br> |
| 1534 | <u>If you use Apache web server</u><br> |
| 1535 | The best way of working is to use the <i>"combined"</i> log format (See the <a href="awstats_setup.html">Setup and Use</a> page |
| 1536 | to know the way to change your Apache server log from <i>"common"</i> log format into <i>"combined"</i>). |
| 1537 | Don't forget to stop Apache, reset your log file and restart Apache to make change into combined |
| 1538 | effective. Then you must setup your AWStats config file with value <a href="awstats_config.html#LogFormat">LogFormat</a>=1.<br> |
| 1539 | If you want to use another format, read the next FAQ to have examples of LogFile value according |
| 1540 | to log files format.<br> |
| 1541 | <u>If you use IIS server or Windows built-in web server</u><br> |
| 1542 | The Internet Information Server default W3C Extended Log Format will not work correctly with AWStats. |
| 1543 | To make it work correctly, start the IIS Snap-in, select the web site and look at it's Properties. |
| 1544 | Choose W3C Extended Log Format, then Properties, then the Tab Extended Properties and uncheck everything |
| 1545 | under Extended Properties. Once they are all unchecked, check off the list given in |
| 1546 | the <a href="awstats_setup.html">Setup and Use</a> page ("With IIS Server" chapter).<br> |
| 1547 | You can also read the next FAQ to have examples of <a href="awstats_config.html#LogFormat">LogFormat</a> value according to log files format.<br> |
| 1548 | <br> |
| 1549 | 2) You are viewing stats for a year or month when no hits was made on your server.<br> |
| 1550 | When you run awstats, the reports is by default for the current month/year.<br> |
| 1551 | If you want to see data for another month/year you must:<br> |
| 1552 | Add -year=YYYY -month=MM on command line when building the html report page from command line.<br> |
| 1553 | Use an URL like http://myserver/cgi-bin/awstats.pl?config=xxx&year=YYYY&month=MM if viewing stats with AWStats used as a CGI.<br> |
| 1554 | <br> |
| 1555 | 3) When you read your statistics, AWStats does not use the same config file than the one |
| 1556 | used for the update process. Scan your disk for files that match <i>awstats.*conf</i> and remove |
| 1557 | all files that are not the config file(s) you need (awstats.conf files, if found, can be deleted. |
| 1558 | It is better to use a config file called awstats.mydomain.conf).<br> |
| 1559 | <br> |
| 1560 | |
| 1561 | <a name="PARTIAL_STATS"></a><br> |
| 1562 | <b><u>FAQ-SET360 : STATISTICS REPORTED EXCEPT FOR OS, BROWSERS, ROBOTS AND KEYWORDS/KEYPHRASES</u></b><br> |
| 1563 | <font class=CProblem>PROBLEM:</font><br> |
| 1564 | AWStats seems to report my statistics however some charts, like robots, os', browsers, search engines, |
| 1565 | or keywords/keyphrases are empty.<br> |
| 1566 | <font class=CSolution>SOLUTION:</font><br> |
| 1567 | If only robots, search engines or keywords/keyphrases are empty, this simply means your web site was not |
| 1568 | yet visited by any robots and noone found your site using a search engines (this happens particularly |
| 1569 | for Intranet which are not referenced on search engines). |
| 1570 | If all of them are empty or with only unknown values, even after several updates, this probably means |
| 1571 | that your logfile does not contains all informations, this happens with Apache when using the |
| 1572 | standard <i>"common"</i> log format instead of the standard <i>"combined"</i> log format.<br> |
| 1573 | You may also use LogFormat=4 into your AWStats config files instead of 1.<br> |
| 1574 | Read AWStats setup documentation to known how to setup your Apache Web server to report logs in |
| 1575 | a "combined" log format then set <a href="awstats_config.html#LogFormat">LogFormat</a>=1 into your |
| 1576 | AWStats config file.<br> |
| 1577 | <br> |
| 1578 | |
| 1579 | <a name="REDIRECT"></a><br> |
| 1580 | <b><u>FAQ-SET400 : PIPE REDIRECTION TO A FILE GIVE ME AN EMPTY FILE</u></b><br> |
| 1581 | <font class=CProblem>PROBLEM:</font><br> |
| 1582 | I want to redirect awstats.pl output to a file with the following command :<br> |
| 1583 | >awstats.pl -config=... [other_options] > myfile.html<br> |
| 1584 | But myfile.html is empty (size is 0). If i remove the redirection, everythings works correctly.<br> |
| 1585 | <font class=CSolution>SOLUTION:</font><br> |
| 1586 | This is not an AWStats bug but a problem between Perl and Windows.<br> |
| 1587 | You can easily solve this running the following command instead:<br> |
| 1588 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1589 | perl awstats.pl -config=... [other_options] > myfile.html |
| 1590 | </td></tr></table> |
| 1591 | <br> |
| 1592 | |
| 1593 | <a name="NO_ICON"></a><br> |
| 1594 | <b><u>FAQ-SET450 : NO PICTURES/GRAPHICS SHOWN</u></b><br> |
| 1595 | <font class=CProblem>PROBLEM:</font><br> |
| 1596 | AWStats seems to work (all data and counters seem to be good) but I have no image shown.<br> |
| 1597 | <font class=CSolution>SOLUTION:</font><br> |
| 1598 | With Apache web server, you might have troubles (no picture shown on stats page) if you use a directory called "icons" (because of Apache |
| 1599 | pre-defined "icons" alias directory), so use instead, for example, a directory called "icon" with |
| 1600 | no s at the end (Rename your directory physically and change the <a href="awstats_config.html#DirIcons">DirIcons</a> parameter in config file |
| 1601 | to reflect this change).<br> |
| 1602 | <br> |
| 1603 | |
| 1604 | <a name="MIGRATEDOUBLED"></a><br> |
| 1605 | <b><u>FAQ-SET700 : MY VISITS ARE DOUBLED FOR OLD MONTH I MIGRATED FROM 3.2 TO 5.X</u></b><br> |
| 1606 | <font class=CProblem>PROBLEM:</font><br> |
| 1607 | After having migrated an old history file for a month, the number of visits for this month is doubled. |
| 1608 | So the number of "visits per visitor" is also doubled and "pages per visit" and "hits per visit" is divided by 2. |
| 1609 | All other data like "pages", "hits" and bandwith are correct.<br> |
| 1610 | <font class=CSolution>SOLUTION:</font><br> |
| 1611 | This problem occurs when migrating history files from 3.2 to 5.x.<br> |
| 1612 | To fix this you can use the following tip (warning, do this only after migrating from |
| 1613 | 3.2 to 5.x and if your visit value is doubled). The goal is to remove the line |
| 1614 | in history file that looks like this<br> |
| 1615 | <i>YYYYMM00 999 999 999 999</i><br> |
| 1616 | where YYYY and MM are year and month of config file and 999 are numerical values.<br> |
| 1617 | <br> |
| 1618 | <b>So if your OS is Unix/Linux</b><br> |
| 1619 | <i> |
| 1620 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1621 | grep -vE '^[0-9]{6}00' oldhistoryfile > newhistoryfile<br> |
| 1622 | mv newhistoryfile oldhistoryfile<br> |
| 1623 | </td></tr></table> |
| 1624 | </i> |
| 1625 | And then run the migrate process again on the file.<br> |
| 1626 | <br> |
| 1627 | <b>If your OS is windows and got cygwin</b><br> |
| 1628 | You must follow same instructions than if OS is Unix/Linux BUT |
| 1629 | you must do this from a cygwin 'sh' shell and not from the DOS |
| 1630 | prompt (because the ^ is not understanded by DOS).<br> |
| 1631 | And then run the migrate process again on the file.<br> |
| 1632 | <br> |
| 1633 | <b>In any other case (in fact works for every OS)</b><br> |
| 1634 | You must remove manually the line <i>YYYYMM00 999 999 999 999</i> (must find one and only |
| 1635 | one such line) and then run the migrate process again on the file.<br> |
| 1636 | <br> |
| 1637 | |
| 1638 | <a name="OUTOFMEMORYCYGWIN"></a><br> |
| 1639 | <b><u>FAQ-SET750 : AWSTATS RUN OUT OF MEMORY DURING UPDATE PROCESS WITH CYGWIN PERL</u></b><br> |
| 1640 | <font class=CProblem>PROBLEM:</font><br> |
| 1641 | When I run the update process on a large log file with cygwin Perl, AWStats run out of memory |
| 1642 | but I am sure that I have enough memory to run AWStats according to the 'memory' column in |
| 1643 | benchmark chart available in AWStats documentation (<a href="awstats_benchmark.html">benchmark page</a>).<br> |
| 1644 | <font class=CSolution>SOLUTION:</font><br> |
| 1645 | It might be a limit inside Cygwin Perl. |
| 1646 | Try to increase the Cygwin parameter <i>heap_chunk_in_mb</i>.<br> |
| 1647 | <br> |
| 1648 | |
| 1649 | <a name="SPEED"></a><br> |
| 1650 | <b><u>FAQ-SET800 : AWSTATS SPEED/TIMEOUT PROBLEMS ?</u></b><br> |
| 1651 | <font class=CProblem>PROBLEM:</font><br> |
| 1652 | When I analyze large log files, processing times are very important (Example: update process from a browser |
| 1653 | returns a timeout/internal error after a long wait). |
| 1654 | Is there a setup or things to do to avoid this and increase speed ?<br> |
| 1655 | <font class=CSolution>SOLUTION:</font><br> |
| 1656 | You really need to understand how a log analyzer works to have good speed. |
| 1657 | There is also major setup changes you can do to decrease your processing time.<br> |
| 1658 | See <a href="awstats_benchmark.html#ADVICES">important advices</a> in benchmark page.<br> |
| 1659 | <br> |
| 1660 | |
| 1661 | |
| 1662 | <hr><br> |
| 1663 | |
| 1664 | |
| 1665 | <a name="CSSATTACK"></a><br> |
| 1666 | <b><u>FAQ-SEC100 : CAN AWSTATS BE USED TO MAKE CROSS SITE SCRIPTING ATTACKS ?</u></b><br> |
| 1667 | <font class=CProblem>PROBLEM:</font><br> |
| 1668 | If a bad user use a browser to make a hit on an URL that include a < SCRIPT > ... < /SCRIPT > |
| 1669 | section in its parameter, when AWStats will show the links on the report page, does the script will be executed ?<br> |
| 1670 | <font class=CSolution>SOLUTION:</font><br> |
| 1671 | No. AWStats use a filter to remove all scripts codes that was included in an URL to make a Cross Site Scripting Attack using a |
| 1672 | log analyzer report page.<br> |
| 1673 | <br> |
| 1674 | |
| 1675 | <a name="SECUSER"></a><br> |
| 1676 | <b><u>FAQ-SEC150 : HOW CAN I PREVENT SOME USERS TO SEE STATISTICS OF OTHER USERS ?</u></b><br> |
| 1677 | <font class=CProblem>PROBLEM:</font><br> |
| 1678 | I don't want a user xxx (having a site www.xxx.com) to see statistics of user yyy (having |
| 1679 | a site www.yyy.com). How can i setup AWStats for this ?<br> |
| 1680 | <font class=CSolution>SOLUTION:</font><br> |
| 1681 | Take a look at the <a href="awstats_security.html">security page</a>.<br> |
| 1682 | <br> |
| 1683 | |
| 1684 | <a name="WORMS"></a><br> |
| 1685 | <b><u>FAQ-SEC200 : HOW TO MANAGE LOG FILES (AND STATISTICS) CORRUPTED BY 'WORMS' ATTACKS ?</u></b><br> |
| 1686 | <font class=CProblem>PROBLEM:</font><br> |
| 1687 | My site is attacked by some worms viruses (like Nimba, Code Red...). This make my log file corrupted |
| 1688 | and full of 404 errors. So my statistics are also full of 404 errors. This make AWStats slower and my history files very large. |
| 1689 | Can I do something to avoid this ?<br> |
| 1690 | <font class=CSolution>SOLUTION:</font><br> |
| 1691 | Yes.<br> |
| 1692 | 'Worms' attacks are infected browsers, robots or server changed into web client that make hits on your site using a very long |
| 1693 | unknown URL like this one:<br> |
| 1694 | <i>/default.ida?XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%40%50...%40%50</i><br> |
| 1695 | URL is generated by the infected robot and the purpose is to exploit a vulnerability of the web server (In most cases, only IIS is vulnerable). |
| 1696 | With such attacks, you will will always find a 'common string' in those URLs. |
| 1697 | For example, with Code Red worm, there is always default.ida in the URL string. Some other worms send URLs with cmd.exe in it.<br> |
| 1698 | With 6.0 version and higher, you can set the <a href="awstats_config.html#LevelFor">LevelForWormsDetection</a> |
| 1699 | parameter to "2" and <a href="awstats_config.html#Show">ShowWormsStats</a> to "HBL" in |
| 1700 | config file to enable the worm filtering nd reporting.<br> |
| 1701 | However, this feature reduce seriously AWStats speed and the worms database (lib/worms.pm file) can't contain |
| 1702 | all worms signatures. So if you still have rubish hits, you can modify the worms.pm file yourself or |
| 1703 | edit your config file to add in the <a href="awstats_config.html#SkipFiles">SkipFiles</a> parameter some |
| 1704 | values to discard the not required records, using a regex syntax like example :<br> |
| 1705 | <table border=1 cellpadding=1 cellspacing=0 bgcolor=#F4F4F4 width="95%" class=CFAQ><tr class=CFAQ><td class=CFAQ> |
| 1706 | SkipFiles="REGEX[^\/default\.ida] REGEX[\/winnt\/system32\/cmd\.exe]"<br> |
| 1707 | </td></tr></table> |
| 1708 | <br> |
| 1709 | |
| 1710 | <hr> |
| 1711 | |
| 1712 | <script language=javascript> |
| 1713 | var date='$Date: 2010/06/22 21:35:24 $'; |
| 1714 | document.writeln("Last revision: "+date); |
| 1715 | </script> |
| 1716 | |
| 1717 | </body> |
| 1718 | </html> |
| 1719 |
