Jump to content


 


Register a free account to unlock additional features at BleepingComputer.com
Welcome to BleepingComputer, a free community where people like yourself come together to discuss and learn how to use their computers. Using the site is easy and fun. As a guest, you can browse and view the various discussions in the forums, but can not create a new topic or reply to an existing one unless you are logged in. Other benefits of registering an account are subscribing to topics and forums, creating a blog, and having no ads shown anywhere on the site.


Click here to Register a free account now! or read our Welcome Guide to learn how to use this site.

Photo

Web server log file column headers.


  • Please log in to reply
3 replies to this topic

#1 Ray Parrish

Ray Parrish

  • Members
  • 91 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Cottage Grove, Oregon
  • Local time:10:36 PM

Posted 01 April 2010 - 04:06 PM

Hello,

I am currently writing a web server log file analyzing program, and would like to make it configurable for more than one server's log file column arrangement. I am currently working with Windows IIS server logs, and would like to add to a configuration file, the column headers for log file formats from other servers such as Apache.

Could some of you who have servers other than Windows IIS please post your column headers for your server's log file in answer to this posting?

The Windows IIS log file has 22 columns with the following headers -

#Software: Microsoft Log Parser
#Version: 1.0
#Date: 2010-03-30 03:59:05
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

Note that the first four lines of these server logs do not contain log data, but rather are log file specific lines describing the column headers, data and time, etc... Please include any details about leading lines in your specific servers log's files, and whether they are comma separated, or space separated. The log files I am working with now are space separated.

:thumbsup: Thank you for your co-operation in providing me with this information. I'm writing the web log analyzer in Python, so pretty much everyone will be able to use it when I'm done, since Python is cross platform. 8-)

Edited by Ray Parrish, 01 April 2010 - 04:16 PM.


BC AdBot (Login to Remove)

 


#2 Ray Parrish

Ray Parrish
  • Topic Starter

  • Members
  • 91 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Cottage Grove, Oregon
  • Local time:10:36 PM

Posted 08 April 2010 - 03:04 AM

Hello again,

No one has added any header definitions as of yet, and I have realized that I have Apache installed on my local machine, so if someone could point me to the folder that Apache keeps it's log files in, I can find them, and check the Apache server log format for myself.

I'm surprised no yet has been able to provide me with the first few lines of their server logs, and the name of their web server software. I expected this group to be full of web masters with that type of information available.

Later, Ray Parrish

#3 groovicus

groovicus

  • Security Colleague
  • 9,963 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Centerville, SD
  • Local time:12:36 AM

Posted 08 April 2010 - 07:09 AM

Apache keeps its log files in the LOG folder by default. You can find information about them by looking here.

EDIT: You also didn't specify which log you want, as there are, umm, a dozen different ones or so.

#4 Ray Parrish

Ray Parrish
  • Topic Starter

  • Members
  • 91 posts
  • OFFLINE
  •  
  • Gender:Male
  • Location:Cottage Grove, Oregon
  • Local time:10:36 PM

Posted 08 April 2010 - 09:13 AM

Apache keeps its log files in the LOG folder by default. You can find information about them by looking here.

EDIT: You also didn't specify which log you want, as there are, umm, a dozen different ones or so.


Sorry, I wanted the access log information. I have used the link you provided to look up the access logs for my version of apache, and in their default configuration they spit out very little information compared to the logs I am used to. Here is a sample line from one -

127.0.0.1 - - [07/Feb/2010:09:37:33 -0800] "GET /favicon.ico HTTP/1.1" 404 283 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.17) Gecko/2010010604 Ubuntu/8.04 (hardy) Firefox/3.0.17"

Note that it's not an entirely space separated file, in that some columns include spaces within quotes, such as the user agent string, which appears in column 9, or the last column of the file, whereas the log files I am used to using are space separated entirely, and have 22 columns of information.

The docs you linked to point out the fact that each Apache administrator can define which information they want output to the server's access logs, so designing for all of the permutations from everyone's differing configurations is not practical, unless the log files themselves could include a column header declaration line similar to the log files I am used to.

I also note that the apache log combines three columns into one quoted column for the request method, requested uri-stem, and protocol versions. This also is configurable to be output in several columns instead of just one.

So evidently it is necessry to know the log format string for an apache log that I need to parse. Here is one example log format line from the apache docs -

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined

These log format lines can be assigned in any order, so it's really up to the server administrator how the access logs get written, and parsing them requires their log format string to set up the parser's configuration.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users