Microsoft Bob

Advanced Log Parser Charts Part 2 - Using Gradient Colors for Area Charts

In Part 2 of this series, I'll show you how to customize the area chart from Part 1 to show the chart area with a gradient. More specifically, there are three different chart gradient methods that we'll take a look at in this blog post:

• SetOneColorGradient
• SetTwoColorGradient
• SetPresetGradient

Before I continue, there is one quick Log Parser convention that you should realize: there are two objects that Log Parser will create and pass to your script. As you look at the sample scripts in this post, you will see these objects in use:

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color.
chart.PlotArea.Interior.Color = "#ffffff";

Before I get started, here's a quick review of VBScript that uses Log Parser COM objects:

Option Explicit

' Declare the variables.
Dim objLogQuery, strSQL
Dim objInputW3CFormat, objOutputChartFormat

' Create the Log Parser objects.
Set objLogQuery = WScript.CreateObject("MSUtil.LogQuery")
Set objInputW3CFormat = WScript.CreateObject("MSUtil.LogQuery.W3CInputFormat")
Set objOutputChartFormat = WScript.CreateObject("MSUtil.LogQuery.ChartOutputFormat")

' Define the SQL query.
strSQL = "SELECT Date, COUNT(*) AS Hits " & _
    " INTO _Part2.gif " & _
    " FROM *.log " & _
    " GROUP BY Date " & _
    " ORDER BY Date"

' Specify the chart options.
objOutputChartFormat.groupSize = "800x600"
objOutputChartFormat.fileType = "GIF"
objOutputChartFormat.chartType = "Area"
objOutputChartFormat.categories = "ON"
objOutputChartFormat.values = "ON"
objOutputChartFormat.legend = "OFF"

' Execute the SQL statement to create the chart.
objLogQuery.ExecuteBatch strSQL, objInputW3CFormat, objOutputChartFormat

As I mentioned in Part 1 of this series, you don't have to use the COM objects, but I chose to do so for this blog series because it makes it a little easier to script. That being said, if I use one month's worth of log files from one of my low-volume websites, Log Parser and this VBScript creates the following rather ugly daily hits chart:

With all of this in mind, let's take a look at some simple configuration scripts.

Setting Fonts and Titles and Such...

The above chart really needs some help, so the first thing that we'll do is change a few things. First things first, we need to specify the name of the chart configuration script in the VBScript sample:

Option Explicit

' Declare the variables.
Dim objLogQuery, strSQL
Dim objInputW3CFormat, objOutputChartFormat

' Create the Log Parser objects.
Set objLogQuery = WScript.CreateObject("MSUtil.LogQuery")
Set objInputW3CFormat = WScript.CreateObject("MSUtil.LogQuery.W3CInputFormat")
Set objOutputChartFormat = WScript.CreateObject("MSUtil.LogQuery.ChartOutputFormat")

' Define the SQL query.
strSQL = "SELECT Date, COUNT(*) AS Hits " & _
    " INTO Part2.gif " & _
    " FROM *.log " & _
    " GROUP BY Date " & _
    " ORDER BY Date"

' Specify the chart options.
objOutputChartFormat.groupSize = "800x600"
objOutputChartFormat.fileType = "GIF"
objOutputChartFormat.chartType = "Area"
objOutputChartFormat.categories = "ON"
objOutputChartFormat.values = "ON"
objOutputChartFormat.legend = "OFF"
objOutputChartFormat.config = "Part2.js"

' Execute the SQL statement to create the chart.
objLogQuery.ExecuteBatch strSQL, objInputW3CFormat, objOutputChartFormat

Next, we need to create the actual chart configuration script, which I wrote in JavaScript; you will need to save this as "Part2.js" in order to use my samples:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#f0f0f0";

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 9;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 9;

This chart configuration script does several things:

• Sets the title above the chart to "Hits by Day"
• Sets a black border style for the chart
• Sets the background color for the plot area to a light gray
• Sets the font size for the chart values to 6-point
• Sets the caption below the chart for the start and end date
• Sets the font styles and captions for the Y and Y axes

When you run the VBScript, the resulting chart looks like the following:

This looks a little more legible, but now let's look at setting some colors.

Setting a One-Color Gradient

Using the same JavaScript sample from earlier, we just need to make a couple of changes to the chart configuration script in order to use the SetOneColorGradient method:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#f0f0f0";

// Specify the chart gradient styles.
chart.SeriesCollection(0).Interior.SetOneColorGradient(
 chartSpace.Constants.chGradientHorizontal,
 chartSpace.Constants.chGradientVariantEnd,
 1.0,
 "#ff0000");

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 9;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 9;

When you run the VBScript, this renders a chart that looks like the following:

There are four parameters for the SetOneColorGradient method to look at:

Let's make some quick changes to parameters that we are passing to the SetOneColorGradient method and alter a few of the colors:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#333333";

// Specify the chart gradient styles.
chart.SeriesCollection(0).Interior.SetOneColorGradient(
    chartSpace.Constants.chGradientHorizontal,
    chartSpace.Constants.chGradientVariantStart,
    0.0,
    "#00ff00");

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Color = "#ffffff";

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 9;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 9;

When you run the VBScript, that results in the following considerably cooler-looking chart:

Setting a Two-Color Gradient

The SetTwoColorGradient method offers more color flexibility than the one-color gradient method, and we only need to make a couple of changes to the JavaScript for the chart configuration script in order to use the new method:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#FFFF99";

// Specify the chart gradient styles.
chart.SeriesCollection(0).Interior.SetTwoColorGradient(
 chartSpace.Constants.chGradientVertical,
 chartSpace.Constants.chGradientVariantStart,
 "#0066FF",
 "#00FFCC");

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 9;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 9;

When you run the VBScript, this will create the following chart:

There are four parameters for the SetTwoColorGradient method to consider:

Using a Preset Gradient

There is an additional gradient method that uses a collection of preset color palettes; this method is appropriately named SetPresetGradient. Once again, we need to make a couple of changes to the JavaScript for the chart configuration script in order to use the new method:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#EEFFDD";

// Specify the chart gradient styles.
chart.SeriesCollection(0).Interior.SetPresetGradient(
 chartSpace.Constants.chGradientHorizontal,
 chartSpace.Constants.chGradientVariantStart,
 chartSpace.Constants.chGradientFire);

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 9;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 9;

When you run the VBScript, this will create the following chart:

There are three parameters for the SetPresetGradient method to look at:

There are several of preset gradients in the ChartPresetGradientTypeEnum enumeration, and a little experimentation will yield the best results.

Using 3-D Area Charts

For one last sample, I'd like to show you what gradients can do for your 3-D area charts. To do so, we first need to make a couple of small changes the VBScript that will create the chart:

Option Explicit

' Declare the variables.
Dim objLogQuery, strSQL
Dim objInputW3CFormat, objOutputChartFormat

' Create the Log Parser objects.
Set objLogQuery = WScript.CreateObject("MSUtil.LogQuery")
Set objInputW3CFormat = WScript.CreateObject("MSUtil.LogQuery.W3CInputFormat")
Set objOutputChartFormat = WScript.CreateObject("MSUtil.LogQuery.ChartOutputFormat")

' Define the SQL query.
strSQL = "SELECT Date, COUNT(*) AS Hits " & _
    " INTO _Part2.gif " & _
    " FROM *.log " & _
    " GROUP BY Date " & _
    " ORDER BY Date"

' Specify the chart options.
objOutputChartFormat.groupSize = "1024x768"
objOutputChartFormat.fileType = "GIF"
objOutputChartFormat.chartType = "Area3D"
objOutputChartFormat.categories = "ON"
objOutputChartFormat.values = "ON"
objOutputChartFormat.legend = "OFF"
objOutputChartFormat.config = "Part2.js"

' Execute the SQL statement to create the chart.
objLogQuery.ExecuteBatch strSQL, objInputW3CFormat, objOutputChartFormat

Next, we need to update the JavaScript for the chart configuration script to work with the new VBScript; for the most part, I'm just updating font sizes and chart colors:

// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = "Hits by Day"

// Clear the caption for the chart series.
chart.SeriesCollection(0).Caption = "";

// Set the border style for the chart.
chartSpace.Border.Color = "#000000";
chartSpace.Border.Weight = 2;

// Change the background color for the plot area.
chart.PlotArea.Interior.Color = "#FFFFCC";

// Specify the chart gradient styles.
chart.SeriesCollection(0).Interior.SetTwoColorGradient(
    chartSpace.Constants.chGradientHorizontal,
    chartSpace.Constants.chGradientVariantEnd,
    "#00CCFF",
    "#FFFFFF");

// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 7;

// Get the start and end dates from the X axis.
var startDate = chart.Axes(0).CategoryLabels.Item(0).Caption;
var endDate = chart.Axes(0).CategoryLabels.Item(chart.Axes(0).CategoryLabels.ItemCount-1).Caption;

// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
    "This chart shows the hits by day from "
    + startDate + " to " + endDate + ".";
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 10;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = "Dates";
chart.Axes(0).Title.Font.Size = 11;

// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 9;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = "Hits";
chart.Axes(1).Title.Font.Size = 11;

When you run the VBScript, this will create the following chart:

Summary

In this blog post, I've written a lot of code samples in order to show you four different ways to set gradients for your Log Parser area charts. In future posts, I'll show you how to do some more cool things with some other types of charts.

;-]

Posted: Jan 29 2012, 16:30 by Bob | Comments (0) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: IIS | LogParser | Scripting

Tags: Log Parser, IIS, JavaScript, VBScript, Scripting, Log Files

Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 1 - Working With Configuration Scripts

I recently had a situation where I wanted to customize the chart output from Log Parser, and after a bunch of research I eventually arrived at the conclusion that configuration scripts to create customized charts are probably the least-documented feature of Log Parser. After a lot of experimentation, (and a bit of frustration), I finally managed to achieve the results that I wanted. With that in mind, I thought that it would make a great blog series if I documented some of the settings that I used.

Log Parser and Chart Configuration Scripts

When you look in the Log Parser help file, it makes mention of using configuration scripts to customize charts, and it provides the following small JavaScript sample:

// Add a caption
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption = "Generated by Log Parser 2.2";
chartSpace.ChartSpaceTitle.Font.Size = 6;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Change the background color
chart.PlotArea.Interior.Color = chartSpace.Constants.chColorNone;

Unfortunately, this sample isn't very useful, although I found dozens of forum posts that quote this sample as a way to do things - but it's the only sample that most people cite. The Log Parser help file mentions looking at the MSDN ChartSpace Object Model documentation, but that documentation is only slightly more useful. These two references are what led me to my earlier conclusion that chart configuration scripts are not well-documented, and especially when you are trying to do something with Log Parser.

What I found to be particularly helpful was to use the Log Parser COM interface and write scripts by using Adersoft's VbsEdit and JsEdit. In case you haven't used either of those applications, they are great IDEs for writing scripts; they both give you a great debugging environment, and they have a great object browser that I used to discover what options were available to me. In the end, these two editors made it possible to create the chart configuration scripts that I will discuss in this blog series.

By the way, chart configuration scripts can be written in VBScript or JavaScript, but for this blog I will use VBScript for the Log Parser COM samples and JavaScript for the configuration script samples. I didn't have to do it that way, but it seemed like a good idea to help differentiate between the samples.

Using COM versus the Command-Line

For the samples in this blog series, I will use Log Parser's COM interface and VBScript to create my charts, but this is not necessary; everything that I am documenting can be done from the command-line version of Log parser, and I'll give you some quick examples to see the differences.

The following examples generate some simple area charts that plot the total number of hits by day, and both examples do exactly the same thing:

Command-Line:

logparser.exe "SELECT Date, COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -fileType:GIF -groupSize:800x600 -chartType:Area -categories:ON -values:ON -legend:OFF

COM Interface:

Option Explicit

Dim objLogQuery, strSQL
Dim objInputW3CFormat, objOutputChartFormat

Set objLogQuery = WScript.CreateObject("MSUtil.LogQuery")
Set objInputW3CFormat = WScript.CreateObject("MSUtil.LogQuery.W3CInputFormat")
Set objOutputChartFormat = WScript.CreateObject("MSUtil.LogQuery.ChartOutputFormat")

strSQL = "SELECT Date, COUNT(*) AS Hits " & _
    " INTO HitsByDay.gif " & _
    " FROM *.log " & _
    " GROUP BY Date " & _
    " ORDER BY Date"

objOutputChartFormat.groupSize = "800x600"
objOutputChartFormat.fileType = "GIF"
objOutputChartFormat.chartType = "Area"
objOutputChartFormat.categories = "ON"
objOutputChartFormat.values = "ON"
objOutputChartFormat.legend = "OFF"

objLogQuery.ExecuteBatch strSQL, objInputW3CFormat, objOutputChartFormat

Ugly Charts

Using some of the log files from one of my websites, the above samples created the following basic chart:

Taking a look at this chart makes it easy to see why you would want to customize your output; that light blue is pretty awful, and those values are pretty hard to read.

Specifying Configuration Scripts

If you remember the incredibly basic configuration script from earlier, you only need to add one parameter to each example in order to specify the configuration script:

Command-Line:

logparser.exe "SELECT Date, COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -fileType:GIF -groupSize:800x600 -chartType:Area -categories:ON -values:ON -legend:OFF -config:HitsByDay.js

COM Interface:

Option Explicit

Dim objLogQuery, strSQL
Dim objInputW3CFormat, objOutputChartFormat

Set objLogQuery = WScript.CreateObject("MSUtil.LogQuery")
Set objInputW3CFormat = WScript.CreateObject("MSUtil.LogQuery.W3CInputFormat")
Set objOutputChartFormat = WScript.CreateObject("MSUtil.LogQuery.ChartOutputFormat")

strSQL = "SELECT Date, COUNT(*) AS Hits " & _
    " INTO HitsByDay.gif " & _
    " FROM *.log " & _
    " GROUP BY Date " & _
    " ORDER BY Date"

objOutputChartFormat.groupSize = "800x600"
objOutputChartFormat.fileType = "GIF"
objOutputChartFormat.chartType = "Area"
objOutputChartFormat.categories = "ON"
objOutputChartFormat.values = "ON"
objOutputChartFormat.legend = "OFF"
objOutputChartFormat.config = "HitsByDay.js"

objLogQuery.ExecuteBatch strSQL, objInputW3CFormat, objOutputChartFormat

Simple Output

Taking a look at the resulting chart, you can see why I mentioned earlier that the configuration script wasn't very useful; all it does is add a centered title to the bottom of the chart:

Yup - that's a pretty useless sample configuration script for chart customization.

Next...

In my subsequent posts, I'll show how to make this chart (and several other types of charts) look a lot better.

Posted: Jan 28 2012, 14:13 by Bob | Comments (0) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: IIS | LogParser | Scripting

Tags: IIS, Log Parser, Log Files, Scripting, VBScript, JavaScript

Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

How to determine if FTP clients are using FTPS

One of my colleagues here at Microsoft, Emmanuel Boersma, just reminded me of an email thread that we had several weeks ago, where a customer had asked him how they could tell if FTPS was being used on their FTP server. He had pointed out that when he looks at his FTP log files, the port number was always 21, so it wasn't as easy as looking at a website's log files and looking for port 80 for HTTP versus port 443 for HTTPS. I had sent him the following notes, and I thought that they might make a good blog. ;-)

As I mentioned earlier, we had discussed the control channel is typically over port 21 for both FTP and FTPS, so you can't rely on the port. But having said that, I mentioned that you will see certain verbs in your FTP logs that will let you know when FTPS is being used, and that’s a reliable way to check.

With that in mind, I suggested the following two methods that you can use to determine if FTPS is being used:

• If the port number is something other than 990, and you see the following verbs being used (and succeeding), then Explicit FTPS is being used:
  • AUTH
  • PBSZ
  • PROT
• If the port is 990, then Implicit FTPS is being used. (This means the FTPS is always on.)

For example, see the highlighted data in following FTP log file excerpts:

Explicit FTPS over port 21:

#Fields: date time c-ip cs-username cs-host s-ip s-port cs-method cs-uri-stem sc-status sc-win32-status sc-substatus sc-bytes cs-bytes time-taken
2011-06-30 22:11:24 ::1 - - ::1 21 ControlChannelOpened - - 0 0 0 0 0
2011-06-30 22:11:24 ::1 - - ::1 21 AUTH TLS 234 0 0 31 10 16
2011-06-30 22:11:27 ::1 - - ::1 21 PBSZ 0 200 0 0 69 8 0
2011-06-30 22:11:27 ::1 - - ::1 21 PROT P 200 0 0 69 8 0
2011-06-30 22:11:36 ::1 - - ::1 21 USER robert 331 0 0 69 13 0
2011-06-30 22:11:42 ::1 robert - ::1 21 PASS *** 230 0 0 53 15 2808

Implicit FTPS over port 990:

#Fields: date time c-ip cs-username cs-host s-ip s-port cs-method cs-uri-stem sc-status sc-win32-status sc-substatus sc-bytes cs-bytes time-taken
2011-06-30 22:16:55 ::1 - - ::1 990 ControlChannelOpened - - 0 0 0 0 0
2011-06-30 22:16:58 ::1 - - ::1 990 USER robert 331 0 0 69 13 0
2011-06-30 22:16:58 ::1 robert - ::1 990 PASS *** 230 0 0 53 15 78
2011-06-30 22:16:58 ::1 robert - ::1 990 SYST - 500 5 51 1005 6 0
2011-06-30 22:16:58 ::1 robert - ::1 990 FEAT - 211 0 0 313 6 0
2011-06-30 22:16:58 ::1 robert - ::1 990 OPTS UTF8+ON 200 0 0 85 14 0
2011-06-30 22:16:58 ::1 robert - ::1 990 PBSZ 0 200 0 0 69 8 0
2011-06-30 22:16:58 ::1 robert - ::1 990 PROT P 200 0 0 69 8 0

FWIW – An explanation about Implicit FTPS and Explicit FTPS can be found in the following articles:

• http://blogs.msdn.com/b/robert_mcmurray/archive/2008/11/10/ftp-clients-part-2-explicit-ftps-versus-implicit-ftps.aspx
• http://www.iis.net/ConfigReference/system.applicationHost/sites/site/ftpServer/security/ssl

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Oct 04 2011, 10:01 by Bob | Comments (0) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: FTP | LogParser

Tags: FTP, Log Files

Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Converting W3C log files to NCSA format

Around a year ago I wrote a blog entry titled "Converting NCSA log files to W3C format", which showed how to use the MSWC.IISLog object to convert log files in the NCSA format back to W3C format. I wrote that blog entry to make up for the fact that the CONVLOG.EXE utility only converts log files to NCSA format, which some older log analysis software packages require. So what happens if you have a bunch of log files in W3C format and you don't have a copy of CONVLOG.EXE on your computer?

This blog entry is something of a reverse direction on my previous post, and shows you how to use the MSUtil.LogQuery object to convert W3C log files to NCSA format. The MSUtil.LogQuery object is shipped with LogParser, which you can download from one of the following URLs:

• Version 2.2 and 2.0: http://www.microsoft.com/download, search for "Log Parser"
• Version 2.1: http://www.microsoft.com/download, search for "IIS 6.0 Resource Kit Tools" 

Once you've downloaded and installed the LogParser package, you will need to manually register the LogParser.dll file in order to use the MSUtil.LogQuery object. Having done so, you can use the Windows Script Host (WSH) code in this blog article to convert a folder filled with W3C log files to NCSA format. 

To use this code, just copy the code into notepad, and save it with a ".vbs" file extension on your system. To run it, copy the script to a folder that contains your W3C log files, (named "ex*.log"), then double-click it.

Option Explicit
Dim objFSO
Dim objFolder
Dim objInputFile
Dim objOutputFile
Dim objLogQuery
Dim objLogRecordSet
Dim objLogRecord
Dim strInputPath
Dim strOutputPath
Dim strLogRecord
Dim strLogTemp

Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(".")

For Each objInputFile In objFolder.Files
 strInputPath = LCase(objInputFile.Name)
 If Left(strInputPath,2) = "ex" And Right(strInputPath,4) = ".log" Then
  strOutputPath = objFolder.Path & "\" & "nc" & Mid(strInputPath,3)
  strInputPath = objFolder.Path & "\" & strInputPath
  Set objLogQuery = CreateObject("MSUtil.LogQuery")
  Set objLogRecordSet = objLogQuery.Execute("SELECT * FROM " & strInputPath)
  Set objOutputFile = objFSO.CreateTextFile(strOutputPath)
  Do While Not objLogRecordSet.atEnd
  
   Set objLogRecord = objLogRecordSet.getRecord
   strLogRecord = FormatField(objLogRecord.getValue("c-ip"))
   strLogRecord = strLogRecord & " " & FormatField("")
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("cs-username"))
   strLogTemp = BuildDateTime(objLogRecord.getValue("date"),objLogRecord.getValue("time"))
   strLogRecord = strLogRecord & " " & FormatField(strLogTemp)
   strLogRecord = strLogRecord & " """ & FormatField(objLogRecord.getValue("cs-method"))
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("cs-uri-stem"))
   strLogTemp = FormatField(objLogRecord.getValue("cs-version"))
   If strLogTemp = "-" Then
    strLogRecord = strLogRecord & " HTTP/1.0"""
   Else
    strLogRecord = strLogRecord & " " & strLogTemp & """"
   End If   
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("sc-status"))
   strLogRecord = strLogRecord & " " & FormatField(objLogRecord.getValue("sc-bytes"))
    objOutputFile.WriteLine strLogRecord
   objLogRecordSet.moveNext
  Loop
  
  Set objLogQuery = Nothing
  objOutputFile.Close
 
 End If
Next

Function FormatField(tmpField)
 On Error Resume Next
 FormatField = "-"
 If Len(tmpField) > 0 Then FormatField = Trim(tmpField)
End Function

Function BuildDateTime(tmpDate,tmpTime)
 On Error Resume Next
 BuildDateTime = "[" & _
  Right("0" & Day(tmpDate),2) & "/" & _
  Left(MonthName(Month(tmpDate)),3) & "/" & _
  Year(tmpDate) & ":" & _
  Right("0" & Hour(tmpTime),2) & ":" & _
  Right("0" & Minute(tmpTime),2) & ":" & _
  Right("0" & Second(tmpTime),2) & _
  " +0000]"
End Function

I hope this helps!

Posted: Nov 06 2006, 21:08 by Bob | Comments (0) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: IIS | LogParser

Tags: IIS, Scripting, Log Files, Log Parser

Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Converting NCSA log files to W3C format

One of the great utilities that ships with IIS is the CONVLOG.EXE application, which converts W3C or MS Internet Standard log files to NCSA format, where they can be processed by any of the applications that only parse NCSA log file information. The trouble is, what happens when you already have NCSA log files and you want W3C log files? You can't use the CONVLOG.EXE application, it only works in the opposite direction.

With that in mind, I wrote the following Windows Script Host (WSH) script that will read the current directory and convert all NCSA-formatted log files to W3C format. To use this code, just copy the code into notepad, and save it with a ".vbs" file extension on your system. To run it, copy the script to a folder that contains NCSA log files, (named "nc*.log"), then double-click it.

Option Explicit

Dim objIISLog
Dim objFSO
Dim objFolder
Dim objFile
Dim objOutputFile
Dim strInputPath
Dim strOutputPath
Dim strLogRecord

Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")

Set objFolder = objFSO.GetFolder(".")

For Each objFile In objFolder.Files

 strInputPath = LCase(objFile.Name)

 If Left(strInputPath,2) = "nc" And Right(strInputPath,4) = ".log" Then

  strOutputPath = objFolder.Path & "\" & "ex" & Mid(strInputPath,3)
  strInputPath = objFolder.Path & "\" & strInputPath

  Set objIISLog = CreateObject("MSWC.IISLog")
  objIISLog.OpenLogFile strInputPath, 1, "", 0, ""
  Set objOutputFile = objFSO.CreateTextFile(strOutputPath)

  objIISLog.ReadLogRecord

  objOutputFile.WriteLine "#Software: Microsoft Internet Information Services 5.0"
  objOutputFile.WriteLine "#Version: 1.0"
  objOutputFile.WriteLine "#Date: " & BuildDateTime(objIISLog.DateTime)
  objOutputFile.WriteLine "#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status cs(User-Agent)"

  Do While Not objIISLog.AtEndOfLog

   strLogRecord = BuildDateTime(objIISLog.DateTime)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ClientIP)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.UserName)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ServerIP)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ServerPort)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.Method)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.URIStem)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.URIQuery)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.ProtocolStatus)
   strLogRecord = strLogRecord & " " & FormatField(objIISLog.UserAgent)
   objOutputFile.WriteLine strLogRecord

   objIISLog.ReadLogRecord

  Loop

  objIISLog.CloseLogFiles 1
  objIISLog = Null
 
 End If

Next

Function FormatField(tmpField)
 On Error Resume Next
 FormatField = "-"
 If Len(tmpField) > 0 Then FormatField = Trim(tmpField)
End Function

Function BuildDateTime(tmpDateTime)
 On Error Resume Next
 tmpDateTime = CDate(tmpDateTime)
 BuildDateTime = Year(tmpDateTime) & "-" & _
  Right("0" & Month(tmpDateTime),2) & "-" & _
  Right("0" & Day(tmpDateTime),2) & " " & _
  Right("0" & Hour(tmpDateTime),2) & ":" & _
  Right("0" & Minute(tmpDateTime),2) & ":" & _
  Right("0" & Second(tmpDateTime),2)
End Function

I hope this helps!

Posted: Nov 30 2005, 18:58 by Bob | Comments (0) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: IIS | Scripting

Tags: IIS, Scripting, Log Files, W3C

Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us