Microsoft Bob

Just a short, simple blog for Bob to share some tips and tricks.

Be sure to check out my non-technical blog at www.bobsbasement.net.

Month List

Advanced Log Parser Charts Part 5 - Creating a Generic Chart Color Script

In Part 5 of this series, I'll show you how to create a generic script that you can use to add some color to your Log Parser charts. As I mentioned in Part 1 of this series, the default colors for Log parser charts are really dull and boring. For example, if I parse one month's worth of log files from one of my low-volume websites with the following query:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:ColumnClustered -chartTitle:"" -q:ON

Log Parser will create the following ugly daily hits chart:

Generic Color Change Script

Here's the background story for this blog: I have a collection of scripts that I use to format my charts, several of which have faithfully served as the fodder for this blog series. With that in mind, I had a situation recently where I was querying logs with a series of data just like this, and of course the resulting charts were kind of hideous to look at. In one of the scripts that I often use, I create an array of colors to use, and then I apply the various colors to the individual data points in the series.

In the past I have always hard-coded the length for the array of colors based on the data that I am working with, but in this situation I had no idea how many data points I would have, so I decided to put together a quick script with an array that would work with a series of any size.

Here's the resulting script:

// Set a default color for the chart's data.
chart.SeriesCollection(0).Interior.Color = "#ffcccc";

// Define a short array of colors.
var colors = [
    "#ffff99", "#ff99ff", "#ff9999",
    "#99ffff", "#99ff99", "#9999ff",
    "#ffffcc", "#ffccff", "#ffcccc",
    "#ccffff", "#ccffcc", "#ccccff"
];

// Loop through the data points in the series.
for (x=0;x<chart.SeriesCollection(0).Points.Count;++x)
{
    // Set the color for the data point based on modulo division of the array length.
    chart.SeriesCollection(0).Points(x).Interior.Color = colors[x % colors.length ];
}

That's all that there is to the script - it's pretty simple. If I take the above script and save it as "FormatChart.js", I can use that script with my Log Parser query from earlier by adding an extra parameter to the command:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:ColumnClustered -chartTitle:"" -q:ON -config:FormatChart.js

Now Log Parser will create the following daily hits chart with a great deal more color to it:

Okay - perhaps that's not the best color palette, but you get the idea. It looks even better when I change the query to use 3D charts:

logparser.exe "SELECT date,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY date ORDER BY date" -i:w3c -o:CHART -chartType:Column3D -chartTitle:"" -q:ON -config:FormatChart.js

The above query creates the following chart:

Color Changing Pie Charts

I'd like to make a quick change to the script in order to make it work a little better with a pie chart:

// Set a default color for the chart's data.
chart.SeriesCollection(0).Interior.Color = "#cccccc";

// Define a short array of colors.
var colors = [
    "#cc3333", "#3333cc", "#33cc33",
    "#33cccc", "#cccc33", "#cc33cc"
];

// Loop through the data points in the series.
for (x=0;x<chart.SeriesCollection(0).Points.Count;++x)
{
    // Set the color for the data point based on modulo division of the array length.
    chart.SeriesCollection(0).Points(x).Interior.Color = colors[x % colors.length ];
}
// Rotate the chart 180 degrees - just so it looks a little better.
chartSpace.Charts(0).PlotArea.RotateClockwise();
chartSpace.Charts(0).PlotArea.RotateClockwise();

For this query I'd like to see a break down by HTTP status, and this necessitates some small change to the Log parser query:

logparser.exe "SELECT sc-status AS Status,COUNT(*) AS Hits INTO HITS.gif FROM *.log GROUP BY Status ORDER BY Status" -i:w3c -o:CHART -chartType:PieExploded3D -chartTitle:"" -q:ON -config:FormatChart.js

The above query creates the following chart:

Summary

That wraps it up for this blog - I hope that I've given you some ideas for ways that you can easily add some colors to some dull-looking Log Parser charts.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Aug 24 2012, 18:53 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: LogParser | Scripting
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Windows Cache 1.3 for PHP 5.4

The IIS team has officially signed off on the Windows Cache Extension (WinCache) version 1.3 for PHP 5.4, and the files have been uploaded to SourceForge. This version addresses all of the problems that were identified with WinCache 1.1 that customers were seeing after they upgraded their systems from PHP 5.3 to PHP 5.4.

With that in mind, you can download WinCache 1.3 for for PHP 5.4 from the following URL:

http://sourceforge.net/projects/wincache/files/wincache-1.3.4/

You can discuss WinCache 1.1 and WinCache 1.3 in the Windows Cache Extension for PHP forum on Microsoft's IIS.net website.

Source Code Availability

Since WinCache is an open source project, the IIS team has uploaded the pre-release source code for WinCache at the following URL:

http://pecl.php.net/package/WinCache

For the instructions on how to build the extension yourself, please refer to the Building WinCache Extension documentation.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Jul 26 2012, 05:37 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | PHP | WinCache
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

WebDAV Website Importer for WebMatrix

The other day I was talking with one of my coworkers, Yishai Galatzer, about Microsoft's WebMatrix. By way of introduction, Yishai is one of our senior developers on the WebMatrix project; I'm not sure if you've used WebMatrix, but it's a pretty handy website editor. Here's a few generic screen shots:

WebMatrix 2 Splash Screen
WebMatrix 2 Quick Start Screen
Editing QDIG in WebMatrix 2

In any event, I was explaining how easy it is to work with WebDAV, and I mentioned that I had written some some blogs about working with WebDAV websites programmatically. (See my Sending WebDAV Requests in .NET Revisited blog for an example.) Since WebMatrix 2 has some pretty cool extensibility, Yishai challenged me to write a WebDAV extension for WebMatrix. His idea was just too good for me to pass up, so I stayed up late that night and I wrote a simple WebDAV Website Import extension for WebMatrix 2.

With that in mind, there are a few things that I need to explain in this blog:

  • What this extension actually does.
  • How to install this extension.
  • How to use this extension.

What the WebDAV Website Importer Extension Actually Does

The WebDAV Website Importer extension does just what its name implies - it allows you to import a website into WebMatrix over WebDAV. This allows you to download your website to your local computer, where you can make changes to your source files and test them on your local system with IIS Express.

It should be noted that this extension is only designed to create a new local website by downloading a copy of your website's files in order to create a local copy of your website - it is not designed to be a website publishing feature like WebMatrix's built-in FTP and Web Deploy features. (I would like to write a full-featured website import/export/sync extension, but that's another project for another day.)

How to Install the WebDAV Website Importer Extension

To install this extension, you first need to install WebMatrix. You can find details about installing WebMatrix at the following URL:

Once you have WebMatrix installed, click the Extensions menu on the ribbon, and then click Gallery.

WebMatrix 2's Extensions Menu

When the Extensions Gallery appears, you will see the WebDAV Website Importer in the list of extensions.

WebDAV Website Importer in the Extensions List

When you click Install, the WebDAV Website Importer details page will be displayed.

WebDAV Website Importer Details

When you click Install, the End User License Agreement for the WebDAV Website Importer will be displayed.

WebDAV Website Importer EULA

When you click I Accept, WebMatrix will download and install the extension.

How to Use the WebDAV Website Importer Extension

Once you have downloaded and installed the WebDAV Website Importer extension, it will show up whenever you are creating a new website in WebMatrix.

Import Site from WebDAV menu

When you click Import Site from WebDAV, WebMatrix will prompt you for the credentials to your WebDAV website.

WebDAV Website Credentials Dialog

Once you enter your credentials and click OK, the extension will import the content from your WebDAV website and save it in a new local website folder.

Summary

So - there you have it; this is a pretty simple extension, but it opens up some WebDAV possibilities for WebMatrix. As I mentioned earlier, this extension is import-only - perhaps I'll write a full-featured import/export/sync extension in the future, but for now - this was a cool test for combining WebMatrix extensibility and WebDAV.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: Jul 20 2012, 18:12 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: WebDAV | WebMatrix
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Creating "Pretty" XML using XSL and VBScript

I was working with an application recently that stored all of its settings in a large XML file, however, when I opened the XML in Windows Notepad, all I saw was a large blob of tags and text - there was no structured formatting to the XML, and that made it very difficult to change some of settings by hand. (Okay - I realize that some of you are probably thinking to yourselves, maybe I wasn't supposed to be editing those settings by hand - but that's just the way I do things around here... if I can't customize every setting to my heart's content, then it's just not worth using.)

In any event, I'll give you an example of what I mean by using the example XML database that's provided on MSDN at the following URL:

http://msdn.microsoft.com/en-us/library/windows/desktop/ms762271.aspx

Note - the entire XML file would be too long to repost here, so I'll just include an unstructured except from that file that resembles what my other XML looked like when I opened the file in Windows Notepad:

<?xml version="1.0"?><catalog><book id="bk101"><author>Gambardella, Matthew</author><title>XML Developer's Guide</title><genre>Computer</genre><price>44.95</price><publish_date>2000-10-01</publish_date><description>An in-depth look at creating applications with XML.</description></book><book id="bk102"><author>Ralls, Kim</author><title>Midnight Rain</title><genre>Fantasy</genre><price>5.95</price><publish_date>2000-12-16</publish_date><description>A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world.</description></book><book id="bk103"><author>Corets, Eva</author><title>Maeve Ascendant</title><genre>Fantasy</genre><price>5.95</price><publish_date>2000-11-17</publish_date><description>After the collapse of a nanotechnology society in England, the young survivors lay the foundation for a new society.</description></book></catalog>

This is obviously difficult to read, and even more so when you are dealing with hundreds or thousands of lines of XML code. What would be considerably easier to read and edit would be something more like the following example:

<?xml version="1.0"?>
<catalog>
  <book id="bk101">
    <author>Gambardella, Matthew</author>
    <title>XML Developer's Guide</title>
    <genre>Computer</genre>
    <price>44.95</price>
    <publish_date>2000-10-01</publish_date>
    <description>An in-depth look at creating applications with XML.</description>
  </book>
  <book id="bk102">
    <author>Ralls, Kim</author>
    <title>Midnight Rain</title>
    <genre>Fantasy</genre>
    <price>5.95</price>
    <publish_date>2000-12-16</publish_date>
    <description>A former architect battles corporate zombies, an evil sorceress, and her own childhood to become queen of the world.</description>
  </book>
  <book id="bk103">
    <author>Corets, Eva</author>
    <title>Maeve Ascendant</title>
    <genre>Fantasy</genre>
    <price>5.95</price>
    <publish_date>2000-11-17</publish_date>
    <description>After the collapse of a nanotechnology society in England, the young survivors lay the foundation for a new society.</description>
  </book>
</catalog>

I had written a "Pretty XML" script sometime around ten years ago that read an XML file, collapsed all of the whitespace between tags, and then inserted CRLF sequences and TAB characters in order to reformat the file. This script worked great for many years, but I decided that it would be more advantageous to use XSL to transform the XML. (e.g. "Why continue to do things the hard way when you really don't need to?");-]

With that in mind, I rewrote my old script as the following example:

' ****************************************
' MAKE PRETTY XML
' ****************************************

Option Explicit

Const strInputFile = "InputFile.xml"
Const strOutputFile = "OutputFile.xml"

' ****************************************

Dim objInputFile, objOutputFile, strXML
Dim objFSO : Set objFSO = WScript.CreateObject("Scripting.FileSystemObject")
Dim objXML : Set objXML = WScript.CreateObject("Msxml2.DOMDocument")
Dim objXSL : Set objXSL = WScript.CreateObject("Msxml2.DOMDocument")

' ****************************************
' Put whitespace between tags. (Required for XSL transformation.)
' ****************************************

Set objInputFile = objFSO.OpenTextFile(strInputFile,1,False,-2)
Set objOutputFile = objFSO.CreateTextFile(strOutputFile,True,False)
strXML = objInputFile.ReadAll
strXML = Replace(strXML,"><",">" & vbCrLf & "<")
objOutputFile.Write strXML
objInputFile.Close
objOutputFile.Close

' ****************************************
' Create an XSL stylesheet for transformation.
' ****************************************

Dim strStylesheet : strStylesheet = _
"<xsl:stylesheet version=""1.0"" xmlns:xsl=""http://www.w3.org/1999/XSL/Transform"">" & _
"<xsl:output method=""xml"" indent=""yes""/>" & _
"<xsl:template match=""/"">" & _
"<xsl:copy-of select="".""/>" & _
"</xsl:template>" & _
"</xsl:stylesheet>"

' ****************************************
' Transform the XML.
' ****************************************

objXSL.loadXML strStylesheet
objXML.load strOutputFile
objXML.transformNode objXSL
objXML.save strOutputFile

WScript.Quit

This script is really straightforward in what it does:

  1. Creates two MSXML DOM Document objects:
    • One for XML
    • One for XSL
  2. Creates two file objects:
    • One for the input/source XML file
    • One for the output/destination XML
  3. Reads all of the source XML from the input file.
  4. Inserts whitespace between all of the XML tags in the source XML; this is required or the XSL transformation will not work properly.
  5. Saves the resulting XML into the output XML file.
  6. Dynamically creates a simple XSL file that will be used for transformation in one of the MSXML DOM Document objects.
  7. Loads the output XML file from earlier into the other MSXML DOM Document object.
  8. Transforms the source XML into well-formatted ("pretty") XML.
  9. Replaces the XML in the output file with the transformed XML.

That's all that there is to it.

Note: For more information about the XSL stylesheet that I used, see http://www.w3.org/TR/xslt.

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jul 06 2012, 06:07 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: Scripting | XML
Tags: , ,
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Working with the Different IIS Express Modes and HTTPS

I had another great question from a customer the other day, and I thought that his question was the perfect impetus for me to write blog that explained the different modes of IIS Express.

The customer's issue was that he was trying to run IIS Express from a command-line by specifying the path to a folder and he wanted to use that with SSL. He couldn't find a way to accomplish that, so he asked Scott Hanselman if there was a switch that he was missing, and Scott sent him my way. In the meantime, he was copying one of the IIS Express template ApplicationHost.config files and configuring SSL by modifying the XML programmatically.

First of all, the short answer is that there isn't some form of "/https" switch for IIS Express that the customer was asking about.

But that being said, this seemed like a great occasion for me to explain a little bit of design architecture for IIS Express, which might help everyone understand a little bit about what's going on behind the scenes when you run IIS Express.

In case you weren't aware, there are actually two modes that you can use with IIS Express:

  • Personal Web Server Mode
  • Application Server Mode

Having said that, I'll explain what both of those fancy titles actually mean, and how you can use IIS Express with SSL.

Personal Web Server Mode

When you are using Personal Web Server Mode, one ApplicationHost.config file is created per user by default, (unless an alternate file is specified on the command-line), and by default that ApplicationHost.config file is kept in your "%UserProfile%\Documents\IISExpress\config" folder.

In this mode, websites are persistent like they are with the full version of IIS, and the template that is used to create the per-user ApplicationHost.config file is located at:

"%ProgramFiles%\IIS Express\config\templates\PersonalWebServer\ApplicationHost.config"

Note: When you are using Personal Web Server Mode, your default website is named "WebSite1".

The general syntax for Personal Web Server Mode is:

iisexpress.exe [/config:config-file] [/site:site-name] [/systray:true|false] [/siteid:site-id] [/userhome:user-home]

If you are using IIS Express from a command-line with no parameters, or you are using IIS Express with WebMatrix or Visual Studio, then you are using Personal Web Server Mode. You can use SSL by enabling HTTPS in either WebMatrix or Visual Studio, or you can modify your ApplicationHost.config file directly and add an HTTPS binding to a website.

Application Server Mode

When you are using "Application Server Mode," a temporary ApplicationHost.config file generated when IIS Express starts in the user's "%TEMP%\iisexpress" folder.

In this mode, sites are transient like they are with Cassini, and the template that is used to create the temporary ApplicationHost.config file is located at:

"%ProgramFiles%\IIS Express\AppServer\ApplicationHost.config"

Note: When you are using Application Server Mode, your default website is named "Development Web Site".

The general syntax for Application Server Mode is:

iisexpress.exe /path:app-path [/port:port-number] [/clr:clr-version] [/systray:true|false]

If you are using IIS Express from a command-line by specifying the path to a folder, then you are using Application Server Mode, and unfortunately you can't use SSL with this mode.

Using SSL with IIS Express

As I have already mentioned, if you are using Personal Web Server Mode, you can use SSL by enabling HTTPS in WebMatrix or Visual Studio if you are using either of those tools, or you can modify your ApplicationHost.config file directly and add an HTTPS binding to a website.

However, there is no way to specify HTTPS for Application Server Mode; but that being said, there are definitely workarounds that you can use.

Copying the template file like the customer was doing is a good place to start. But I need to state an important warning: you should never modify the actual template files that are installed with IIS Express! However, if you copy the template files somewhere else on your system, you can modify the copied files as much as you want.

If you are using IIS 8 Express, we've made it possible to use AppCmd.exe with any ApplicationHost.config file by using the "/apphostconfig" switch. So instead of modifying the XML directly, you can use AppCmd.exe to make your changes for you.

For example, the following batch file creates a temporary website and sets it up for use with HTTPS:

@echo off

pushd "%~dp0"

REM Create the website's folders.

md %SystemDrive%\myhttpstemp
md %SystemDrive%\myhttpstemp\wwwroot
md %SystemDrive%\myhttpstemp\config

REM Copy the template configuration file.

copy "%ProgramFiles%\IIS Express\AppServer\ApplicationHost.config" %SystemDrive%\myhttpstemp\config

REM Configure the website's home directory.

"%ProgramFiles%\IIS Express\appcmd.exe" set config -section:system.ApplicationHost/sites /"[name='Development Web Site'].[path='/'].[path='/'].physicalPath:%SystemDrive%\myhttpstemp\wwwroot" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Configure the website for SSL.

"%ProgramFiles%\IIS Express\appcmd.exe" set config -section:system.ApplicationHost/sites /+"[name='Development Web Site'].bindings.[protocol='https',bindingInformation='127.0.0.1:8443:']" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Enable directory browsing so this example works without a home page.

"%ProgramFiles%\IIS Express\appcmd.exe" set config "Development Web Site" -section:system.webServer/directoryBrowse /enabled:"True" /commit:apphost /apphostconfig:%SystemDrive%\myhttpstemp\config\ApplicationHost.config

REM Run the website with IIS Express.

"%ProgramFiles%\IIS Express\iisexpress.exe" /config:%SystemDrive%\myhttpstemp\config\ApplicationHost.config /siteid:1 /systray:false

REM Clean up the website folders.

rd /q /s %SystemDrive%\myhttpstemp

popd

As you can see in the above example, this is a little more involved than simply invoking Application Server Mode with a switch to enable HTTPS, but it's still very easy to do. The changes that we've made in IIS 8 Express make it easy to script Personal Web Server Mode in order to enable SSL for a temporary website.

In Closing...

I hope this information makes using the various IIS Express modes and SSL a little clearer, and you can get IIS 8 Express by following the link in the following blog post:

http://blogs.msdn.com/b/robert_mcmurray/archive/2012/05/31/microsoft-iis-8-0-express-release-candidate-is-released.aspx

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jul 03 2012, 07:06 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS Express | Scripting
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Error: Class Not Registered (0x80040154) when Querying FTP Runtime State

I had a great question from a customer earlier today, and I thought that it was worth blogging about. The problem that he was running into was that he was seeing the following error when he was trying to query the runtime state for the FTP service in an application that he was writing:

Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))

He was using Visual Basic, and his code looked okay to me, so for the moment I was stumped.

I'm more of a C# guy, and I remembered that I had written the following blog many years ago:

Viewing current FTP7 sessions using C#

I copied the code from that blog into a new Visual Studio project, and I got the same error that he was seeing when I ran my code - this had me a little more confused. Have you ever said to yourself, "Darn - I know that worked the other day...?" ;-]

I knew that there is more than one way to access the runtime state, so I rewrote my sample application using two different approaches:

Method #1:

AppHostAdminManager objAdminManager = new AppHostAdminManager();
IAppHostElement objSitesElement =
  objAdminManager.GetAdminSection("system.applicationHost/sites",
  "MACHINE/WEBROOT/APPHOST");
uint intSiteCount = objSitesElement.Collection.Count;
for (int intSite = 0; intSite < intSiteCount; ++intSite)
{
    IAppHostElement objFtpSite = objSitesElement.Collection[intSite];
    Console.WriteLine("Name: " + objFtpSite.Properties["name"].StringValue);
    IAppHostElement objFtpSiteElement = objFtpSite.ChildElements["ftpServer"];
    IAppHostPropertyCollection objProperties = objFtpSiteElement.Properties;
    try
    {
        IAppHostProperty objState = objProperties["state"];
        string ftpState = objState.StringValue;
        Console.WriteLine("State: " + ftpState);
    }
    catch (System.Exception ex)
    {
        Console.WriteLine("\r\nError: {0}", ex.Message);
    }
}

Method #2:

ServerManager manager = new ServerManager();
foreach (Site site in manager.Sites)
{
    Console.WriteLine("Name: " + site.Name);
    ConfigurationElement ftpServer = site.GetChildElement("ftpServer");
    try
    {
        foreach (ConfigurationAttribute attrib in ftpServer.Attributes)
        {
            Console.WriteLine(attrib.Name + ": " + attrib.Value);
        }
    }
    catch (System.Exception ex)
    {
        Console.WriteLine("\r\nError: {0}", ex.Message);
    }
}

Both of these methods returned the same COM error, so this was getting weird for me. Hmm...

The FTP runtime state is exposed through a COM interface, and that is implemented in a DLL that is named "ftpconfigext.dll". That file should be registered when you install IIS, and I re-registered it on my system just for good measure, but that didn't resolve the issue.

I had a brief conversation with one of my coworkers, Eok Kim, about the error that I was seeing. He also suggested re-registering the DLL, but something else that he said about searching the registry for the InprocServer32 entry made me wonder if the whole problem was related to the bitness of my application.

To make a long story short - that was the whole problem.

Both the customer and I were creating 32-bit .NET applications, and the COM interface for the FTP runtime state is implemented in a 64-bit-only DLL. Once we both changed our projects to compile for 64-bit platforms, we were both able to get the code to run. (Coincidentally, all I had was a 32-bit system when I wrote my original blog, so I probably would have run into this sooner if I had owned a 64-bit system way back then. ;-])

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jun 29 2012, 12:01 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: FTP | IIS
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Using URL Rewrite with QDIG

One of the applications that I like to use on my websites it the Quick Digital Image Gallery (QDIG), which is a simple PHP-based image gallery that has just enough features to be really useful without a lot of work on my part to get it working. (Simple is always better - ;-].) Here's a screenshot of QDIG in action with some Bing photos:

(Click to enlarge photo.)

The trouble is, QDIG creates some really heinous query string lines; see the URL line in the following screenshot for an example:

(Click to enlarge photo.)

I don't know about you, but in today's SEO-friendly world, I hate long and convoluted query strings. Which brings me to one of my favorite subjects: URL Rewrite for IIS

If you've been around IIS for a while, you probably already know that there are a lot of great things that you can do with the IIS URL Rewrite module, and one of the things that URL Rewrite is great at is cleaning up complex query strings into something that's a little more intuitive.

It would take way to long to describe all of the steps to create the following rules with the URL Rewrite interface, so I'll just include the contents of my web.config file for my QDIG directory - which is a physical folder called "QDIG" that is under the root of my website:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <system.webServer>
    <rewrite>
      <rules>
        <!-- Rewrite the inbound URLs into the correct query string. -->
        <rule name="RewriteInboundQdigURLs"  stopProcessing="true">
          <match url="Qif/(.*)/Qiv/(.*)/Qis/(.*)/Qwd/(.*)" />
          <conditions>
            <add input="{REQUEST_FILENAME}"  matchType="IsFile" negate="true" />
            <add input="{REQUEST_FILENAME}"  matchType="IsDirectory" negate="true" />
          </conditions>
          <action type="Rewrite"  url="/QDIG/?Qif={R:1}&amp;Qiv={R:2}&amp;Qis={R:3}&amp;Qwd={R:4}"  appendQueryString="false" />
        </rule>
      </rules>
      <outboundRules>
        <!-- Rewrite the outbound URLs into user-friendly URLs. -->
        <rule name="RewriteOutboundQdigURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="A, Img, Link"  pattern="^(.*)\?Qwd=([^=&amp;]+)&amp;(?:amp;)?Qif=([^=&amp;]+)&amp;(?:amp;)?Qiv=([^=&amp;]+)&amp;(?:amp;)?Qis=([^=&amp;]+)(.*)" />
          <action type="Rewrite"  value="/QDIG/Qif/{R:3}/Qiv/{R:4}/Qis/{R:5}/Qwd/{R:2}" />
        </rule>
        <!-- Rewrite the outbound relative QDIG URLs for the correct path. -->
        <rule name="RewriteOutboundRelativeQdigFileURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="Img"  pattern="^\.\/qdig-files/(.*)$" />
          <action type="Rewrite"  value="/QDIG/qdig-files/{R:1}" />
        </rule>
        <!-- Rewrite the outbound relative file URLs for the correct path. -->
        <rule name="RewriteOutboundRelativeFileURLs"  preCondition="ResponseIsHTML"  enabled="true">
          <match filterByTags="Img"  pattern="^\.\/(.*)$" />
          <action type="Rewrite"  value="/QDIG/{R:1}" />
        </rule>
        <preConditions>
          <!-- Define a precondition so the outbound rules only apply to HTML responses. -->
          <preCondition name="ResponseIsHTML">
            <add input="{RESPONSE_CONTENT_TYPE}"  pattern="^text/html" />
          </preCondition>
        </preConditions>
      </outboundRules>
    </rewrite>
  </system.webServer>
</configuration>

Here's the breakdown of what all of the rules do:

  • RewriteInboundQdigURLs - This rule will rewrite inbound user-friendly URLs into the appropriate query string values that QDIG expects. I should point out that I rearrange the parameters from the way that QDIG would normally define them; more specifically, I pass the value Qwd parameter last, and I do this so that the current directory "." does not get ignored by browsers and break the functionality.
  • RewriteOutboundQdigURLs - This rule will rewrite outbound HTML so that all anchor, link, and image tags are in the new format. This is where I actually rearrange the parameters that I mentioned earlier.
  • RewriteOutboundRelativeQdigFileURLs - There are several files that QDIG creates in the "/qdig-files/" folder of your application; when the application paths are rewritten, you need to make sure that the those paths won't just break. For example, once you have a path that is rewritten as http://localhost/QDIG/Qif/foo.jpg/Qiv/name/Qis/M/Qwd/, the relative paths will seem to be offset from that URL space as though it were a physical path; since it isn't, you'd get HTTP 404 errors throughout your application.
  • RewriteOutboundRelativeFileURLs - This rule is related to the previous rule, although this works for the files in your actual gallery. Since the paths are relative, you need to make sure that they will work in the rewritten URL namespace.
  • ResponseIsHTML - This pre-condition verifies if an outbound response is HTML; this is used by the three outbound rules to make sure that URL Rewrite doesn't try to rewrite responses where it's not warranted.

Once you have these rules in place, you get nice user-friendly URLs in QDIG:

(Click to enlarge photo.)

I should also point out that these rules also support changing the style from thumbnails to file names to file numbers, etc.

(Click to enlarge photo.)

All of that being said, there is one thing that these rules do not support - and that's nested folders under my QDIG application. I don't like to use folders under my QDIG folder - I like to use separate folders with the QDIG file in it, because this makes each gallery self-contained and easily transportable. That being said, after I had written the text for this blog, I tried to use a subfolder under my QDIG application and that didn't work. By looking at what was going on, I'm pretty sure that it would be pretty trivial to write some URL Rewrite rules that would accommodate using subfolders, but that's another project for another day. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: Jun 28 2012, 13:10 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | URL Rewrite
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Microsoft IIS 8.0 Express Release Candidate is Released!

In addition to all of the other great products that Microsoft has released today, I’m happy to announce the release of the Internet Information Services (IIS) 8.0 Express Release Candidate. You can install the IIS 8.0 Express Release Candidate through the Microsoft Download Center by using the following URL:

http://go.microsoft.com/fwlink/?LinkId=254247

IIS 8.0 Express Release Candidate supports the following new features:

X64 Support
IIS 8.0 Express supports both 32-bit and 64-bit architectures.
Shared Features with IIS 8.0
IIS 8.0 Express shares a great number of features with the full version of IIS; for example, IIS 8.0 Express uses many of the same binaries, configuration files, and error codes as the full version of IIS, so migrating projects between your development environment and your production environment should be easier than ever.
Changing the User Home Directory
IIS 8.0 Express supports changing the user's home directory, which is mapped to the %IIS_USER_HOME% variable in configuration. By default this path is located at %UserProfile%\Documents\IISExpress, but users can change this by setting a CustomUserHome registry property in HKCU\Software\Microsoft\IISExpress, or by specifying the "/userhome" parameter when launching iisexpress.exe.
WebSocket Support
IIS 8.0 Express supports writing WebSocket applications. For more information, see the WebSocket protocol specification or the System.Net.WebSockets Namespace topic on Microsoft's MSDN website.
AppCmd Support for editing multiple configuration files
The AppCmd utility now supports a "/AppHostConfig" parameter, which makes it possible to use AppCmd to edit multiple ApplicationHost.config files and not just the primary configuration file.

For more information, see the IIS 8.0 Express ReadMe file.

My thanks to Eok Kim, Jeong Hwan Kim, Yamini Jagadeesan, Wade Hilmo, and Won Yoo!

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/

Posted: May 31 2012, 11:29 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS Express
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 4 - Adding Custom Input Formats

In Part 4 of this series, I'll show you how you can do a couple of cool things:

  • Create a new input format for Log Parser (and I'll use FTP RSCA data as an example)
  • Create charts from your custom input format

For the data source for my custom plug-in, I thought that it would be cool to consume the data from FTP 7's Runtime Status (RSCA). If you've followed some of my old blogs, you would have seen that around five years ago I wrote the following pair of blogs about programmatically viewing FTP 7 sessions:

I'm going to recycle some of the FTP RSCA concepts from those blogs in order to create my COM plug-in.

Log Parser Input Formats

If you're like me, you already realize that the existing features of Log Parser simply rock. But what most people don't realize is that Log Parser lets you extend the functionality by adding new input formats, so you can consume the data from any place where you feel compelled to sit down and write your own Log Parser module.

As a quick reminder, Log Parser supports the following built-in input formats:

  • IIS Log File Input Formats
    • IISW3C: parses IIS log files in the W3C Extended Log File Format.
    • IIS: parses IIS log files in the Microsoft IIS Log File Format.
    • BIN: parses IIS log files in the Centralized Binary Log File Format.
    • IISODBC: returns database records from the tables logged to by IIS when configured to log in the ODBC Log Format.
    • HTTPERR: parses HTTP error log files generated by Http.sys.
    • URLSCAN: parses log files generated by the URLScan IIS filter.
  • Generic Text File Input Formats
    • CSV: parses comma-separated values text files.
    • TSV: parses tab-separated and space-separated values text files.
    • XML: parses XML text files.
    • W3C: parses text files in the W3C Extended Log File Format.
    • NCSA: parses web server log files in the NCSA Common, Combined, and Extended Log File Formats.
    • TEXTLINE: returns lines from generic text files.
    • TEXTWORD: returns words from generic text files.
  • System Information Input Formats
    • EVT: returns events from the Windows Event Log and from Event Log backup files (.evt files).
    • FS: returns information on files and directories.
    • REG: returns information on registry values.
    • ADS: returns information on Active Directory objects.
  • Special-purpose Input Formats
    • NETMON: parses network capture files created by NetMon.
    • ETW: parses Enterprise Tracing for Windows trace log files and live sessions.
    • COM: provides an interface to Custom Input Format COM plug-ins.

This last input format, COM, is how you interface with Log Parser in order to create your own input formats. When you install Log Parser, there are a few COM-based samples in the Log Parser directory, and you can take a look at those when you get the chance.

A Brief Introduction to Creating a COM plug-ins for Log Parser

To start with, your COM plug-in has to support a few public methods - and each of these will be more clear when I create my plug-in later:

Method Name Description
OpenInput Opens your data source and sets up any initial environment settings.
GetFieldCount Returns the number of fields that your plug-in will provide.
GetFieldName Returns the name of a specified field.
GetFieldType Returns the datatype of a specified field.
GetValue Returns the value of a specified field.
ReadRecord Reads the next record from your data source.
CloseInput Closes your data source and cleans up any environment settings.

After you've created and registered your COM plug-in, you will call it by using something like the following syntax:

logparser "SELECT * FROM FOO" -i:COM -iProgID:BAR

In this example, FOO is some data source that makes sense to your plug-in, and BAR is the COM class name for your plug-in.

Creating the COM plug-in for FTP RSCA Data

I'm going to demonstrate how to create a COM component as a scriptlet, and then I'll call that from Log Parser to process the data. I chose to use a scriptlet for this demo because they are quick to design and they're easily portable. Since no compilation is required, updates take place on the fly. All of that being said, if I were writing a real COM plug-in for Log Parser, I would use C# or C++.

To create the sample COM plug-in, copy the following code into a text file, and save that file as "MSUtil.LogQuery.FtpRscaScriptlet.sct" to your computer. (Note: The *.SCT file extension tells Windows that this is a scriptlet file.)

<SCRIPTLET>
  <registration
    Description="FTP RSCA for Log Parser Scriptlet"
    Progid="MSUtil.LogQuery.FtpRscaScriptlet"
    Classid="{4e616d65-6f6e-6d65-6973-526f62657274}"
    Version="1.00"
    Remotable="False" />
  <comment>
  EXAMPLE 1: logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  EXAMPLE 2: logparser "SELECT * FROM 1" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet
  </comment>
  <implements id="Automation" type="Automation">
    <method name="OpenInput">
      <parameter name="strValue"/>
    </method>
    <method name="GetFieldCount" />
    <method name="GetFieldName">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="GetFieldType">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="ReadRecord" />
    <method name="GetValue">
      <parameter name="intFieldIndex"/>
    </method>
    <method name="CloseInput">
      <parameter name="blnAbort"/>
    </method>
  </implements>
  <SCRIPT LANGUAGE="VBScript">

Option Explicit

Dim objAdminManager,objSessionDictionary
Dim objSitesSection,objSitesCollection
Dim objSiteElement,objFtpServerElement
Dim objSessionsElement,objSessionElement
Dim intSiteElementPos,intSession,intRecordIndex
Dim clsSession

intRecordIndex = -1

' --------------------------------------------------------------------------------
' Open an input session that reads FTP RSCA data and stores it in a dictionary object.
' --------------------------------------------------------------------------------

Public Function OpenInput(strValue)
  Set objSessionDictionary = CreateObject("Scripting.Dictionary")
  Set objAdminManager = CreateObject("Microsoft.ApplicationHost.WritableAdminManager")
  objAdminManager.CommitPath = "MACHINE/WEBROOT/APPHOST"
  Set objSitesSection = objAdminManager.GetAdminSection("system.applicationHost/sites", "MACHINE/WEBROOT/APPHOST")
  Set objSitesCollection = objSitesSection.Collection
  If IsNumeric(strValue) Then
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("id", strValue))
  Else
    intSiteElementPos = FindElement(objSitesCollection, "site", Array("name", strValue))
  End If
  If intSiteElementPos > -1 Then
    Set objSiteElement = objSitesCollection.Item(intSiteElementPos)
    Set objFtpServerElement = objSiteElement.ChildElements.Item("ftpServer")
    Set objSessionsElement = objFtpServerElement.ChildElements.Item("sessions").Collection
    For intSession = 0 To CLng(objSessionsElement.Count)-1
       Set objSessionElement = objSessionsElement.Item(intSession)
       Set clsSession = New Session
       clsSession.CurrentDateTime = GetUtcDate()
       clsSession.ClientIp = objSessionElement.GetPropertyByName("clientIp").Value
       clsSession.SessionId = objSessionElement.GetPropertyByName("sessionId").Value
       clsSession.SessionStartTime = objSessionElement.GetPropertyByName("sessionStartTime").Value
       clsSession.UserName = objSessionElement.GetPropertyByName("userName").Value
       clsSession.CurrentCommand = objSessionElement.GetPropertyByName("currentCommand").Value
       clsSession.PreviousCommand = objSessionElement.GetPropertyByName("previousCommand").Value
       clsSession.CommandStartTime = objSessionElement.GetPropertyByName("commandStartTime").Value
       clsSession.BytesSent = objSessionElement.GetPropertyByName("bytesSent").Value
       clsSession.BytesReceived = objSessionElement.GetPropertyByName("bytesReceived").Value
       clsSession.LastErrorStatus = objSessionElement.GetPropertyByName("lastErrorStatus").Value
       objSessionDictionary.Add intSession,clsSession
    Next
  End If
End Function

' --------------------------------------------------------------------------------
' Close the input session.
' --------------------------------------------------------------------------------

Public Function CloseInput(blnAbort)
  intRecordIndex = -1
  objSessionDictionary.RemoveAll
End Function

' --------------------------------------------------------------------------------
' Return the count of fields.
' --------------------------------------------------------------------------------

Public Function GetFieldCount()
  GetFieldCount = 11
End Function

' --------------------------------------------------------------------------------
' Return the specified field's name.
' --------------------------------------------------------------------------------

Public Function GetFieldName(intFieldIndex)
  Select Case intFieldIndex
    Case 0
      GetFieldName = "currentDateTime"
    Case 1
      GetFieldName = "clientIp"
    Case 2
      GetFieldName = "sessionId"
    Case 3
      GetFieldName = "sessionStartTime"
    Case 4
      GetFieldName = "userName"
    Case 5
      GetFieldName = "currentCommand"
    Case 6
      GetFieldName = "previousCommand"
    Case 7
      GetFieldName = "commandStartTime"
    Case 8
      GetFieldName = "bytesSent"
    Case 9
      GetFieldName = "bytesReceived"
    Case 10
      GetFieldName = "lastErrorStatus"
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's type.
' --------------------------------------------------------------------------------

Public Function GetFieldType(intFieldIndex)
  Const TYPE_INTEGER   = 1
  Const TYPE_REAL      = 2
  Const TYPE_STRING    = 3
  Const TYPE_TIMESTAMP = 4
  Const TYPE_NULL      = 5
  
  Select Case intFieldIndex
    Case 0
      GetFieldType = TYPE_STRING
    Case 1
      GetFieldType = TYPE_STRING
    Case 2
      GetFieldType = TYPE_STRING
    Case 3
      GetFieldType = TYPE_STRING
    Case 4
      GetFieldType = TYPE_STRING
    Case 5
      GetFieldType = TYPE_STRING
    Case 6
      GetFieldType = TYPE_STRING
    Case 7
      GetFieldType = TYPE_STRING
    Case 8
      GetFieldType = TYPE_INTEGER
    Case 9
      GetFieldType = TYPE_INTEGER
    Case 10
      GetFieldType = TYPE_INTEGER
  End Select
End Function

' --------------------------------------------------------------------------------
' Return the specified field's value.
' --------------------------------------------------------------------------------

Public Function GetValue(intFieldIndex)
  If objSessionDictionary.Count > 0 Then
    Select Case intFieldIndex
      Case 0
        GetValue = objSessionDictionary(intRecordIndex).CurrentDateTime
      Case 1
        GetValue = objSessionDictionary(intRecordIndex).ClientIp
      Case 2
        GetValue = objSessionDictionary(intRecordIndex).SessionId
      Case 3
        GetValue = objSessionDictionary(intRecordIndex).SessionStartTime
      Case 4
        GetValue = objSessionDictionary(intRecordIndex).UserName
      Case 5
        GetValue = objSessionDictionary(intRecordIndex).CurrentCommand
      Case 6
        GetValue = objSessionDictionary(intRecordIndex).PreviousCommand
      Case 7
        GetValue = objSessionDictionary(intRecordIndex).CommandStartTime
      Case 8
        GetValue = objSessionDictionary(intRecordIndex).BytesSent
      Case 9
        GetValue = objSessionDictionary(intRecordIndex).BytesReceived
      Case 10
        GetValue = objSessionDictionary(intRecordIndex).LastErrorStatus
    End Select
  End If
End Function
  
' --------------------------------------------------------------------------------
' Read the next record, and return true or false if there is more data.
' --------------------------------------------------------------------------------

Public Function ReadRecord()
  If objSessionDictionary.Count > 0 Then
    If intRecordIndex < (objSessionDictionary.Count-1) Then
      intRecordIndex = intRecordIndex + 1
      ReadRecord = True
    Else
      ReadRecord = False
    End If
  End If
End Function

' --------------------------------------------------------------------------------
' Return the current UTC date/time.
' --------------------------------------------------------------------------------

Private Function GetUtcDate()
  Dim dtmNow,dtmUtc,strUtc
  Dim objShell,lngActiveTimeBias
  dtmNow = Now()
  Set objShell = CreateObject("WScript.Shell")
  lngActiveTimeBias = CLng(objShell.RegRead("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation\ActiveTimeBias"))
  dtmUtc = DateAdd("n",lngActiveTimeBias,dtmNow)
  strUtc = Year(dtmUtc) & "-" & _
    Right("0" & Month(dtmUtc),2) & "-" & _
    Right("0" & Day(dtmUtc),2) & "T" & _
    Right("0" & Hour(dtmUtc),2) & ":" & _
    Right("0" & Minute(dtmUtc),2) & ":" & _
    Right("0" & Second(dtmUtc),2) & ".000Z"
  GetUtcDate = strUtc
End Function

' --------------------------------------------------------------------------------
' Return an element's position in a collection.
' --------------------------------------------------------------------------------

Private Function FindElement(objCollection, strElementTagName, arrValuesToMatch)
  Dim i,elem,matches,j,prop,value
  For i = 0 To CInt(objCollection.Count) - 1
    Set elem = objCollection.Item(i)
    If elem.Name = strElementTagName Then
      matches = True
      For j = 0 To UBound(arrValuesToMatch) Step 2
        Set prop = elem.GetPropertyByName(arrValuesToMatch(j))
        value = prop.Value
        If Not IsNull(value) Then
          value = CStr(value)
        End If
        If Not value = CStr(arrValuesToMatch(j + 1)) Then
          matches = False
          Exit For
        End If
      Next
      If matches Then
        Exit For
      End If
    End If
  Next
  If matches Then
    FindElement = i
  Else
    FindElement = -1
  End If
End Function

' --------------------------------------------------------------------------------
' Define a generic class for holding session data.
' --------------------------------------------------------------------------------

Class Session
  Public CurrentDateTime
  Public ClientIp
  Public SessionId
  Public SessionStartTime
  Public UserName
  Public CurrentCommand
  Public PreviousCommand
  Public CommandStartTime
  Public BytesSent
  Public BytesReceived
  Public LastErrorStatus
End Class

  </SCRIPT>

</SCRIPTLET>

After you've saved the scriptlet code to your computer, you will register it by using the following syntax:

regsvr32 MSUtil.LogQuery.FtpRscaScriptlet.sct

At the very minimum, you can now use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

Next, we'll analyze what the scriptlet does.

Examining the COM plug-in in Detail

Here are the different parts of the scriptlet and what they do:

  • The <registration> section of the scriptlet sets up the COM registration information; you'll notice the COM component class name and GUID, as well as version information and a general description.
  • The <implements> section declares the public methods that the COM plug-in has to support.
  • The <script> section contains the actual implementation:
    • The first part of the script section declares the global variables that will be used.
    • The second part of the script contains the required methods:
      • The OpenInput() method opens the FTP RSCA data for a specific FTP site:
        • The site will be specified in your Log Parser query, and the COM plug-in supports using either the site name or the site's numeric ID:
          • "SELECT * FROM ftp.example.com"
          • "SELECT * FROM 1"
        • All of the RSCA data for the site in stored in classes that will be cached in a global dictionary for quick access
      • The  CloseInput() method doesn't do much in this script, but your COM plug-ins may require more clean up depending on your data source.
      • The GetFieldCount() method simply returns the number of data fields in each record of your data.
      • The GetFieldName() method returns the name of a field that is passed to the method as a number.
      • The GetFieldType() method returns the data type of a field that is passed to the method as a number; Log Parser supports the following five data types for COM plug-ins:
        • TYPE_INTEGER
        • TYPE_REAL
        • TYPE_STRING
        • TYPE_TIMESTAMP
        • TYPE_NULL
      • The GetValue() method returns the data value of a field that is passed to the method as a number.
      • The ReadRecord() method moves to the next record in your data set; this method returns True if there is data to read, or False when the end of data is reached.
    • The third part of the script contains some helper features:
      • The GetUtcDate() method returns the current date and time in Universal Coordinated Time (UTC) format.
      • The FindElement() method locates a specified element's position within an IIS collection, or -1 if the element cannot be found. This method is used to determine the specified FTP site within the IIS configuration.
      • The Session class is a generic construct to hold the information for a single FTP RSCA data record.

This wraps up the description of how the scriptlet works as a COM plug-in, in the next part of my blog we'll look at how to actually use it.

Using the COM plug-in with Log Parser

Earlier I showed you how you can use the COM plug-in with Log Parser by using syntax like the following:

logparser "SELECT * FROM ftp.example.com" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet

This will return output that resembles something like the following:

currentDateTime clientIp sessionId sessionStartTime userName currentCommand previousCommand commandStartTime bytesSent bytesReceived lastErrorStatus
---------------- -------- --------- ---------------- -------- -------------- --------------- ---------------- --------- ------------- ---------------
2012-05-25T11:42:11.000Z 10.121.75.26 3950d1e5-3e94-4734-a89a-9768c52aa924 2012-05-25T10:08:09.861Z robert PASS USER 2012-05-25T11:42:06.080Z 6049 1193 0
2012-05-25T11:42:11.000Z 10.121.75.26 d1591fa8-3b09-4afd-b2c0-950421ba79fe 2012-05-25T10:08:18.184Z robert RETR NLST 2012-05-25T11:42:07.172Z 5887 1169 0
2012-05-25T11:42:11.000Z 10.121.75.26 0f92b5ed-920a-441d-a15d-39056a36f2a4 2012-05-25T10:08:22.327Z robert NOOP NLST 2012-05-25T11:41:40.917Z 5857 1163 0
2012-05-25T11:42:11.000Z 10.121.75.26 16925f0d-1fc5-4cb7-be19-ab33face2da9 2012-05-25T10:08:48.756Z   NLST SYST 2012-05-25T11:41:44.770Z 6026 1192 0
2012-05-25T11:42:11.000Z 10.121.75.26 aeb68389-869b-4afc-8c81-47b578e74824 2012-05-25T10:08:54.214Z   USER HOST 2012-05-25T11:41:42.087Z 5864 1168 0
2012-05-25T11:42:11.000Z 10.121.75.26 4ed55569-ee25-47d1-8388-12cdb90a1c07 2012-05-25T10:12:31.555Z alice RETR NLST 2012-05-25T11:42:01.789Z 5780 1138 0
2012-05-25T11:42:11.000Z 10.121.75.26 d6b16bb4-cb65-492d-a9fa-fbd6b72de0f3 2012-05-25T10:12:54.591Z bob NOOP NLST 2012-05-25T11:41:46.563Z 5748 1130 0
                     
Statistics:                    
-----------                    
Elements processed: 7                  
Elements output: 7                  
Execution time: 0.12 seconds                  

That information is something of a jumbled mess, and we can clean that up a bit by simply choosing the fields that we might be interested in:

userName currentCommand commandStartTime
-------- -------------- ----------------
robert PASS 2012-05-25T11:42:06.080Z
robert RETR 2012-05-25T11:42:07.172Z
robert NOOP 2012-05-25T11:41:40.917Z
  NLST 2012-05-25T11:41:44.770Z
  USER 2012-05-25T11:41:42.087Z
alice RETR 2012-05-25T11:42:01.789Z
bob NOOP 2012-05-25T11:41:46.563Z
     
Statistics:    
-----------    
Elements processed: 7  
Elements output: 7  
Execution time: 0.12 seconds  

Now let's look at some interesting data - one of the main focuses for this blog series is charting  with Log Parser, so let's look at doing something useful with the data. To start with, here's how to create a pie chart that counts the number of sessions by user name:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"User Sessions"

This will generate a chart like the following:

Here's a variation on that script that illustrates how to create a pie chart that counts the number of authenticated sessions versus anonymous sessions:

logparser "SELECT CASE UserName WHEN '' THEN 'Anonymous' ELSE 'Authenticated' END AS AuthType,COUNT(*) AS Sessions INTO AuthenticatedSessions.gif FROM 1 GROUP BY AuthType" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:PieExploded -o:CHART -values:on -categories:off -legend:on -chartTitle:"Authenticated Sessions"

This will generate a chart like the following:

We can also do line, bar, and column charts with the data:

logparser "SELECT CASE UserName WHEN '' THEN 'anonymous' ELSE TO_LOWERCASE(UserName) END AS User,COUNT(*) AS Sessions INTO SessionCountByUser.gif FROM 1 GROUP BY User" -i:COM -iProgID:MSUtil.LogQuery.FtpRscaScriptlet -chartType:Column3D -o:CHART -values:on -legend:off -chartTitle:"User Sessions"

The above code sample will generate a chart like the following:

There's a lot more that we could do with this, but eventually I have to get some sleep, so I think that's enough fun for the day.

Summary

In this blog post, I've shown you how to add your own custom input format to Log Parser by creating scriptlet as a COM plug-in. I hope that you take this information and create some great Log Parser plug-ins of your own.

;-]

Posted: May 24 2012, 22:47 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | LogParser
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us

Advanced Log Parser Charts Part 3 - Missing Office Web Components for Charting

In Part 3 of this series, I'll explain what to do when you're missing the Office Web Components that are required for creating the charts that I have been demonstrating in this series.

Here's a brief explanation of the symptoms: you try a simple query that will create a chart like the following example:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

And you get the following error message:

Error creating output format "CHART": This output format requires a licensed Microsoft Office Chart Web Component to be installed on the local machine

More often than not, this simply means that you have Office 2007 or Office 2010, which do not contain the Office Web Components that are used by Log Parser to create charts. Fortunately, you can download the missing components from the following URL on Microsoft's website:

Office 2003 Add-in: Office Web Components
http://www.microsoft.com/en-us/download/details.aspx?id=22276

When you run the installation, you will see the following license agreement:

When you check the box to accept the license agreement and click Install, you will eventually receive the following dialog box to let you know that the Office 2003 Web Components have been installed:

Once you have the Office 2003 Web Components installed, you can run the same query successfully:

logparser.exe "SELECT Date,COUNT(*) AS Hits INTO HitsByDay.gif FROM *.log GROUP BY Date ORDER BY Date" -i:W3C -o:CHART -chartType:Line -legend:off -chartTitle:"Hits By Day"

Statistics:
-----------
Elements processed: 123330
Elements output: 14
Execution time: 0.57 seconds

Note: The above query generates the following somewhat uninteresting chart:

That being said, the point of this blog was to let you know how to get charting back, not how to make pretty charts. I'll save pretty charts for a future blog. ;-]

Note: This blog was originally posted at http://blogs.msdn.com/robert_mcmurray/
Posted: May 24 2012, 19:40 by Bob | Comments (0)
  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Filed under: IIS | LogParser
Social Bookmarks: E-mail | Kick it! | DZone it! | del.icio.us