David Foderick's Blog - OnMaterialize()

LINQ/DLINQ release available for download

An updated version of LINQ (including DLINQ) for Whidbey (Visual Studio 2005) RTM is avilable for download from Microsoft.

LINQ is the Language Integrated Query which provides a syntax for querying over collections and database tables with a SQL-like syntax integrated right into your favorite .NET language. DLINQ is the "lightweight" ORM (Object-Relational Mapper) that will translate your query into a database query to return data back to your application from the database.

A high resolution timer utility

Have you ever needed to measure the performance of your code? One simple way is to get the current time (DateTime.Now) at the start and at the end of the test and get the difference, giving you a TimeSpan object. The TimeSpan class has a Milliseconds property that gives you the elapsed time in milliseconds. What if your measurement is smaller than milliseconds? Is there anything that can give you micro or nanosecond accuracy?

It turns out there are Win32 calls that can give you nanosecond (or at least microsecond) accuracy. The methods are called QueryPerformanceCounter and QueryPerformanceFrequency. To use these, call QueryPerformanceCounter at the beginning and at the end of your test. Divide the difference by the QueryPerformanceFrequency (that's your sampling rate). The result will be the number of seconds elapsed.

I've culled the internet for the best sources of information and distilled the findings into a reusable class that you can easily use in your code. Note the following interesting points about the code:

• The class calls the Win32 routines through interop. .NET 2.0 or future versions of .NET may have these calls in a regular .NET namespace.
• I call the Start and Stop methods in the constructor. This is to ensure that the methods are jitted by .NET.
• There is an overhead in calling the Start and Stop methods. This difference is somewhat accounted for when calculating the difference.
• Because of overhead and the fact that your machine does multiprocessing, you will never get the full accuracy of the resolution. The best resolution available to .NET programmers on Windows is probably in the low microsecond range. Do your own testing! I wouldn't trust any numbers smaller than that.

Here is the class:

using System;

namespace QueryPerfCounter
{
 using System;
 using System.ComponentModel;
 using System.Runtime.InteropServices;

 // A class for performance counting
 public class QueryPerfCounter
 {
  [DllImport("KERNEL32")]
  private static extern bool QueryPerformanceCounter(
   out long lpPerformanceCount);
  [DllImport("Kernel32.dll")]
  private static extern bool QueryPerformanceFrequency(out long lpFrequency);

  private long start;
  private long stop;
  //Seconds to nanoseconds multiplier. There are 10e9 nanoseconds/sec
  Decimal multiplier = new Decimal(1.0e9);
  private long frequency;

  #region Properties
  /// <summary>
  /// Gets the number of samples per second
  /// </summary>
  public long Frequency
  {
   get { return frequency; }
  }

  //Best possible resolution that this class is capable of
  public double Resolution
  {
   get { return (1.0/frequency) * (double)multiplier; }
  }

  // Total elapsed time for the test (including all iterations)
  public double TotalElapsedTimeInSeconds
  {
   get
   {
    return (Duration() / (double)frequency);
   }
  }

  //returns total time in nanoseconds of the test
  public double TotalElapsedTimeInNanoseconds
  {
   get 
   {
    return ((Duration() * (double) multiplier) / (double)frequency);
   }
  }
  #endregion

  #region Constructor
  public QueryPerfCounter()
  {
   if (QueryPerformanceFrequency(out frequency) == false)
   {
    // Frequency not supported
    throw new Win32Exception();
   }

   //Call methods so they are jitted!
   Start();
   Stop();
  }
  #endregion

  #region Methods
  //Start the timer
  public void Start()
  {
   QueryPerformanceCounter(out start);
  }

  //End the timer
  public void Stop()
  {
   QueryPerformanceCounter(out stop);
  }

  // returns time in nanoseconds
  public double TimePerIterationInNanoseconds(int iterations)
  {
   return (((Duration() * (double) multiplier) / (double)frequency)/iterations);
  }

  // returns 
  public double TimePerIterationInSeconds(int iterations)
  {
   return ((Duration() * (double) multiplier)/iterations);
  }

  // total duration of the test (in sampling units)
  private double Duration()
  {
   return (double)(stop - start - Overhead());
  }

  // calculate overhead of starting/stoping
  private long Overhead()
  {
   long begin;
   long end;
   QueryPerformanceCounter(out begin);
   QueryPerformanceCounter(out end);
   return (end - begin);
  }
  #endregion
 }
}

In general, you will use it like this:

QueryPerfCounter.QueryPerfCounter myTimer = new QueryPerfCounter.QueryPerfCounter();
myTimer.Start();
RunYourMethod();
myTimer.Stop();
delay = myTimer.TimePerIterationInNanoseconds(1);

And here is how to use it to measure the round trip time to your database (including the time to walk up the SQL Server protocol stack and run a query):

  static void Main(string[] args)
  {

   using (SqlConnection conn = new SqlConnection("..."))
   {
    int iter = 10;
    Console.WriteLine(string.Format("{0} iterations",iter));
    double delay = 0;
    QueryPerfCounter.QueryPerfCounter myTimer = new QueryPerfCounter.QueryPerfCounter();
    SqlCommand cmd = new SqlCommand("select GetDate()",conn);
    conn.Open();
    OneTripToDB(conn,cmd); // throw away the first round trip
    myTimer.Start();
    for (int i = 0; i < iter; i++)
    {
     OneTripToDB(conn,cmd);
    }
    myTimer.Stop();
    delay = myTimer.TimePerIterationInNanoseconds(iter);
    conn.Close();
    Console.WriteLine(String.Format("Resolution: {0}",myTimer.Resolution));
    Console.WriteLine(String.Format("Total  Sec: {0}",myTimer.TotalElapsedTimeInSeconds));
    Console.WriteLine(String.Format("Nanoseconds: {0}",delay));
    Console.WriteLine(String.Format("Milliseconds: {0}",delay/(double)(new Decimal(1.0e6))));
   }
  }

  private static void OneTripToDB(SqlConnection conn, SqlCommand cmd)
  {
//   System.Threading.Thread.Sleep(1);
   SqlDataReader dr = cmd.ExecuteReader();
   while (dr.Read())
   {
   }
   dr.Close();
  }

Finally, I would like to acknowledge the source of this information:
How To Use QueryPerformanceCounter to Time Code
How To: Time Managed Code Using QueryPerformanceCounter and QueryPerformanceFrequency
High-Performance Timer in C#

Object-Relational Mapping: Introductory article

Vanatec, a spin-off from Versant, has published my article (Object-Relational Mapping - First Steps) in their community support area.

The article was written for the Versant version, but apparently the Vanatec version hasn't changed that much yet. I expect to see some more significant changes to their product for Visual Studio 2005 that may date the article.

Nevertheless, the primary message or the article is that business logic should not be muddied with data access logic and that principle holds for any system that you are building.

Beta Exam 70-528 Study links

Here are the resources I am studying for the Microsoft ASP.NET 2.0 Web exam.

Skills measured by exam 70-528
Creating and Programming a Web Application
Create and configure a Web application.
Create a new Web application.
Add Web Forms pages to a Web application.
          
Add and configure Web server controls.
Add Web server controls to a Web Form.
Configure the properties of Web server controls programmatically.
Configure Web server control properties by using the Microsoft Visual Studio Property Editor.
Specify whether events of a control cause a Web Form to post to the server.
Configure a control to receive postback events.
Access controls in Web Forms pages when working with naming containers and child controls.
Create HTML server controls in the designer.
Set HTML server control properties programmatically.
Use HTML server controls to programmatically access HTML tags.
Create HTML controls as elements in an HTML document.
Use the AdRotator Web server control to manage banners and pop-up windows.
Use the Button Web server control to send a command to the server when a button is clicked.
Display a calendar on a Web page by using the Calendar Web server control.
Implement the CheckBox Web server control.
Implement the FileUpload Web server control.
Create and manipulate links on a Web Form by using the HyperLink Web server control.
Display an image on a Web Form by using the Image Web server control.
Implement a button on a Web Form by using the ImageButton Web server control.
Define hotspot regions within an image by using the ImageMap Web server control.
Use the Label Web server control to display customized text on a Web page.
Display a hyperlink style button on a Web Form by using the LinkButton Web server control.
Display lists of information by using controls that derive from the ListControl class.
Create a Web Form with static text by using the Literal Web server control.
Implement pagination for controls on a page by using the Pager Web server control.
Use the Panel Web server control to arrange controls in groups on a page.
Create a container for a group of View controls by using the MultiView Web server control.
Use the View Web server control to create a Web application.
Create a mutually exclusive set of choices by using the RadioButton Web server control.
Construct a table by using the Table, TableRow, and TableCell Web server controls.
Enable users to type information into a Web Form by using the TextBox Web server control.
Create a wizard by using the Wizard Web server control to collect data through multiple steps of a process.
Use the XML Web server control to create XML data at the location of the control.
Customize the appearance of Web server controls by using Web control templates.
Programmatically edit settings in a Web site's configuration file.
Dynamically add Web server controls to a Web Forms page.
          
Create event handlers for pages and controls.
Create event handlers for a page or control at design time.
Respond to application and session events.
          
Manage state and application data.
Manage state of an application by using client-based state management options.
Manage state of an application by using server-based state management options.
Maintain state of an application by using database technology.
          
Implement globalization and accessibility.            
Implement site navigation and input validation.
Use the SiteMap Web server control to display a representation of a Web site's navigation structure.
Use validation controls to perform Web Forms validation.
Validate against values in a database for server controls by using a CustomValidator control.
Create a CustomValidator control and tie it to a custom function.
Test programmatically whether a user's input passed validation before running code.
Specify the location of a validation error message for server controls.
Format validation error messages for server controls.
Specify the layout for in-place messages on server controls.
Disable validation for server controls.
Display custom error messages for server controls.
Validate server controls programmatically.
            
Write an ASP.NET handler to generate images dynamically for display on a Web page.           
Configure settings for a Web application.
Configure system-wide settings in the Machine.config file.
Configure settings for a Web application in the Web.config file.
Manage a Web application's configuration by using the Web Site Administration Tool.
        
Program a Web application.
Redirect users to another Web page by using a server-side method.
Detect browser types in Web Forms.
Ascertain the cause of an unhandled exception at the page level.
Programmatically access the header of a Web page.
Implement cross-page postbacks.
Assign focus to a control on a page when the page is displayed.
Avoid performing unnecessary processing on a round trip by using a page's IsPostBack property.
Access encapsulated page and application context.
Avoid unnecessary client-side redirection by using the HttpServerUtility.Transfer method.
Avoid round trips by using client-side scripts.
Use a page's Async attribute to create a page that has built-in asynchronous capabilities.
Use a page's Async attribute to create a page that has built-in asynchronous capabilities.
Convert HTML server controls to HTML elements.
        
Integrating Data in a Web Application by Using ADO.NET, XML, and Data-Bound Controls
Implement data-bound controls.
Use tabular data source controls to return tabular data.
Use hierarchical data source controls to display hierarchical data.
Display data by using simple data-bound controls.
Display data by using composite data-bound controls.
Display data by using hierarchical data-bound controls.
Use the FormView control to display the values of a single table record from a data source.
          
Manage connections and transactions of databases.
Configure a connection to a database graphically by using the Connection Wizard.
Configure a connection by using Server Explorer.
Configure a connection to a database by using the connection class.
Connect to a database by using specific database connection objects.
Enumerate through instances of Microsoft SQL Server by using the DbProviderFactories.GetFactoryClasses method.
Enumerate through instances of Microsoft SQL Server by using the DbProviderFactories.GetFactoryClasses method.
Enumerate through instances of Microsoft SQL Server by using the DbProviderFactories.GetFactoryClasses method.
 
Open a connection by using the Open method of a connection object.
Close a connection by using the connection object.
Secure a connection to protect access to your data source.
Create a connection designed for reuse in a connection pool.
Control connection pooling by configuring ConnectionString values based on database type.
Use connection events to detect database information.
Handle connection exceptions when connecting to a database.
Perform transactions by using the ADO.NET Transaction object.
          
Create, delete, and edit data in a connected environment.
Retrieve data by using a DataReader object.
Build SQL commands visually in Server Explorer.
Build SQL commands in code.
Create parameters for a command object.
Perform database operations by using a command object.
Retrieve data from a database by using a command object.
Perform asynchronous operations by using a command object.
Perform asynchronous operations by using a command object.
Perform bulk copy operations to copy data to a SQL Server computer.
Perform bulk copy operations to copy data to a SQL Server computer.
Store and retrieve binary large object (BLOB) data types in a database.
        
Create, delete, and edit data in a disconnected environment.
Create an instance of the DataSet class programmatically.
Create a DataSet graphically.
Create a DataSet programmatically.
Add a DataTable to a DataSet.
Add a relationship between tables.
Navigate a relationship between tables.
Merge DataSet contents.
Copy DataSet contents.
Create a strongly typed DataSet.
Create DataTables.
Manage data within a DataTable.
Create and use DataViews.
Represent data in a DataSet by using XML.
Access an ADO Recordset or Record by using the OleDbDataAdapter object.
Generate DataAdapter commands automatically by using the CommandBuilder object.
Generate DataAdapter commands programmatically.
Populate a DataSet by using a DataAdapter.
Update a database by using a DataAdapter.
Resolve conflicts between a DataSet and a database by using the DataAdapter.
Respond to changes made to data at the data source by using DataAdapter events.
Perform batch operations by using DataAdapters. 
 
Manage XML data with the XML Document Object Model (DOM).
Read XML data into the DOM by using the Load method.
Modify an XML document by adding and removing nodes.
Modify nodes in an XML document.
Write data in XML format from the DOM.
Work with nodes in the XML DOM by using XmlNamedNodeMap and the XmlNodeList.
Work with nodes in the XML DOM by using XmlNamedNodeMap and the XmlNodeList.
Work with nodes in the XML DOM by using XmlNamedNodeMap and the XmlNodeList.
Work with nodes in the XML DOM by using XmlNamedNodeMap and the XmlNodeList.
Work with nodes in the XML DOM by using XmlNamedNodeMap and the XmlNodeList.
Handle DOM events.
Modify XML declaration.
           
Read and write XML data by using the XmlReader and XmlWriter.
Read XML data by using the XmlReader.
Read all XML element and attribute content.
Read specific element and attribute content.
Read XML data by using the XmlTextReader class.
Read node trees by using the XmlNodeReader.
Validate XML data by using the XmlValidatingReader.
Write XML data by using the XmlWriter. 

Creating Custom Web Controls
Creating Custom Web Controls
Create a composite Web application control.
Create a user control.
Convert a Web Forms page to a user control.
Include a user control in a Web Forms page.
Manipulate user control properties.
Handle user control events within the user control code-declaration block or code-behind file.
Create instances of user controls programmatically.
Develop user controls in a code-behind file.
Create a templated user control.
          
Create a custom Web control that inherits from the WebControl class.
Create a custom Web control.
Add a custom Web control to the Toolbox.
Individualize a custom Web control.
Create a custom designer for a custom Web control.
          
Create a composite server control.
Create a base class for composite controls.
Create a composite control.
          
Develop a templated control.
Create a templated control.
Develop a templated data-bound control.
          
Tracing, Configuring, and Deploying Applications
Use a Web setup project to deploy a Web application to a target server.
Create a Web setup project.
Configure deployment properties for a Web setup project.
Install a Web application on a target server.
           
Copy a Web application to a target server by using the Copy Web tool.          
Precompile a Web application by using the Publish Web utility.
Optimize and troubleshoot a Web application.
Customize event-level analysis by using the ASP.NET health-monitoring API.
Use performance counters to track the execution of an application.
Troubleshoot a Web application by using ASP.NET tracing.
Optimize performance by using the ASP.NET Cache object.
           
Customizing and Personalizing a Web Application 
Implement a consistent page design by using master pages.
Implement a consistent page design by using master pages.
Create a master page.
Add a ContentPlaceHolder control to a master page.
Specify default content for a ContentPlaceHolder.
Reference external resources in a master page.
Define the content of a particular page in a content page.
Create a content page.
Add content to a content page.
Reference a master page member from a content page.
Handle events when using master pages.
Create a nested master page.
Change master pages dynamically.
          
Customize a Web page by using themes and user profiles.
Customize a Web page by using themes and user profiles.
Apply a theme declaratively.
Apply a theme programmatically.
Apply a user-selected theme programmatically.
Define custom themes.
Define the appearance of a control by using skins.
Enable users to personalize an application by using Web Parts.
Track and store user-specific information by using user profiles.
Personalize a Web page by dynamically adding or removing child controls in a Placeholder control at run time.
          
Implement Web Parts in a Web application.
Track and coordinate all Web Parts controls on a page by adding a WebPartManager control.
Connect Web Parts to each other by using connection objects.
Divide a page that uses Web Parts into zones by using WebPartZones.
Present a list of available Web Parts controls to users by using CatalogPart controls.
Enable users to edit and personalize Web Parts controls on a page by using EditorPart controls.
           
Implementing Authentication and Authorization
Establish a user's identity by using forms authentication.
Configure forms authentication for a Web application by using a configuration file.
Enable cookieless forms authentication by setting the cookieless attribute.
Use membership APIs and the Membership class to manage users.
Enable anonymous identification.
           
Use authorization to establish the rights of an authenticated user.
Manage roles in the Web Site Administration Tool.
Ascertain whether a specific user is in role.
Get the roles for a specific user by using the Roles object or the User object.
Store role information in a cookie.
Restrict access to files by using file authorization.
Restrict access to portions of an application by using URL authorization.
           
Implement Microsoft Windows authentication and impersonation.
Establish a user's identity by using Windows authentication.
Use impersonation to control access to resources.
 
Use login controls to control access to a Web application.
Use the Login Web server control.
Use the LoginView Web server control to view a user's login status.
Use the PasswordRecovery Web server control to allow a user to recover a password.
Use the LoginStatus Web server control to display either a login or logout link.
Use the LoginName Web server control to display a user's login name on a Web page.
Use the CreateUserWizard Web server control as a UI for creating new Web application user accounts.
Use the ChangePassword Web server control to allow users to change their passwords.
Specify the membership provider used for logging on.
Configure a mail server so that login controls can be used to send e-mail messages to users.
           
Creating ASP.NET Mobile Web Applications
Create a mobile Web application project.           
Use device-specific rendering to display controls on a variety of devices.           
Use adaptive rendering to modify the appearance of Web server controls.            
Use the mobile Web controls to display content on a device.

How to pass a Microsoft Beta Exam

How to target your study
The exam data is fast approaching and you only have an hour or two each night to study. How should you target your efforts? Easy. Microsoft tells you exactly what is going to be on the exam! For each exam you should go through the published “Skills to be measured” bullet point. Read an article or two (or as many as you can get your hands on) for each item. Yeah, it’s a lot of reading. But usually you can skim the articles and just pick on the major points of each article. If you have time, try out the code samples.

• Get a good night’s sleep before the exam. Do not try to take the exam after a full day’s work. Your brain will be fried. I like to take the exam on the weekends early in the morning.
• Ask for a quiet spot. As the exam monitor is setting you up don’t be shy to request another machine in a location that is more private. Stay away from the sickies – the people who are coughing and fidgeting. They will only interrupt your concentration. Also, don’t be afraid to switch chairs or computer equipment. Nothing is worse than trying to write the exam with a sticky mouse!
• This one may surprise you. You don’t have to know the correct answer in order to answer the question correctly! In other words, be an intelligent guesser. If the question has four possible answers and you make a random guess you have a 25% chance. If you can, use your knowledge to eliminate one answer that is definitely wrong then you have 33% chance. Finally, if you can eliminate two obviously wrong answers then you have a 50% change of getting it correct. You will increase your chances of passing the exam dramatically.
• Pick the best answer, not the correct answer. All the answers may solve the problem posed by the question but only one will be the best answer. Make sure you read all the answers before choosing your answer.
• Use the pad and paper! If it helps you to make a diagram with little arrows on it, or boxes or circles then do it. I am a visual person and I can think better about a problem when I can draw in up in a diagram.
• Keep track of your time! For a 90 minute exam with 45 questions on it that will give you approximately 2 minutes per question. At 30 minutes you should be at least through question 15. At 60 minutes you should be at least through question 30. Leave yourself enough time at the end to go back the recheck the questions that you were not sure about the first pass through the exam.

Myth Busters: Business Objects

Above all, there is one extremely pernicious misconception about Business Objects. This misconception pervades online architecture forums. I even see it in my conversations with other developers whose architectural and OOP skills I admire immensely. Its conclusion is so simple and direct that it is unquestioned. It is the misconception that Business Objects should implement their own data access - that is, Business Objects should have Save and Load methods on them.

The first lesson of object-oriented programming is that objects should encapsulate state and behavior. In other words, objects should be responsible for their own internal state by protecting it with public methods. A client may only change the state of an object through those well defined entry points - the public interface.

It is this principle that leads developers to assume that business objects should have Load and Save methods on them. A load method retrieves state from the database and "materializes" the business object by populating all the persistent fields. A save method persists the changes in the state of the business object while it is in memory and saves it to the database. It would seem natural that a persistent object, as the protector of its own state, would know how to load and save itself from the database.

But this premise leads to some immediate problems. How can you load an object if the object doesn't already exist? There is no object yet in memory to call Load on! Simple, you create a static method on the class or some factory for each class that creates instances. Goodbye encapsulation. Now you have static and instance methods on your classes. Or you have factories for each persistent class that have to be maintained. Or better yet, you just new up the object and then call load with the primary key from the database. Now how do you do polymorphism?

Then there is the problem of loading collections. To have any hope of a scalable solution you have to be able to retrieve a set of records from the database to load a collection otherwise you would have to make a trip to the database for each item in the collection (the N+1 problem). So many developers give their collections the ability to load all the child items in the collection. So whose responsibility is it to load the children? Is it the children or the parent or the collection? In a class that is supposed to have pure business logic there is this additional aspect of how to deal with loading and saving instances. The persistence aspect dirties and muddles the mind of the business logic programmer.

Suddenly they realize they are caught in the object-relational impedence mismatch. They backtrack. They think some OOP principles have to be abandoned. They violate every other OOP principle but cling to the misguided notion that their business objects must have Save and Load methods!

But let's re-examine this premise that has led us into this quagmire. What is this data access doing on the business object. It is taking field values and storing them in the database or it is populating field values with data from the database. Its just moving bits in and out of memory from the database. Is this something that just your application needs or is it something that every business application needs? Every app that uses persistent business objects needs this type of data access logic. The inescapable conclusion is that data access for business objects (object persistence) is infrastructure logic! It is to be handled at the very lowest level of your solution, preferably in a way that can be reused from project to project.

So what? Why am I making such a big deal about classifying the data access? It is because you are designing a Business Object. On a Business Object you expect to find business logic, not infrastructure. There is another object-oriented principle called cohesion. Cohesion is that idea that methods on an object are functionaly related. When you mix business logic and infrastucture logic, your objects are no longer cohesive! Your business logic is coupled with infrastucture. If you change your infrasturcture then your business object will have to change. This is a bad thing. A misguided attempt at following the principle of encapsulation has lead to an unencapsulated, non-cohesive mess. Do not put Load and Save methods on your business objects! Completely divorce your business and infrastucture logic.

How? Any proper Object Relational Mapping solution has gone out of its way to help you separate business and data access logic. The persistence aspect of objects is handled by the mapper. Any vendor that forces you to write MyObject.Save is doing you a disservice. If you are writing your own home grown mapper then you no longer have any excuses. Put responsibility for persistence where it belongs, with the persistence manager.

My favorite source for properly defining responsibilites for my application layers is Domain Driven Design by Eric Evans.

Removing Performance counters

Performance Counters are a great way to instument your .NET application. Sometimes while adding your custom performance counters you may want to remove them without custom programming. Or, sometimes Windows tries to be helpful by putting a (1) after the counter instance which causes your counter not to get tracked.

If you ever need to remove performance counters there is a command line utility called unlodctr just for this purpose. In you command line, run

unlodctr "YourPerformanceCounterHere"

By the way, if you ever loose your CLR performance counters, you can reinstall them with these commands

unlodctr .NETFramework
lodctr c:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\corperfmonsymbols.ini

Thanks to Mike Dodd's blog post for showing me how!