• | Table of Contents
A First Look at ADO.NET and System.Xml v. 2.0
By Alex Homer, Dave Sussman, Mark Fussell
Publisher | : Addison Wesley
Pub Date | : October 22, 2003
ISBN | : 0-321-22839-1
Pages | : 320

Version 2.0 of the .NET Framework will offer powerful enhancements to ADO.NET that will give application and service developers unprecedented control over their data. In A First Look at ADO.NET and System.Xml v. 2.0, Microsoft's lead program manager on XML technologies joins with two leading .NET and XML experts to present a comprehensive preview of tomorrow's ADO.NET and System.Xml classes.

Drawing on the first .NET Framework 2.0 Technology Preview Release, the authors introduce powerful new techniques for simplifying development, enhancing flexibility, and leveraging .NET's new support for emerging XML standards. They also preview important new synergies between .NET Framework 2.0, XML, and the next version of SQL Server (code-named "Yukon").

This book's detailed coverage includes:

  • Clear explanations of Microsoft's data-access intentions and directions-so you'll be ready when .NET 2.0 arrives

  • A brief overview of ObjectSpaces--the exciting new technology for using objects that represent and hold their own data

  • Major improvements in bulk loading, batch execution, and paging Support for truly asynchronous connection and command operations

  • In-process server-side cursors for programmatic data processing within stored procedures

  • XML class improvements that enhance standards support, promote integration, and maximize performance

  • Beyond SQLXML 3.0: leveraging XQuery and XML Views in distributed query processing

  • Better support for storing XML data and integrating relational and XML data management

  • "Yukon's" new XML data type: using SQL Server as an XML document store

Whether or not you're already developing with ADO.NET, this book brings together all the insights, best practices, and sample code you'll need to get a running start with ADO.NET 2.0 and the System.Xml v. 2.0 classes.


• | Table of Contents
A First Look at ADO.NET and System.Xml v. 2.0
By Alex Homer, Dave Sussman, Mark Fussell
Publisher | : Addison Wesley
Pub Date | : October 22, 2003
ISBN | : 0-321-22839-1
Pages | : 320

Copyright
Figures
Tables
Foreword
Microsoft .NET Development Series
 | Titles in the Series
Chapter 1. ADO.NET and System.Xml Version 2.0
 | The Version 2.0 Technology Preview Release
 | Summary of New Features in Version 2.0
 | The Growing Importance of XML
 | Relational Data Binding in ASP.NET
 | A Roadmap for This Book
 | SUMMARY
Chapter 2. Bulk Loading, Batch Execution, and Paging
 | The New Bulk Data Copy Features
 | The New Batched Update Features
 | Data Paging with the Command Object
 | Retrieving Connection Statistics
 | SUMMARY
Chapter 3. MARS, Asynchronous Commands, and ObjectSpaces
 | Expanded and Improved Data Access Features
 | MARS in Action
 | Asynchronous Command Execution
 | An Overview of ObjectSpaces
 | SUMMARY
Chapter 4. ADO.NET and SQL Server "Yukon"
 | SQL Server Query Notifications
 | SQL Server User-Defined Types
 | Managed Code, Server-Side Cursors, and Result Sets
 | SUMMARY
Chapter 5. New Features of System.Xml
 | Why Should I Care about XML?
 | System.Xml Version 1.x
 | The Quest for Universal Data Access
 | Which Programming Data Model: Relational, Object, or XML?
 | The Changing XML Landscape
 | What's New in System.Xml Version 2.0
 | SUMMARY
Chapter 6. The XPathDocument2 Class
 | Creating and Updating the XPathDocument2 Class
 | An Overview of the XPathDocument2 Class
 | Properties and Methods for the XPathDocument2 Class and Dependent Classes
 | The XPathEditor and XPathChangeNavigator Classes
 | Using the XPathDocument2 Class
 | SUMMARY
Chapter 7. The XmlAdapter and SqlXml Classes
 | The XmlAdapter Class and XML Views
 | Updating SQL Server with the XmlAdapter Class
 | Migrating from SQLXML 3.0 to the .NET Framework
 | XML Data Source Controls and Data Binding
 | SUMMARY
Chapter 8. XQuery and SQL Server
 | A Very Brief Overview of the XQuery Language
 | The XQueryProcessor Class
 | The XsltProcessor Class
 | The Common Query Architecture
 | SQL Server as an XML Database
 | SUMMARY


Copyright

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Addison-Wesley was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals.

The .NET logo is either a registered trademark or trademark of Microsoft Corporation in the United States and/or other countries and is used under license from Microsoft.

The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

The publisher offers discounts on this book when ordered in quantity for bulk purchases and special sales. For more information, please contact:

U.S. Corporate and Government Sales

(800) 382-3419

corpsales@pearsontechgroup.com

For sales outside of the U.S., please contact:

International Sales

(317) 581-3793

international@pearsontechgroup.com

Visit Addison-Wesley on the Web:

www.awprofessional.com

Library of Congress Cataloging-in-Publication Data

Homer, Alex.

A first look at ADO.NET and system.XML v. 2.0 / Alex Homer, Dave Sussman, Mark Fussell.

p. cm.

Includes bibliographical references and index.

ISBN 0-321-22839-1

1. Internet programming. 2. ActiveX. 3.

XML (Document markup language) I. Sussman, David. II. Fussell, Mark. III. Title.

QA76.625.H655 2003

006.7'6—dc22

2003066460

Copyright © 2004 by Microsoft Corporation

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. Published simultaneously in Canada.

For information on obtaining permission for use of material from this work, please submit a written request to:

Pearson Education, Inc.

Rights and Contracts Department

75 Arlington Street, Suite 300

Boston, MA 02116

Fax: (617) 848-7047

Text printed on recycled paper

1 2 3 4 5 6 7 8 9 10—CRS—0706050403

First printing, October 2003


Figures

Figure 1.1. Access to and conversion of relational data as XML

Figure 1.2. Serialization of XML data over a network

Figure 1.3. A data source and GridView control in action

Figure 2.1. A simple bulk copy operation with a single table

Figure 2.2. A multiple-table bulk copy operation

Figure 2.3. Using batch commands with a DataSet and DataAdapter

Figure 2.4. Specifying different batch sizes

Figure 2.5. A batch size of 1, as shown in SQL Profiler

Figure 2.6. A batch size larger than 1, as shown in SQL Profiler

Figure 2.7. Data paging with the ExecutePageReader method

Figure 2.8. Retrieving database connection statistics

Figure 3.1. Multiple commands to the same database through a single connection

Figure 3.2. Multiple commands for separate DataReader instances through a single connection

Figure 3.3. Multiple commands to different databases through separate connections

Figure 3.4. A mix of commands and connections for different databases

Figure 4.1. Simple use of the SqlResultSet class

Figure 4.2. Inserting a row into a SqlResultSet

Figure 5.1. Converting to XML 1.0 to allow data interchange

Figure 5.2. Exposing a data store as a logical XML document via an XML provider API

Figure 5.3. Layering XML technologies onto an XML provider

Figure 5.4. XML views as a type of XML provider

Figure 5.5. Mapping the Customer table to a Customer element with attributes

Figure 5.6. Mapping the Customer table to the Customer and Address elements

Figure 5.7. Mapping the Customer and Order tables

Figure 5.8. Representing each domain by its own schema format

Figure 5.9. Updating changes made to the XPathDocument2 to SQL Server via the XmlAdapter

Figure 5.10 . Comparing the ADO.NET relational and XML APIs to access data

Figure 6.1. The bookstore XML document viewed within Internet Explorer

Figure 6.2. Inserting a new book with the XmlWriter

Figure 6.3. The XPathDocument2 class and associated classes

Figure 6.4. Updating price nodes in a document with the XPathEditor

Figure 6.5. Deleting price nodes in a document with the XPathEditor

Figure 6.6. Inserting a sibling node in a document with the XPathEditor

Figure 6.7. Inserting child nodes into a document with the XPathEditor

Figure 6.8. The books.xml document before changes with the XPathEditor

Figure 6.9. Navigating the changes to the books.xml document with the XPathChangeNavigator

Figure 6.10. Validation failure on the document with an XML schema

Figure 6.11. Failure to insert a new book without a publicationdate attribute

Figure 6.12. Failed to set the price of a book to 0

Figure 6.13. Removing the title from each book whose publication date is not 1981

Figure 7.1. Using the XmlAdapter to fill an XPathDocument2

Figure 7.2. Using the XmlAdapter to fill an XPathDocument2

Figure 7.3. Using the XmlAdapter to update SQL Server

Figure 7.4. Handling the UpdateError event for the XmlAdapter

Figure 7.5. Using the XmlBulkLoad class to populate a database table

Figure 7.6. Retrieving a value with an XPath and a DBObject

Figure 7.7. Updating a value with an XPath and a DBObject

Figure 7.8. Filling a DataSet with an XmlRowsetAdapter

Figure 7.9. Binding XML data to an ASP.NET TreeView control

Figure 8.1. Using the XQueryProcessor to query an XML document

Figure 8.2. Using the XQueryProcessor to query multiple XML documents

Figure 8.3. Using an XQueryProcessor to query an XML view

Figure 8.4. Using the XQueryProcessor to query multiple XML views

Figure 8.5. Using an XQueryProcessor to query multiple XML views via multiple database connections

Figure 8.6. Using the XsltProcessor class to transform an XML document into HTML

Figure 8.7. The Common Query Architecture showing multiple XML query languages


Tables

Table 2.1. The SqlBulkCopyOperation Constructors

Table 2.2. The SqlBulkCopyOperation Properties

Table 2.3. The SqlBulkCopyOperation Methods

Table 2.4. The SqlRowsCopiedEventHandler Properties

Table 2.5. The SqlBulkCopyColumnAssociator Constructors

Table 2.6. The SqlBulkCopyColumnAssociator Properties

Table 2.7. The SqlBulkCopyColumnAssociatorCollection Properties

Table 2.8. The SqlBulkCopyColumnAssociatorCollection Methods

Table 2.9. New Property and Methods for the SqlConnection Class

Table 3.1. Asynchronous Execution Methods of the SqlCommand Class

Table 3.2. Asynchronous Execution Properties of the SqlCommand Class

Table 3.3. The AsyncCallback Constructor

Table 3.4. The Properties of the WaitHandle Class

Table 3.5. The Methods of the WaitHandle Class

Table 3.6. The Asynchronous Properties of the SqlConnection Class

Table 3.7. The Asynchronous Methods of the SqlConnection Class

Table 4.1. The Notification Property of the SqlCommand Class

Table 4.2. The Constructors for the SqlNotificationRequest Class

Table 4.3. The Properties of the SqlNotificationRequest Class

Table 4.4. The Constructors for the SqlDependency Class

Table 4.5. The Properties of the SqlDependency Class

Table 4.6. The Methods of the SqlDependency Class

Table 4.7. The Event of the SqlDependency Class

Table 4.8. The Constructors for the SqlParameter Class for UDT Support

Table 4.9. The Properties of the SqlParameter Class for UDT Support

Table 4.10. The Methods of the SqlDataReader Class for UDT Support

Table 4.11. The SqlMetaData Class UdtTypeName Field

Table 4.12. The Methods of the SqlCommand Class to Support Result Sets

Table 4.13. The ResultSetOptions Enumeration

Table 4.14. The Properties of the SqlResultSet Class

Table 4.15. The Column Value Access Methods of the SqlResultSet Class

Table 4.16. The Column Property Access Methods of the SqlResultSet Class

Table 4.17. The Navigation Methods of the SqlResultSet Class

Table 4.18. The Row Manipulation Methods of the SqlResultSet Class

Table 5.1. Typical Scenarios for XML

Table 6.1. The Constructors for the XPathDocument2 Class

Table 6.2. The Properties of the XPathDocument2 Class

Table 6.3. The Methods of the XPathDocument2 Class

Table 6.4. The Events of the XPathDocument2 Class

Table 6.5. The Properties of the XPathDocument2ChangedEventArgs Class

Table 6.6. The XPathDocument2ChangedEventAction Enumeration

Table 6.7. The XmlNodeChangeType Enumeration

Table 6.8. The XmlChangeFilters Enumeration

Table 6.9. The Methods of the XPathEditor Class

Table 6.10. The Properties of the XPathChangeNavigator Class

Table 6.11. The Methods of the XPathChangeNavigator Class

Table 6.12. Differences between the XmlSchemaCollection and the XmlSchemaSet

Table 7.1. The Constructors for the XmlAdapter Class

Table 7.2. The Properties of the XmlAdapter Class

Table 7.3. The Methods of the XmlAdapter Class

Table 7.4. The Event of the XmlAdapter Class

Table 7.5. The Properties of the XmlUpdateEventArgs Class

Table 7.6. The Constructors for the XmlDataSourceResolver Class

Table 7.7. The Properties of the XmlDataSourceResolver Class

Table 7.8. The Methods of the XmlDataSourceResolver Class

Table 7.9. A Comparison of Features in SQLXML 3.0 and the .NET Framework

Table 7.10. The Constructor for the XmlBulkLoad Class

Table 7.11. The Properties of the XmlBulkLoad Class

Table 7.12. The Methods of the XmlBulkLoad Class

Table 7.13. The Constructor for the DBObject Class

Table 7.14. The Property of the DBObject Class

Table 7.15. The Methods of the DBObject Class

Table 7.16. The Methods of the XmlQueryArgumentList Class

Table 7.17. The Constructor for the XmlRowsetAdapter Class

Table 7.18. The Property of the XmlRowsetAdapter Class

Table 7.19. The Methods of the XmlRowsetAdapter Class

Table 8.1. The Constructor for the XQueryProcessor Class

Table 8.2. The Properties of the XQueryProcessor Class

Table 8.3. The Methods of the XQueryProcessor Class

Table 8.4. The Execute Method of the XmlCommand Class

Table 8.5. The Constructor for the XsltProcessor Class

Table 8.6. The Property of the XsltProcessor Class

Table 8.7. The Methods of the XsltProcessor Class


Foreword

It's 2:30 p.m. and I'm on a flight bound for Michigan, wondering why I agreed to spend the first hours of my five-day vacation writing the foreword for a book about data and XML.

The setting reminds me of a flight to Sydney a few years ago to give a presentation on Microsoft's latest set of data interfaces. In preparing for the presentation I thought back to how I had first gotten involved with data access.

It all started innocently enough about 12 years ago when I was a fresh young program manager working on Microsoft Excel. While we had invested heavily in making Excel a great tool for analyzing data in the spreadsheet, a vast amount of corporate data was locked away in relational databases—outside of the reach of applications like Excel. The challenge of working with disparate databases led to my involvement with Microsoft's Open Data Base Connectivity (ODBC) specification, a common call-level interface for issuing queries to and retrieving results from relational SQL databases. ODBC afforded me the opportunity to participate in various committees working toward an international standard for how applications could communicate with relational databases. Untold hours of work by industry leaders from different companies across the world, united in a common vision of data accessibility, cumulated in the approval of an extension to the ANSI/ISO SQL 92 specification known as SQL/CLI (SQL Call-Level Interface).

But before the ink was dry on the new standard, we were already thinking about how to take data access to the next level. By viewing a database as a set of services, all connected through a common interface, we hoped to provide common relational services over any type of store. The result was OLE-DB, which, along with the higher-level ActiveX Data Objects (ADO), became the core of Microsoft's Universal Data Access platform and the topic of my presentation in Sydney.

This reflection brought me to the sudden realization that I actually found accessing and working with data interesting. This was a somewhat disturbing discovery. Up until that point, I had always considered myself pretty "hip"; I had been active in sports, a letterman in track and field, seemingly fun at parties, and I loved the outdoors. Yet I feared that this recent discovery meant that, somewhere along the line, I had become a nerd.

But why not take an interest in data? Just about any application, when you get down to it, is about accessing, manipulating, or storing data of one type or another. Information (data) is what makes the Internet so powerful, and harnessing that power by building better ways not just to access but also to work with and manage that data is the key to harnessing that power.

Still, I thought to myself, this newly realized love for data was not something to bring up at a party or dwell overly on during a first date.

Since that flight, however, I've noticed that I'm not alone in my affliction. There are others equally passionate about enabling new and innovative ways to bring data to life. My coworkers, individuals within the standards community, people I meet at conferences, the gurus who assist others on newsgroups and mailing lists, and even the guy currently sitting across from me engrossed in his book on building data-oriented Web sites all do what they do because they share that same passion for data.

And so do the authors of this book.

I first met Mark Fussell when he was interviewing for Microsoft. It was immediately apparent that Mark was a fellow data-phile through his enthusiasm for finding ways to enrich how customers access and work with data. Understanding the incredible potential in XML's ability to model data in a human-readable, self-describing format, and the tools and opportunities this enabled, Microsoft had merged the XML team into the Data team responsible for more traditional relational data access, and Mark quickly took on a lead role in that team. In his three years at Microsoft his enthusiasm and customer focus have contributed significantly to the design and development of ADO.NET and the System.Xml classes that shipped in version 1.0 and beyond.

I first became acquainted with Alex Homer's name through the ADO.NET 1.0 beta newsgroups. ADO.NET 1.0 provided an unprecedented level of control to the developer. Rather than hide data access logic behind higher-level concepts, ADO.NET defines explicit interactions between connected objects optimized for accessing a database and an in-memory object specifically designed for working with a disconnected set of data. This new factoring, however, required a conceptual shift for a number of developers familiar with the previous ADO objects. It was interesting to watch the early beta newsgroups as developers caught on to the power of this new factoring and started evangelizing the model to others. Alex was one such evangelist who took an early interest in ADO.NET, ultimately writing one of the first books dedicated to the subject.

Dave Sussman interacts with customers daily as a consultant, trainer, and writer. He, too, has been working with the .NET Framework since before its release and shares the same enthusiasm and excitement over the Framework.

The benefits of these three authors' personal experiences, as well as those of the customers they represent, show through in this book's unique customer perspective of the evolution of the ADO.NET and XML features described in this book.

This is why, I realize now, I agreed to write this foreword. If these authors can share their insights into some of the exciting new features that comprise the next evolutionary step in our ability to access and interact with data, then maybe more people will discover a hidden passion for data. And who knows; perhaps in time I can even feel cool again.

But in the meantime, I wouldn't leave this book around during your next cocktail party.

Michael Pizzo
Software Architect
WebData Team
Microsoft Corporation


Microsoft .NET Development Series

John Montgomery, Series Advisor

Don Box, Series Advisor

Martin Heller, Series Editor

The Microsoft .NET Development Series is supported and developed by the leaders and experts of Microsoft development technologies including Microsoft architects and DevelopMentor instructors. The books in this series provide a core resource of information and understanding every developer needs in order to write effective applications and managed code. Learn from the leaders how to maximize your use of the .NET Framework and its programming languages.


Titles in the Series

KeithBallinger, .NET Web Services: Architecture and Implementation, 0-321-11359-4

DonBox with ChrisSells, Essential .NET Volume 1: The Common Language Runtime, 0-201-73411-7

MaheshChand, Graphics Programming with GDI+, 0-321-16077-0

AndersHejlsberg, ScottWiltamuth, PeterGolde, C# Language Specification, 0-321-15491-6

AlexHomer, DaveSussman, MarkFussell, A First Look at ADO.NET and System.Xml v. 2.0, 0-321-22839-1

AlexHomer, DaveSussman, RobHoward, A First Look at ASP.NET v. 2.0, 0-321-22896-0

Microsoft Common Language Runtime Team, The Common Language Runtime Annotated Reference and Specification, 0-321-15493-2

Microsoft .NET Framework Class Libraries Team, The .NET Framework CLI Standard Class Library Annotated Reference, 0-321-15489-4

Microsoft Visual C# Development Team, The C# Annotated Reference and Specification, 0-321-15491-6

JamesS.Miller and SusannRagsdale, The Common Language Infrastructure Annotated Standard, 0-321-15493-2

FritzOnion, Essential ASP.NET with Examples in C#, 0-201-76040-1

FritzOnion, Essential ASP.NET with Examples in Visual Basic .NET, 0-201-76039-8

TedPattison and Dr. JoeHummel, Building Applications and Components with Visual Basic .NET, 0-201-73495-8

ChrisSells and JustinGehtland, Windows Forms Programming in Visual Basic .NET, 0-321-12519-3

ChrisSells, Windows Forms Programming in C#, 0-321-11620-8

DamienWatkins, MarkHammond, BradAbrams, Programming in the .NET Environment, 0-201-77018-0

ShawnWildermuth, Pragmatic ADO.NET: Data Access for the Internet World, 0-201-74568-2

www.awprofessional.com/msdotnetseries/


Chapter 1. ADO.NET and System.Xml Version 2.0

One of the fundamental design goals of ADO.NET and the classes from the System.Xml namespaces is to give the developer complete control over how data is accessed and used. In designing ADO.NET, Microsoft concentrated on providing the right core building blocks by factoring functionality into discrete, optimized objects, with the idea that classes could be built in later versions that use these building blocks to make common tasks easier.

When Microsoft first introduced version 1.0 of the .NET Framework, many developers familiar with ADO initially found this new factoring of classes for accessing and working with data unfamiliar. The one-size-fits-all Recordset object that formed the core of ADO and previous data access technologies was replaced with a series of new objects optimized for specific scenarios, such as the DataReader and DataSet. Meanwhile, a raft of new classes was added to the Framework for working with Extensible Markup Language (XML) documents and XML-formatted data. While this factoring provides added flexibility and control, many common tasks require understanding and tying together multiple independent objects, as opposed to the simplicity of the single Recordset object in ADO.

Now, in version 2.0 of the Framework, we get some exciting new additions to the ADO.NET and XML classes that build on the original version 1.0 classes, adding ease of use for common scenarios as well as introducing a host of new features. There is also support for the new features in the forthcoming version of SQL Server (code-named "Yukon") and the introduction of many core data access techniques that developers have been asking for since version 1.0.

This book covers all these new features. However, bear in mind that, as we described in the introduction to the book, the version of the Framework being released at the time of writing (late 2003) is a Technology Preview edition rather than a final release.

So, in this first chapter, we will:

  • Overview the version 2.0 Technology Preview release

  • Summarize the new data management features in version 2.0

  • Look at the reasons for the growing importance of XML

  • Summarize how the techniques for binding to data in ASP.NET are changing

  • Provide a roadmap for the remainder of the book

We start with an overview of the Technology Preview release of version 2.0 of the .NET Framework. Bear in mind that this book is about what's new in version 2.0 of the .NET Framework, so we'll be assuming that you are familiar in general with the version 1.0 data access techniques and classes.


The Version 2.0 Technology Preview Release

The Technology Preview version contains the core changes planned for version 2.0 of the .NET Framework, including the provision of new features prompted both by Microsoft's own development team's inspirations and by feedback from the version 1.0 customer base. It also, of course, includes fixes to known issues with the version 1.0 classes and in some places a tidying up of the class interfaces.

But this is by no means a definitive view of what the final product will contain. It is a good general guide to the way the product is changing and to the kinds of features that will be encompassed in the release version (due toward the end of 2004).

So, the Technology Preview release—and this book—will help you as a developer to understand how data access in .NET is moving and to be more prepared for the changes when they arrive. It means that you can see how the Microsoft team is thinking about the future of the product and will also allow you to design your applications to suit this evolution. Just bear in mind the proviso that the final release version will no doubt be subtly different from what you see here.

Enhancements in Relational Data Management

Although some developers initially struggled with the move away from the simplicity of the single ADO Recordset object in version 1.0 of the Framework, the .NET relational data access model offers proven advantages in several key areas.

  • As application development has moved from the connected client/server model toward Web-based applications, the focus has increasingly been on a disconnected data access approach. The .NET relational data access techniques provide a rich disconnected environment that makes it easy to remote data to clients, access data server-side, and perform staged or bulk data updates.

  • The explicit factoring between objects that connect to and use database resources and objects that contain pure state provides the developer with more control over database interactions, performance characteristics, and application behavior.

  • Programming in the .NET environment predominantly involves managed code, which runs under the control of the Framework runtime. The ADO.NET classes within the .NET Framework are exposed as fully managed code, providing a safe, consistent development platform with excellent error handling and debugging capabilities.

  • The DataSet object introduced in version 1.0 provides a "container" for storing data from multiple tables, and optionally the relationships between these tables, together with associated classes and methods that provide easy updating of the source data and serialization to XML for transmission across networks. In version 2.0, new classes are added to the Framework to make this approach even more flexible.

  • The use of data binding to display data from a variety of sources has grown in importance, especially when building Web applications. ASP.NET provides a range of controls that can perform data binding in different ways, and the DataReader object introduced in ADO.NET makes this a highly efficient approach for displaying relational data in grid or free-form formats.

  • New classes provide a tighter integration with XML data and components.

These are only a few of the reasons that the .NET data access model is proving successful. There are plenty more new features in version 2.0 of the Framework. We'll be summarizing them later in this chapter, and then we'll go on to examine them in more depth in later chapters.

Enhancements in XML Data and Document Management

While the move toward Web-based application development has changed the requirements we have for relational data access, there have been more fundamental changes in the area of document and unstructured data management. Over the last few years, XML has become the de facto way to persist this kind of data in a way that is human-readable, can be machine-processed, and avoids custom binary representations.

Since its introduction, XML has evolved in several different directions. While the unstructured information storage paradigm is still extremely important, we've also come to see XML as a way to serialize and persist structured and hierarchical data (for example, rows and columns from a data table, or more than one related data table).

Prior to .NET, the usual way to manage XML was through the MSXML component, either via the XML Document Object Model (DOM) class, used to build an in-memory tree representation, or through the Simple API for XML (SAX) model, which "pushes" events to the application. Although this works well enough, .NET version 1.0 introduced several exciting new alternatives for manipulating XML data.

  • A fast, forward-only, "pull" model XML parser named XmlReader that is able to read streams of XML in a highly efficient manner. This innovative API provides a simpler and more intuitive approach to reading XML documents. The complementary class XmlWriter is used to generate XML to a stream, including automatic indentation of the output and white-space control.

  • An XML data store that is used to load XML documents into memory for editing and updating. This is the XmlDocument class, and it provides full standardization with the World Wide Web Consortium (W3C) recommendations for the XML DOM level 2.

  • A cursor-style API for navigating through XML documents. The XPathNavigator class implements the XPath 1.0 specification for querying XML documents. This is combined with a read-only store optimized for XPath queries, called the XPathDocument. In version 2.0 of the Framework, the XPathDocument is extended to provide more features, including change tracking and the ability to update data stores in a manner similar to the ADO.NET DataSet. The XPathNavigator is also extended to support the more powerful XML query languages for XPath 2.0 and XML Query Language (XQuery).

  • The XmlDataDocument object, which acts as a bridge between XML and relational data by allowing data to be viewed and manipulated as XML or as a DataSet. While this class is useful, the new features of the XPathDocument will probably prove to be more flexible in this area, as well as allowing the data to be remoted across a network.

  • A series of objects that allow developers to create XML schemas and validate XML documents against supplied XML schemas, along with objects such as the XslTransform class to perform transformations using XML Stylesheet Language Transformations (XSLT) stylesheets.

In version 2.0 of the Framework, there are many enhancements in the XML data management arena. In addition to improvements to the XPath Document class, there are new features for working with XQuery and better integration with the new version of SQL Server ("Yukon") to be released around the same time as version 2.0 of the Framework.


Summary of New Features in Version 2.0

The remainder of this book looks in detail at what's new and the fundamental changes to existing techniques for working with relational and XML data in version 2.0 of the .NET Framework. We've divided the content into the two obvious sections:

  1. The relational data management features in ADO.NET, covering the namespace System.Data and its subsidiary namespaces

  2. The XML data management features in the .NET Framework, covering the namespace System.Xml and its subsidiary namespaces

These two sections provide a summary of what is new in both ADO.NET and System.Xml. However, bear in mind that one of the aims in the .NET Framework is to integrate data from independent data sources and data in different formats, so the distinction between the two approaches listed above is not absolute. Often you will work with XML data through the various ADO.NET objects or manipulate a relational data store using objects from the System.Xml namespaces.

Summary of New Features in ADO.NET

The new features of ADO.NET that are covered in this book are listed below.

  • Multiple Active Results Sets (MARS) allows a connection to support multiple concurrent commands and multiple open sets of results (for example, multiple open DataReader instances). You can open a results set from a query, and then execute subsequent commands over the same connection while the previous one is still open and being accessed.

  • The addition of truly asynchronous operations for connections and commands, allowing code to concurrently fetch and process data from multiple sources. This can provide a considerable performance boost in many situations—particularly when combined with the MARS feature.

  • Support for paging data rows is now built in. The new ExecutePage Reader method of the Command class provides more efficient access to data that is displayed in separate pages, rather than as a single list.

  • New features for executing SQL batched commands. Commands that contain more than one SQL statement can now be executed in a more efficient manner with no extra work required from the developer.

  • The provision of in-process server-side cursors for programmatic processing of data within a stored procedure. Because SQL Server "Yukon" can host the Common Language Runtime (CLR), you can write code that uses server-side cursors without losing out on performance to the same extent as traditional ADO methods.

  • The addition of change notifications for SQL Server, allowing code to cache data and results sets, but automatically react to any changes that invalidate the original data within the database.

  • SQL Server "Yukon" now supports User-Defined Types (UDTs), allowing you to build your own data types as .NET assemblies and install them into the database server and client so that information can be accessed in a more natural and efficient manner.

  • A set of new classes, called ObjectSpaces, that allow data to be handled directly as objects, rather than as individual values.

  • New classes in ADO.NET support programmatic batch updates and bulk data loading from various data sources into SQL Server.

Summary of New Features in System.Xml

The System.Xml namespaces have been extended considerably in version 2.0, in order to implement the main aims listed below.

  • Integrating XML data with a range of data sources, and exposing a programming model that follows the same paradigm as that used in ADO.NET for reading and updating data. This is, in many ways, a parallel with the DataAdapter techniques used with the ADO.NET DataSet class but based on the XML data model rather than the relational data model.

  • Adding built-in support for XQuery and XML Views. This provides a technique for distributed query processing over multiple different data sources, though for the current release this is targeted at SQL Server and XML documents. This is in effect the migration to .NET of the SQLXML 3.0 technology available in SQL Server 2000 today, but with several enhancements in the mapping technology and the query capabilities. Updates via XML Views are supported through the use of declarative "updategrams" that detail the changes that have occurred to the data.

  • Providing better support for the storage of XML data and the integration of relational and XML data management techniques for the new version of SQL Server ("Yukon").

  • Enhancing existing version 1.0 XML classes to support recent W3C standards to provide closer integration between the components in System.Xml; significantly improve the performance of XML parsing, XML schema validation, and XSLT transformations; and enable XML schema types for the classes.

To meet these aims, the System.Xml and subsidiary namespaces in version 2.0 of .NET contain several new classes and improvements to existing ones.

  • There are major changes to the XPathDocument class. The content can now be edited, and change control is built in so that updates, inserts, and deletes are automatically tracked. It can also be remoted while preserving the XML data model and XML schema types.

  • A new XPathChangeNavigator class is introduced to allow code to access the changes in an XPathDocument, and a new class named XPathEditor can be used to perform changes.

  • The introduction of XML Views, a mapping between relational data types and XML data type definitions, allows relational data stored in SQL Server to be viewed and handled as XML.

  • A new XmlAdapter class mirrors the concepts of the ADO.NET DataAdapter class. It connects an XML document to a database via an XML View, allowing data to be read and changes to be persisted to the database through auto-generation of SQL statements.

  • Updates to the XmlReader and XmlWriter classes support the XML schema types, and a new XmlFactory class provides a mechanism for generating XmlReader, XmlWriter, and XPathNavigator instances based on user-defined settings.

  • XQuery is supported through the new XQueryProcessor class, used in conjunction with an XPathNavigator class to query and return information from XML documents using the W3C XQuery language. This language has a more readable (non-XML) syntax for ease of authoring. XQuery is to XML what SQL is to relational databases. XQuery is set to become the preferred query language when querying over XML documents, XML Views, or any data source exposed as XML.

  • The latest version of SQL Server ("Yukon") adds features that integrate with version 2.0 of System.Xml. This includes the provision of a new first-class data type for XML, the xml data type, allowing SQL Server to be used as a store for XML documents. This allows columns in a table to be marked as an xml data type, and XQuery queries can be performed over the XML and associated schema to provide type information.

These brief listings of the new features should whet your appetite to learn more, and that's what we'll be doing in the remainder of this book. To help you find what you need, a later section of this chapter presents a roadmap of the contents of the remaining chapters.


The Growing Importance of XML

You can see from the list of new data-oriented features in version 2.0 of the .NET Framework that there is considerable emphasis on XML as a data persistence, transmission, and manipulation format. XML was originally seen as a way to expose information that may or may not follow some structured or hierarchical format, and the XML standards allow XML to be used in a huge range of ways for storing information of almost any type.

As XML becomes more mainstream, it fulfills an increased role as an information integrator. XML is the bridge between relational data, objects, documents, Web Services, and more.

The Changing XML Landscape

The System.Xml version 1.0 APIs were developed at a time when XML standards were quite different than those that exist today. The predominant standards at the end of 1999 and early 2000, when the initial APIs for version 1.0 were laid down, were XML 1.0, DOM 1.0 and 2.0, XPath 1.0, and XSLT 1.0. Significant standards such as the XML Information Set (October 24, 2001, http://www.w3.org/TR/xml-infoset/), XML Schema (May 2001, http://www.w3.org/TR/xmlschema-1/), and SOAP 1.2 (CR December 2002, http://www.w3.org/TR/SOAP/) subsequently emerged. These standards are having a positive impact across the industry.

In the last two years there has been a significant shift from viewing XML purely as a text-based format to a data representation format epitomized by the XML Information Set specification, which was developed in recognition of the fact that the DOM API did not define the relationships between the information item types found within an XML document. An XML Information Set (often described just as an XML InfoSet) refers to the data types that can be found in an XML document after it has been parsed by an XML parser. This shift of emphasis toward XML defined in terms of information items now appears in many of the W3C specifications. For example, to quote from the SOAP 1.2 specification:

A SOAP message is specified as an XML Information Set or XML InfoSet (http://www.w3.org/TR/soap12-part1/#XMLInfoSet#XMLInfoSet). While all SOAP message examples in this document are shown using XML 1.0 syntax, other representations MAY be used to transmit SOAP messages between nodes (see SOAP Protocol Binding Framework, http://www.w3.org/TR/soap12-part1/#transpbindframew#transpbindframew).

What's more, this has been reflected in all of the significant emerging W3C specifications, such as XQuery, XPath 2.0, and XML Schema 1.0. Each of these specifications is dependent on the definition of XML as InfoSet items.

To draw a parallel: When working with a relational database, the way the data is actually written to disk (serialized) is completely hidden from the developer and the database administrator. This is desirable because it means that the method in use can be changed over time to make storage more efficient without breaking existing code. Rather than having to know about tracks and sectors on the disk, the developer or database administrator deals with the database through a level of abstraction that consists of tables, rows, and columns. This is the relational data model.

The XML InfoSet performs the same role of abstraction. The data may or may not be written as serialized XML 1.0 with angle brackets, but either way the data types that are exposed always appear consistent to the processing application. Handling XML InfoSets provides the ability to expose a multitude of data sources as XML, a technique often referred to as virtualization. The XPathNavigator class provides XML virtualization over a data source in System.Xml today, based on the XPath 1.0 data model, which uses the XML InfoSet item types.

Perhaps one of the most significant specifications to emerge recently is the XQuery and XPath 2.0 Data Model (http://www.w3.org/TR/query-datamodel/), which, being based on the XML InfoSet, defines the data model for the XQuery language in the same fashion as the relational data model does for the SQL query language. In many ways this should really be considered the XML data model, since its importance goes beyond the bounds of just the XQuery and XPath 2.0 languages.

XML and Relational Databases

Relational databases such as SQL Server are the origin of much of the data that is interchanged today and then typically stored again in other relational databases. Since relational systems today are overwhelmingly used to manage structured data, most of the XML exchanged also fits the relational model of structured data. Hence it is becoming increasingly common for XML to be focused in scenarios that involve structured data storage and transmission, for example, exposing relational data (the traditional "rows and columns" format), and hierarchical data (such as multiple related data tables).

This movement has also tended to simplify the XML content by concentrating mainly on the use of elements and attributes and avoiding what might be considered the more "esoteric" capabilities of XML (such as entities, notations, and so on). This is the domain of XML as a data interchange format rather than as a document markup language (which was the original driving force behind the introduction of XML).

An XML View is a mapping of a data source, which typically is in a non-XML format, through to an XML document—such as mapping a set of relational tables from a database. These XML Views virtualize the data, in that it isn't actually converted into an XML (serialized) format but merely shaped and transformed into a structure as if it were XML, based on the XML InfoSet data types. The use of XML Views over relational databases allows generation of XML documents and the hierarchical data structure, and also enables the use of XML technologies such as XQuery to provide heterogeneous queries over disparate data sources.

XML is also a more flexible data format than its relational equivalent because it has the ability to express the semi-structured and unstructured format that applies to much of the data lying outside of relational databases. Semi-structured refers, for example, to properties that appear only on a certain number of elements and not in a regular repeating (structured) fashion. Unstructured format is like that of a Microsoft Word document, typically consisting of "marked-up" data such as the content of the paragraphs in the text. XML has both the simplicity and flexibility to easily represent these differing data structures, and as a result XML has established itself as the lingua franca of data interchange for business and applications.

XML in Web Applications

XML as a data format is well suited to the environment in which Web-based applications live, being a platform-independent syntax for which simple and efficient parsers are widely available. In particular, any situation that involves remoting data (moving it across the network to a client or another server, as we describe in more detail later) can benefit from the use of XML. For example, clients or servers that run on disparate operating systems or software platforms can share XML-formatted data easily. Being persisted as Unicode means that even a 7-bit wire format (such as the "text-only" nature of the Web) for the intervening network does not impede simple and efficient data transmission.

In contrast to this, application- or operating system–specific data formats, particularly binary formats, may well cause all kinds of issues when used with disparate clients. As a simple example, some operating systems may treat 16-bit numbers as big-endian (the leftmost byte, at the lower memory address, is the most significant) while others treat them as little-endian (the leftmost byte is the least significant).

XML is also the ideal format for simply sending data to a client or receiving it from a client. For example, the Web site or application can provide a browser client with an XML document that is downloaded to the user's browser. Then it can be manipulated within the browser to display the data and (if required) edited and submitted back to the server for subsequent processing. Because the data is in a platform- and operating system–independent format, the code or application at either end of the network can be adapted and changed without reference to changes at the other end, as long as the data format remains the same.

Data Description through Schemas

Even more useful is the ability to use an XML schema with the XML data. A schema specifies the structure and allowable content of the XML document. This permits applications to be built where the client or recipient uses the schema to interpret the structure of the data, with the result that this data format can be changed independently as long as an appropriate schema is available.

An example of this approach is found in the new XML-format metabase used by Internet Information Services (IIS) 6.0. Each time the configuration of IIS is changed, an XML file containing the complete configuration of all the installed services is written to disk. However, as you install and remove services or add sites or virtual roots to IIS, the structure of this file changes in subtle ways to reflect the content it will store.

So, as well as writing the XML data file to disk, IIS also writes the current schema to disk. This means that, irrespective of which services are installed and how the current configuration looks, the saved configuration data can be correctly interpreted and restored each time.

Transformation and Presentation through XSLT Stylesheets

The concept of applying presentation information to an XML document has been one of the core aims of W3C almost since the inception of XML. The original proposals concentrated on two aspects: (1) transformations, which can change the content, structure, and ordering of the XML (to produce, for example, HTML output), and (2) the definition of a formatting language used by specialist client applications to apply style and layout information directly to the XML content.

In data management terms, the first of these two scenarios is the interesting one. The ability to apply a transformation that optionally can change the content, structure, and ordering of the elements means that an XSL or XSLT stylesheet can not only generate output that is aimed at presentation (for example, adding style definitions or HTML elements to the content) but also output a different XML document from that applied as the input.

XSLT is increasingly being used to "process" XML documents. Its recursive nature and reasonably simple syntax mean that it's possible to write generic stylesheets that can process different documents, and it has found a home in many areas and applications that need to convert XML data from one format to another (for example, Microsoft BizTalk).

XML Data Querying through XQuery

One of the more recent areas of development for XML is XML Query Language, or XQuery. This is a project aimed at deriving a technique for applying queries to an XML document that (in the broadest sense) more closely resemble SQL statements. While SQL is ideal as the query language for relational data, it is not designed to be a query language for XML data, since a query language should have as little impedance mismatch to the data model it queries as possible. As mentioned earlier, XQuery is to the XML data model what SQL is to the relational data model and is set to become the preferred query language for XML data querying. This will make XML much more approachable to those developers and administrators more used to working in a relational data environment.

Although XSLT can already perform most of the tasks that XQuery is aimed at, it's already becoming clear that XQuery can provide more options, as well as broader and simpler techniques for accessing multiple documents, than XSLT does. However, the choice between the two is more likely in the long term to be based on the developer's grasp of the techniques and language choice, in an identical manner to the language choice available in the .NET Framework today.

In a nutshell, XQuery provides support for:

  • A more readable, SQL-like syntax designed to support query expressions across several XML data sources

  • Type checking so that, for example, all the Order types defined in an XML schema can be found in an XML document

  • A set of functions and operators defined for the built-in XML Schema (XSD) types (for example, the result of adding an XSD integer type to an XSD string type)

The current recommendations and guides to all the specifications we've mentioned here (XML, XSLT, XQuery, and XML Schema) can be found at the W3C Web site at http://www.w3.org.

XML Content Publishing

While Web applications are a major beneficiary of XML, they are not alone. Another arena where XML is growing is in content publishing. This is a wide and difficult area to define exactly, but in general the term applies to application and service integration that is aimed at exposing data or information to users.

Microsoft's own Content Management Server is an example, including features to build and deploy Web sites and Web Services, manage updates, perform workflow management, and integrate with Microsoft Office applications. It uses XML to transport and expose information in conjunction with templates, and to package content objects.

Similarly, SharePoint Server and Microsoft Office 2003 contain many features based on XML. In fact, XML is at the forefront of integration in Office 2003, with the new InfoPath data collection application storing data as XML and exposing it to the other "traditional" Office applications such as Word and Excel through Smart Documents based on XML.

XML in the .NET Framework

XML has been at the heart of the .NET Framework since version 1.0. Many aspects of data persistence, storage, and transmission depend wholly or partly on XML. For example, while the ADO Recordset object provided a custom binary format by default for persisted data, XML is the only data persistence format supported by the ADO.NET DataSet. The GetXml and WriteXml methods always return XML-formatted data.

Conversions between Relational and XML Data

Figure 1.1 demonstrates some of the ways that XML is used within the .NET Framework for access to and conversion of relational data. The ADO.NET objects SqlCommand and DataSet can export or expose their content as XML, and the DataSet can also export the matching schema.

Figure 1.1. Access to and conversion of relational data as XML

You can also see that the XmlDataDocument can be used to expose XML from relational data, using an existing DataSet as the source of the XmlDataDocument instance. What's new and exciting in version 2.0, however, is shown in the lower section of the schematic. These are the new features of the XPathDocument and its associated new classes. An XmlAdapter provides the link between the XPathDocument and a relational database. The XPathChangeNavigator and XPathEditor are used to navigate through and update the XML content, after which the XmlAdapter can push the changes back into the database.

XML Serialization and Remoting

The .NET Framework also includes a class called XmlSerializer (see Figure 1.2) that can be used to serialize objects and class instances to XML—including custom classes you declare—and deserialize them from XML. This makes it easy to build applications that interchange data represented as objects, such as a purchase order or invoice object generated from custom classes.

Figure 1.2. Serialization of XML data over a network

XML serialization is useful when you need to remote objects (rather than just plain data) to another location, or even just to another tier of your application. However, if you want to remote only the data itself, another useful feature of the XML classes is that the XmlDataDocument can be used as an alternative to a DataSet. It will maintain all of the schema information required to resurrect the data after transmission — including the data types, relations, and so on. The XmlDataDocument can be filled directly with XML of the required format, which can include nested or related data. Alternatively, it can be instantiated from an existing DataSet, as long as a schema is available.

However, as mentioned earlier, the XPathDocument can also be remoted. Because it implements change tracking, the remoting format has to be able to persist the changes as well as the "current" values of nodes in the document in case this information is required to be persisted in an XPathDocument regenerated from the serialized data. Three scenarios are supported.

  1. Save or serialize just the data content, not details of the changes made to the values.

  2. Save or serialize all the data and all details of the changes made to the values.

  3. Save or serialize only the values where changes have occurred, not the "unchanged" values.

The XPathDocument serializes values at Node level. In other words, every node in the document gets serialized and is then available when the document is rebuilt. In an XML document, every element and attribute is a node, so the complete data content of the document is maintained.

Serialization is actually useful not only when remoting data but also for supporting off-line data access and editing. After using an XmlAdapter to fill an XPathDocument, you can save the data to a local disk and disconnect from the database, then work with the data without maintaining a connection. Later you can reconnect and resynchronize the data in the database using the changes in the XPathDocument in much the same way as you would in the relational world.

Migration of the SQLXML 3.0 Technology to .NET

One of the aims of version 2.0 of the .NET Framework is to move the existing SQLXML technology to the .NET platform. SQLXML was originally developed as an add-on for SQL Server 7.0. The aim was to make it easy to extract data from SQL Server as XML and to use specially formatted XML updategrams to push data updates back into SQL Server.

The original Technology Preview of SQLXML involved installing a "filter" that sits in SQL Server and responds to queries that contain the FOR XML keywords. SQL Server 2000 has the SQLXML version 3.0 technology built in, and it automatically detects and processes SQL statements that are XML queries. For example, the following SQL statement:

SELECT * FROM Country WHERE Name LIKE 'U%' FOR XML AUTO

returns a series of XML elements that represent each matching row in the database table, with each column represented as an attribute of the row:

<Country Name="USA" Continent="America" Size="Very large"/>
<Country Name="UK" Continent="Europe" Size="Quite small" />
... etc ...

These elements can easily be wrapped in a suitable preamble and root element to turn them into an XML document. In .NET, SQLXML queries that return XML elements are handled by an XmlReader returned from a call to the ExecuteXmlReader method of a Command object connected to the database.

However, it would be useful to be able to use this type of data query, and return XML, with databases other than SQL Server and to make it more compatible with the techniques used in the Framework. This is achieved in version 2.0 of the .NET Framework through the XmlAdapter and XPathDocument you saw in Figure 1.1, though in the current release this is limited to the SQL Server "Yukon" version. Instead of having to learn complex rules for specifying the format of the data returned from the database and all the different options available for updategrams, you use an approach and a syntax that are similar to those used in relational data access through the ASP.NET DataAdapter.

Server-Side Data Binding to XML

In version 2.0, the .NET Framework also supports server-side UI data binding in Web Forms (ASP.NET) and Windows Forms (executable) applications to XML documents stored in the XPathDocument and XmlDocument classes. This provides a fast and efficient technique for displaying data that is persisted as XML in Web pages and Web applications and in your .NET executable programs.


Relational Data Binding in ASP.NET

One of the great features introduced into ASP.NET in version 1.0 was server-side data binding, through a range of ASP.NET server controls and to many different kinds of data structures. This includes the ADO.NET DataTable and DataReader, as well as other collection-based structures.

In version 2.0 of ASP.NET, this concept has been extended to allow developers to build data-bound pages that no longer require any code to be written—even for complex operations such as paging, sorting, and updating the original data.

ASP.NET server controls expose a property named DataSource to which you assign the source data structure. For example, if you have an active instance of a DataReader, you simply assign it to the DataSource property of the control and then call the DataBind method (on the control or on the Page object that contains it) to cause the control to iterate through the data rows and generate the output in the page:

MyDataGrid.DataSource = MyDataReader
MyDataGrid.DataBind()

If the data source contains more than one table, for example, when using a DataSet, the DataMember property of the control is used to specify which table is to be bound:

MyDataGrid.DataSource = MyDataReader
MyDataGrid.DataMember = "TableName"
MyDataGrid.DataBind()

ASP.NET Data Source Controls and the GridView Control

In version 2.0, ASP.NET introduces a range of new server controls to handle data management and server-side data binding. These controls fall into two distinct groups:

  1. Data source controls that connect to a data source and expose the data

  2. The new GridView and DetailsView controls that consume the data and display it in a Web page

Several data source controls are included with ASP.NET, all residing in the System.Web.UI.WebControls namespace. These include controls designed to work with relational data from SQL Server, OLE-DB and ODBC databases, Microsoft Access, and Oracle. There is also a special data source control designed to work with the new ObjectSpaces technology to enable server-side data binding to custom business objects. Some data source controls consume XML from a stream or disk file. The XML Data Source Controls and Data Binding section in Chapter 7 briefly discusses the controls that consume XML, and the companion book to this one, A First Look at ASP.NET v. 2.0 (Boston, MA: Addison-Wesley, 2004, ISBN 0-321-22896-0), contains full details of all the data source controls.

The new ASP.NET GridView and DetailsView controls expose Data Source and DataMember properties, just like most other "bindable" server controls. However, they also expose a DataSourceID property, which can be set to the ID of a data source control that exposes data in relational (columns and rows) format. When this is done, the two controls work together to extract the data and perform the server-side data binding automatically, with no code required.

You'll also be pleased to know that all the ASP.NET server controls that support data binding now expose the DataSourceID attribute in version 2.0, so they can be used interchangeably with any data source control that exposes the data in the appropriate format.

An Example of the Data Source and GridView Controls

As an example of how these two controls change the whole approach to server-side data binding, the code in Listing 1.1 (taken from the book referred to above) shows the complete <form> section of an ASP.NET page that extracts some rows from the Northwind database and displays it in a GridView control.

Listing 1.1 Using a Data Source Control and a GridView Control
<form runat="server">

<asp:SqlDataSource id="ds1" runat="server"
  ConnectionString="server=localhost;database=Northwind;uid=x;pwd=x"
  SelectCommand="SELECT ProductID, ProductName, QuantityPerUnit,
                 UnitPrice, UnitsInStock, Discontinued FROM Products"
/>

<asp:GridView id="grid1" DataSourceID="ds1" runat="server" />

</form>

The SqlDataSource control uses a connection string and a SQL statement (it could alternatively be a stored procedure name), plus "sensible" default settings, to connect to a database and extract the data rows. It reacts to the page-level events that occur when the page is requested and internally builds the usual ADO.NET objects it needs (Connection, DataAdapter, DataSet, and so on).

The GridView control performs the data binding and displays the data. The screenshot in Figure 1.3 shows the result. It displays all the rows from the Products table in the Northwind database, just as specified in the SQL statement. However, notice the Discontinued column. This is a Boolean field in the table, and the control automatically displays it using read-only (disabled) checkboxes.

Figure 1.3. A data source and GridView control in action

Do I Still Need to Write ADO.NET Code?

In ASP.NET, you may be able to avoid writing code to interact with a database and just rely on the data source controls to do all the work. However, that doesn't mean you can stop reading now! As is always the case, knowing more about how the data access technologies actually work, as well as being able to take advantage of the features they offer to tailor performance and behavior to suit your requirements, is part of the development process.

In particular, you may prefer to take advantage of specific new features in ADO.NET such as MARS, asynchronous processing, and batch command execution to access and expose your data in exactly the way you require. In this case, having generated your data structure, you can continue to benefit from server-side data binding by using the new or existing controls through their DataSource and DataMember properties.


A Roadmap for This Book

The remainder of this book is divided into two sections, along the same lines as the feature summaries you saw earlier in this chapter. In the next three chapters, we look in more detail at the new features in ADO.NET version 2.0.

  • Chapter 2, Bulk Loading, Batch Execution, and Paging, looks at the new classes, methods, and other additions to ADO.NET in version 2.0 that enable some exciting new techniques to be applied. This includes bulk inserts and updates in SQL Server, batched execution of commands, and data paging support.

  • Chapter 3, MARS, Asynchronous Commands, and ObjectSpaces, looks at three more of the new core data access techniques and features added to ADO.NET 2.0. MARS provides the opportunity to open more than one result set over the same connection and access them concurrently, while asynchronous execution of multiple commands over the same Connection instance is also possible now. This chapter also introduces the new ObjectSpaces technology added to ADO.NET, which allows data to be defined as instances of a class but handled just like other intrinsic data types.

  • Chapter 4, ADO.NET and SQL Server "Yukon," looks at ADO.NET integration with the new version of Microsoft SQL Server (code-named "Yukon"). It examines notifications, managed code within SQL Server, user-defined data types, and the new feature that allows server-side cursors to be used to directly access and modify data.

Then, in the final four chapters, we look at the changes in the System.Xml and subsidiary namespaces.

  • Chapter 5, New Features of System.Xml, explores and overviews the main basic concepts—such as XML programming and the XML data model, disconnected XML database access, XML Views, XQuery—and how to get started using them.

  • Chapter 6, The XPathDocument2 Class, concentrates in more depth on the new features of the updated XPathDocument class and its associated classes, such as XPathChangeNavigator and XPathEditor, that provide read/write access to XML documents.

  • Chapter 7, The XmlAdapter and SqlXml Classes, is a detailed look at the XmlAdapter class that, together with the XPathDocument, enables disconnected data manipulation in XML. It also covers XML schemas and the three-part mapping technology that links relational and XML data. This chapter also looks at the .NET replacements for the unmanaged SQLXML technology implemented in SQL Server 2000.

  • Finally, Chapter 8, XQuery and SQL Server, looks at what XQuery is, and the support for it in version 2.0 of the .NET Framework. This includes examination of the XQuery architecture, the new XQueryProcessor class, and query composition. This chapter also looks at the new mapping format for defining XML Views over data sources that enable queries and updates and briefly discusses the XML data type in SQL Server "Yukon."


SUMMARY

This short first chapter aims to get you familiar with the overall enhancements to version 2.0 of the .NET Framework over version 1.0 that concern data management using ADO.NET and the classes from the System.Xml and subsidiary namespaces. As well as summarizing these as simple lists, we've attempted to fill out the picture a little by including some back ground on why they have come about and Microsoft's aims as far as developers are concerned.

We also spent some time looking at the area of XML data management within the .NET Framework, to make sure you appreciate the growing importance and widening acceptance of XML as a data persistence, manipulation, and transmission format. As new business applications and software suites appear (such as Microsoft Office System, SharePoint, and so on), the use of and dependence on XML can only continue to grow.

We then briefly overviewed the way that XML is managed within .NET, in both versions 1.0 and 2.0, to reinforce the direction in which development of the Framework is moving. The concept of relational and XML data as being two different entities is rapidly disappearing, and the forthcoming opportunities to construct XML data and schema repositories based on SQL Server will only help to accelerate this change.

Finally, after a quick look at how the changes to ADO.NET affect (or, in fact, do not affect) ASP.NET data binding, we ended the chapter with a brief outline of the content in the remainder of this book. This should help you to dive in and see the areas that are of particular interest to you.

As we stated earlier, the Technology Preview release of the .NET Framework version 2.0 is by no means a complete view of all the features that will be present in the final release. One area that Microsoft is working on, but which has not been finalised yet, is a new class that simplifies relational data management. Early prototypes indicate that it will remove the need for developers to explicitly create intermediate class instances such as Connection, Command, and DataAdapter by using sensible default values and internal heuristics. This feature, providing characteristics similar to but extending the existing DataTable class, should find its way into forthcoming Beta releases of the Framework.


Chapter 2. Bulk Loading, Batch Execution, and Paging

As we discussed in Chapter 1, there are many changes, updates, and improved features in ADO.NET version 2.0. In this chapter we'll look in detail at some of these, including:

  • New classes that enable managed code to perform bulk copying of data, achieving the same kind of effects as the command-line BCP utility

  • The ability to send update commands in batches using a DataAdapter

  • Built-in support for data paging using the new ExecutePageReader method of the Command class

  • Retrieving from a connection the statistics that describe the processes it is handling

This isn't a comprehensive list of the new features, however. You'll see more in the following chapters as well. But we start here with a look at the new bulk data copy features in ADO.NET 2.0.


The New Bulk Data Copy Features

SQL Server comes with a command-line utility named BCP that can be used to copy rows into a database table. This utility is popular with administrators for moving data between servers and generally managing their databases, and most database server suppliers provide a similar routine.

In ADO.NET 2.0, the ability to bulk-load a database table with data has been added, using entirely managed code through several new .NET classes. The target (at the moment) is restricted to SQL Server. However, sources of data for a bulk copy operation can be:

  • Any DataReader instance, from the same or any other type of database

  • Any DataTable, or array of DataRow instances

  • Any data source that can be used to load a DataTable, for example, XML documents or application data

So this technique makes it easy to efficiently copy data from a multitude of sources, including other relational sources, into SQL Server.

The bulk copy process is implemented using classes in the System .Data.SqlClient namespace. The main class is SqlBulkCopyOperation, which exposes properties, methods, and events that perform the tasks required for a bulk copy operation. Subsidiary classes are the SqlBulkCopy ColumnAssociator class, which is used to associate each column in the source table with the target column in the destination table, and the Sql BulkCopyColumnAssociatorCollection class, which contains a collection of all the SqlBulkCopyColumnAssociator instances for one bulk copy operation.

The SqlBulkCopyOperation Class

The SqlBulkCopyOperation class is the core object for performing a bulk copy, exposing properties that you set to define the overall behavior of the process. By default, it starts a new transaction in the destination database for each copy operation it performs, and all the inserts are done under this transaction. If there is an error, the copy process halts and all the rows copied during this operation are removed when the transaction is rolled back.

The SqlBulkCopyOperation constructors provide a variety of ways to instantiate the class, including the ability to use your own transaction rather than the default. The methods are used to perform the copy and close the class instance afterward. There is also one event that you can handle to get information about the progress of the operation.

Table 2.1. The SqlBulkCopyOperation Constructors

Constructor

Description

SqlBulkCopyOperation(connect-string)

Creates an instance of the class and an associated Connection instance, given a String that contains the connection string of the destination database.

SqlBulkCopyOperation(connection)

Creates an instance of the class that uses an existing Connection instance specified as the connection parameter.

SqlBulkCopyOperation(connect-string, copy-options)

Creates an instance of the class and an associated Connection instance, given a String that contains the connection string of the destination database. The second parameter is a combination of values from the SqlBulkCopyOptions enumeration, as described below.

  • Default: None of the following rules are applied.

  • CheckConstraints: Constraints defined in the destination table are enforced while the data is being inserted.

  • KeepIdentity: Identity column values in the source data table are preserved in the destination table. If not specified, new values are auto-generated in the destination table.

  • KeepNulls: null values are preserved in the destination table even if the destination column has a default value specified. If not specified, any null values are replaced by the column default value where this is defined.

  • TableLock: A bulk update lock is obtained on the table for the duration of the bulk copy operation. If not specified, row-level locking is used instead.

SqlBulkCopyOperation(connection, copy-options, transaction)

Creates an instance of the class that uses an existing Connection instance specified as the connection parameter, a value from the SqlBulkCopyOptions enumeration, and a reference to an existing Transaction instance under which the copy will take place.

Constructors

Table 2.1 shows the four constructors for the SqlBulkCopyOperation class.

Properties

The properties of the SqlBulkCopyOperation class are shown in Table 2.2. They define details of how the copy operation will take place, such as the batch size and the timeout, plus a reference to the collection of column associations to be used during the bulk copy operation.

Table 2.2. The SqlBulkCopyOperation Properties

Property

Description

BatchSize

Sets or returns an Integer value that specifies the number of rows to be copied in each batch. If the destination database supports batched updates, the commands are batched together to improve efficiency and reduce network traffic. If not, the SqlBulkCopyOperation instance manages and executes each batch itself. A batch size of 0 (the default) indicates that all the updates will be performed as a single batch.

BulkCopyTimeout

Sets or returns an Integer value that specifies the number of seconds for the complete operation to complete before it times out. Following a timeout the transaction is not committed, so all copied rows are removed from the destination table.

ColumnAssociators

Returns a reference to the collection of SqlBulkCopy ColumnAssociator instances for this operation.

DestinationTableName

Sets or returns a String value that specifies the name of the destination table to which the rows will be copied.

NotifyAfter

Sets or returns an Integer value that specifies how often the SqlRowsCopied event is raised. The default is 0, meaning that the event is raised only at the end of the operation. Any other positive value is the number of rows after which the event should be fired.

Methods

Table 2.3 shows the four methods for the SqlBulkCopyOperation class that start a bulk copy operation, each taking a different type of object that acts as the source of the rows to be copied. The final method is used to close the SqlBulkCopyOperation instance after use. There are no return values for these methods.

The SqlRowsCopied Event Handler

As a bulk copy process is running, and depending on the value of the NotifyAfter property, the SqlRowsCopied event will be raised at various intervals. This event can be handled to monitor progress of the operation or cancel it partway through if required. The event passes an instance of a SqlRowsCopiedEventHandler to the event handler code. Table 2.4 shows the properties of this event handler.

Table 2.3. The SqlBulkCopyOperation Methods

Method

Description

WriteDataReaderToServer(DataReader)

Copies the rows from an open DataReader instance into the destination table.

WriteDataRowsToServer(DataRow-array)

Copies the rows referenced as an array of DataRow objects into the destination table.

WriteDataTableToServer(DataTable)

Copies the rows from a DataTable instance into the destination table.

WriteDataTableToServer(DataTable, DataRowState)

Copies only the rows in DataTable instances that have the specified value for their DataRowState (i.e., Unchanged, Modified, Inserted, or Deleted) into the destination table.

Close()

Closes the SqlBulkCopyOperation instance.

Bulk Copy Column Associations

When performing a bulk copy, you may be lucky and find that the source and destination tables have the same number and ordering of columns, with the same column names. In this case, the SqlBulkCopyOperation class will automatically map the source table to the destination table and insert the values into the correct columns.

Table 2.4. The SqlRowsCopiedEventHandler Properties

Property

Description

RowsCopied

Returns an Integer indicating the number of rows copied to the destination table during the current bulk copy operation.

Abort

Provides a Boolean value that can be set to True to cancel the current operation. If it is executing under the default transaction, the rows are removed from the destination database when the transaction is rolled back. If you specified an existing transaction when creating the SqlBulkCopyOperation instance, you must manage the commit or rollback yourself.

However, it's often the case that the destination table has different column names and perhaps more or fewer columns. To perform a bulk copy in these cases, you must create and populate a collection of SqlBulkCopy ColumnAssociator instances that define the mappings or associations between the columns.

The SqlBulkCopyColumnAssociator Class

Each SqlBulkCopyColumnAssociator instance defined for a bulk copy operation associates a source and a destination column so that data is inserted into the correct column. If none are defined for a copy operation (i.e., the SqlBulkCopyColumnAssociatorCollection is empty), the default process will copy the data from every column in the source table into the same column in the destination table. If the schemas for the two tables don't match, an exception is raised.

However, as soon as you add at least one SqlBulkCopyColumn Associator instance to the SqlBulkCopyColumnAssociatorCollection, the default process is abandoned. Data is copied from the source table to the destination table only for columns that are associated by a SqlBulk CopyColumnAssociator instance. This means that you can easily map one table to a different table and copy data to specific columns.

Constructors

There are five constructors for the SqlBulkCopyColumnAssociator class, documented in Table 2.5. They allow the source and destination columns to be identified by a zero-based ordinal position or by the column name.

Properties

Table 2.6 shows the four properties of the SqlBulkCopyColumnAssociator class that define the source and destination columns, either as a column name or as a zero-based ordinal position.

The SqlBulkCopyColumnAssociatorCollection Class

The SqlBulkCopyColumnAssociatorCollection class contains the collection of SqlBulkCopyColumnAssociator instances that define the current bulk copy operation. The collection has properties for the overall capacity and the current number of members, as well as a range of methods to manage the content of the collection.

Table 2.5. The SqlBulkCopyColumnAssociator Constructors

Constructor

Description

SqlBulkCopyColumnAssociator()

Creates an instance of the class with no column association set. The properties of the class can be set later to define the association.

SqlBulkCopyColumnAssociator(source-name, destination-name)

Creates an instance of the class with the source and destination columns identified by the (String) column names.

SqlBulkCopyColumnAssociator(source-index, destination-name)

Creates an instance of the class with the source column identified by the (Integer) column index and the destination column identified by the (String) column name.

SqlBulkCopyColumnAssociator(source-name, destination-index)

Creates an instance of the class with the source column identified by the (String) column name and the destination column identified by the (Integer) column index.

SqlBulkCopyColumnAssociator(source-index, destination-index)

Creates an instance of the class with the source and destination column identified by the (Integer) column indexes.

Table 2.6. The SqlBulkCopyColumnAssociator Properties

Property

Description

SourceColumn

Sets or returns the source column for this association, using the column name specified as a String.

SourceOrdinal

Sets or returns the source column for this association, using the column index specified as an Integer.

DestinationColumn

Sets or returns the destination column for this association, using the column name specified as a String.

DestinationOrdinal

Sets or returns the destination column for this association, using the column index specified as an Integer.

Properties

The three properties of the SqlBulkCopyColumnAssociatorCollection class documented in Table 2.7 provide information about the collection and access to individual SqlBulkCopyColumnAssociator instances within it.

Table 2.7. The SqlBulkCopyColumnAssociatorCollection Properties

Property

Description

Capacity

Sets or returns the capacity of the collection as an Integer.

Count

Returns the number of SqlBulkCopyColumnAssociator instances currently in the collection as an Integer.

Item

The indexer for the collection. Returns a reference to the SqlBulkCopyColumnAssociator at the specified index.

Methods

Table 2.8 shows the five methods for adding SqlBulkCopyColumn Associator instances to the collection, the method for removing them, and the method that removes all instances from the collection.

A Simple Bulk Copy Example Using a DataReader

To see how the bulk copy process can be used, Listings 2.1, 2.2, and 2.3 show a simple example where the source and destination tables are in the same database and have the same number of columns, the same column order, and the same column names.

In a "real" application, you may be appending rows to an existing table that is not empty. However, to ensure that the example code can be executed more than once without causing identity column or constraint violations, the first task in Listing 2.1 is to clear any existing rows from the destination table by executing a SQL DELETE statement.

After this, a Command that will extract rows from the source table is created, and the results are ready to be referenced through the DataReader returned from the ExecuteReader method. Then the code creates and opens a second connection that it will use to access the destination table.

Listing 2.1 A Simple Example of a Bulk Copy Using a DataReader
' create connection to database to extract source rows
Dim sConnectString As String = "your-connection-string"
Dim oSrcConn As New SqlConnection(sConnectString)
oSrcConn.Open()

' remove any existing rows from table first so page can
' run more than once without causing integrity error in table
Dim oCmd As New SqlCommand("DELETE FROM Copy_Of_Products", oSrcConn)
oCmd.ExecuteNonQuery()

' fetch some rows from database into a DataReader

Table 2.8. The SqlBulkCopyColumnAssociatorCollection Methods

Method

Description

Add(source-name, destination-name)

Creates a new SqlBulkCopyColumnAssociator instance using the source and destination column names specified as a String, and adds it to the collection. Returns the newly added instance.

Add(source-name, destination-index)

Creates a new SqlBulkCopyColumnAssociator instance using the source column specified by name as a String and the destination column specified by index as an Integer, and adds it to the collection. Returns the newly added instance.

Add(source-index, destination-name)

Creates a new SqlBulkCopyColumnAssociator instance using the source column specified by index as an Integer and the destination column specified by name as a String, and adds it to the collection. Returns the newly added instance.

Add(source-index, destination-index)

Creates a new SqlBulkCopyColumnAssociator instance using the source and destination column indexes specified as an Integer, and adds it to the collection. Returns the newly added instance.

Add(column-associator)

Adds an existing SqlBulkCopyColumnAssociator instance to the collection. Returns the newly added instance.

RemoveAt(index)

Removes the SqlBulkCopyColumnAssociator instance at the specified (Integer) index from the collection. No return value.

Clear()

Removes all SqlBulkCopyColumnAssociator instances from the collection. No return value.

oCmd.CommandText = "SELECT * FROM Products"
Dim oReader As SqlDataReader = oCmd.ExecuteReader()

' define and open connection to use to push data into new table
' in this example, the destination table is in the same database
' so the code can use the same connection string again
Dim oDestConn As New SqlConnection(sConnectString)
oDestConn.Open()
...
Performing the Bulk Copy Operation

The core of the operation comes next (see Listing 2.2). A new SqlBulkCopy Operation instance is created using the open destination connection, and the name of the destination table is specified. No column associations are required in this case because the table schemas match, so the next step is to execute the WriteDataReaderToServer method—passing in the DataReader that is already open against the source table.

With Multiple Active Results Set (MARS) support, as described in Chapter 3, you can use the same connection for the source and destination tables if they are on the same server. However, the code we've used here can easily be adapted if the tables are in different SQL Server servers—in which case you will use two different connection strings. And if the tables are on different types of database servers, you will have to use the appropriate .NET Data Provider classes for the source reader (e.g., OleDbDataReader or Odbc DataReader).

Once the method returns, the code closes the SqlBulkCopyOperation and then closes the DataReader and the two connections to the database.

Listing 2.2 Writing the Data to the Destination Table
...
' create new SqlBulkCopyOperation instance
Dim oSqlBCP As New SqlBulkCopyOperation(oDestConn)

' specify destination table name
oSqlBCP.DestinationTableName = "Copy_Of_Products"

' write the data to the destination table
oSqlBCP.WriteDataReaderToServer(oReader)

' close SqlBulkCopyOperation instance and dispose of it
oSqlBCP.Close()

' close DataReader and connections
oReader.Close()
oDestConn.Close()
oSrcConn.Close()
...

To see the results, we added some extra code that reconnects to the database and extracts the rows that have just been copied from the destination table named Copy_Of_Products (see Listing 2.3). These rows are displayed using an ASP.NET DataGrid control that is defined elsewhere in the page.

Listing 2.3 Reading the New Data Back and Displaying It
...
' read data back from destination table and display it
' can use source connection as it points to same database
oCmd.CommandText = "SELECT * FROM Copy_of_Products"
oSrcConn.Open()
oReader = oCmd.ExecuteReader(CommandBehavior.CloseConnection)
Grid1.DataSource = oReader
Grid1.DataBind()
Label1.Text = "Contents of table 'Copy_Of_Products'"

' close DataReader and destination table connection
oReader.Close()

The screenshot in Figure 2.1 shows the results.

Figure 2.1. A simple bulk copy operation with a single table

Multiple Bulk Copy Operations and Events Using a DataSet

The previous example showed the simplest case, where we used the Sql BulkCopyOperation class to perform a bulk copy from one table to another within the same database. The next example (see Listings 2.4 to 2.9) demonstrates other uses of this class, including creating associations between tables that do not have the same schemas, copying rows from other sources (which are loaded into DataTable instances), and handling the SqlRowsCopied event.

As in the previous example, two connections are used. However, this time they are to two different databases—though the only difference as far as the code is concerned is that the two connection strings are different. Then the code removes all the existing rows from the three target tables using SQL DELETE statements. To avoid duplication, we haven't listed this code.

Fetching the Source Data

The data in this example comes from three DataTable instances within a DataSet. To demonstrate another useful approach for using the bulk copy feature, we're taking the data from an XML document stored as a disk file. We also have a schema available, so the code in Listing 2.4 loads this and the data into the DataSet by calling its ReadXmlSchema and ReadXml methods.

Listing 2.4 Loading the XML Schema and Data
...
' create DataSet and fill three tables from an XML file
' containing rows for Customers, Products, and Shippers tables
Dim oDataSet As New DataSet("Source Tables")
oDataSet.ReadXmlSchema("c:\temp\nwindcpsdata.xsd")
oDataSet.ReadXml("c:\temp\nwindcpsdata.xml")
...
Creating the SqlBulkCopyOperation Instance

The next step (see Listing 2.5) is to create the SqlBulkCopyOperation instance, using the existing destination connection. In order to handle the SqlRowsCopied event, you must add a reference to an event handler to the SqlBulkCopyOperation instance. This event handler will be executed when the event is raised. The event handler routine in this example, called OnRowsCopied, is shown in Listing 2.9 later in this section of the chapter.

By default, the SqlRowsCopied event is raised only when a Write xxx ToServer method call completes. However, the event can be raised after every specific number of rows has been copied by setting the NotifyAfter property to an Integer value that indicates this number of rows (in our example, 20). Finally, the code opens the connection ready to start the bulk copy operation.

Listing 2.5 Creating the SqlBulkCopyOperation Instance
...
' create new SqlBulkCopyOperation instance
Dim oSqlBCP As New SqlBulkCopyOperation(oDestConn)

' add an event handler for the SqlRowsCopied event
AddHandler oSqlBCP.SqlRowsCopied, _
           New SqlRowsCopiedEventHandler(AddressOf OnRowsCopied)

oSqlBCP.NotifyAfter = 20

' open connection to destination database
oDestConn.Open()
...

It's important to note that we are handling the SqlRowsCopied event purely so that you can see the tables being processed and the rows being copied into the target database tables. You do not have to handle this event when using the bulk copy feature unless you need to carry out some action (as we're doing) while the bulk copy is in progress.

Processing the Shippers Table

The first table processed is the Shippers table. The DestinationTableName property of the SqlBulkCopyOperation instance is set to the name of the target table (see Listing 2.6). This is the same as the source table, but there are differences in the column names between the source and target tables in this example.

Therefore, three SqlBulkCopyColumnAssociator instances are created, each one specifying the source and destination column names, and these are added to the collection exposed by the ColumnAssociators property of the SqlBulkCopyOperation instance. Then the code calls the WriteData TableToServer method, specifying the Shippers column in the DataSet as the source for the rows to be copied.

Listing 2.6 Processing the Shippers Table
...
' in this case two of the three column names are different
' but destination table name is the same
oSqlBCP.DestinationTableName = "Shippers"
Label1.Text &= "Copying 'Shippers' table...<br />"

oSqlBCP.ColumnAssociators.Add("ShipperID", "ShipperID")
oSqlBCP.ColumnAssociators.Add("CompanyName", "ShipperName")
oSqlBCP.ColumnAssociators.Add("Phone", "ShipperPhone")

' write the data to the destination table
oSqlBCP.WriteDataTableToServer(oDataSet.Tables("Shippers"))
...
Processing the Customers Table

To demonstrate some other ways to create column associations, the Customers table is copied to a destination table that has fewer columns, and the SqlBulkCopyColumnAssociator instances that are required use a mixture of the column names and their ordinal indexes.

We're reusing the existing SqlBulkCopyOperation instance, rather than closing it and creating a new one. This means that the column associations used for the Shippers table are still in the SqlBulkCopyColumnAssociator Collection. Before adding any new associations for the Customers table, the code in Listing 2.7 removes all the existing associations by calling the Clear method. It also has to update the DestinationTableName property.

Then the code generates the first four associations using the column index for both the source and destination columns, via a For..Next loop, and adds them to the collection. For the remaining five columns it iterates through the Columns collection exposed by the DataTable, extracting the column name and using this to specify the destination columns for the association. After all the associations are set up, it calls WriteDataTableToServer with the Customers table as the source of the rows to be copied.

Listing 2.7 Processing the Customers Table
...
' specify destination table name; in this case the column
' names are the same in the source and destination tables
oSqlBCP.DestinationTableName = "Customers"
Label1.Text &= "Copying 'Customers' table...<br />"

' have to remove existing associations first
oSqlBCP.ColumnAssociators.Clear()

' create associations for columns showing different techniques
' use Integer ordinal column positions for first four columns
For iLoop As Integer = 0 To 3
  oSqlBCP.ColumnAssociators.Add(iLoop, iLoop)
Next

' use ordinal and name for rest
Dim sColName As String
For iLoop As Integer = 4 To 8
  sColName = oDataSet.Tables("Customers").Columns(iLoop).ColumnName
  oSqlBCP.ColumnAssociators.Add(iLoop, sColName)
Next

' write rows to destination database from DataTable
oSqlBCP.WriteDataTableToServer(oDataSet.Tables("Customers"))
...
Processing the Products Table

For the final table, the code in Listing 2.8 first clears the collection of column associations and specifies the new destination table name. This time the table name is different from that of the source table, but the schemas are the same. Because the collection of column associations is now empty, the copy operation will use the existing source table schema to create the default associations automatically. So it's simply a matter of executing WriteDataTableToServer with the Products table, then closing the Sql BulkCopyOperation instance and the destination connection.

Listing 2.8 Processing the Products Table
...
' in this case the columns are the same, but the table
' name in the destination database is different
oSqlBCP.DestinationTableName = "ProductList"
Label1.Text &= "Copying 'Products' table...<br />"

' remove any existing associations - when collection
' is empty will copy all columns if column names match
oSqlBCP.ColumnAssociators.Clear()

' write rows to destination database from DataTable
oSqlBCP.WriteDataTableToServer(oDataSet.Tables("Products"))

' close SqlBulkCopyOperation instance and dispose of it
oSqlBCP.Close()
oSqlBCP = Nothing

' close destination connection
oDestConn.Close()

Label1.Text &= "Bulk copy complete."
Handling the SqlRowsCopied Event

The remaining code in this example (see Listing 2.9) is the event handler executed when the SqlRowsCopied event is raised by the SqlBulkCopy Operation instance as rows are copied to the destination tables. The event handler receives the usual reference to the "sender" object, plus an instance of the SqlRowsCopiedEventArgs class that contains information about the event.

The event handler is named OnRowsCopied in this example, and code within it can access the RowsCopied property of the SqlRowsCopiedEvent Args instance to see how many rows in total have been copied so far.

Listing 2.9 Handling the SqlRowsCopied Event
Sub OnRowsCopied(oSender As Object, oArgs As SqlRowsCopiedEventArgs)
  Label1.Text &= "Processing table '" & oSender.DestinationTableName _
              & "' - copied " & oArgs.RowsCopied.ToString() _
              & " rows so far...<br />"
End Sub

Throughout the copying of the data into the three destination tables, a Label control in the page is updated with details of the process. This includes the text generated by the code in the OnRowsCopied event handler. The screenshot in Figure 2.2 shows the result, with the NotifyAfter property of the SqlBulkCopyOperation instance set to 20 (rows).

Figure 2.2. A multiple-table bulk copy operation

    [ Team LiB ]


The New Batched Update Features

The bulk copy classes discussed in the previous section provide the most efficient way to load a set of data into SQL Server. A related capability added to ADO.NET 2.0 enables more efficient updating of data from a DataSet (via a DataAdapter) by batching inserts, updates, and deletes.

Using batched updates can provide huge improvements in performance because they require only one round-trip to the database to perform all the updates in the batch. Without this feature, each update is sent to the database individually.

In this section of this chapter we look at the new functionality implemented by the DataAdapter to support batched updates. The DataAdapter, which is used to submit updates from a DataSet, exposes a property named UpdateBatchSize. As you'll guess from the name, this property sets the number of commands that will be executed as a single batch when the Update method is executed.

Obviously, this relies on the target database or data source supporting batch command execution. If it does not, the commands can only be executed one at a time. All the classes in ADO.NET that support batch execution will take advantage of this feature where the database supports it. However, if it is not supported by the target data source, they silently degrade and use single command execution—without requiring any modification to the code or producing any change in the results visible to the user.

Batch Command Execution with a DataSet and DataAdapter

To demonstrate batch command execution in its simplest form, we look first at the way it works with a DataSet and DataAdapter. The DataAdapter uses Command objects that are assigned to the UpdateCommand, Insert Command, and DeleteCommand properties to perform the updates in the database. All that's required to change the behavior of these Command objects to use batched updates is to set the DataAdapter object's UpdateBatchSize property.

By default, this property is set to 1, giving behavior that is the same as in ADO.NET version 1.x. Each update is sent to the database individually. If the UpdateBatchSize property is set to 0, the DataAdapter will instruct the Command objects to execute all the updates for the current Update method call as a single batch. Other positive values specify the number of commands to execute as a batch before starting a new batch.

Using the DataAdapter.UpdateBatchSize Property

The results of changing the UpdateBatchSize value can be seen in the example page we provide with all the samples you can download for this book. It loads some rows from the database into a table in a DataSet, changes some of the values in these rows, and then displays the rows. Then it calls the Update method of the DataAdapter to push the changes back into the database. The default, when the UpdateBatchSize property is set to 1, is shown in Figure 2.3.

Figure 2.3. Using batch commands with a DataSet and DataAdapter

In order to show how the statements are being sent in a batch, our example code handles the RowUpdated event of the DataAdapter. Each time this event is raised as the Update method is executing, it displays details of the event in a Label control at the bottom of the page in Figure 2.3. When the value of UpdateBatchSize is 1 (the default), the RowUpdated event is raised as the UpdateCommand for each row is executed.

Similar to the bulk copy example in the previous section of this chapter, it's important to note that we are handling the RowUpdated event purely so that you can see the batching of the commands. You do not have to handle the event when performing batched updates unless you need to carry out some action (as we're doing) when each batch is completed.

Figure 2.4 shows that setting the UpdateBatchSize property to 0 causes the RowUpdated event to be called only once, when the complete batch has been executed to update all the rows. Details of the row being updated are no longer available—the properties of the "argument instance" sent to the event handler contain only the number of rows affected. Meanwhile, setting the UpdateBatchSize property to an intermediate value causes the updates to be executed in more than one batch.

Figure 2.4. Specifying different batch sizes

The Code for the Example Page

Much of the code in the example page uses standard and straightforward ADO.NET techniques to connect to a database, extract some rows, and change some values in these rows. To make it easier to see what is actually relevant, that code isn't listed here.

The important step in the process comes next (see Listing 2.10). The current setting of the option buttons at the top of the page is converted to an Integer and assigned to the UpdateBatchSize property of the DataAdapter. Then the code adds the event handler named OnRowUpdated to the DataAdapter and calls the Update method (using a CommandBuilder to create the update statements). This process is repeated each time the page is loaded in response to clicking the Go button on the page, which simply resubmits the page.

Listing 2.10 Batch Execution with a DataSet and DataAdapter
...
' set the batch size for the updates from
' the selected option button
oDA.UpdateBatchSize = Decimal.Parse(optMaxRows.SelectedValue)

' add event handlers for Updated events
AddHandler oDA.RowUpdated, _
  New SqlRowUpdatedEventHandler(AddressOf OnRowUpdated)

' create an auto-generated command builder that will
' build commands to update, insert, and delete rows
Dim oCmdBuilder As New SqlCommandBuilder(oDA)

' push changed rows back into database
oDA.Update(oDataSet, "Orders")

The event handler named OnRowUpdated is shown in Listing 2.11. It attempts to read and display the value of the OrderID column in the current row, then displays the number of rows affected by the update. When the UpdateBatchSize property is set to a value other than 1, the number of rows affected reflects the number of rows in the batch.

If the event has been fired after all the rows have been updated, at the end of the batch update, there is no row available. The Try..Catch construct prevents the exception that is raised from halting execution.

Listing 2.11 The RowUpdated Event Handler
Sub OnRowUpdated(oSender As Object, oArgs As SqlRowUpdatedEventArgs)
  Try
    Label1.Text &= "RowUpdated event fired for row " _
                & oArgs.Row("OrderID") & " &nbsp; "
  Catch
  End Try
  Label1.Text &= "Updated " & oArgs.RecordsAffected.ToString() _
              & " row(s)<br />"
End Sub
Viewing the Batch Execution

There is another way to see exactly what is going on when the DataSet (or any other class for that matter) interacts with SQL Server. Open Profiler from the SQL Server section in your Start menu, or from the Tools menu in Enterprise Manager. Select File | New | Trace to start a new trace, and in the Events tab of the Properties for the new trace select the Transactions and Stored Procedures sections. Then run your code, and you'll see the SQL statement that is being generated appear each time.

When the example page in Figure 2.4 is executed with the batch size set to 1, each update can be seen executing as a separate process within the sp_executesql stored procedure that SQL Server uses to execute a SQL statement (see Figure 2.5).

Figure 2.5. A batch size of 1, as shown in SQL Profiler

However, as you can see in Figure 2.6, setting the UpdateBatchSize to a value greater than the number of rows that have been changed results in a single execution of the sp_executesql stored procedure, with all the updates applied in one batch.

Figure 2.6. A batch size larger than 1, as shown in SQL Profiler

    [ Team LiB ]


Data Paging with the Command Object

An exciting new feature added to ADO.NET is the enabling of individual "pages" of data to be fetched from a data source. Where an application presents a rowset of data broken down into separate pages, this feature can provide a useful solution by reducing the number of rows passed over the network without requiring the application to cache them all itself.

The ExecutePageReader Method

In ADO.NET 2.0, the Command object gains a new method named Execute PageReader. This currently applies only to the SqlCommand and Oracle Command, and not to the OleDbComand and OdbcCommand. There is a single overload of this method, which takes three parameters:

Command.ExecutePageReader(command-behavior, start-index, _
                          max-page-rows)

where:

  • command-behavior is a value from the CommandBehavior enumeration, such as SingleRow or CloseConnection.

  • start-index is the zero-based index of the first row to return.

  • max-page-rows is the number of rows to return.

If there are less than max-page-rows in the rowset following the specified start-index row, only these following rows are returned. Also note that the ExecutePageReader method returns the page or set of rows according to the current values in the database. If rows have been added to or removed from the table between calls to the method, the set of rows returned may contain rows from the previous page (where rows have been added to the table prior to the current page), or rows may be skipped (where rows prior to the current page have been deleted from the table). This means that paging is not well suited for very dynamic data when exact positioning is required, but it works well for relatively static data (e.g., publishing read-only data) or where consistency from one page to the next is not required.

Using the ExecutePageReader Method

Listing 2.12 shows an example of using the ExecutePageReader method of the Command class. The DataReader returned by the ExecutePageReader method is bound to a DataGrid within the page to display the rows. Specifying the value CloseConnection for the CommandBehavior parameter of the ExecutePageReader method means that the connection is automatically closed when the DataReader is closed at the end of the code routine.

Listing 2.12 Paging with the ExecutePageReader Method
' get connection string and specify SQL statement
Dim sConnectString As String = your-connection-string "
Dim sQueryText As String = "SELECT ProductID, ProductName, " _
  & "QuantityPerUnit, UnitPrice, UnitsInStock FROM Products"

' create connection and command statement
Dim oConn As New SqlConnection(sConnectString)
Dim oCmd As New SqlCommand(sQueryText, oConn)

' figure out row index of first row for current page
Dim iStart As Integer = (iPageNumber - 1) * iPageMaxRows

' get required page of rows from database
oConn.Open()
Dim oReader As SqlDataReader
oReader = oCmd.ExecutePageReader(CommandBehavior.CloseConnection, _
                                 iStart, iPageMaxRows)

' see if there are any rows
If Not(oReader.HasRows) And iPageNumber > 1 Then
  ' prevent moving back beyond page 1
  iPageNumber -= 1
Else
  ' display rows in DataGrid on page
  dgrGrid1.DataSource = oReader
  dgrGrid1.DataBind()
End If
oReader.Close()

Notice how the code detects whether there are any rows returned for the current page, by taking advantage of the HasRows property that was added to the DataReader in version 1.1 of the .NET Framework. The result is shown in Figure 2.7.

Figure 2.7. Data paging with the ExecutePageReader method

    [ Team LiB ]


Retrieving Connection Statistics

A relatively simple technique that may be useful for monitoring the processes in progress across a connection to a data store has been added to ADO.NET 2.0 in the form of connection statistics retrieval. However, bear in mind that this produces a considerable performance hit, so it should be used only when necessary—for example, during page monitoring or debugging exercises. Note that the feature described here is enabled only for the SqlConnection class at present.

Before executing one or more queries, whether they return rows or update existing data, you can turn on statistics collection for the connection they use. Then, once the process is complete, you collect these statistics into any data structure that exposes the IDictionary interface. The simplest of these is a HashTable, as used in the example coming up shortly.

To support statistics retrieval, the SqlConnection class gains a new property and a couple of new methods in ADO.NET version 2.0 (see Table 2.9).

Table 2.9. New Property and Methods for the SqlConnection Class

Property

Description

StatisticsEnabled

Sets or returns a Boolean value indicating whether statistics retrieval is currently enabled for this Connection. The default is False.

Methods

ResetStatistics()

Removes any cached statistics for this Connection so that the next process will be recorded with values starting at 0 or the default. No return value.

RetrieveStatistics()

Retrieves the statistics from this Connection and returns them as a data structure, such as a HashTable, that supports the IDictionary interface.

A Simple Statistics Retrieval Example

Listing 2.13 shows the basic principle for collecting statistics information from a SqlConnection instance. After creating the Connection, the code sets the StatisticsEnabled property to True and calls the ResetStatistics method to zero the counters.

Then it opens the connection, executes a SQL UPDATE statement, and closes the connection afterward. The statistics are then retrieved into a HashTable, and the code iterates through all the values, displaying them in a Label control on the page.

Listing 2.13 Retrieving Connection Statistics
' create connection to database to extract some rows
Dim sConnectString As String = "your-connection-string"
Dim oConn As New SqlConnection(sConnectString)

' turn on statistics collection for connection and reset buffer
oConn.StatisticsEnabled = True
oConn.ResetStatistics()

' fetch some rows from database into a DataReader
oConn.Open()
Dim oCmd As New SqlCommand("SELECT * FROM Products", oConn)
Dim oReader As SqlDataReader = oCmd.ExecuteReader()
oReader.Close()
oConn.Close()

' retrieve Dictionary containing statistics
Dim htStats As Hashtable = oConn.RetrieveStatistics()

' display values in Label control
For Each oKey As String In htStats.Keys
  Label1.Text &= "Key: " & oKey & " = " _
              & htStats(oKey) & "<br />"
Next

The result is shown in Figure 2.8 on the next page. You can see that there are plenty of details returned, though probably the most useful ones when you are testing applications are the BytesReceived and the ExecutionTime values.

Figure 2.8. Retrieving database connection statistics

    [ Team LiB ]


SUMMARY

This chapter has covered a range of different topics and features that are new in ADO.NET version 2.0. These features are designed both to make application construction easier and to improve the usability and reach of ADO.NET.

The ability to bulk-load a database table with data has been added to ADO.NET version 2.0, using entirely managed code through several new .NET classes. This chapter demonstrated how it's done and how it can be adapted to match a range of different data management and administrative requirements.

A new "batch execution" feature has been added to ADO.NET 2.0. This feature is implemented in the DataAdapter class in order to reduce the number of round-trips made to the database when updating changes from a DataSet.

This chapter also demonstrates how the ExecutePageReader method of the Command class allows pages of data to be fetched from a data source so that an application can easily present a rowset of data broken down into separate pages.

In version 2.0 of ADO.NET, you can now easily monitor the processes that are in progress across a connection to a data store by retrieving the execution statistics from the database once the process is complete. We showed a simple example, connecting to a SQL Server database and using a SqlConnection (this is the only namespace that supports this feature at the moment).

In the next chapter, we'll look at some more new features of ADO.NET 2.0, including MARS, asynchronous connection and commands, and a brief overview of the new ObjectSpaces technology


Chapter 3. MARS, Asynchronous Commands, and ObjectSpaces

The data access classes introduced into ADO.NET version 1.0 and the techniques for using them are aimed at the needs of modern applications—in particular the two distinct requirements that these applications encounter. The first is to be able to operate in a disconnected environment, where data needs to be remoted to clients across a network. The second is to provide fast, simple, and clean access to data in read-only fashion for populating controls and other data consumers.

However, there will always be many more specific scenarios in which your code will operate. To provide more opportunities for extracting better performance from your applications and to add new features that have been requested by users, ADO.NET version 2.0 supports some new approaches to accessing and updating data. The features we'll be covering in this chapter include the following:

  • Multiple Active Results Sets (MARS), which provide the opportunity to open more than one results set over the same connection and access them concurrently

  • Asynchronous access to the results of multiple commands over the same Connection instance, plus the ability to open connections asynchronously

  • The new ObjectSpaces technology added to ADO.NET, which allows data to be defined as instances of a class but handled just like other intrinsic data types

The first two of these topics are related in that the ability to execute multiple commands asynchronously and to have more than one set of results active over the same connection provides huge opportunities for increasing performance when complex multiquery processes are required. We start this chapter with an overview of these two technologies and how they fit in with the existing data access capabilities of ADO.NET.


Expanded and Improved Data Access Features

To understand what the topics covered in this chapter are really about, how they fit together, and what kinds of problems they are designed to solve, you need to think about how existing data access methods are used when all the data an application needs resides in a single database and when it is in more than one database. These issues are becoming more common as applications have to work with multiple sources of data.

While the existing techniques from version 1.x of ADO.NET can easily accomplish the required tasks, both ease of access and performance can be improved. In particular, when the data sources are not local (as is becoming increasingly common in modern client-side applications and multiple-server Web farms), the procedural synchronous nature of current data access techniques can cause processing bottlenecks. You have to consider the individual steps that the code must go through to access the data when more than one query to the database is required.

Accessing a Single Data Source

In the simplest case, where data comes from a single server, the same connection can be used for all the commands that need to be executed to read or update data. Taking a DataSet as an example, the commands would be those required to fetch data from the database (the SelectCommand) and to update the data (the InsertCommand, UpdateCommand, and DeleteCommand). Figure 3.1 shows a simple scenario.

Figure 3.1. Multiple commands to the same database through a single connection

This process supports only synchronous access to the data. You can't read data through the connection while concurrently updating rows, and you can't execute concurrent insert, delete, and update operations. When the Fill method is called, no other access is possible through the connection until the method completes. And the Update operation automatically executes the commands sequentially when pushing updates back into the database. In most cases, however, this behavior is perfectly satisfactory and matches the requirements of the application.

Executing Multiple Commands Sequentially

An alternative situation is when you use several object instances to access data (here multiple DataReader instances, but it could be a mix of DataReader and DataSet instances) over the same connection. Again, only one of these can be executing a query over the single connection at a time. But because the ADO.NET code you write will usually call methods on the DataReader or other objects sequentially and will block on each method call until it is complete, there is no problem with using the same connection. Figure 3.2 shows this scenario.

Figure 3.2. Multiple commands for separate DataReader instances through a single connection

In this case, however, there are two areas where performance and usability issues can arise:

  1. If you open a rowset with a DataReader over the connection, you must close it before you attempt to open another DataReader or execute another command—if not, you'll get a "Connection is busy . . ." error. To be able to open more than one rowset concurrently in ADO.NET version 1.x requires that each command have its own separate connection to the database. Because the number of database connections available is limited and they are expensive in terms of resource usage, this often isn't a feasible approach.

  2. If one of the commands you're executing takes a long time to return results, the code in the application will block and wait until the query process is complete before it can execute the next command. So your code may be standing idle when it could be doing something else in the meantime.
The Requirements for Asynchronous Execution

It would be nice to be able to start the three commands together, allowing them to execute asynchronously and then handling the results as they are returned. That way the data or the results can be processed and/or displayed as they are received—making the applications appear more responsive and usable. But that requires two changes to the existing classes to make it work, and (not coincidentally) these are two of the new features in ADO.NET that we're covering in this chapter.

  1. The MARS feature in ADO.NET 2.0 allows you to have more than one rowset or command open over a single connection. In other words, you can open a rowset with a DataReader and then send other commands to the database over the same connection while the DataReader is open. You can even have more than one rowset open in separate DataReader instances at the same time and read from either one as required.

  2. The introduction of asynchronous processing in ASP.NET 2.0 for the Command class allows you to start more than one command executing concurrently and then wait until one or all of them have completed. You can also open connections asynchronously, though only a single connection request can be outstanding at any one time.

The combination of the two techniques means that, using only a single database connection, you can execute multiple commands concurrently and have the rowsets they return open at the same time. And, of course, update commands can also be executed over the same connection at the same time, even when rowsets are open on that connection. This provides a huge opportunity for increased performance and responsiveness for your applications and Web pages. However, bear in mind that there are scenarios, such as when working with multiple separate data sources, when MARS is not directly useful.

Accessing Multiple Data Sources

When it comes to accessing multiple different data sources in an application, you have no option but to use separate connections. Of course, a connection can be to only a single database. So, for multiple data access operations against separate databases, you end up with something like the situation shown in Figure 3.3.

Figure 3.3. Multiple commands to different databases through separate connections

The MARS feature may not provide any advantage in this situation because each command will use its own dedicated connection. However, it may be that you have some hybrid scenario, where some commands are to the same database (in which case they can share a connection) while others are to a different database through a different connection (see Figure 3.4).

Figure 3.4. A mix of commands and connections for different databases

Here, MARS can provide the opportunity for improved access, for example, being able to open a rowset with a DataReader and then execute commands against the same database while the DataReader is open. But what is also useful, in both of these scenarios (Figures 3.3 and 3.4), is the ability to execute commands asynchronously. For example, you can start execution of the command that fetches the rows for the DataReader, then—while it is running—execute the Fill method for the DataSet.

Opening Connections Asynchronously

In ADO.NET 2.0 it's now also possible to open connections asynchronously. This offers even more opportunity to increase efficiency, especially when access to different data sources is required. It means that your code can continue to execute while waiting for a connection to be established—particularly useful when connecting to a remote data source rather than a local one.

When multiple connections are required, a common scenario is for the code to open these one by one. In fact, there's no alternative to this in ADO.NET 1.x. Being able to open each one asynchronously means that you can start to retrieve or update the data over that connection while waiting for other connections to open. You may even end up actually using fewer connections because you may be able to close some and return them to the connection pool before the last one has opened. Running the same code synchronously would mean holding all of the connections open until all of the processing completes.

Of course, you can open the connection asynchronously in situations where you need only one connection, but the advantage is not as obvious unless you start the opening process at the beginning of the code and then do some other work while you wait for it to open. The danger here is that you may hold on to the connection for longer than necessary, robbing other applications (and other instances of this application) of available connections and producing no overall advantage.


MARS in Action

ADO.NET 2.0 provides a great new feature called MARS. We discussed how useful this is in the first sections of this chapter, and here you'll see an example of it in action.

Listing 3.1 demonstrates how useful MARS can be when dealing with related tables (though this is not, of course, the only way it can be used). The sample Northwind database we're using here contains an Orders table and a related Order Details table. The example fetches the first five rows from the Orders table, displays a couple of column values, and then—while this rowset is open over the connection—executes another command that returns a rowset of the related Order Details rows for the current order.

In future releases (following the Technology Preview release) you won't have to do anything to enable MARS. As long as you have version 9.0 of the Microsoft Data Access Components (MDAC) library on the client machine, MARS will just work transparently. However, for the Technology Preview version, you must include in the connection string for your database the token "use mdac9=true;". For Windows Forms (compiled) applications, see the topic Using New ADO.NET Features That Require MDAC 9.0 in the Overview of ADO.NET section of the SDK for details of how to create an application manifest that specifies MDAC 9.0.

The code starts in the usual way by creating a single Connection and then two Command instances. The first Command selects the first five rows from the Orders table. The second selects the details of each line on an order, using a parameter named @orderid that will be set to the ID of the current order when this statement is executed. A parameter that corresponds to the @orderid parameter in the SQL statement is then added to the Command object's Parameters collection.

Listing 3.1 An Example of MARS in Action
' get connection string and specify SQL statements
Dim sConnectString As String = "your-connection-string"
Dim sOrderSQL As String = "SELECT TOP 5 OrderID, ShipName FROM Orders"
Dim sOrderLineSQL As String = "SELECT OrderID, Quantity, " _
           & "[Order Details].UnitPrice As Price, ProductName " _
           & "FROM [Order Details] JOIN Products " _
           & "ON [Order Details].ProductID = Products.ProductID " _
           & "WHERE OrderID = @orderid"

' create connection and two commands
Dim oConn As New SqlConnection(sConnectString)
Dim oCmdOrder As New SqlCommand(sOrderSQL, oConn)
Dim oCmdOrderLine As New SqlCommand(sOrderLineSQL, oConn)

' add parameter to second command to use for selecting order
oCmdOrderLine.Parameters.Add("@orderid", SqlDbType.Char, 5)
...

Fetching the Orders Rows

In Listing 3.2 the code continues by declaring a couple of DataReader variables and executing the command to get the Orders rowset. The While construct then starts by reading the first row and displaying the order ID and customer name.

Listing 3.2 Executing the Command to Get the Orders Rowset
...
' declare two DataReader variables and open connection
Dim oDROrder, oDROrderLine As SqlDataReader
oConn.Open()

' get rowset from Orders table
oDROrder = oCmdOrder.ExecuteReader(CommandBehavior.CloseConnection)

' iterate through orders
While oDROrder.Read()

  ' display order ID and name
  lblResult.Text &= oDROrder("OrderID").ToString() & " " _
                 & oDROrder("ShipName") & "<br />"
  ...

Fetching the Order Details Rows

Having discovered the ID of the current order, the code in Listing 3.3 now sets the value of the @orderid parameter and executes the second command to fetch the matching rows from the Order Details table. It iterates through this rowset, displaying the product name and quantity, and then closes this DataReader. However, the first DataReader is still open against the Orders table, so when the While loop ends, the code goes back and reads the next Orders row, then repeats the process.

Listing 3.3 Executing the Command to Get the Order Details Rowset
  ...
  ' set parameter for second command to value of order ID
  oCmdOrderLine.Parameters("@orderid").Value = oDROrder("OrderID")

  ' get list of lines for this order from Order Details table
  oDROrderLine = oCmdOrderLine.ExecuteReader()

  ' iterate through order lines displaying details
  While oDROrderLine.Read()
    lblResult.Text &= oDROrderLine("Quantity") & " " _
                   & oDROrderLine("ProductName") & " at " _
                   & oDROrderLine("Price") & "<br />"
  End While

  ' close DataReader for Order Details table
  oDROrderLine.Close()
  lblResult.Text &= "<br />"

End While

' close DataReader for Orders table
oDROrder.Close()

Once all the Orders rows have been read and the related Order Details rows for each one fetched and displayed, this DataReader is closed. Because the CloseConnection option was specified when this DataReader was opened, the connection to the database is closed automatically at this point.


Asynchronous Command Execution

When accessing data in a data store such as a database using ADO or ADO.NET 1.x, each query is executed in turn, and the code waits for each one to complete before processing the next one. This allows the same connection to be used if the queries access the same data store or a different connection if they access different data stores.

In future releases (following the Technology Preview release) you won't have to do anything to enable the asynchronous command execution feature. As long as you have version 9.0 of the MDAC library on the client machine, asynchronous command execution will just work transparently. However, for the Technology Preview version, you must include in the connection string for your database the token "use mdac9=true;". For Windows Forms (compiled) applications, see the topic Using New ADO.NET Features That Require MDAC 9.0 in the Overview of ADO.NET section of the SDK for details of how to create an application manifest that specifies MDAC 9.0.

The previous section of this chapter demonstrates how MARS can be used to enable a single connection to handle multiple sets of results from the same data store, but this still doesn't solve the issue of executing more than one command concurrently. Neither does it help when you want to access different data stores concurrently.

However, ADO.NET 2.0 now includes features that allow multiple commands to execute concurrently and asynchronously. This means, in essence, that code can create and start execution of several commands without having to wait for them to complete individually. However, unless you use the MARS feature described earlier in this chapter, you must provide a separate connection for each one.

The asynchronous model adopted in ADO.NET 2.0 aims to give maximum flexibility when using this feature by implementing three distinct techniques:

  1. The polling model, where you start all the processes and then repeatedly test each one for completion. Other tasks can be accomplished between each test.

  2. The callback model, where you specify a routine that will be executed when each process is complete.

  3. The wait model, where a wait handle is created for each method that starts the command execution. This allows control of the timeout for each process, and the code can sleep while waiting for a result to be returned or for a process to time out.

It's also possible to open connections to a data source asynchronously, allowing code to continue with other work while waiting for the connection to be made. The process is much the same as executing a command, and we look at this topic later in the chapter. But before we examine the actual code required to perform asynchronous command execution, the next section lists the classes, properties, and methods used. At present, this feature is implemented only for the classes in the SqlClient namespace.

Asynchronous Classes, Methods, and Properties

To a greater extent, all three asynchronous execution techniques use the same methods that are added to the SqlCommand class in ADO.NET 2.0 to start a process and then capture the results. These methods are listed in Table 3.1.

Table 3.1. Asynchronous Execution Methods of the SqlCommand Class

Method

Description

BeginExecuteNonQuery

Starts an asynchronous query to the data source that is not expected to return any rows. The return value is a reference to an object that implements the IAsyncResult interface, in this case an instance of the SqlAsyncResult class, which is used to monitor and access the process as it runs and when it is complete. There are two overloads for this method:

async-result = command.BeginExecuteNonQuery()

Starts execution of this command and returns a reference to a SqlAsyncResult instance for this process.

async-result = command.BeginExecuteNonQuery(callback, state)

As above, but takes an AsyncCallback instance that specifies the callback routine to be executed when the process is complete, plus an Object that defines state information for the process.

EndExecuteNonQuery

Once the command execution started by a BeginExecute NonQuery call has completed, this method is called to access the results. The single parameter is a reference to the SqlAsyncResult for the command, and the method returns an Integer that is the number of rows affected (in the same way the ExecuteNonQuery method does for synchronous processes).

rows-affected = command.EndExecuteNonQuery(async-result)

BeginExecuteReader

Starts an asynchronous query to the data source that is expected to return some rows. The return value is a reference to an object that implements the IAsyncResult interface, in this case an instance of the SqlAsyncResult class, which is used to monitor and access the process as it runs and when it is complete. There are four overloads for this method:

async-result = command.BeginExecuteReader()

Starts execution of this command and returns a reference to a SqlAsyncResult instance for this process.

async-result = command.BeginExecuteReader(command-behavior)

Starts execution of this command using the specified command behavior value, such as CloseConnection, and returns a reference to a SqlAsyncResult instance for this process.

async-result = command.BeginExecuteReader(callback, state)

As above, but takes an AsyncCallback instance that specifies the callback routine to be executed when the process is complete, plus an Object that defines state information for the process.

async-result = command.BeginExecuteReader(callback, state, command-behavior)

As above, but using the specified command behavior value, such as CloseConnection.

EndExecuteReader

Once the command execution started by a BeginExecute Reader call has completed, this method is called to access the results. The single parameter is a reference to the SqlAsyncResult for the command, and the method returns a DataReader that references the rowset(s) returned by the query (in the same way the ExecuteReader method does for synchronous processes). The single overload is:

data-reader = command.EndExecuteReader(async-result)

BeginExecuteXmlReader

Starts an asynchronous query to the data source that is expected to return some rows as XML elements. The return value is a reference to an object that implements the IAsyncResult interface, in this case an instance of the SqlAsyncResult class, which is used to monitor and access the process as it runs and when it is complete. There are two overloads for this method:

async-result = command.BeginExecuteXmlReader()

Starts execution of this command and returns a reference to a SqlAsyncResult instance for this process.

async-result = command.BeginExecuteXmlReader(callback, state)

As above, but takes an AsyncCallback instance that specifies the callback routine to be executed when the process is complete, plus an Object that defines state information for the process.

EndExecuteXmlReader

Once the command execution started by a BeginExecute XmlReader call has completed, this method is called to access the results. The single parameter is a reference to the SqlAsyncResult for the command, and the method returns an XmlReader that references the XML data returned by the query (in the same way the ExecuteXml Reader method does for synchronous processes). The single overload is:

Xml-reader = command.EndExecuteXmlReader(async-result)

The IAsyncResult Interface and SqlAsyncResult Class

All of the methods for the SqlCommand class listed in the previous section that start an asynchronous process return a reference to an object that exposes the IAsyncResult interface. The specific class used for asynchronous command execution is SqlAsyncResult, and the properties of this class are shown in Table 3.2.

Table 3.2. Asynchronous Execution Properties of the SqlCommand Class

Property

Description

AsyncState

Returns an Object that describes the state of the process. Read-only.

AsyncWaitHandle

Returns a WaitHandle instance that can be used to set the timeout, force code to wait for completion, and test when and how the process completed. Read-only.

CompletedSynchronously

Returns a Boolean value indicating whether the process was executed synchronously (True) or asynchronously (False). Read-only.

IsCompleted

Returns a Boolean value indicating whether the process is complete (True) or is still executing (False). Read-only.

The AsyncCallback Delegate

The AsyncCallback delegate is a system class that is not specifically allied to ADO.NET but is used by many objects within the Framework. It is used to specify the callback routine or function that will be executed when the asynchronous process completes. The only member of this class that's important here is the constructor, shown in Table 3.3, which is used to generate an instance of the class to pass to the BeginExecuteNonQuery, BeginExecute Reader, and BeginExecuteXmlReader methods of the Command.

Table 3.3. The AsyncCallback Constructor

Constructor

Description

AsyncCallback(callback-handler)

Creates a new AsyncCallback instance that defines the name of the method, function, or routine that will be called when the process is complete.

See the section on the asynchronous callback model later in this chapter for more details.

The WaitHandle Class

The WaitHandle class is an abstract system class used for many purposes within the Framework. By calling the methods of the WaitHandle, you effectively tell the code to wait for the specified process to complete. If there are multiple processes involved, you can create an array of WaitHandle instances and then tell the code to "sleep" until any one of them completes. The WaitHandle class exposes a Static property, shown in Table 3.4, for the timeout that will be allocated to the process.

There are also three methods that allow you to start the wait process and one to close and release the handle afterwards, as shown in Table 3.5.

Table 3.4. The Properties of the WaitHandle Class

Property

Description

WaitTimeout

An Integer value that is the timeout for the process in milliseconds.

Table 3.5. The Methods of the WaitHandle Class

Method

Description

WaitOne

Waits for a single process to complete or time out. Returns a Boolean value that is True if the process completed or False if it timed out. There are three overloads of this method:

completed = WaitHandle.WaitOne()

Starts the wait process using the default value for the timeout.

completed = wait-handle.WaitOne(milliseconds, exit-context)

Starts the wait process, with the timeout set to the number of milliseconds specified as an Integer. The exit-context parameter specifies whether the method will reacquire a synchronized context and should be set to False for the ADO.NET asynchronous execution.

completed = WaitHandle.WaitOne(time-span, exit-context)

Starts the wait process, with the timeout specified using a TimeSpan instance that describes the timeout in terms of the time unit and value. The exit-context parameter is as described above.

WaitAny

A Static method that waits for any one of the processes in an array of WaitHandle instances to complete or time out. Returns an Integer value that is the index within the array of the first process that completed. If any of the processes times out, the method returns the value of the timeout instead of the handle index. There are three overloads of this method:

index = WaitHandle.WaitAny(wait-handle-array)

Starts the wait process for any of the WaitHandles, using the default value for the timeout.

index = WaitHandle.WaitAny(wait-handle-array, milliseconds, exit-context)

Starts the wait process for any of the WaitHandles, with the timeout in milliseconds and the exit-context parameter as described for the WaitOne method.

index = WaitHandle.WaitAny(wait-handle-array, time-span, exit-context)

Starts the wait process for any of the WaitHandles, with the timeout specified as a TimeSpan instance and the exit-context parameter as described for the WaitOne method.

WaitAll

A Static method that waits for all of the processes in an array of WaitHandle instances to complete or time out. Returns a Boolean value that is True if all the processes completed or False if any one of them timed out. There are three overloads of this method:

completed = WaitHandle.WaitAll(wait-handle-array)

Starts the wait process for all the WaitHandles, using the default value for the timeout.

completed = WaitHandle.WaitAll(wait-handle-array, milliseconds, exit-context)

Starts the wait process for all the WaitHandles, with the timeout in milliseconds and the exit-context parameter as described for the WaitOne method.

completed = WaitHandle.WaitAll(wait-handle-array, time-span, exit-context)

Starts the wait process for all the WaitHandles, with the timeout specified as a TimeSpan instance and the exit-context parameter as described for the WaitOne method.

Close

Closes and releases the WaitHandle instance and any resources it is using.

See the section on the asynchronous wait model later in this chapter for more details.

The Asynchronous Polling Model

The simplest approach to handling asynchronous execution of one or more commands is through the polling model. It simply involves starting off the process (in Listing 3.4 this is a SQL UPDATE statement, but it could be any statement or stored procedure that does not return rows) and then repeatedly checking the IsCompleted property of the SqlAsyncResult instance until it returns True.

Of course, the code can go off and do other things between checking to see whether the process is complete. However, this approach is not recommended unless it is a simple and "tight" loop that handles a specific required task and uses only minimal processing time. If there are large or unrelated tasks to accomplish, you should consider using the callback or wait models instead.

Listing 3.4 An Example of the Asynchronous Polling Model
' create connection and command as usual
Dim oConn As New SqlConnection("your-connection-string")
oConn.Open()
Dim oCmd As New SqlCommand("UPDATE table SET ....", oConn)

' start the command execution, and collect the AsyncResult instance
Dim oResult As SqlAsyncResult = oCmd.BeginExecuteNonQuery()

While Not oResult.IsCompleted

  ' do something else

End While

' must now be complete, so get result from AsyncResult instance
Dim iRowsAffected As Integer = oCmd.EndExecuteNonQuery(oResult)

If the query you execute will return rows, you just swap the BeginExecuteNonQuery and EndExecuteNonQuery method calls with BeginExecuteReader and EndExecuteReader, then access the DataReader that's returned after the EndExecuteReader call. Similar logic applies if you want to return an XmlReader, whereupon you would use the BeginExecuteXmlReader and EndExecute XmlReader methods.

All this assumes that you want to execute only one command and that you need to wait for it to complete. It's also possible to start off more than one Command execution process; however, in this case you must execute each one over a separate connection unless you are using the MARS feature of ADO.NET 2.0.

The Asynchronous Callback Model

The second approach to asynchronous execution involves providing a routine in the code that will act as a callback. It will be executed when the action you specify occurs, rather like the way that an event handler is called to handle a user's interaction with an application.

To specify the callback routine you create a new instance of the Async Callback class, providing the name of that routine as the parameter, and pass this AsyncCallback instance into the method you use to start execution of the command. In Listing 3.5, we use a query that returns some rows, so it is executed with the BeginExecuteReader method.

Listing 3.5 An Example of the Asynchronous Callback Model
' create connection and command as usual
Dim oConn As New SqlConnection("your-connection-string")
oConn.Open()
Dim oCmd As New SqlCommand("SELECT * FROM table", oConn)

' start the command execution, and collect the AsyncResult instance
oCmd.BeginExecuteReader(New AsyncCallback(AddressOf DisplayRows), _
                        CommandBehavior.CloseConnection)

' do something else here
' ...
' and then finish

Code can continue to do other things after the query in the Command has been executed. During this period, as soon as the query has completed, the callback handler named DisplayRows is called (on an arbitrary thread, which is unlikely to be the same thread as the rest of the code). The callback event handler receives a reference to an AsyncResult instance that is created automatically by the BeginExecute xxx method, from which it can obtain a reference to the results of the query by calling the appropriate EndExecute xxx method, as shown in Listing 3.6.

Listing 3.6 The DisplayRows Callback Event Handler
' callback handler routine to display results
Sub DisplayRows(oResult As SqlAsyncResult)

  ' must now be complete, so get result from AsyncResult instance
  Dim oReader As DataReader = oResult.EndExecuteReader(oResult)

  ' display results using oReader - which is a normal DataReader
  ' ...

End Sub

The Asynchronous Wait Model

The most complex of the asynchronous methods is also the most efficient if all you want to do is start some commands running against one or more data sources (they can all use separate connections and therefore different databases if required) and not execute other code in the meantime. You simply want to wait until one, more, or all of the commands have completed and then perhaps display some results.

In this case, you start each process in the same way as the previous examples but then use the AsyncResult to create a WaitHandle that you use to monitor each process. If there is more than one process and you want to wait for one or all of these to complete, you use an array of WaitHandle references. The code in Listing 3.7 shows the simplest case of waiting for one command to complete.

Listing 3.7 A Simple Example of the Asynchronous Wait Model
' create connection and command as usual
Dim oConn As New SqlConnection("your-connection-string")
oConn.Open()
Dim oCmd As New SqlCommand("UPDATE table SET ...", oConn)

' execute command and get back AsyncResult instance
Dim oResult As SqlAsyncResult = oCmd.BeginExecuteNonQuery()

' use AsyncResult instance to create a WaitHandle
Dim oHandle As WaitHandle = oResult.AsyncWaitHandle

' tell code to sleep here until process is complete
oHandle.WaitOne()

' process is complete, so get results
Dim iRowsAffected As Integer = com.EndExecuteNonQuery(oResult)
Checking Whether the Process Timed Out

In the code in Listing 3.7, there is no check to see whether the process completed successfully or failed through a timeout. It's easy enough to specify the timeout you want and then to detect whether the process actually did complete within that period, as shown in Listing 3.8.

Listing 3.8 Waiting for a Single Process to Complete or Time Out
...
' use AsyncResult instance to create a WaitHandle
Dim oHandle As WaitHandle = oResult.AsyncWaitHandle

' set the timeout to 10 seconds
oHandle.WaitTimeout = 10000

' tell code to sleep here until the process is complete
' on return, check the status of the process
If oHandle.WaitOne() = True Then

  ' process is complete, so get results
  Dim iRowsAffected As Integer = oCmd.EndExecuteNonQuery(oResult)

Else

  ' process timed out
  lblError.Text = "Process failed to complete within 10 seconds"

End If
Using Multiple Wait Handles

If you are executing more than one process at a time, you use an array of WaitHandle instances to detect when one or all of the processes have completed. First you need to create the separate Connection and Command instances (unless you are using MARS over a single connection), as shown in Listing 3.9.

Listing 3.9 An Example of Multiple Wait Handles
' create three connections and three commands, and open connections
Dim oConnA As New SqlConnection("connection-string-A")
Dim oCmdA As New SqlCommand("SELECT * FROM tableA", oConnA)
oConnA.Open()

Dim oConnB As New SqlConnection("connection-string-B")
Dim oCmdB As New SqlCommand("UPDATE tableB SET ...", oConnB)
oConnB.Open()

Dim oConnC As New SqlConnection("connection-string-C")
Dim oCmdC As SqlCommand
oCmdC = New SqlCommand("SELECT * FROM tableC FOR XML AUTO", oConnC)
oConnC.Open()
...

Now the code can start the three processes running, collecting an AsyncResult instance for each one (Listing 3.10). These AsyncResult instances are queried to get the three WaitHandles, and they are assigned to an array named aHandle. In Listing 3.10, the code will wait for all three processes to complete because it uses the static WaitAll method of the WaitHandle class and passes in the array of three WaitHandle instances. At the point where all are complete, the code "wakes up" and retrieves the three results using the appropriate EndExecute xxx methods.

Listing 3.10 Executing the Commands with Multiple Wait Handles
...
' execute commands and get back AsyncResult instances
Dim oResultA As SqlAsyncResult = oCmdA.BeginExecuteReader()
Dim oResultB As SqlAsyncResult = oCmdB.BeginExecuteNonQuery()
Dim oResultC As SqlAsyncResult = oCmdC.BeginExecuteXmlReader()

' use all three AsyncResult instances to create WaitHandle array
Dim aHandle(2) As WaitHandle
aHandle(0) = oResultA.AsyncWaitHandle
aHandle(1) = oResultB.AsyncWaitHandle
aHandle(2) = oResultC.AsyncWaitHandle

' tell code to sleep here until all the processes are complete
WaitHandle.WaitAll(aHandle)

' processes are all complete, so get results
Dim oDataReader As DataReader = oCmdA.EndExecuteNonQuery(oResultA)
Dim iRowCount As Integer = oCmdB.EndExecuteNonQuery(oResultB)
Dim oXmlReader As XmlTextReader = oCmdC.EndExecuteXmlReader(oResultC)

If you want to test whether any of the processes did not complete because of a timeout, you just check the returned value from the WaitAll method. If it's False, at least one of the processes timed out. And if you want to specify the timeout, you can do so in the call to the WaitAll method. The following code sets the timeout to 10 seconds:

WaitHandle.WaitAll(aHandle, 10000, False)
Handling Multiple Processes as Each One Completes

Rather than waiting for all processes to complete, you can improve your code efficiency even more by using the WaitAny method. This wakes up your code as soon as any one of the processes completes. You handle the results of this process and then put the code back to sleep again until the next process completes. This must be repeated until all the processes in the array of WaitHandle instances are complete or have timed out.

The code in Listing 3.11 shows how this works. The code that creates the three connections and three commands, executes these commands, and creates the array of WaitHandle instances is not repeated here. All this is the same as in Listings 3.9 and 3.10.

What differs is that the code calls the WaitAny method this time. The Integer value this method returns is either the index within the WaitHandle array of the process that completed successfully or the value of the timeout if one of the processes timed out before it was complete. Note that the order in which processes complete or time out will probably not be the same as the order of their WaitHandle instances within the array.

To be sure of handling every one of the commands that are executing asynchronously, you must force the code to call the WaitAny method once for every command that is executing. If you call WaitAny only once, you'll get only one indication of a process completion. Trying to access all the results at this point is likely to cause an error because the rest of the commands may not have completed when you call the respective End Execute xxx method.

So, in Listing 3.11, a For..Next loop executes the WaitAny method three times. As each process completes, the code checks the index value to see which process just finished and calls the appropriate EndExecute xxx method. (The results can also be displayed at this point to take maximum advantage of the asynchronous processing model.)

Listing 3.11 Handling Completion of Multiple Asynchronous Processes
' code as before to create and execute three commands
...

' code here to create array of WaitHandle instances
...

' following code is executed once for each WaitHandle instance
' so that all three end up being handled at some point
For iLoop As Integer = 1 To 3

  ' tell code to sleep here until any one of the processes completes
  ' collect the index of the next one to complete successfully
  ' NB: this will be the (Static) timeout value if one times out
  Dim iIndex As Integer = WaitHandle.WaitAny(aHandle, 5000, False)

  ' one of the processes has completed or timed out
  Select Case iIndex

    Case 0:
      ' command A completed successfully
      Dim oDataReader As DataReader
      oDataReader = oCmdA.EndExecuteReader(oResultA)

    Case 1:
      ' command B completed successfully
      Dim iRowCount As Integer
      iRowCount = oCmdB.EndExecuteNonQuery(oResultB)

    Case 2:
      ' command C completed successfully
      Dim oXmlReader As XmlTextReader
      oXmlReader = oCmdC.EndExecuteXmlReader(oResultC)

    Case WaitHandle.WaitTimeout:
      ' this one timed out - could use Case Else here instead
      lblError.Text &= "One process has timed out"

  End Select

Next

Canceling Processing for Asynchronous Commands

There may be an occasion where a process is executing asynchronously and you want to provide the user with the opportunity to cancel it partway through. To do so, all you have to do is call the Cancel method of the Command instance, as shown in Listing 3.12.

Listing 3.12 Canceling Asynchronous Command Execution
...
Dim oResult As SqlAsyncResult = oCmd.BeginExecuteNonQuery()

' ... do something else here ...

If (some-condition-is-met) Then
  oCmd.Cancel()
End If

Asynchronously Opening Connections

As well as executing a Command asynchronously, ADO.NET 2.0 also allows you to open connections asynchronously. This is useful, of course, when you are accessing more than one database at a time so that you can execute multiple asynchronous commands. The principle for asynchronous connections is the same as that we looked at for commands, with the same three options of using the polling, callback, or wait approaches. And, like the asynchronous command execution feature, asynchronous connection opening applies only to the SqlClient namespace classes—in this case, SqlConnection.

The SqlConnection class exposes a couple of properties that are useful when working asynchronously, as shown in Table 3.6.

There is also a method that allows you to start the process of opening the connection and one to complete the process, as shown in Table 3.7.

Table 3.6. The Asynchronous Properties of the SqlConnection Class

Property

Description

Asynchronous

Returns a Boolean value indicating whether the connection has been opened asynchronously. Read-only.

State

Returns a value from the System.Data.Connection State enumeration indicating the state of the connection. The possible values are:

Closed, Connecting, Open, Executing, Fetching, and Broken.

Table 3.7. The Asynchronous Methods of the SqlConnection Class

Method

Description

BeginOpen

Starts opening a connection asynchronously. The return value is a reference to an object that implements the IAsyncResult interface, in this case an instance of the SqlAsyncResult class, which is used to monitor and access the process as it runs and when it is complete. There are two overloads for this method:

async-result = connection.BeginOpen()

Starts opening the connection as detailed above.

async-result = connection.BeginOpen(callback, state)

As above, but takes an AsyncCallback instance that specifies the callback routine to be executed when the process is complete, plus an Object that defines state information for the process.

EndOpen

Ends the process of opening the connection specified in the IAsyncResult class instance passed to the method. The single overload is:

connection.EndOpen(async-result)

Asynchronous Connection Examples

The code in Listings 3.13, 3.14, and 3.15 demonstrates the three asynchronous models for opening a connection. The polling model, shown in Listing 3.13, collects the AsyncResult instance from the BeginOpen method. Then it repeatedly tests the connection state until the connection is open and calls the EndOpen method with the AsyncResult instance.

Listing 3.13 Opening a Connection Using the Asynchronous Polling Model
' create a new connection and open asynchronously
Dim oConn As New SqlConnection("your-connection-string")
Dim oResult As SqlAsyncResult = oConn.BeginOpen()

While Not oConn.State = ConnectionState.Open

  ' do something else

End While

oConn.EndOpen(oResult)

The callback model, shown in Listing 3.14, creates a new AsyncCallback instance that defines the name of the callback routine and passes this to the BeginOpen method to start opening the connection. Once the connection is open, the callback routine (named ConnectionOpened in this example) is called.

Listing 3.14 Opening a Connection Using the Asynchronous Callback Model
' create a new connection and open asynchronously
Dim oConn As New SqlConnection("your-connection-string")
oConn.BeginOpen(New AsyncCallback(AddressOf ConnectionOpened))

' do something else here
' ...
' and then finish
'----------------

' callback handler routine
Sub ConnectionOpened(oResult As SqlAsyncResult)

  oConn.EndOpen(oResult)

End Sub

Finally, the wait model, shown in Listing 3.15, creates a new WaitHandle for the process from the AsyncResult instance and calls its WaitOne method to "sleep" the code until the connection is open, then calls the EndOpen method.

Listing 3.15 Opening a Connection Using the Asynchronous Wait Model
' create a new connection and open asynchronously
Dim oConn As New SqlConnection("your-connection-string")
Dim oResult As SqlAsyncResult = oConn.BeginOpen()

' use AsyncResult instance to create a WaitHandle
Dim oHandle As WaitHandle = oResult.AsyncWaitHandle

' tell code to sleep here until process is complete
oHandle.WaitOne()

' process is complete
oConn.EndOpen(oResult)

Catching Asynchronous Processing Errors

One thing that the examples of asynchronous processing do not demonstrate is handling any errors that might occur. Connection errors are particularly common in data access applications, and you will probably decide to use a Try..Catch construct and error handling code in every case.

However, remember that in asynchronous processing situations, the error will be raised somewhere between calling the Begin xxx and End xxx methods—rather than at the specific point where you call the Open method (for a Connection) or the Execute xxx method (for a Command) when using synchronous processing. This means that you must enclose the complete section of code in a Try..Catch construct and then use smaller nested constructs to catch specific errors as and where required.


An Overview of ObjectSpaces

ObjectSpaces is probably one of the most revolutionary and exciting of the new techniques being introduced into .NET as far as data access is concerned. At the moment it is less than fully mature, but it already provides a useful framework for building applications that access real-world data.

The concept of ObjectSpaces is to allow you to access, manipulate, and update data in a relational database using standard .NET "objects," rather than having to write stored procedures or build SQL statements that specifically match the data you want to work with to the structure of the source table in the database. And, in the future, the technology will be extended to support access to information in other types of data stores, not just relational databases. However, in the current release, support extends only to SQL Server 2000 and SQL Server "Yukon."

The objects you use to store your data are ordinary .NET classes that can contain public and private methods and properties and expose one or more constructors. The ObjectSpaces technology then maps these objects to the tables and columns in the database and automatically wires them up so that you can read, manipulate, and update the data in the tables simply by calling methods within the ObjectSpaces interface.

For all this to work, ObjectSpaces exposes a series of classes that include the following:

  • A base class called ObjectSpace that performs the basic operations on the data and from which the class instances you require to manipulate the data can be created

  • A class named ObjectSources that contains the connection and transaction information to be used when accessing databases

  • A class named ObjectQuery, together with a language called OPath (broadly based on XPath), which is used to select objects from a results set

  • A class named ObjectReader that follows the same broad principles as other Reader classes and is used to fetch objects from the database and expose them as a stream

  • A class named ObjectSet that broadly mirrors the DataSet but is used to hold copies of objects, allowing them to be remoted or accessed in a disconnected environment

  • Two classes named ObjectList and ObjectHolder, which are used to manage delayed loading for objects and hence maximize performance

Basic ObjectSpaces Techniques

The basic principles for using ObjectSpaces are much the same as the existing .NET data access approaches. You create an ObjectSpace instance, specifying the mappings to the data and the ObjectSources connection and transaction information:

Dim oSpace As ObjectSpace
oSpace = New ObjectSpace(mappings-file, connection)

The mappings-file parameter can be the path to a file containing the mappings or a MappingSchema instance that defines the mappings. The connection parameter can be a standard database connection (Sql Connection or DbConnection) or an instance of an ObjectSources object that defines connection and transaction information.

Then you can create an ObjectReader instance from the ObjectSpace. Any filtering or selection operation you need to accomplish is specified in an ObjectQuery instance that indicates the object type and uses OPath as the query language. The following code retrieves objects of type Customer, where the region value is Europe:

Dim oQuery As New ObjectQuery(Customer, "region='Europe'")
Dim oReader As ObjectReader = oSpace.GetObjectReader(oQuery)

The Customer object instances can then be retrieved from the DataReader using the Read method. Each instance is cast (converted) to the correct Customer type as it is retrieved from the ObjectReader (see Listing 3.16).

Listing 3.16 Retrieving and Accessing a Customer Object
...
Dim oCust As Customer

While oReader.Read()

  ' retrieve object and cast to Customer type
  oCust = CType(oReader.Current, Customer)

  ' display or use values in Customer object
  lblResult.Text &= oCust.CustomerName

End While
oReader.Close

In future releases, ObjectSpace will also be able to return an Object Reader that uses paging, rather like the new ExecutePageReader method does for a standard DataReader. However, the exact details of how or when this will be implemented are still under discussion.

Using an ObjectReader

An ObjectReader provides stream-based access to objects, rather like a DataReader does to data rows when executing a normal SQL query. You can use it for data binding to Windows Forms and Web Forms controls. However, bear in mind that, like all Reader objects, it works in the connected scenario only, and you must be sure to close it when you have finished using it.

Filling and Manipulating an ObjectSet

To hold copies of objects for remoting, processing, and updating, you use an ObjectSet. The ObjectSet is basically a collection of your objects, but it also performs other tasks like tracking the original values when objects are updated and allowing the collection to be sorted.

An ObjectSet is created using the GetObjectSet method of the current ObjectSpace instance. The objects you require can be specified using an existing ObjectQuery instance:

Dim oSet As ObjectSet = oSpace.GetObjectSet(oQuery)

or by specifying the object type as a Type instance, and an OPath query as a String:

Dim oSet As ObjectSet = oSpace.GetObjectSet(object-type, query)

If you only want to retrieve a single instance of an object from the database, you can use the GetObject method of the ObjectSpace instead. It uses the same combination of parameters as the Get ObjectSet method but just returns a new instance of the object. However, GetObject will raise an error if the query results in more than one object being returned.

The ObjectSet also exposes methods that allow you to add objects to the collection or remove them from the collection. The Add method takes either a single object instance or a collection of object instances and adds them to the ObjectSet collection. To remove an object you use the MarkFor Deletion method. As the name suggests, the object remains in the ObjectSet but is marked as being deleted. When the changes to the ObjectSet are persisted to the database, the object is removed from the database (this is, of course, just the same as the way the standard ADO.NET DataSet works).

Saving the Changes to an ObjectSet

One of the features that make ObjectSpaces such an exciting technology is that the ObjectSet, like the ADO.NET DataSet, provides methods that push changes to the objects back into the database. The PersistChanges method of the ObjectSpace pushes the changes made in an ObjectSet back into the database, and the Resync method synchronizes the Object Set and the database. Both methods can be applied to a single object or to a collection of objects (or the complete ObjectSet contents).

Listing 3.17 shows a simple example of the ObjectSet in action. It collects a set of Customer objects from the database into an ObjectSet using the GetObjectSet method. Then, for each one, it updates the value of the Status field to Checked. Next, one of the Customer objects is marked for deletion, and a new one added, before the changes are persisted to the database.

Listing 3.17 Manipulating Customer Objects
' get ObjectSpace using mappings and connection
Dim oSpace As ObjectSpace = New ObjectSpace(mappings-file, connection)

' create ObjectSet from ObjectSpace using OPath query
Dim oSet As New ObjectSet _
    = oSpace.GetObjectSet(Customer, "State='WA'")

' iterate through objects in ObjectSet setting the Status field
For Each oCust As Customer In oSet
  oCust.Status = "Checked"
Next

' mark the Customer object at index 7 for deletion
oSet.MarkForDeletion(oSet(7))

' add a new Customer object
oSet.Add(New Customer("name", "address", "status"))

' persist the changes into the database
oSpace.PersistChanges(oSet)

Object Mappings

The ObjectSpaces technology seems like a natural progression from existing ADO.NET data management techniques, and indeed it is. The power comes from the mappings that are created between the objects and the columns and tables in the source database. These mappings are defined in XML and take advantage of a new approach that Microsoft has introduced to help blur the distinctions between different types of data and different types of data stores.

This new approach uses a three-part mapping system to map different types of data sources and data consumers. The details of the generic three-part mapping technology are described in Chapter 7, where they are applied to mapping XML to relational data. However, exactly the same approach is taken to map objects to relational data stores.

The process involves three mapping files. The relational database schema model describes the way data is stored in the relational database, and an object schema model describes how the objects themselves store and view the data they hold. The third file actually maps the elements in each of the other two to each other, to provide the "bridge" between the two disparate models.

Hierarchical Data

In this brief introduction we've concentrated on the way that ObjectSpaces can be used and what the technology offers to developers. We haven't spent time looking at the actual details of the objects it can manipulate because in essence they are just standard .NET classes. The important feature of ObjectSpaces is that it is designed from the ground up to make it much easier to work with hierarchical or related data in the form of objects.

What Are Hierarchical Objects?

As an example of hierarchical objects, a Customer object may be related to one or more Contact objects, each of which describes a person at the company defined by the Customer object. A property exposed on the Customer class—for example a Contacts property—can define this relationship. The Contacts property is simply a reference to a collection of Contact objects.

Looked at from the relational database point of view, however, the Customer object is the parent in a one-to-many relationship with child Contact objects. ObjectSpaces provides methods to work with related objects like these; for example (using the description of the objects in the previous paragraph), you can add a new Contact object as a child of an existing Customer object using the following code:

Dim oCust As Customer = oSpace.GetObject(Customer, "ID=2756")
oCust.Contacts.Add(New Contact("name", "position", "phone"))

More than that, ObjectSpaces is designed to handle graphs of interrelated objects. Even in cases where an XML-type hierarchy is not natural (because of multiple parents for a given object), you can easily represent and build the data model using ObjectSpaces. Likewise, it can handle many-to-many relationships that are not always expressed naturally as a hierarchy, for example, a Customers to Products relationship.

Loading Child Objects

One issue that springs to mind where hierarchical or related-object data access is concerned is performance. By default, a request that fetches objects matching an OPath query (such as "State=NV") will return only the objects that match the query, not any child or related objects. However, this means that you have to go back to the database each time you want to access a child object.

ObjectSpaces addresses this by using the concept of a span. Basically a span is a comma-delimited list of the objects you want to retrieve and is used in conjunction with the ObjectQuery class. So, if you specify Customer, Customer.Contact as the span for the operation, the related child objects are extracted and returned as well, without requiring a further query to the database.

A third option, called lazy loading, is also worth considering. In this case, where objects are stored in an ObjectHolder and ObjectList, a request for an object will be intercepted and all of the related target objects are automatically retrieved as well.


SUMMARY

This chapter has taken a focused look at three new topics in ADO.NET 2.0 that can really improve the performance of your data access code—in a variety of ways. As discussed at the start of the chapter, accessing multiple databases, or executing multiple commands against the same database, can lead to code blocking as it waits for expensive processes like opening a database connection or fetching rows from a database to complete.

Instead, the new MARS and asynchronous command execution features in ADO.NET 2.0 can be used individually or together, in a range of scenarios, to reduce the occurrence of code blocking and to minimize the number of connections required by your code.

We also looked at one of the most exciting of the new technologies to come to ADO.NET—ObjectSpaces. This technology allows developers to work easily with objects that expose data in a structured and natural way. On top of this, objects can be reusable, and this approach suits modern application design methodologies. ObjectSpaces removes the need for developers to build complex code into their objects to support serialization, data store persistence, and interaction with Web Services. By simply defining mappings between the objects and the data source (tools will be available to automate this), you can retrieve, manipulate, and persist data as pure .NET objects.

The next chapter completes our tour of the ADO.NET relational data features in version 2.0 of the Microsoft .NET Framework with a look at some more new topics. This time, however, they are all related to the new version of SQL Server, code-named "Yukon."


Chapter 4. ADO.NET and SQL Server "Yukon"

To coincide with the Technology Preview release of the .NET Framework, Microsoft has released a beta of the forthcoming version of SQL Server, code-named "Yukon." As well as a raft of improvements to the database engine and the management environment, "Yukon" brings with it new integration features for managed code written in any of the .NET languages.

This chapter looks at the new ADO.NET/"Yukon" integration features, not at "Yukon" itself. Here are the topics we'll be considering:

  • SQL Server Query Notifications, which allow the database to alert code to changes in the data and other events occurring in the database

  • The addition of User-Defined Types (UDTs) to SQL Server, allowing you to store objects and custom data structures within the database

  • The addition of a new cursor-based data access model, primarily designed for use within SQL Server managed code procedures

To use the features described in this chapter, you must be accessing the beta "Yukon" version of SQL Server and have version 9.0 of the MDAC data access library installed on the machine that is executing the code (i.e., the client that is accessing the database server, if they are on different machines).


SQL Server Query Notifications

In environments where different users regularly update data, there is often a need to refresh cached data when the original data changes. One approach to this is to poll the server at preset intervals to detect changes or to simply fetch a new copy of the data after a prescribed interval has elapsed.

However, these techniques are not the most efficient approach. If the source data has not changed, this approach leads to extra work and resource consumption that are simply not required. To reduce this extra load, wait times between polling or refreshing the data are often increased, which has the negative effect of increasing the latency in seeing changes and, as a result, the frequency with which outdated data is served. Arranging for the server to notify the application that is caching the data of any changes to the source data when, and only when, these changes occur can eliminate this waste of processing and resources.

Note that there is a cost associated with registering for and receiving notifications. Query Notifications are not designed to be associated with every ad hoc query (where each result is different) but are intended for use where multiple users or requests share the same query results.

The caching features that are implemented in ASP.NET 2.0 and the changes to the underlying request handling mechanism in IIS 6.0 both take this approach, automatically invalidating the cache content based on user-defined rules. Now, in conjunction with updates to some of the classes in the SqlClient namespace, SQL Server "Yukon" offers the same opportunities for relational data.

The notification infrastructure scales well to support large numbers of similar queries that vary only on parameter values. It is not designed to scale to support hundreds of different ad hoc queries. As a result, notifications work well for invalidating cached Web responses based on common database queries but not for notifying any arbitrary queries posed by numerous users.

An Overview of Query Notifications

When a query that generates a rowset is executed, code can instruct SQL Server that it wants to be notified of any change to the source data included in the rowset returned. The changes that cause a notification to occur include the following:

  • Tables being truncated, altered, or dropped

  • Data being inserted into, updated, or deleted from the tables, which would result in different values being returned if the query were reexecuted

  • An internal server error, or the server being restarted

When a notifiable event occurs, SQL Server creates a notification message and places it into a queue on the server. In some cases a default queue is used, while in other cases you can create and use custom queues (see Tables 4.3 and 4.4 later in this chapter).

You can think of a notifiable event as anything that occurs on the server that would cause the results of the query to be different from the original if it were executed again. A notification can occur immediately when a command is executed. Examples include but are not limited to situations such as a query that is not a suitable SELECT query (see the next subsection for details) or a query that causes an error.

Suitable Query Types

Query notifications are supported only for suitable SELECT queries. This means a single SQL SELECT statement that returns rows, or a stored procedure or batch statement that contains one or more SELECT queries. However, there are other limitations. So that the notifications architecture can track the tables and columns in the database correctly, the SELECT statement must define the columns explicitly and not use the asterisk (all columns) syntax. The name of the table must also be fully qualified as database-name.owner.table-name.

For example, the following SELECT statement is not suitable and may cause a notification to be sent immediately:

SELECT * FROM table

The next SELECT statement is suitable (assuming that the named table and columns exist in the database), and a notification will be sent only when the data changes, the table is modified, or a server error occurs:

SELECT ColumnA, ColumnB FROM MyDatabase.dbo.MyTable

Note that the syntax requirements stated above apply to the current release, although breaking these "rules" may not result in an error or immediate notification in the Technology Preview version of ADO.NET 2.0. And it may be possible to use a more relaxed syntax in future releases.

Detecting a Notification

How the application detects and reacts to a change notification depends on the type of application and the way that the notification is set up in the first place. Applications can poll the queue to see if a notification has occurred. OK, so this doesn't seem like an advantage over polling the server in some other way to check whether the source data has changed. However, the process of polling the queue for a notification message is far quicker and far more efficient than querying the source data again.

It's likely that you would only poll the queue just before your application actually uses the data it cached from the original request, rather than at regular intervals, though both approaches are perfectly valid. You might even set it up as a user-controlled action, perhaps with a Refresh button or command, so that users can check for changes when they want to.

Alternatively, with the new asynchronous execution model, you can execute a query against the queue that will block until a change notification is posted while allowing other code in the application to continue to execute. Since change notifications for multiple queries can be delivered to the same queue, you can have this single query listening for a change from multiple queries. Because "Yukon" supports ADO.NET 2.0 asynchronous command execution (as demonstrated in Chapter 3), executing this blocking query asynchronously means that you don't tie up a thread on the client.

Finally, with the new MARS support in ADO.NET 2.0 (also described in Chapter 3), you can have several such queries simultaneously checking for notifications against different queues, all on the same connection.

Subscribing to Notification Events

An even better way to maximize efficiency is to arrange for the notifications service to raise an event that an application can react to, rather than having to poll the queue. This approach is also implemented in the notifications feature of SQL Server "Yukon" and ADO.NET 2.0.

Setting Up a Notification

The way that your application arranges to receive notifications determines which of the two available techniques you must use. Setting up a notification to raise an event requires a different set of classes to be used from setting up a notification where your application will poll the queue directly.

To set up a notification that raises an event, you simply create an instance of the SqlDependency class and specify an event handler that will be called when the source data changes. This event handler can be associated with a cache entry. For example, in ASP.NET 2.0 the event can invalidate any cached data, forcing the application to refresh the rowset from the database to get the updated data.

Alternatively, you can use the SqlNotificationRequest class to send a request for notifications to the server. The notification messages are placed in a queue that you have previously created in the database and that you specify when you create the SqlNotificationRequest instance. You then poll the queue (as just described) to detect a notification.

Under the hood, the SqlDependency class uses a SqlNotification Request instance to register for notifications with the server. The Sql Dependency class effectively acts as a higher-level abstraction, making it easier to create applications that work with notifications using the default behavior. When you need to tailor the behavior, however, you may prefer to use the SqlNotificationRequest directly instead. We'll look at this class first.

The SqlNotificationRequest Class

The SQL notifications feature in ADO.NET 2.0 adds the new Sql NotificationRequest class to the SqlClient namespace and also adds a new property to the SqlCommand class that allows it to interface with notifications.

The SqlCommand.Notification Property

The SqlCommand class in ADO.NET 2.0 can now be bound to an instance of the SqlNotification class through a new property named Notification (see Table 4.1).

Table 4.1. The Notification Property of the SqlCommand Class

Property

Description

Notification

Sets or returns the SqlNotificationRequest instance that is bound to this command and must be set before the command is executed. By default the value is null, indicating that no notification request will be registered.

The SqlNotificationRequest Constructors

The two overloads of the constructor for the SqlNotificationRequest class are shown in Table 4.2.

Table 4.2. The Constructors for the SqlNotificationRequest Class

Constructor

Description

SqlNotificationRequest()

Creates a new SqlNotificationRequest instance with the properties set to null, ready for values to be assigned to them.

SqlNotificationRequest (id, service, timeout)

Creates a new SqlNotificationRequest instance and sets the properties. See the property descriptions in Table 4.3 for details of the parameters.

The SqlNotificationRequest Properties

The three properties of the Sql NotificationRequest class, which are equivalent to the parameters to the second constructor in Table 4.2, are shown in Table 4.3.

Table 4.3. The Properties of the SqlNotificationRequest Class

Property

Description

Id

Sets or returns a String that defines application-specific information for this notification. Although not actually used by the notifications system, the ID allows the user to associate a notification with specific parts of the application's state. The value is included in the queue message when a notification occurs, and the application can then invalidate the appropriate cached data. This is useful if notifications from several commands are posted to the same queue.

Service

Sets or returns a String that is the name of the service queue to which notification messages will be posted. The service (queue) must be configured on the server. If not specified, the default queue is used.

Timeout

Sets or returns an Integer that specifies the length of time, in seconds, that SQL Server will continue to monitor for changes. When the timeout expires a timeout notification is sent, even if no change to the data has taken place. The default is 0, which means that the default timeout defined for the server will be used.

Using a SqlNotificationRequest

To specify that an application should be notified of changes to the result of executing a query that returns rows, you create a connection and command as usual but then bind a new or an existing SqlNotification Request to the command before executing it.

The code in Listing 4.1 uses the simpler of the two SqlNotification Request constructors and then sets the Id and Service properties before binding the new SqlNotificationRequest instance to the Command and calling the ExecuteReader method to get back a DataReader containing the result.

Listing 4.1 Creating and Registering a Notification Request
' create a new Command object from an existing Connection instance
Dim sSQL As String = "SELECT colA, colB FROM database.dbo.table"
Dim oCmd As New SqlCommand(sSQL, oConn)

' create a notification object
Dim oNotifyRequest As New SqlNotificationRequest()

' specify an ID that app will use to identify this notification
oNotifyRequest.Id = "MyNotificationID"

' must specify a custom queue
oNotifyRequest.Service = "MyQueue"

' bind the notification request to the command
oCmd.Notification = oNotifyRequest

' get the results and use them
Dim oReader As DataReader = oCmd.ExecuteReader()
...

The DataReader can then be used in the application to display the data. Alternatively, you can use the Command to fill a DataSet with the returned rows.

Polling the Queue

Because the code has defined a notification request and sent it to the server, any changes to the original data will cause a notification message to be sent to the queue named MyQueue that was specified for the Service property. This message will contain the value MyNotificationID that was assigned to the Id property.

There is no built-in client-side support for polling the notification queue. However, SQL Server "Yukon" does have the ability to poll the queue and return a notification message in response to a SQL query. The format of this query is:

RECEIVE <field-list> FROM <queue-name>

For example, in order to set up an asynchronous (blocking) command that will return only when a message is delivered to the queue, you might write code such as that shown in Listing 4.2. It uses a Command object that has a polling query defined as the CommandText and executes this command asynchronously with the BeginExecuteReader method. An event handler named GetNotify is declared as the callback for this method.

Listing 4.2 Creating a Blocking Command to Receive a Notification
' create a blocking command to receive a notification from the queue
oCmd.CommandText = "WAITFOR(RECEIVE * FROM myDatabase.dbo.MyQueue)"
oCmd.Connection.Open()
oCmd.BeginExecuteReader(New AsyncCallback(AddressOf GetNotify), _
                            CommandBehavior.CloseConnection)

When a change to the data occurs and a notification message is delivered to the queue, the GetNotify event handler is automatically called (see Listing 4.3). It can retrieve the message from the queue using the DataReader returned by the EndExecuteReader method. For more details on asynchronous command execution, see Chapter 3.

Listing 4.3 Handling the Delivery of a Notification
' called when a notification message is delivered to MyQueue
Sub GetNotify(oResult As IAsyncResult)
  Dim oReader As DataReader = oCmd.EndExecuteReader(oResult)
  While(oReader.Read())
    Console.WriteLine("Data Changed!")
  End While
  oReader.Close()
End Sub
The SqlDependency Class

Instead of creating and binding a SqlNotificationRequest instance to the Command, you can use the alternative SqlDependency approach to define a required notification. The SqlDependency class has three overloads for the constructor and exposes three properties, two methods, and a single event.

The SqlDependency Constructors

The constructor for the SqlDependency class creates a new SqlDependency and automatically binds it to a Command instance if one is specified. The three overloads are shown in Table 4.4.

Table 4.4. The Constructors for the SqlDependency Class

Constructor

Description

SqlDependency()

Creates a new SqlDependency instance with the properties set to null, ready for values to be assigned to them.

SqlDependency(command)

Creates a new SqlDependency instance that is bound to the specified SqlCommand instance. The remaining properties are set to null, ready for values to be assigned to them.

SqlDependency(command, service-name, authentication, encryption, transport, timeout)

Creates a new SqlDependency instance that is bound to the specified SqlCommand instance. The remaining parameters are:

service-name: The name of the service queue as a String, if not using the default queue.

authentication: The authentication type to be used by the client listener. This value is one from the SqlNotificationAuthType enumeration:

  • None: The connection to the client doesn't use authentication. Any connection will be considered valid. This is the default.

  • Kerberos: Use NT integrated authentication. Note: This value is not available in the Technology Preview release.

  • Certificate: Use an X509 certificate for server authentication. Note: This value is not available in the Technology Preview release.

  • Any: Allow authentication using either Kerberos or Certificate. Note: This value is not available in the Technology Preview release.

encryption: The encryption type to be used when sending notification data from the server to the client. This is a value from the SqlNotification EncryptionType enumeration:

  • None: Data is exchanged in clear text. This is the default.

  • Certificate: Use a server certificate to encrypt the SSL channel. Note: This value is not available in the Technology Preview release.

  • Kerberos: Use Kerberos to obtain the session key. Note: This value is not available in the Technology Preview release.

transport: the method used by the server to send a notification to the client. This value is one from the SqlNotificationTransport enumeration:

  • Tcp: Use raw sockets as the transport. This is useful where a kernel-mode HTTP listener is not available. A separate port is required for each application domain that uses notifications, which can reduce scalability and performance.

  • Http: Use HTTP through the kernel-mode HTTP listener in IIS 6.0, allowing a single port to be shared across all application domains.

  • Any: Use HTTP if available, or use TCP otherwise. This is the default.

timeout: the timeout in seconds for this notification. If not specified, the server default timeout is used.

Note that the Technology Preview release does not support authentication or encryption. These will be added in future releases and may also become the default.

The SqlDependency Properties

The three properties of the SqlDependency class are shown in Table 4.5.

Table 4.5. The Properties of the SqlDependency Class

Property

Description

Id

Returns a String representation of the GUID that identified this dependency. Used with the computer name, authentication type, and encryption type to define the Id property of the underlying SqlNotificationRequest object.

HasChanges

Returns True if any of the results bound to this dependency have changed, or returns False otherwise.

InvalidationString

Returns the information required by SQL Server to send back the notification, as an XML document fragment such as:

[View full width]
<MachineAddress>address</MachineAddress>
   <AuthType>authentication_type</AuthType>
   <EncryptionType>encryption_type</EncryptionType>
   <Key>id</Key>

The SqlDependency Methods

There are two methods for the Sql Dependency class, which can be used to bind a dependency to a command or retrieve the dependency assigned to a command (see Table 4.6).

Table 4.6. The Methods of the SqlDependency Class

Method

Description

AddCommandDependency(command)

Binds this dependency to the specified SqlCommand instance. More than one command can be added to a dependency if required. The dependency will raise a notification when any result fetched by any of the commands changes. No return value.

GetSqlDependency(id)

A Static method that returns the SqlDependency, given a String value for the Id property of that dependency.

The SqlDependency Event

There is a single event raised by the Sql Dependency instance when the values in the source data change or when any other event in the database (such as an error or a timeout) requires a notification to be sent to the client (see Table 4.7). Remember that a notification can be raised when the command is executed if there is an error or if the command is not valid for notifications.

Table 4.7. The Event of the SqlDependency Class

Event

Description

OnChanged

Calls the specified event handler when a notifiable event occurs in the database. Passes an instance of a SqlNotificationEventArgs class to that event handler. The properties of the SqlNotificationEventArgs class are:

Info: Indicates the reason for the notification. This is a value from the SqlNotificationInfo enumeration:

  • Alter: An underlying object related to the query was modified.

  • Truncate: One or more tables were truncated.

  • Drop: An underlying object related to the query was dropped.

  • Delete: Data was changed by a DELETE statement.

  • Insert: Data was changed by an INSERT statement.

  • Update: Data was changed by an UPDATE statement.

  • Restart: The database server was started or restarted (notifications are sent when it starts).

  • Error: An internal error occurred in the server.

  • Query: A statement for which notifications are enabled was provided (i.e., a suitable SELECT statement).

  • Invalid: A non-notifiable statement was provided (e.g., an UPDATE statement).

Source: Indicates the item in the database that generated the notification or the reason it was generated. This is a value from the SqlNotificationSource enumeration:

  • Data: The source data has changed; for example, there was an INSERT, UPDATE, DELETE, TRUNCATE, or similar statement executed that affected the source data.

  • Object: A database object has changed; for example, a table containing the source data was modified or dropped from the database.

  • Owner: The ownership of an object in the database that affects the source data has changed.

  • Database: The state of a database that affects the source data has changed; for example, the database was detached or dropped.

  • Timeout: The specified timeout period for the notification has expired.

  • System: A system event occurred; for example, an internal error, the server being restarted, or lack of resources caused invalidation of data.

  • Statement: The statement being executed is not valid for notifications, for example, a non-notifiable SELECT statement or any statement other than a SELECT statement.

  • Environment: The runtime environment was not compatible with notifications; for example, snapshot isolation or incompatible SET options were specified.

  • Execution: A runtime error occurred while executing the statement.

Type: Indicates whether this notification is due to an actual change in the data. This is a value from the SqlNotificationType enumeration:

  • Change: The notification occurred because the data has changed.

  • Subscribe: The notification occurred when the command was executed or when the notification request was being processed.

Using a SqlDependency

Specifying and requesting notifications using the SqlDependency class is just as easy as using the SqlNotificationRequest class. You can create the dependency and then bind it to the Command object. Or, as shown in Listing 4.4, you can specify the Command instance in the constructor for the SqlDependency. Before executing the command, you have to attach the event handler that will be called when a notification occurs. Then you use the command in the normal way to return a DataReader or to fill a DataSet.

Listing 4.4 Creating a SqlDependency
' create a new Command object from an existing Connection instance
Dim sSQL As String = "SELECT colA, colB FROM database.dbo.table"
Dim oCmd As New SqlCommand(sSQL, oConn)

' create a dependency and associate it with the command
Dim oDepend As New SqlDependency(oCmd)

' subscribe to the dependency event
AddHandler oDepend.OnChanged, _
           New OnChangedEventHandler(AddressOf MyHandler)

' get the results and use them
Dim oReader As DataReader = oCmd.ExecuteReader()
...

The difference between the SqlDependency and SqlNotification Request classes is that you don't have to poll the notifications queue when using the SqlDependency class. Your code can continue to execute, and the event handler you specified is called only when a notification occurs. In the event handler you can access the three properties, Info, Source, and Type, to see what caused the event. If the data has changed, you can invalidate any cached data and even perform a refresh to fetch the updated data (see Listing 4.5).

Listing 4.5 Handling the OnChanged Event of a SqlDependency
' event handler routine
Sub MyHandler(oSender As Object, oArgs As SqlNotificationEventArgs)

  ' use the properties of the SqlNotificationEventArgs event...
  ' probably to invalidate the cached data, and maybe refresh it
  ' maybe even use the asynchronous feature new in ADO.NET 2.0!
  If oArgs.Type = SqlNotificationType.Change Then
    ' invalidate cached data, and maybe refresh
    ...
  End If

End Sub

Using SqlDependency in ASP.NET Applications

ASP.NET has provided a Cache object for which dependencies can be specified since version 1.0. In version 2.0 of the Framework, the ASP.NET cache infrastructure is extended to work with the new SqlDependency class. This means that you can now specify Cache dependencies as being dependent on a result returned from SQL Server "Yukon." For example, when saving data in the Cache instance that is exposed for the application, a SqlDependency instance can be used to specify when the cache will be invalidated.

The code in Listing 4.6 creates a DataSet and fills it with rows, after binding a SqlDependency to the Command that the DataSet uses. This SqlDependency is then specified in the call to the Insert method of the Cache instance. When the notification of a change is received, the DataSet is invalidated and removed from the cache automatically.

Listing 4.6 Using a SqlDependency in ASP.NET
' create a new Command object from an existing Connection instance
Dim sSQL As String = "SELECT colA, colB FROM database.dbo.table"
Dim oCmd As New SqlCommand(sSQL, oConn)

' create a dependency and associate it with the command
Dim oDepend As New SqlDependency(oCmd)

' create a new DataSet and fill with rows
Dim oDataSet As New DataSet()
Dim oAdapter As New SqlDataAdapter(oCmd)
oAdapter.Fill(oDataSet, "TableName")

' insert into cache to use as and when required
Cache.Insert("MyDataSet", oDataSet, oDepend)
...

Of course, this means that the code will have to check whether the DataSet is in the cache before attempting to use it later and recreate it if it's not there.

Page-Level Output Caching Dependencies

SQL notifications can also be used to invalidate a response generated by an ASP.NET page. An attribute is specified in the @OutputCache directive of the page to denote that the cached response is dependent on all SqlClient commands used to retrieve and generate the content for the page. If the results of any of these commands change, the page is automatically removed from the cache.

When ASP.NET sees SqlDependency="CommandNotification" in the @OutputCache page directive, it automatically creates a SqlDependency under the covers. All commands created within the page register with that SqlDependency. Then, if any one of the results becomes invalid because of changes in the database or to other notifiable events on the database server, the ASP.NET page output is invalidated on the Web server. The next request will then cause fresh and up-to-date copies of the data to be fetched from the database. Here's an example of such a directive:

<%@ OutputCache Duration="300" VaryByParam="none"
    SqlDependency="CommandNotification" %>

The same kind of process can also be implemented for individual commands in code at runtime, rather than being automatically applied to all commands through the @OutputCache directive. The AddCacheDependency method adds the SqlDependency to the response (output) cache, causing the cached data to be invalidated when the data retrieved by the Command changes in the database (see Listing 4.7).

Listing 4.7 Using a SqlDependency to Invalidate Cache Entries
[View full width]
...
' create a dependency and associate it with the command as before
Dim oDepend As New SqlDependency(oCmd)
...

' set any other values required for response cache
Response.Cache.SetExpires(DateTime.Now.AddSeconds(300))
Response.Cache.VaryByParams("None") = True Response.Cache.SetCacheability(HttpCacheability
  .Public)
Response.Cache.SetValidUntilExpires(True)

' add the SqlDependency
Response.AddCacheDependency(oDepend)
...

See the companion book A First Look at ASP.NET v. 2.0 (Boston, MA: Addison-Wesley, 2004, ISBN 0-321-22896-0) for a more detailed look at how object-level and output (page-level) caching works in ASP.NET 2.0.


SQL Server User-Defined Types

In conjunction with the changes taking place to SQL Server in the forthcoming "Yukon" release, it's now possible to execute managed code—in any of the supported .NET languages—on the Web or application server, the client, and within SQL Server itself. One opportunity this raises is an extension of the data types that can be stored in a SQL Server database.

User-defined data types can easily be created in .NET by defining a class that exposes the individual items of data as properties. For example, you might create a class named Vehicle that exposes properties such as WheelCount and Weight. It's then possible to register this class in SQL Server "Yukon" as a UDT. You must add a few other attributes and methods to the class to enable the correct integration, but other than that the class provides a new data type that you can declare for columns in the database tables.

On the client (which may be a server-based application such as ASP.NET or a client-based Windows Forms application), you simply install the same UDT class assembly. Instances of the class can then be exposed by the database through a DataReader—just as though they were native data types.

It's also possible to arrange for the UDT class assembly to be downloaded from SQL Server if it's not already present on the client, or even to read and access the data in the UDT without the class assembly being installed on the client. This section of the chapter describes three scenarios:

  1. Accessing UDT instances when both client and database have the assembly installed

  2. Downloading the UDT assembly to the client when it is not already installed

  3. Accessing the data in a UDT instance without installing the assembly

Following on from these, we look at the interface changes to the Parameter class that allow UDTs to be used as parameters to a query.

Accessing UDTs Installed on the Client and the Database

If both the client and the database have the assembly implementing the UDT installed, data can be handled as that type on the client. This is an early-bound scenario, where both client and database can instantiate the assembly and can access the properties directly. Bear in mind that the client can be server-based, such as an ASP.NET application, and may even be on the same machine as the database. It is still classed as the client.

You can use a DataReader to fetch instances of the UDT from the database using a standard SELECT query and then access the rows in the DataReader in the usual way. The UDT instance can be accessed by using methods of the DataReader, such as GetValue, or by simply specifying the index of the column within the row (see Listing 4.8).

Listing 4.8 Accessing an Early-Bound UDT
' create a new Command object from an existing Connection instance
' statement gets column named Vehicles, defined as UDT Vehicle
Dim oCmd As New SqlCommand("SELECT Vehicles FROM table", oConn)

' declare a variable to hold a Vehicle UDT
Dim oVeh As Vehicle

' fetch the results and iterate through them
Dim oReader As SqlDataReader = oCmd.ExecuteReader()
While oReader.Read()

  ' get the UDT column value
  oVeh = CType(oReader(0), Vehicle)

  ' display property values
  lblResult.Text &= "WheelCount = " & oVeh.WheelCount _
                 &  " Weight = " & oVeh.Weight
End While

Once the instance of the UDT has been retrieved from the column (the code in Listing 4.8 references it using the column index), the values can be displayed. They are accessed (as shown) in exactly the same way as any other class—by referencing the properties by name.

Downloading the UDT Assembly on Demand

If the client does not already have the assembly that implements the UDT installed, it can be downloaded from the SQL Server machine on demand. This is a late-bound scenario and requires a slightly different approach to displaying the values. It also requires that the connection string include the name/value pair UDT Assembly Download=true to allow the client to download the assembly from the server.

As in the early-bound scenario, the UDT instance can be accessed by using methods of the DataReader, such as GetValue, or by simply using the index of the column within the row. However, it cannot be cast to the correct type because the assembly is not present on the client at compile time, and so it is treated as being of the generic Object type. The simplest solution is to call the ToString method of the Object type to extract the values (see Listing 4.9).

Listing 4.9 Accessing a Late-Bound UDT
...
' declare a variable to hold a generic Object type
Dim oVeh As Object

Dim oReader As SqlDataReader = oCmd.ExecuteReader()
While oReader.Read()

  ' get the UDT column value
  oVeh = oReader(0)

  ' display property values
  lblResult.Text &= "Values: " & oVeh.ToString()

End While

Alternative approaches are to use reflection to examine the object at runtime and discover the properties it exposes or to design the UDT so that it exposes the same interface as (or derives from) an existing .NET installed class.

Note that if the application has been compiled on a machine that does have the UDT installed but is deployed to a machine that does not have it installed, the technique shown in Listing 4.8 will still work. The UDT is downloaded and installed, then used in an early-bound fashion.

Accessing UDTs Not Available on the Client

If the UDT assembly is not available on the client and cannot be downloaded, the only way to access the values of the UDT instances is as a binary stream of bytes. This also means that the standard techniques of accessing the column value in a DataReader (such as the GetValue method or the column index) cannot be used. Instead, the GetBytes method is used to read the contents of the column into a Byte array (see Listing 4.10). This also indicates to the runtime that it should not attempt to instantiate or download the UDT.

Listing 4.10 Accessing a UDT as a Binary Byte Stream
...
While oReader.Read()

  Dim iCount As Integer   ' for number of bytes returned

  ' get the raw bytes of the UDT into an array
  ' assume UDT is 32 bytes (2 x Int16 property values)
  Dim aVals(32) As Byte
  iCount = oReader.GetBytes(0, 0, aVals, 0, 32)

  ' display property values as hex digits
  lblResult.Text &= "Values: "
  For iLoop As Integer = 0 To iCount - 1
    lblResult.Text &= aVals(iLoop).ToString("x")
  Next

End While

Using UDTs in Query Parameters

When you work with UDTs in ADO.NET, you'll come across the situation where you need to use a UDT as a parameter to a SQL statement or stored procedure. The SqlParameter class has been modified in ADO.NET 2.0 to support a new constructor and new methods that allow UDTs to be used. They hinge on the way that a UDT is identified.

A UDT is identified (at least in the Technology Preview release) using a three-part dotted name, for example, MyTestApp.MyDBTypes.Vehicle. This string is assigned to a new property on the SqlParameter class named UdtTypeName, and the new addition to the SqlDbType enumeration, Udt, is specified for the DbType property to identify the data type as being a UDT. For example:

Dim oParam As New SqlParameter()
oParam.DbType = SqlDbType.Udt
oParam.UdtTypeName = "MyTestApp.MyDBTypes.Vehicle"

The constructors for a SqlParameter accept a value from the SqlDb Type enumeration, so a parameter for a Vehicle type can alternatively be created using:

oParam = oCmd.Parameters.Add("@MyVehicle", SqlDbType.Udt)
oParam.UdtTypeName = "MyTestApp.MyDBTypes.Vehicle"

Therefore, you could create a new Vehicle UDT instance and pass it to a SQL statement or a stored procedure as a UDT parameter, using code like that in Listing 4.11.

Listing 4.11 Using a UDT as a Query Parameter
' create a new Command object from an existing Connection instance
' SQL statement updates a UDT Vehicle column value in a table
Dim oCmd As New SqlCommand("UPDATE table SET column = @MyVehicle " _
                         & "WHERE ID = @RowID", oConn)

Dim oParam As SqlParameter

' add an Integer parameter for the RowID
oParam = oCmd.Parameters.Add("@RowID", SqlDbType.Int)
oParam.Value = 1205

' now add a UDT parameter of type Vehicle
oParam = oCmd.Parameters.Add("@MyVehicle", SqlDbType.Udt)
oParam.UdtTypeName = "MyTestApp.MyDBTypes.Vehicle"

' set value of parameter using constructor for Vehicle class
' i.e., the number of wheels and the weight
oParam.Value = New Vehicle(6, 4295)

' execute the command
Dim iRowsAffected As Integer = oCmd.ExecuteNonQuery()

Future revisions of ADO.NET are likely to move away from the three-part dotted name format for identifying a UDT to use a System.Type instead.

Interface Changes to Support UDTs

Several changes have been made to the classes in the SqlClient namespace to support UDTs. This includes changes to the SqlParameter and SqlDataReader classes, and a class called SqlMetaData that is added to SQL Server "Yukon" as a general extension.

Changes to the SqlParameter Class

The SqlParameter class allows a UDT to be specified as a parameter to a stored procedure or SQL statement. There are two new constructors, two new properties, and changes to the way that a couple of the existing properties work to support UDTs. The new constructors are documented in Table 4.8.

The two new properties for the SqlParameter are UdtTypeName and MetaData. The two properties that have been updated are SqlDbType and Value (see Table 4.9).

Changes to the SqlDataReader Class

The SqlDataReader class exposes one new method to return metadata about a column, while six of the existing methods have been updated to support the use of UDTs (see Table 4.10).

Table 4.8. The Constructors for the SqlParameter Class for UDT Support

Constructor

Description

SqlParameter(name, dbtype, size, direction, nullable, precision, scale, source-col, source-version, value, offset, compare-info, locale, database, schema, udt-type-name)

All the parameters to this constructor are the same as existing constructors with two exceptions. The dbtype is a value from the SqlDbType enumeration, which now contains the value Udt. The new udt-type-name parameter takes a String that contains the name of the UDT type. If SqlDbType.Udt is specified for dbtype, then udt-type-name must be a non-null value that is the type. If dbtype is any other value, then udt-type-name must be null.

SqlParameter(meta-data)

This takes a SqlMetaData instance that fully describes the data type for the parameter. It can be used to create a parameter of other types as well, but for UDT parameters the SqlMetaData instance must include the UDT type name.

Table 4.9. The Properties of the SqlParameter Class for UDT Support

Property

Description

UdtTypeName

Sets or returns the UDT type as a String. In the Technology Preview release this is a three-part dotted name such as MyTestApp.MyDBTypes.Vehicle. In future releases it's likely that this will change to be exposed as a System.Type instance instead. New in this release.

MetaData

Returns a SqlMetaData instance that fully describes the metadata for the parameter, or sets it using a SqlMetaData instance. To change the parameter type you must create a new SqlMetaData instance and assign it to the parameter. You cannot change individual values for the metadata. New in this release.

SqlDbType

Sets or returns the parameter type using values from the SqlDbType enumeration, which now includes the value Udt for a UDT. If this is set, the UdtTypeName must be set to a UDT type name. If any other value is used for this property, the UdtTypeName property must be null.

Value

Sets or returns the value of the parameter. When the parameter is a UDT, this property accepts or returns an instance of the UDT. Note that the value is a reference type, so changes to the values of this instance of the UDT will be reflected in the parameter value.

Table 4.10. The Methods of the SqlDataReader Class for UDT Support

Method

Description

GetSqlMetaData(index)

Returns the SqlMetaData instance that describes the column at the specified index. New in this release.

GetDataTypeName(index)

This method has been updated to return the three-part type name for the specified column if the column contains a UDT.

GetFieldType(index)

This method has been updated to return a System.Type instance that represents the UDT type for the specified column. Calling this method may trigger an assembly download if the assembly for the UDT is not present locally and the option to download assemblies is enabled.

GetValue(index)
GetSqlValue(index)
GetValues(values-array)
GetSqlValues(values-array)

These methods have been updated to return a new instance of the UDT with the values for the specified column if it contains a UDT. Calling them may trigger an assembly download if the assembly for the UDT is not present locally and the option to download assemblies is enabled. This will block the DataReader even when asynchronous execution is taking place.

The SqlMetaData Class

The SqlMetaData class introduced in this release of "Yukon" can be used to hold metadata about a column in a DataReader or the value in a Sql Parameter, and for other purposes within the SqlClient namespace of the framework. It exposes a Static field named UdtTypeName (see Table 4.11).

Table 4.11. The SqlMetaData Class UdtTypeName Field

Field

Description

UdtTypeName

Returns the metadata for a column or a parameter. In the Technology Preview release, this is the three-part dotted name of the UDT as defined in SQL Server. In future releases, the property will expose a System.Type instance instead. Read-only.


Managed Code, Server-Side Cursors, and Result Sets

One of the new features of SQL Server "Yukon" is the ability to write stored procedures in managed code, using any of the supported .NET languages. This presents new opportunities for building more complex stored procedures that are not ideally suited to ordinary Transact SQL (T-SQL) programming techniques, as well as providing a level of portability for your code. For example, existing logic can more easily be moved into or out of the database as required.

Of particular importance is the fact that moving data manipulation code into the database can provide improved performance because the data does not have to be moved over the wire for processing—only the results are exposed by the database server. Of course, this approach may not directly match your own application design methodology, but it can be used judiciously in many scenarios to maximize performance.

The code you write in a stored procedure can use the same ADO.NET logic and programming techniques (the same classes and methods) as your data layer or business layer code. However, one technique that disappeared from ADO in the move to the .NET Framework (and has been requested by users) has been reintroduced primarily for use within the managed code stored procedure model. This is support for server-side cursors.

Using Server-Side Cursors to Access Data Rows Directly

ADO.NET version 1.0 pretty much did away with database cursors as far as developers were concerned. The data access classes DataSet, DataReader, and DataTable are fundamentally aimed at handling rowsets of data that are constructed using SQL statements or stored procedures. Data within the DataSet and DataTable can be accessed and modified directly by finding and selecting rows or by using the row index. In a DataReader, all you can do is read the rows sequentially as a forward-only "firehose" cursor-style rowset.

Microsoft moved away from database cursors and the ability to scroll through the rows directly (as you can with the ADO Recordset) on the grounds that it almost always introduces performance penalties. Modern disconnected applications should connect to the database, extract the rows they need, and then disconnect. Updates in a DataTable are flushed back into the database using the Update method, not through some server-side cursor that points to rows in the target table of the database.

However, to provide better support for code running inside the database, server-side cursors have resurfaced in ADO.NET 2.0 under the guise of the SqlResultSet class. This feature is enabled only for SQL Server at present, through new classes and methods in the SqlClient namespace. They also work client-side with all versions of SQL Server from 7.0 onward, as we demonstrate in the examples later in this chapter, but they are not generally recommended for use in this way because of the inherent inefficiency in this scenario.

Always bear in mind the usual caveat that applies to using cursors. They are resource intensive and are certainly not the most efficient technique for client-side data access unless you need to hit only a few rows. The Result Sets implementation in ADO.NET 2.0 uses system stored procedures within SQL Server that access data on a row-by-row basis, which means repeated requests are made to the database as you navigate the rowset.

Server-Side Cursors and Result Sets

ADO.NET 2.0 adds a new feature that allows developers to work with data in a SQL Server database in a similar way to an ADO.NET Recordset object, using a cursor to access specific rows and read or update the data in these rows. Using the classes from the SqlClient namespace, you can open, scroll through, and update rows in a database table.

The technique makes use of the new concept of a result set and is implemented through extensions to the SqlCommand and the new SqlResultSet and SqlRecord classes. The general goals of the technology are:

  • To implement the new classes and extensions as a thin layer over the top of the existing infrastructure so as to expose a server-side cursor, but to maintain the simplicity of use found in other data access classes in the Framework.

  • To avoid caching any rows on the client and instead use the concept of a "current row." Only one row at a time can be processed.

  • To allow rows to be read in any order and to skip to specific rows, using only the cursor positioning features built into SQL Server.

As you'll see here, the Result Sets feature is an exciting new addition to ADO.NET that enables some of the techniques you can use with a Recordset to be accomplished in managed code against SQL Server.

The Classes and Methods That Support Result Sets

The interface changes required to the existing ADO.NET classes to support result sets are minimal, with two new methods added to the SqlCommand class to create a result set and an individual Record, and a couple of new classes to interface with the content of the result set.

The New SqlCommand Methods

The two new SqlCommand methods are listed in Table 4.12.

Table 4.12. The Methods of the SqlCommand Class to Support Result Sets

Method

Description

ExecuteResultSet

Returns a reference to a SqlResultSet instance through which the data can be accessed and updated. There are two overloads of this method:

results-set = command.ExecuteResultSet(options) Accepts as the single parameter a value from the ResultSetOptions enumeration, which specifies the type of results set to create.

results-set = command.ExecuteResultSet (options, cursor-name)

As above, plus accepts a String value that is used as the name for the cursor.

ExecuteRow

Returns a SqlRecord instance containing the single row returned by the command query.

record = command.ExecuteRow()

The ResultSetOptions Enumeration of the ExecuteResultSet Method

The ExecuteResultSet method takes a value from the ResultSet Options enumeration to define the options that will be available for the result set. The members of the enumeration are shown in Table 4.13 on the next page.

The SqlResultSet Class

The ExecuteResultSet method of a Command object returns a reference to a SqlResultSet instance that is used to manipulate the data rows specified by the Command. One common way that humans describe something new is by comparing it to things that are already familiar. We can take this same approach to understanding broadly what a SqlResultSet looks like and what it does.

Table 4.13. The ResultSetOptions Enumeration

Member

Description

Sensitive

The SqlResultSet contents are sensitive to changes made to the source data and will change in line with changes made to the rows in the database by other applications. This is the default.

Insensitive

The SqlResultSet contents are not affected by changes made to the source data in the database by other applications.

Scrollable

Backward scrolling and relative and absolute positioning within the SqlResultSet are permitted.

Updatable

Update, insert, and delete operations are permitted against the SqlResultSet.

None

None of the above options are applied.

In some respects the SqlResultSet looks like the existing ADO.NET DataReader class because it exposes the current row as a collection of columns so that you can access and update values in that row. It also exposes a series of Get xxx methods (such as GetDouble, GetString, and GetValues) that return individual column values as specific data types or as an array of values. And the SqlResultSet provides information about the current row with the same methods as the DataReader, such as GetName, GetOrdinal, IsDbNull, and so on. Finally, it also exposes navigation methods, such as Read and NextResult.

However, the SqlResultSet also looks somewhat like a "classic" ADO Recordset object, in that it contains methods and properties that allow you to find out about the complete rowset, navigate from row to row (including backward if you specify this when you create it), and insert and delete rows. And, possibly one of the most important aspects in the SqlResultSet, it exposes the Update method so that you can update values in the source database table.

Other new features do not mirror existing classes or techniques. There is a series of methods that extract column values as their native SQL Server data type (such as GetSqlByte, GetSqlChars, GetSqlMoney, Get SqlValues, and so on). Plus, there are methods to set the values of columns using .NET or SQL Server data types (such as SetDouble, SetString, Set Values, SetSqlByte, SetSqlChars, and SetSqlValues).

SQL Server "Yukon" supports new "types" of columns, which can be user-defined data types (such as collections) or XML documents. To support this, you can now access columns through a SqlResultSet by using a Reader instance. The GetData method returns a DataReader that allows you to stream the value of a column to another object or (for example) use it with data binding. The GetSqlXmlReader method returns the column value as XML elements via an XmlReader, and you can even set the column value using an XmlReader with the SetSqlXmlReader method.

The SqlResultSet Properties

The SqlResultSet class exposes several properties that are the same as the DataReader class, such as Depth, FieldCount, IsClosed, and so on. However, it also has properties that are specific to the requirements of the SqlResultSet. The full list of properties is shown in Table 4.14.

Table 4.14. The Properties of the SqlResultSet Class

Property

Description

Item(index)

Provides access to the value of a column in the current row as its native data type, using the zero-based index of the column within the row. The column value can be read and can be updated if the SqlResultSet is updatable.

Item(name)

Provides access to the value of a column in the current row as its native data type, using the column name. The column value can be read and can be updated if the SqlResultSet is updatable.

Depth

Returns an Integer value that indicates the nesting level of the current row. The main (outermost) table is at depth 0. Read-only.

HasRows

Returns True if there are any rows in the SqlResultSet, or returns False if not. Read-only.

FieldCount

Returns an Integer value that indicates the number of columns in each row. Read-only.

HiddenFieldCount

Returns an Integer value that indicates the number of columns in the table that are not in the SELECT statement used to create the SqlResultSet. Read-only.

IsClosed

Returns True if the SqlResultSet is closed, or returns False if it is open. Read-only.

RecordsAffected

Returns an Integer value that indicates the number of rows affected by the last call to the Update method. Read-only.

Scrollable

Returns True if the SqlResultSet is scrollable, or returns False if not. Read-only.

Updatable

Returns True if the SqlResultSet is updatable, or returns False if not. Read-only.

Sensitivity

Returns a value from the ResultSetSensitivity enumeration that indicates whether rows change when the underlying data in the database changes. Read-only. The possible values are:

Insensitive: The SqlResultSet contents are not affected by changes made to the data source by other applications.

Sensitive: The SqlResultSet contents are sensitive to changes made to the data source by other applications and will change when it does.

Asensitive: The sensitivity of the SqlResultSet cannot be determined.

The SqlResultSet Column Value Access Methods

The SqlResultSet class exposes a large number of methods. To make it easier to grasp these and to understand what they do, we have divided them into several separate tables.

The methods that are used to access column values are shown in Table 4.15. Each method accepts an Integer value that is the zero-based index of the column within the row. The exceptions are the GetValues, SetValues, GetSqlValues, and SetSqlValues methods, which use an array of the Object type to get or set all the columns in the row.

Table 4.15. The Column Value Access Methods of the SqlResultSet Class

Method

Description

Get xxx(index)

Returns the value of the column as a native .NET data type. The available methods are:

[View full width]
GetBoolean, GetByte, GetBytes, GetChar, GetChars,
   GetDateTime, GetDecimal, GetDouble, GetFloat, Get
   Guid, GetInt16, GetInt32, GetInt64, GetObjectRef,
   GetString

The GetValue method returns the value as the data type defined for the column.

The GetValues method returns an array of Object types containing all the column values.

GetSql xxx(index)

Returns the value of the column as the corresponding SqlType data type. The available methods are:

[View full width]
GetSqlBinary, GetSqlBoolean, GetSqlByte,
   GetSqlBytes, GetSqlBytesRef, GetSqlChars,
   GetSqlCharsRef, GetSqlDate, GetSqlDateTime,
   GetSqlDecimal, GetSqlDouble, GetSqlGuid,
   GetSqlInt16, GetSqlInt32, GetSqlInt64,
   GetSqlMetaData, GetSqlMoney, GetSqlSingle,
   GetSqlString, GetSqlTime, GetSqlUtcDateTime

The GetSqlValue method returns the value as the SqlType defined for the column.

The GetSqlValues method returns an array of Object types containing all the column values.

GetData(index)

Returns a DataReader instance that references the specified column.

GetSqlXmlReader (index)

Returns an xmlReader instance that references the specified column. The column must be defined as the new xml data type.

Setxxx (index, value)

Sets the value of the column (when the SqlResultSet is updatable) using a native .NET data type. The available methods are:

[View full width]
SetBoolean, SetByte, SetBytes, SetChar, SetChars,
   SetDateTime, SetDecimal, SetDouble, SetFloat,
   SetGuid, SetInt16, SetInt32, SetInt64,
   SetObjectRef, SetString

The SetValue method takes an Object data type.

The SetValues method takes an array of Object types containing all the column values.

SetSqlxxx (index, value)

Sets the value of the column (when the SqlResultSet is updatable) using the corresponding SqlType data type. The available methods are:

[View full width]
SetSqlBinary, SetSqlBoolean, SetSqlByte,
   SetSqlBytes, SetSqlBytesRef, SetSqlChars,
   SetSqlCharsRef, SetSqlDate, SetSqlDateTime,
   SetSqlDecimal, SetSqlDouble, SetSqlGuid,
   SetSqlInt16, SetSqlInt32, SetSqlInt64, SetSqlMoney
  , SetSqlSingle, SetSqlString, SetSqlTime,
   SetSqlUtcDateTime

SetSqlXmlReader (index, reader)

Takes an xmlReader reference and sets the value of the column to the contents of the reader. The column must be defined as the new xml data type.

SetDefault(index)

Sets the column to the default value if one is defined for this column.

IsSetAsDefault(index)

Returns True if the column value is the same as the default for the column if one is defined for this column. In all other cases, returns False.

IsDBNull(index)

Returns True if the column contains null, or returns False otherwise.

The SqlResultSet Methods for Getting Column Properties

Several methods that expose information about a column in the DataReader are implemented in the SqlResultSet class (see Table 4.16).

The SqlResultSet Navigation Methods

The SqlResultSet allows navigation forward, just like a DataReader, using the Read method. And, if the Scrollable option was specified when it was created, it can also implement direct access to a specific row and support navigation backward through the rows. The methods that implement navigation are shown in Table 4.17.

Table 4.16. The Column Property Access Methods of the SqlResultSet Class

Method

Description

GetDataTypeName(index)

Returns the data type name of the specified column as a String.

GetFieldType(index)

Returns the data type of the specified column as a Type instance.

GetName(index)

Returns the name of the specified column as a String.

GetOrdinal(name)

Returns the zero-based Integer index of the column with the specified name.

GetSchemaTable()

Returns a reference to the DataTable that contains metadata about the columns in the SqlResultSet. Unlike a DataReader, it returns information only on the columns included in the SELECT statement that was used to create the SqlResultSet. This includes the data type, name, unaliased base name, and ordinal position. It does not include key or constraint information.

Table 4.17. The Navigation Methods of the SqlResultSet Class

Method

Description

Read()

Reads the next row in the SqlResultSet. Returns True on success, or returns False if there is no row to read. This is the only navigation method that can be used if the SqlResultSet is not opened with the Scrollable option specified.

ReadAbsolute(index)

Reads the row at the specified zero-based index within the SqlResultSet. Returns True on success, or returns False if there is no row at this position.

ReadFirst()

Reads the first row in the SqlResultSet. Returns True on success, or returns False if there are no rows in the SqlResultSet.

ReadLast()

Reads the last row in the SqlResultSet. Returns True on success, or returns False if there are no rows in the SqlResultSet.

ReadPrevious()

Reads the previous row in the SqlResultSet. Returns True on success, or returns False if there is no previous row.

ReadRelative(skip)

Skips the specified number of rows and reads the next one. Returns True on success, or returns False if there is no row at this position. The skip value can be positive or negative.

NextResult()

Moves to the next rowset in the SqlResultSet. Returns True on success, or returns False if there is no following rowset.

Close()

Closes the SqlResultSet.

The SqlResultSet Row Manipulation Methods

Provided that the SqlResultSet instance was created with the Updatable option set, you can insert rows into and delete rows from the results it references. You can also push changes to the current row back into the database (see Table 4.18 on the next page).

Table 4.18. The Row Manipulation Methods of the SqlResultSet Class

Method

Description

CreateRecord()

Returns an instance of a SqlUpdatableRecord that matches the current rowset schema in the SqlResultSet. No return value.

Insert(sql-record)

Inserts the specified SqlUpdatableRecord instance at the current position in the SqlResultSet. No return value.

Delete()

Deletes the row at the current position from the SqlResultSet. No return value.

Update()

Pushes the values in the current row in the SqlResultSet into the database. No return value.

The SqlRecord and SqlUpdatableRecord Classes

A SqlResultSet instance can contain either instances of the SqlRecord class or instances of the SqlUpdatableRecord class (but not both in the same SqlResultSet). The SqlRecord class provides a lighter-weight object that is useful when you only want to read the data in the rows and expose it as data types from the SqlTypes namespace. The SqlUpdatableRecord class is the generic class used when you want to be able to update the rows and access the data using native .NET data types.

The interface exposed by the SqlRecord class is relatively small. There are four properties—FieldCount, HiddenFieldCount, Item(name), and Item(index)—as described earlier in Table 4.14 for the SqlResultSet class. And the only methods it exposes are the GetSql xxx methods listed earlier in Table 4.15 for the SqlResultSet.

The SqlUpdatableRecord has a more complete interface. As well as the same four properties as the SqlRecord class, it exposes the same Updatable property as the SqlResultSet class. It also exposes a full set of methods for accessing and changing column values, which includes all the Get xxx, Set xxx, GetSql xxx, and SetSql xxx variations listed for the SqlResultSet in Table 4.15.

Using the SqlResultSet Class

To demonstrate how the Result Sets feature can be used, this section contains a couple of examples. We've used ASP.NET code here so that you can more easily see what's going on. Just bear in mind the caveat about the general use of result sets and server-side cursors and their adverse affect on performance when used outside of stored procedures.

Navigating through a SqlResultSet

The first example (Listing 4.12) creates a SqlResultSet that is both scrollable and updatable and then displays the values of its properties.

Listing 4.12 Creating and Displaying Details of a SqlResultSet Instance
' get connection string and create connection
Dim sConnectString As String = "your-connection-string"
Dim oConn As New SqlConnection(sConnectString)

'create Command using SQL statement
Dim sQueryText As String = "SELECT * FROM [Order Details]"
Dim oCmd As New SqlCommand(sQueryText, oConn)

' open the connection
oConn.Open()

' declare a SqlResultSet variable and create an updatable
' and scrollable SqlResultSet
Dim oRS As SqlResultSet
oRS = oCmd.ExecuteResultSet(ResultSetOptions.Scrollable _
                            Or ResultSetOptions.Updatable)

' display some properties
lblProperties.Text = "Depth:" & oRs.Depth.ToString() & "<br />" _
  & "HasRows:" & oRs.HasRows.ToString() & "<br />" _
  & "FieldCount:" & oRs. FieldCount.ToString() & "<br />" _
  & "HiddenFieldCount:" & oRs.HiddenFieldCount.ToString() & "<br />" _
  & "IsClosed:" & oRs.IsClosed.ToString() & "<br />" _
  & "Scrollable:" & oRs.Scrollable.ToString() & "<br />" _
  & "Updatable:" & oRs.Updatable.ToString() & "<br />" _
  & "Sensitivity:" & oRs.Sensitivity.ToString() & "<br />"
...

To demonstrate navigation through the SqlResultSet, the code next uses the ReadAbsolute method to go to the row at index 23, displays the values of the columns in that row, and then deletes the row (see Listing 4.13). An error message is displayed if the ReadAbsolute method cannot move to the specified row.

Listing 4.13 Navigating through a SqlResultSet
...
' move to the row at index 23
If oRS.ReadAbsolute(23) Then
  lblResult.Text = "ReadAbsolute(23) succeeded<br />"

  ' iterate through the columns displaying the values
  For iCol As Integer = 0 To oRS.FieldCount - 1
    lblResult.Text &= "Column: " & oRS.GetName(iCol) _
                   & " = " & oRS.GetValue(iCol) & "<br />"
  Next

  ' delete this row
  oRS.Delete()
  lblResult.Text &= "Deleted the row at index 23"

Else

  lblResult.Text &= "No row at index 23"

End If

' remember to close the SqlResultSet after use
oRS.Close()
oConn.Close()

At the end of the process, the SqlResultSet and connection are closed. The screenshot in Figure 4.1 shows the results.

Figure 4.1. Simple use of the SqlResultSet class

Inserting a New Row

Like the previous example, the example in Listings 4.14 through 4.17 is not a real-world scenario and is not a recommended approach for working with data outside of a stored procedure. However, it does neatly demonstrate some of the other techniques available for working with the SqlResultSet and SqlUpdatableRecord classes and makes it easy to see what's going on.

The code starts in Listing 4.14 by creating a SqlResultSet in the same way as the previous example, then checks to see whether there are any rows in it by querying the HasRows property. Then it uses the ReadRelative method to skip forward five rows and display the value of the CompanyName column in that row.

Listing 4.14 Creating the SqlResultSet and Checking for Rows
' create and open connection and create command as before but
' this time using the Customer table from Northwind database
'...

' create an updatable and scrollable SqlResultSet
Dim oRS As SqlResultSet
oRS  = oCmd.ExecuteResultSet(ResultSetOptions.Updatable _
                             Or ResultSetOptions.Scrollable)
If Not oRS.HasRows Then
  lblResult.Text = "Result Set is empty"
  Exit Sub
End If

' skip forward five rows and display CompanyName column value
If oRs.ReadRelative(5) Then
  lblResult.Text = " Executed ReadRelative(5) to " & oRS.GetName(1) _
                 & " = " & oRS.GetValue(1) & "<br />"
Else
  lblResult.Text &= "Cannot move relative +5 rows"
End If
...

Now the code declares an Object array to hold the values from a row, dimensioning it using the FieldCount property minus 1, and then moves to the last row in the SqlResultSet. After displaying the CompanyName column value, it collects all the values into the array using the GetValues method (see Listing 4.15).

Listing 4.15 Retrieving a New Row in a SqlResultSet
...
' declare array to hold row values
Dim aVals(oRS.FieldCount - 1) As Object

' go to last row
If oRS.ReadLast() Then
  lblResult.Text &= "Executed ReadLast() to " & oRS.GetName(1) _
                 & " = " & oRS.GetValue(1) & "<br />"

  ' get all values into an array
  oRS.GetValues(aVals)

Else
  lblResult.Text &= "Cannot move to last row "
End If
...

Next, in Listing 4.16, a new SqlUpdatableRecord is created and filled with values using the same Object array. However, the table uses the first column as a primary key, so its value must be changed before the row can be inserted into the table. The value is set to ZZZZZ, using the SetString method, to ensure it appears at the end of the table and can therefore easily be located again. Then the row is inserted into the SqlResultSet and pushed back into the database using the Update method.

Listing 4.16 Inserting a New Row into a SqlResultSet
...
' create a new row and fill it with saved values
Dim oRR As SqlUpdatableRecord = oRS.CreateRecord()
oRR.SetValues(aVals)

' have to use different value for Primary Key column
' make it last row
oRR.SetString(0, "ZZZZZ")

' insert new row into SqlResultSet
oRS.Insert(oRR)

' push update into database
oRS.Update()
...

Remember that the SqlResultSet doesn't actually hold a copy of the rows. It uses stored procedures to access the table within the database each time. So the row will appear in the table in the location appropriate to the value of its primary key (or the sorting order specified by any table index).

Reading the Row Values

To prove that it worked, the code now moves to the last row again (the one just inserted) and gets the values in the database using the GetValues method (Listing 4.17). After displaying them, it deletes the new row so that the code can be executed again without causing a duplicate primary key error.

Listing 4.17 Reading the New Row Values
...

' go to last row (the one just inserted)
oRS.ReadLast()

' display values just inserted
oRS.GetValues(aVals)
lblResult.Text &= "Inserted new row - values are:<br />"
For Each oVal As Object In aVals
  lblResult.Text &= "&nbsp; " & oVal & "<br />"
Next

' then delete this row so code can run again
oRS.Delete()
lblResult.Text &= "Deleted the new row"

' remember to close the SqlResultSet after use
oRS.Close()

The result is shown in the screenshot in Figure 4.2. You can see the company names from the rows at index 5 and at the end of the table. You can also see the values of the new row that was inserted and then deleted.

Figure 4.2. Inserting a row into a SqlResultSet

    [ Team LiB ]


SUMMARY

In this chapter, we looked at three topics that are related to the forthcoming release of SQL Server "Yukon." SQL Server notifications will prove invaluable to developers of applications that use large quantities of data and where there is pressure on resource usage when caching data extracted from SQL Server. To support more and more concurrent users, it's important to maximize the benefits of caching while minimizing the number of refresh operations made to check that data is still up-to-date.

This is a two-edged sword in some ways, in that the more users there are concurrently updating the data, the shorter the period before data is likely to be out of date. But, at the same time, you want to lengthen the time between requeries to reduce server and database loading. By using SQL notifications, you only need to requery the data when you know that it has been invalidated by a concurrent update or other database event.

The next two topics we covered, user-defined data types and the new Result Sets feature for use in managed code stored procedures, allow more useful types of data to be sent between server and client and more complex queries and code to run within the database. These two technologies can provide opportunities for better performance and a better abstraction between data and application.

With this chapter we've come to the end of the coverage of ADO.NET topics, and it's time to move on to look at the changes and new features in the System.Xml namespace classes. However, as you saw from the somewhat brief summary of ObjectSpaces at the end of Chapter 3, relational data, objects, and XML are all merging ever closer together. Details of this merging of technologies continue in more depth in the following chapters.


Chapter 5. New Features of System.Xml

The System.Xml namespace in the .NET Framework is a set of XML classes that allow you to build XML support into your applications. It consists of a number of essential classes that enable you to read, write, manipulate, and transform XML. After an introduction to the "why" of XML, this chapter provides a brief overview of the classes present in System.Xml 1.x (version 1.0 and version 1.1 of the .NET Framework) and the driving requirements for their development. It then discusses how the XML landscape is changing and how new requirements have driven the design of the classes in System.Xml version 2.0, including emerging standards and the need for a better XML data access model to access the vast array of disparate data sources.

So this chapter covers the following topics:

  • Why XML should be a part of your data access arsenal today

  • What version 1.x of System.Xml was designed to achieve

  • How the quest for universal data access is being addressed

  • An overview of what's new in System.Xml version 2.0

However, we start with a question: "Why should I care about XML?"


Why Should I Care about XML?

As a developer, you cannot escape from XML when developing applications—be it simply reading an XML configuration file on application start-up, sharing data between applications on different platforms, or creating a data-driven application using XSLT to render Web pages dynamically. XML has become a core development skill within enterprises, and hence an understanding of the System.Xml namespace within the .NET Framework is essential for application development.

XML is a versatile metadata language, capable of marking up the information content of diverse data sources. This includes structured and semi-structured documents, relational databases, and object repositories. It is not only useful for describing document formats but also suitable for describing structured data. Examples of structured data include information that is typically contained in spreadsheets, configuration files, and relational databases. Examples of semi-structured data are medical reports, documentation manuals, and e-mail.

XML has advantages over previous data formats in that it can easily represent both tabular data, such as relational data from a database, and semi-structured data, such as a Web page or business document. Other formats, such as CSV or EDI, are not as flexible. XML is also easy to handle because it is just text that often requires only simple string manipulation —significantly lowering the bar for working with it. This has led to the widespread adoption of XML as the lingua franca for information interchange. Table 5.1 details some typical scenarios for XML with examples.

Table 5.1. Typical Scenarios for XML

XML Scenarios

Examples

Content management: The separation of information content from display

Human-readable documents, e.g., documentation manuals, specifications, and e-mail

Data exchange: A platform-independent transport format

Integration of heterogeneous data from numerous data sources, data transformation, and business-to-business (B2B) and application-to-application (A2A) data exchange

Messaging: SOAP, WSDL, and XML Web Services

XML as a mechanism for a distributed, message-based computing infrastructure using XML Schema to describe the types

Since XML has broken down the walls between disparate systems, what we need now are efficient mechanisms for moving this data around and accessing it. XML Web Services enables the transport of XML, but this is only one part of the picture. Accessing XML from storage is even more significant because most meaningful applications need to write the data to a persistent state—and relational databases are without doubt the best choice. Relational databases are unlikely to be replaced as the primary storage mechanism for data, including XML. In fact, all the major databases, including SQL Server, are embracing XML as a native storage type, making it easier to work with. Hence, getting XML data into and out of storage needs to be made easier and more flexible.

So why do you need to care about XML? Because it is now the primary form of data representation and interchange. The XML data model is becoming the predominant way of representing all types of data, not just structured data that easily fits into a relational database. You need to learn to work with XML and understand the significance of the XML data model, since it is now pervasive in application development.


System.Xml Version 1.x

In this section we are going to briefly overview System.Xml version 1.x, as delivered in previous releases of the .NET Framework. Many of the innovations in version 2.0 build on the version 1.x designs and methodologies and continue with the driving requirements of the System.Xml version 1.x architecture.

Version 1.x Design Goals

The following list of requirements was used to drive the design of System.Xml in version 1.x.

  • W3C standards compliance: Support for the major W3C XML standards that provide cross-platform interoperability is necessary. These are XML 1.0, Namespaces in XML, XSLT 1.0, XPath 1.0, and W3C XML Schema. Also included is the DOM level 2 specification, but since this is an API specification, it has little to do with interoperability—as demonstrated by the large number of different DOM implementations on each platform. It does provide some level of API familiarity to developers, which is useful, but the API is limited for use only in particular scenarios.

  • XML provider model: When data access APIs are discussed, these are often thought of in terms of relational APIs such as ADO, OLE-DB, ODBC, and ADO.NET. However, a primary goal was to provide a base to build XML data access providers and hence create an extensible and pluggable architecture. This was achieved through the use of abstract classes to define the XML API and the behavior (or semantics) of the providers as well as concrete implementations. The three abstract classes that constitute this XML provider model are the XmlReader, the XmlWriter, and the XPathNavigator. The latter is of particular significance because it combines a cursor-style API with an XPath 1.0 query engine. The other supporting API that becomes more significant in version 2.0 is the XmlResolver.

  • Integration with ADO.NET: The System.Xml classes can really be considered part of ADO.NET because they are data access APIs, and it was a goal to provide as seamless an experience as possible when moving between XML and relational data. This is epitomized in the flexibility of the DataSet class to both read and write XML into relational table structures.

When the XML 1.0 standard is referred to, this indicates that the format consists of angle brackets (< and >) as a serialized, text format. Contrast this with a data model, which specifies the information in the document that is accessible but not the APIs to represent or access the data. Neither does it describe how the data is written.

Before diving into more detail on these requirements, we'll examine the main classes in System.Xml version 1.x in relation to their usage scenarios.

XML Reading and Writing

The XmlReader and XmlWriter classes provide input/output (I/O) support for XML by providing the abstract definition of how to read or parse XML and how to generate a stream of XML content. The implementation of these are the XmlTextReader, for reading streams of text characters serialized to the W3C XML 1.0 specification, and the XmlNodeReader, which has the ability to read an XmlDocument (DOM) in a streaming fashion. A major innovation in the .NET Framework was the implementation of the XmlReader as a pull model parser, which significantly reduces the complexity of the code needed to read an XML document. This was a first for any platform.

The XmlTextWriter generates XML according to the W3C XML 1.0 specification, checking the document structure according to XML well-formedness rules. The push model XmlWriter provides an API that is significantly easier to use when generating XML. Generally you should always prefer to use these classes over the XmlDocument and other DOMs, such as the XPathDocument, when you do not need the ability to edit an entire document.

XML Document Editing

The W3C DOM was the first widely used API for manipulating XML, and it is still used in many XML applications. The XmlDocument class implements the W3C DOM level 2 specification, but with a .NET feel. This means that it uses the .NET types and provides many useful additional methods, such as Load, Save, and SelectNodes for XPath support, as well as node-level properties such as InnerXml and OuterXml to easily extract and build node trees from strings. You should use the XmlDocument only when you need to load the whole XML document into memory and random access is needed by the application to all parts of the document (such as required by XPath).

The term "in-memory XML store" or just "XML store" is used in this book to describe a document that has been loaded into memory and is accessed via an XML API. For example, the XmlDocument, XPathDocument, and XmlDataDocument classes are all XML stores in the .NET Framework. By contrast, the ADO.NET DataSet class is a relational store because it has a relational API and storage model.

XML Validation or Content Checking

The ability to check, or validate, the structure of an XML document according to a Document Type Definition (DTD) or an XML schema is provided by the XmlValidatingReader class, which layers validation support on top of an XmlTextReader. This class ensures that when an XML instance document is read, the structure and (in the case of an XML schema) the types are matched according to the schema definition. Any irregularities cause errors to be thrown. When using XML schemas to perform this validation, there is the added benefit of being able to return .NET Common Language Runtime (CLR) types mapped from the XML schema types. For example, a DateTime CLR type is returned for the numerous W3C XML schema date types, such as gDay, gMonthDay, gYear, and so on. The W3C XML schemas are represented via the XmlSchema class and its related classes, such as XmlSchema Element, XmlSchemaAttribute, and so on. The XmlSchemaCollection class is a library of XML schemas loaded into memory that can be used by the XmlValidatingReader.

XML Querying

XPath 1.0 has established itself as the preferred query language for XML, in part because of its similarity to a file path–like syntax. The XPathNavigator class is another major innovation in System.Xml version 1.x because not only does this class separate the XPath query engine from the XSLT engine but it also provides a cursor-style API for random access retrieval within the XML. Importantly, this class implements the XPath 1.0 data model via the XPathNodeType enumeration, which means that it has a much simpler set of node types based on the XML Information Set (or InfoSet) types, rather than the overburdened DOM node types represented by the XmlNodeType enumeration.

The XPathDocument Class: A Better XML Store for XPath Queries

In order to support faster querying, and hence faster XSLT, an XML store that is based on the XPath data model is provided. This is the XPathDocument. The XPathDocument is typically 20%–30% faster than the XmlDocument for queries and is the preferred store for XSLT transformations. The Xml Document should be used only if you need to edit the XML document before performing the transformation. In System.Xml version 1.x the XPath Document is a read-only store, and one of the significant changes in version 2.0 is to make this editable via a cursor-style API. This now makes the XPathDocument the preferred class not only for querying but also for editing. Given its other new features (which we'll see in Chapter 6), the XPath Document will supplant the XmlDocument class as the main XML store.

XSL Transformation

The XslTransform class is an XSLT processor providing query and transformations, via the XSLT language written as XSL stylesheets. XSLT is a heavily used XML technology because it solves a range of development issues centered on shaping XML. The classic use of XSLT is for data-driven Web sites. Here the data layer is used to generate the presentation layer, typically as HTML, thereby allowing a clean UI separation (i.e., a data/ view separation). As new data feeds are introduced, these may or may not be rendered in the UI differently, depending on the generation rules embedded in the XSLT stylesheets.

The innovation of the XslTransform class is its ability to provide a streaming output via an XmlReader (pull) or an XmlWriter (push), rather than as an XML store such as an XPathDocument or XmlDocument as required by most other XSLT processors. This means that the output of the XSL transformation does not need to be cached unnecessarily if it is being streamed, for example, via an XML Web Service.


The Quest for Universal Data Access

Given that much of the world's data is stored in a huge variety of data sources, such as HTML pages, file systems, e-mail, documents, spreadsheets, and project files, it has always been a desire to provide a universal data access mechanism for this data. Typically this has been achieved by defining a "universal" API for accessing this data regardless of where it lives. This provides the advantages of having to learn only a single API, hides the native storage of the data source, and allows tools and services to evolve independently.

In the Windows DNA era, the universal data access mechanism was OLE-DB; however, it never completely succeeded in fulfilling this role, primarily because it was a Microsoft technology and not an industry-agreed standard, combined with being based on COM, which was also only on Microsoft Windows platforms. Implementing an OLE-DB provider was hard, and writing the providers was far from trivial. Then XML came onto the scene and changed everything.

Now XML is the universal data access mechanism, and the world of data and data interchange can be regarded as XML documents. The primary driving factors for the growth of XML were its industry-agreed standard, its language and API neutrality, and (best of all) its human-readability and ease of manipulation. These factors led initially to the use of XML parsers as the data access API—data stores wrote their data to the XML 1.0 format and then consumers could use their favorite XML parser to read this generated XML. Since there were many parsers available, virtually any platform could consume this data. Figure 5.1 on the next page illustrates this concept of rewriting the data in the data store to the XML 1.0 format and then accessing this from the application with an XML parser.

Figure 5.1. Converting to XML 1.0 to allow data interchange

However, mapping large stores of data to XML 1.0 is not feasible, and typically implementers wanted to expose the data not as XML 1.0 but as an XML data model via some API that could be consumed. This need drove the requirement for XML providers.

XML Providers

In order to address the growing need to provide a better data access story for XML, System.Xml version 1.x introduced the XML provider model, which translates the data store interface API into an XML API. This is effectively a mapping between one API and another. It allows for non-XML data stores to be exposed as "logical" XML documents, which do not contain any angle brackets when accessing the data. Figure 5.2 illustrates this concept of mapping the stores' data access API to an XML API without writing the document as XML 1.0.

Figure 5.2. Exposing a data store as a logical XML document via an XML provider API

Once data has been exposed via a specific XML API, XML services can be layered on top to provide additional XML processing. These services can be other XML APIs, thereby creating a "pluggable" architecture utilizing other XML technologies, such as XSLT 1.0 and XQuery. Figure 5.3 illustrates this.

Figure 5.3. Layering XML technologies onto an XML provider

System.Xml version 1.x makes it easy to write your own custom XML providers that can be plugged into the overall architecture. These providers are either stream based or random access based. The streaming provider APIs are the XmlReader for reading and the XmlWriter for writing. The random access–based API is the XPathNavigator. It is worth noting that the W3C DOM (represented by the XmlDocument class) could be considered a provider, but it suffers from the major limitation that the API is large—and hence not realistic or easy to implement. Significantly, it also forces the implementer to build a tree of nodes, which is another duplication of the data store. The DOM API is a very poor XML provider, and the other APIs in System.Xml version 1.x do not suffer from these limitations.

If you implement the XmlReader API over a data store, you have a streaming pull-based API for reading XML. If you implement the XmlWriter API, you have the ability to push XML into the data store in a very efficient, noncaching manner.

If you implement an XPathNavigator provider API over a data store (e.g., a file system, the CLR metadata in a .NET assembly, or any CLR object graph), you have the ability to perform random access data retrieval as well as XPath 1.0 and XSLT over this data store.

The advent of XML data providers has started to elevate XML from being not just the interchange format on the wire but also a universal data access mechanism to retrieve and update data back to different and disparate data sources. With the advent of XQuery this process will be further accelerated, as it provides a common data query and aggregation language.

Integration with ADO.NET

ADO.NET has deeply integrated support for XML today, which is shown by the DataSet class's ability to read and write XML and to persist its relational table structure as W3C XML schemas. The DataSet has a fixed mapping whereby XML element and attribute names are loaded in tables and columns with the same name. Also, through the ExecuteXmlReader method on the SqlClientCommand class, you can issue a FOR XML SQL command against SQL Server to return XML to the client, using the server's support for generating XML. Much of the new work in System.Xml version 2.0 is about further providing deeply integrated support in ADO.NET for XML via the XmlAdapter class and XQuery.


Which Programming Data Model: Relational, Object, or XML?

It is all about programming data model choice. Similar to the way that Neo in The Matrix had to choose between the red pill and the blue pill, often an application consideration now involves choosing among relational, XML, and object data models. The .NET CLR and Microsoft Intermediate Language (MSIL) may have dismissed many of the language wars between developers (e.g., arguments about whether C++ is better or worse than Visual Basic), only to find data model wars breaking out instead (e.g., arguments about which is the best way to work with and represent data). The .NET platform enables you to have this choice and, when necessary, to move data as easily as possible between these often quite differing data representations.

If your data is structured, stored within a relational database, and displayed on a form as a grid, and you have spent many years learning the SQL query language, the relational data model is for you.

The relational data model has been in existence for well over 30 years. It is a tried and trusted model for storing very regular or structured data. With the huge investment in relational DBMS technology, the advances in query optimization, and a mathematically understood data model, relational DBMSs are the kings of data storage. The relational data model is flat in nature, using two-dimensional tables with explicit relationships that describe how to navigate between these tables. This model works very well because it is very straightforward. However, the data stored in databases probably represents less than 20% of the world's generated data, most residing outside as documents in file storage. The Microsoft Office 2003 release typifies this by providing a development platform centered on the generation and reuse of XML documents, thereby reducing the duplication of data and providing the ability to aggregate, search, and manage your data more easily. Despite its dominance in structured data, the relational data model has never been universally accepted for data interchange precisely because it is not able to represent all forms of data, particularly documents that are typically semi-structured in nature.

If you are in an enterprise that has built architectures based around object-oriented methodologies and practices in order to provide a layer of business object abstraction to an underlying data store, programming with object-oriented languages and object technologies with object persistence to relational databases makes perfect sense. These provide tremendous value and, being based on the type system and data model of the commonly used object-oriented languages like VB, C++, and C#, there is a natural ease of programming that allows you to work with the objects as data and then persist these to a data store. The ObjectSpaces technology briefly discussed in Chapter 3 is an example of this, with the primary charter for this technology being to provide data storage access to graph or object-oriented data models. However, like relational data, there has never been an agreement on a persistable object representation for all platforms, so you will never see objects being used for data interchange across platforms in the same way as XML.

Although the dominant data model for storage is relational, it has been a holy grail of the IT industry for decades to provide a richer mainstream data model. There is demand for data stores that work more naturally with hierarchical, semi-structured, and graph (object) data models. However, attempts to provide hierarchical or object data stores have failed to attract broad customer bases and tend to be fragmented into many incompatible niche players. XML represents a semi-structured data model that technologies like the DOM, XSLT, XPath, and XQuery can target, in the same manner that SQL targets the relational data model. Already XPath has seen far greater industry adoption than any of the previous semi-structured niche programming models. With XQuery enjoying the support of large vendors, including Microsoft, it is highly probable that XML as the semi-structured data and programming model will succeed in the mainstream.

It is very unlikely that semi-structured storage in the long run will be an alternative to relational storage, but rather there will be an evolution of the relational model. In other words, XML has challenged the industry to provide more flexible data models in the mainstream storage engines, and the industry has faced up to the challenge by providing XML type support. As relational DBMSs evolve to the semi-structured data model, there needs to be improvement in the way developers access this data. This is the primary goal of the System.Xml version 2.0 release as part of ADO.NET.

The XML data model is very expressive, in that it is able to represent structured, semi-structured, and totally irregular data. And, being hierarchical, it implicitly defines the relationships between entities within its structure. This means that it is able to represent all forms of complex data—be it flat, tree, recursive, or graph-like (references between entities) data. As a metadata language it is self-describing, has a versatile expressive type system (XML Schema), enforces document order, and is extensible through your own schema definitions and names. With the growing number of applications unifying under XML-based interfaces, because of decentralized data requirements, the XML model also promotes a loosely coupled architecture, which in turn makes designs more flexible. We are really still at the base of the curve for the XML data model, but with the advent of relational DBMS native storage of XML and better XML query languages such as XQuery, the trend is set to continue across the IT industry.


The Changing XML Landscape

A number of XML technologies have emerged that are redefining how XML is generated, stored, and queried. All of these have a direct impact on application development both now and in the future.

Growing XML Content Generation

Do you remember four years ago, when you received only a hundred e-mails per day? Now you probably receive closer to four hundred. The same parallels can be drawn for XML. Although its origins were as a metadata markup language for documents, XML has now proliferated as the primary form of data interchange, as configuration and log files, in the design of loosely coupled service-based applications, and as the primary format for representing hierarchical data.

The release of Microsoft Office 2003 will significantly fuel the generation of more XML content because it is positioned as an application development platform to unleash the content of documents as XML, thereby allowing the information to be aggregated, searched, managed, and reused. This requires developers to be aware of XML when building applications that integrate with Microsoft Office.

XML Storage

In Chapter 8 we'll see how the new version of SQL Server (code-named "Yukon") now includes an XML data type that can be used to store XML documents. Now all those Microsoft Office 2003 XML documents have a place to be centrally stored, queried, and managed.

The Need for Another XML Query Language

XML has query languages available today in the form of XPath 1.0 and XSLT 1.0, both of which are hugely popular, to the point that XPath has become the de facto middle-tier query language. Although positioned as an XML-to-XML transformation language, XSLT is capable of performing queries across XML data sources. However, despite the availability of these technologies, the W3C decided that it was necessary to introduce a new XML query language. Enter XQuery. The W3C XQuery language specification includes this justification for XQuery:

A query language that uses the structure of XML intelligently can express queries across all kinds of data, whether physically stored in XML or viewed as XML via middleware. This specification describes a query language called XQuery, which is designed to be broadly applicable across many types of XML data sources. . . .

XQuery is designed to meet the requirements identified by the W3C XML Query Working Group XML Query 1.0 Requirements and the use cases in XML Query Use Cases. It is designed to be a language in which queries are concise and easily understood. It is also flexible enough to query a broad spectrum of XML information sources, including both databases and documents.[1]

Essentially XQuery can be paraphrased in the following statement:

[1] From the W3C's "XQuery 1.0: An XML Query Language." Accessed in May 2003 at http://www.w3.org/TR/xquery/.

XQuery is to XML as the SQL language is to relational databases.

Chapter 8 details more on the XQuery language, but primarily it was designed to provide the following benefits to XML.

  • XQuery has a greater expressiveness than SQL in its ability to perform complex query operations such as joins, ordering, sorting, and so on.

  • It has a human-friendly syntax, as opposed to XSLT (which has XML syntax). Note that XQuery does include an XML syntax called XQueryX, but this is unlikely to be supported in Microsoft products because of the availability of XSLT.

  • It is strongly typed at both runtime and compile time; that is, through the use of W3C XML Schema types you can query for particular types of data in the data source.

  • It has a rich set of functions and operators based on the W3C XML Schema types.

From an application development perspective, XQuery provides distributed queries across numerous data sources that are exposed as XML. By taking advantage of the XML provider model mapped over data stores, XQuery is set to become a universal query language for data integration across disparate sources.

The landscape for XML is changing as rapidly now as when XML was first unleashed five years ago. More XML content is being produced at an increasingly rapid rate, relational DBMSs have evolved to store this content as a native type, and the emergence of XQuery provides a language to easily query, aggregate, and manipulate this data.


What's New in System.Xml Version 2.0

In this section of the chapter we overview what is new in System.Xml version 2.0 and the System.Xml.Query and System.Data.SqlXml namespaces that have been introduced. Greater detail and usage scenarios are provided for each of these in the next three chapters.

Middle-Tier XML Data Access

Middle-tier XML programming is the ability to query data sources exposed through an XML provider; manipulate the XML with the DOM, XSLT, and XQuery; maybe share the data as a Web Service to another application; and then push the changes back to the original data source. The primary aim is to maintain the fidelity and shape of the XML, taking advantage of the hierarchical data model and its interoperable cross-platform nature with the layered XML technology services, such as querying, validation, and APIs.

When considering SQL Server as the DBMS, there is rich XML support available—both built directly into SQL Server and available through the middle-tier native SQLXML 3.0 technology. Although this book does not cover the existing XML support in SQL Server, which has substantial coverage in The Guru's Guide to SQL Server Stored Procedures, XML, and HTML by Ken Henderson (Boston, MA: Addison-Wesley, 2002, ISBN 0-201-70046-8), there is a deep synergy between the new support for XML in the middle tier introduced in System.Xml version 2.0 and SQLXML. The combination of XML Views (which we discuss next) and XQuery is a migration of the SQLXML technology to the .NET Framework, providing richer querying support and a more flexible mapping format to cope with the increasing demands of building XML providers over relational and other sources.

XML Views: Extending the XML Provider Model

Since there are and always will be large amounts of data not stored as XML, being able to represent this data as XML without physically converting it to XML 1.0 is essential. This concept was briefly mentioned earlier. In order to virtualize the data stored in a relational DBMS, the concept of XML Views is introduced. XML Views are a declarative format to map data in one data model to the XML data model, and can be defined as follows.

XML Views are the translation, using a declarative mapping syntax, of data represented in one data model into an XML data model. The XML View maps structure and relationships between these differing data models in order to provide a virtualized XML document over the data source.

XML views have been introduced to System.Xml version 2.0 to provide the ability to map data stored in SQL Server tables to XML, thereby enabling queries and updates. Although primarily aimed at mapping relational database tables to XML, this virtualized XML is not constrained to just this use. For example, the same view technology is used by ObjectSpaces to map relational data to object graphs.

XML views are present in SQLXML through annotated XML schemas, so this concept is not new; however, the implementation in the .NET Framework takes XML views to new levels of functionality and flexibility by introducing a format called three-part mapping, which we will discuss later in this chapter. Figure 5.4 on the next page shows how XML views are another type of XML provider that can be built to virtualize the data in a data store to be accessed via middle-tier XML APIs.

Figure 5.4. XML views as a type of XML provider

XML views offer the following benefits.

  • They act as a replacement for custom procedural mapping code. When you build an XML provider in code, you have to decide how to map between the different structures. For example, with a relational database you have to decide whether to map the table name as an element name and the tables and columns as attributes or elements. XML views replace the need to write this code by using a declarative design time representation.

  • They are easily updated without requiring code recompilation because they are a declarative file format.

  • They are bidirectional, in that they can be used for both queries and updates against the data store. Many XML providers are read-only.

  • They are ideal for use with design time tools.

XML views are syntax for transformation between data models. This means they are ideal for compilation in the same way that XML query languages are compiled. The System.Xml.Query.XmlViewSchema class represents a compiled XML view, with the System.Xml.Query.XmlView SchemaDictionary class being a dictionary of these that can be referenced in queries. We will use these classes when we do queries and updates to SQL Server in the following chapters.

An XML View Example

In order to provide some visualization of an XML view, the following series of figures provide an example of mapping relational tables to an XML schema to create an XML view. Here we have chosen to map the Customer and Order tables from the SQL Server Northwind sample database. This example will continue into the following chapters, using code to query and update SQL Server.

Figure 5.5 shows the mappings for the Customer table. Each of its columns (for example, CustomerID) is mapped to an attribute on an element named Customer. The attributes represent each of the columns, so there is a CustomerID attribute. This is a simple XML view to construct and clearly shows that XML views are a data model structural transformation process.

Figure 5.5. Mapping the Customer table to a Customer element with attributes

The Customer element maps to a table, and its attributes map to columns. The structures in the different data models match. The number of Customer elements created depends on the number of rows in the Customer table. If you delete an element, you delete a row from the table. If you update an attribute, this updates a column in the table.

We can extend this concept further by mapping to a different XML structure. The mapping in Figure 5.6 on the next page shows that the Customer table is mapped this time across two elements, named Customer and Address, where the Address element contains the address-related attributes. Here there is a transformation that creates an Address element for every Customer element. For every row in the Customer table, we are creating a Customer element, so the Address element also depends on the Customer table.

Figure 5.6. Mapping the Customer table to the Customer and Address elements

If you delete an Address element, that does not mean that you will delete the row from the Customer table, although deleting the Customer element does mean deleting a row from the Customer table. The Address element is simply owned by the Customer element and is just another XML shape for the table. This again shows that XML views are a structural transformation.

In the final example, in Figure 5.7, we introduce a child element for the Customer element named Order, which is mapped to the Order table. In the Northwind database, the Customer and Order tables are joined via a relationship based on the CustomerID column. Hence all the orders for a given customer are listed as child elements of the Customer element.

Figure 5.7. Mapping the Customer and Order tables

The behavior of this mapping is different than the previous examples. Deleting a Customer element has no effect on the Orders table, although the Address element is deleted. In other words, the Order element and Customer element behave independently from each other for update operations. XML views can get complicated because they add new behavior to the data model independent of its shape. In this case, if desired, it can be indicated in the mapping to cascade-delete the child Order elements when deleting a Customer element, but this needs to be done explicitly. These examples show that XML views provide a very powerful approach to accessing and updating relational data as XML.

The Three-Part Mapping Format for XML Views

Let's now look at how XML views are defined as a file format. To achieve the cross-domain mapping, three different formats are used—otherwise known as "three-part mapping." These three parts are a W3C XML Schema for the XML domain (XSD), a Relational Schema Definition (RSD) for the relational domain, and a Mapping Schema Definition (MSD) for the mapping transformation.

Taking the first mapping example we discussed previously, we can divide the domains shown in Figure 5.5 among the mapping schema formats as shown in Figure 5.8 on the next page.

Figure 5.8. Representing each domain by its own schema format

Chapters 7 and 8 provide examples of each of these formats for part of the Northwind database. The XSD is simply an XML schema with complex and simple types. The RSD is the persisted format of a relational database, with sufficient information to perform the necessary domain mapping functionality. RSD is a subset of the database's metadata that describes one or more database structures (such as tables, views, or stored procedures) and the relationships between these structures. The MSD format is syntax for expressing mapping between each of the domains, to transform relational tables to elements and columns to attributes or elements. All three formats are persisted as XML.

The XPathDocument2 Class

System.Xml version 1.x has an XML store called the XPathDocument, which is built on the XPath data model and, in turn, is based on the XML InfoSet specification. Hence it does not have any of the XML 1.0 serialization constraints. This makes it ideal for representing XML as just data. The XPathDocument2 is a duplicate of the XPathDocument class but has been extended to allow updates and change tracking at a node level.

The XPathDocument2 class needs an API that reflects the XPath data model in order to manipulate and update the XML. In System.Xml version 2.0 the XPathEditor class provides this ability with an updatable cursor-style API, deriving from the XPathNavigator2 class. The typical editing model is to select parts of your document to update with XPath queries and then edit these through the XPathEditor. Furthermore, the XPathEditor exposes an XmlWriter class used to build a particular node tree at the current location. This involves the more intuitive and usable top-down push model approach, rather than the bottom-up DOM approach of creating, constructing, and attaching node trees.

For developers this has the added benefit of needing to learn only a single API, the XmlWriter, to generate XML—whether for a stream or a node tree. (For those of you who have followed the history of System.Xml development, this is the same as the fabled XmlNodeWriter.) Since the XPathDocument2 can be accessed only via XPathNavigators and XPath Editors, which can be considered as pointers to the underlying data, the internal node structure is not exposed. Hence the underlying XML store structure can be optimized for queries now and in future releases, as XML query optimization technologies evolve.

Another feature of the XPathDocument2 class is its ability to track changes to individual nodes, keeping current and previous values for inserted, updated, and deleted nodes. This enables you to determine changes that have occurred to the XML document and to push these updates back into a data source. A class called XPathChangeNavigator has been introduced to allow navigation of the changed nodes, which also derives from the XPathNavigator2 class. This navigator can be considered as a view of the document changes, along with its current state. In effect, every node has both a current value and an original value, which are maintained and viewed through the XPathChangeNavigator.

Like the XPathDocument class, the XPathDocument2 (being based on the XPath data model) is the preferred store for all queries, both XSLT and XQuery, and is significantly faster than the XmlDocument class. Now that it is editable as well, there is much less need to use the XmlDocument for the editing and query scenarios—the XPathDocument2 does it all.

So, you might be asking, where does this leave the XmlDocument class? Well, having an object-oriented model based on XmlNodes is useful for extensibility to define your own type of XmlDocument store, but apart from that it has come to an innovation dead end. In fact, as discussed earlier, the very fact that it does expose a node-based API means that it cannot easily represent virtualized XML like the XPathNavigator can.

IMPORTANT

In System.Xml version 2.0, the XPathDocument2 and XPathNavigator2 are duplicates of the existing XPathDocument and XPathNavigator classes. In the Technology Preview release, this is an intermediate stage, and both these classes will be merged into the XPathDocument and XPathNavigator classes, respectively, in future beta releases.

Querying and Updating Data with the XmlAdapter

The System.Data.DataSet class is one of the most used data access components in ADO.NET, and rightly so. It represents a relational store on the client that can easily be synchronized to relational DBMSs and was built to support disconnected application scenarios. It also has the ability to read a W3C XML schema to create its table structure, read XML directly into the tables, persist the structure as a W3C XML schema, and write out the data as XML. This tight integration between relational data and XML was a primary goal of ADO.NET.

Despite its great XML support, the DataSet is fundamentally still a relational store. Once the XML has been loaded, you must use relational APIs to manipulate the data. This is fine when the XML that was loaded is regular and maps perfectly into DataSet tables, which in turn map to tables in the DBMS that are going to be updated. However, the world of XML, by its very nature, is typically not that structured. Even if you have defined and agreed on a schema with your business partners, there can be a variety of differently shaped XML documents that correctly conform to that schema.

And, of course, the database administrator defined the database table structure three years ago, and he or she is not going to change that just because some application is now pushing out XML to consume. This forcing of a square XML peg into a round database hole is described as an impedance mismatch. So what is the solution? Typically, developers today do one of the following.

  • Transform the XML, using XSLT, into another XML document that represents the correct relational table structure, load this into a DataSet class, and update the database.

  • Parse the XML using an XmlReader or XPathNavigator and generate insert, update, and delete SQL statements using ADO.NET for the specific database table. This is more efficient than the DataSet because it is not cached, but it requires more code and tends to be easier to break. This is a custom mapping solution (mentioned earlier), which the XML View technology is designed to replace.

  • Take advantage of the built-in database support, such as the SQLXML bulk XML import or the OPENXML features in SQL Server 2000. These are reasonable options, but they tend to be geared toward loading large amounts of data in a single shot.

To overcome these limitations, a new class called the XmlAdapter has been added to System.Xml version 2.0, which—combined with the XPath Document2—provides a powerful way to retrieve, manipulate, and update relational data as XML from SQL Server. The XmlAdapter is really just another data access class in the ADO.NET model, and it further accomplishes the goal of tight XML integration between relational data and XML in ADO.NET.

The XmlAdapter, like the ADO.NET DataAdapter, provides the ability to issue queries to SQL Server and return results. It uses XQuery as the query language and XML views to map the SQL Server tables to XML, filling an XPathDocument2 as the disconnected store. Figure 5.9 illustrates this.

Figure 5.9. Updating changes made to the XPathDocument2 to SQL Server via the XmlAdapter

Once the XPathDocument2 is filled, you can take advantage of other XML technologies such as XQuery and XSLT, as well as use the XPath Editor API to update the XML store. And once updated, the changes to the XPathDocument2 are tracked and used to update SQL Server via the XML view that automatically generates the necessary SQL statements to update the respective tables. In effect, once you have defined an XML view, you are able to seamlessly treat your data in SQL Server as XML using the XPathDocument2 as a disconnected XML store.

The XmlAdapter overcomes all the limitations previously mentioned of storing XML in SQL Server. Also, since it streams relational data out as XML through the XML view, there is no need to cache the data into an intermediate DataSet in order to generate XML. The XML view provides complete control over the structure of the XML generated.

This design parallels the ADO.NET relational classes for XML, as shown in Figure 5.10 on the next page. In version 2.0 of the .NET Framework, there is now an impressive middle-tier XML data access API for working with SQL Server.

Figure 5.10. Comparing the ADO.NET relational and XML APIs to access data

XQuery Programming

Previously we discussed the advent of XQuery as the emerging query language for XML. In order to accommodate this in the .NET Framework, a new class called the XQueryProcessor has been introduced into the System .Xml.Query namespace.

This class allows you to perform queries over either XML in-memory documents or XML views. Whereas the XmlAdapter is used to fill and update the XPathDocument2 store through an XML view, the XQueryProcessor class can also query the XPathDocument2 once loaded.

Another ability of the XQueryProcessor class is to stream the output of the query through a TextWriter class, without requiring this to be loaded into a client-side XML store. (Note that there will be support for Xml Reader and XmlWriter as output types in future releases.) We will see more details of the XQueryProcessor class in Chapter 8.

XQuery and XML Views

By combining the XQueryProcessor class with XML views, XQuery can be used as a data aggregation language over different data sources. An XML view can target multiple data sources via different connections and hence is able to query and aggregate this on the middle tier.

For example, the XQuery in Listing 5.1 creates a list of <customer> elements that each contain a list of customer <name> elements. Loading all the customers and all the orders in the Northwind database, via an XML view called Northwind, generates the names. A filter or predicate is applied through the where clause so that each CustomerID that is matched with the order's CustomerID returns the name for the customer. If a customer has multiple orders, there will be multiple names returned.

Listing 5.1 An XQuery to Extract Customer Names
<customer>
{
  for $cust in map:view('Northwind')/customers,
    $order in map:view ('Northwind')/orders
  where ($cust/Customer/@CustomerID = $order/CustomerID)
  return <name>{ $cust/Customer/@name } </name>
}
</customer>

Listing 5.2 shows an example of output from this query.

Listing 5.2 The Result of an XQuery to Extract Customer Names
<customer>
<name>Mark Fussell</name>
<name>Alex Homer</name>
<name>Dave Sussman</name>
</customer>

We will discuss XQuery and XML views in more depth and with examples in Chapters 7 and 8.

XSLT Programming

With the advent of the System.Xml.Query namespace, there has also been a significant innovation in XSLT processing. System.Xml version 1.x introduced the XslTransform class to execute XSLT stylesheets. The System .Xml.Query namespace was created in order to combine XML languages into a common query processing engine for XSLT, XQuery, and XPath— including XML Views, which are simply another XML transformation or query language. This treats XML languages like other CLR languages, in the sense that they are compiled and optimized and then generate MSIL instruction codes that can be executed, thereby significantly increasing the performance of the queries with some sacrifice for compilation time.

The System.Xml.Query.XsltProcessor class takes advantage of this improved query engine to perform XSL transformations over the XPath Document2 class. The XsltProcessor class should be regarded as still under development, and we will discuss it more with examples in Chapter 8.


SUMMARY

This chapter began by discussing why XML is becoming more and more prevalent in application and systems development through the use of XML as the data interchange format for documents and via Web Services.

We then briefly overviewed the System.Xml version 1.x classes, detailing the driving design requirements (W3C standards compliance, an XML provider model, and close integration with ADO.NET). We also discussed how these aims have continued with the design of the System.Xml version 2.0 classes. Given that XML is the de facto data interchange format, we then discussed how XML is now the universal data access API, with the world of data and data interchange regarded as XML documents. Classes such as the XmlReader, XmlWriter, and XPathNavigator provide data access APIs for an XML provider model.

We then saw how the landscape for XML is changing as rapidly now as it was five years ago. More XML content is being produced at an increasingly rapid rate. And this rate will only increase because of the advent of Microsoft Office 2003, the evolution of relational DBMSs such as SQL Server to store XML as a native type, and the emergence of the XQuery language as a means to query, aggregate, and manipulate this content.

Next we overviewed the new classes introduced in System.Xml version 2.0, which center on the need for better middle-tier XML programming via XML providers. This topic introduced XML views, which are the translation, using a declarative mapping syntax, of data represented in one data model into an XML data model. XML views provide the benefits of replacing custom procedural mapping code and enabling bidirectional queries and updates to the data store.

We then looked at the XPathDocument2 class that allows updates via the XPathEditor class, which together provide an updatable cursor-style API. The XPathDocument2 also has the ability to track changes to individual nodes, keeping current and original values for inserted, updated, and deleted nodes via the XPathChangeNavigator.

The XmlAdapter, like the ADO.NET DataAdapter, provides the ability to issue queries to SQL Server and return results. It uses XQuery as the query language and XML views to map the SQL Server tables to XML, filling an XPathDocument2 as the disconnected store. When combined with the XPathDocument2 class, the XmlAdapter class provides a parallel set of XML APIs to access data in the middle tier in a manner similar to the relational ADO.NET APIs. The XmlAdapter further integrates deep XML support into ADO.NET.

Finally, we touched on the XQueryProcessor and the XsltProcessor classes introduced into the new System.Xml.Query namespace. These classes provide XQuery and XSLT language support over SQL Server via XML Views and over in-memory XML documents via the XPathDocument2 class.

In the next chapter, we move on to look at the new XPathDocument class in more detail.


Chapter 6. The XPathDocument2 Class

The previous chapter provided an overview of the improvements to System.Xml 2.0 and the reasons behind them. This chapter and the next two provide code examples and scenarios for using these new classes.

In this chapter we'll look at the XPathDocument2 class, which is based on the existing XPathDocument class in System.Xml 1.x. We will also examine the XPathEditor and XPathChangeNavigator classes that allow you to perform editing within the XPathDocument2 class and view changes that have been made to the document. The XPathDocument2 class is an XML store that can be validated with a W3C XML schema, so we will also be using the new XmlSchemaSet class to validate the updates to the document.

The topics for this chapter are:

  • Using the XPathDocument2 class and its properties and methods

  • Using the XPathEditor class and its properties and methods

  • Using the XPathChangeNavigator class and its properties and methods

We start off with a look at what the XPathDocument2 class can achieve. XPathDocument2 is a duplicate of the existing XPathDocument class, but it has been extended to allow updates and change tracking at a node level. We will first introduce the editing capabilities of the XPathDocument2 before delving into the rest of its functionality.


Creating and Updating the XPathDocument2 Class

As an example of how the XPathEditor makes it easy to update the XPathDocument2, we can load an XML document and make some changes. Throughout this chapter we will use the same XML document, which is a list of books from a bookstore. This may be familiar; it is taken from the XML QuickStart samples in the .NET Framework SDK. The books.xml document is shown in Listing 6.1.

Listing 6.1 The books.xml File
<?xml version="1.0" encoding="utf-8"?>
<!-- This file represents a fragment of a bookstore database -->
<bookstore>
  <book genre="autobiography" publicationdate="1981"
        ISBN="1-861-11-0">
    <title>The Autobiography of Benjamin Franklin</title>
    <author>
      <first-name>Benjamin</first-name>
      <last-name>Franklin</last-name>
    </author>
    <price>8.99</price>
  </book>
  <book genre="novel" publicationdate="1967" ISBN="0-201-63361-2">
    <title>The Confidence Man</title>
    <author>
      <first-name>Herman</first-name>
      <last-name>Melville</last-name>
    </author>
    <price>11.99</price>
  </book>
  <book genre="philosophy" publicationdate="1991"
        ISBN="1-861001-57-6">
    <title>The Gorgias</title>
    <author>
      <name>Plato</name>
    </author>
    <price>9.99</price>
  </book>
</bookstore>

Figure 6.1 shows the books.xml document displayed in Internet Explorer.

Figure 6.1. The bookstore XML document viewed within Internet Explorer

The code in Listing 6.2 creates and loads an XPathDocument2 with the books.xml file, creates an XPathEditor, navigates to the <bookstore> element, and inserts a new book as a child element. It uses an XmlWriter to build the node tree for this and then saves the XML back to the same file.

Listing 6.2 Inserting a New Element with an XPathEditor
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim xmledit As XPathEditor = doc.CreateXPathEditor()

xmledit.MoveToFirstChild()
xmledit.MoveToNextSibling()

Dim writer As XmlWriter = xmledit.CreateFirstChild()

writer.WriteStartElement("book")
writer.WriteAttributeString("genre", "technology")
writer.WriteAttributeString("publicationdate", "10-27-2003")
writer.WriteAttributeString("ISBN", "1-861003-11-1")
writer.WriteElementString("title", "ADO.NET and System.Xml V2")
writer.WriteStartElement("author")
writer.WriteElementString("first-name", "Alex")
writer.WriteElementString("last-name", "Homer")
writer.WriteEndElement()
writer.WriteStartElement("author")
writer.WriteElementString("first-name", "Mark")
writer.WriteElementString("last-name", "Fussell")
writer.WriteEndElement()
writer.WriteElementString("price", "29.99")
writer.WriteEndElement()

writer.Close()
doc.Save("books.xml")

Notice how a document can now be built using the XmlWriter class, meaning that this approach is identical whether you are writing to a file stream using the XmlTextWriter or writing to an XPathDocument2 (which is an in-memory node tree). Figure 6.2 shows the new book titled ADO.NET and System.Xml V2 that has been added to the bookstore.

Figure 6.2. Inserting a new book with the XmlWriter

    [ Team LiB ]


An Overview of the XPathDocument2 Class

The XPathDocument2 class is used to load XML documents into memory and provides optimized storage for XML query languages such as XPath because it is based on the XPath data model. For example, if the document to be loaded contains entity references from a DTD, these become text nodes. And if there are consecutive text nodes, these are merged into a single text node. The XPath data model is based on the XML InfoSet specification and is typically simpler to use. The XPathDocument2 provides the following support:

  • XPath queries via any of the XPathNavigator2 classes

  • Read-only navigation via the XPathNavigator2 class

  • Editable navigation via the XPathEditor class

  • Change navigation via the XPathChangeNavigator class

  • W3C XML Schema validation via the Validate method

  • Event notifications for updates, inserts, and deletes

  • The ability to load and save XML documents to files and URLs

Figure 6.3 illustrates these features.

Figure 6.3. The XPathDocument2 class and associated classes

    [ Team LiB ]


Properties and Methods for the XPathDocument2 Class and Dependent Classes

Before we look at more examples of using the XPathDocument2 class, this section contains a concise reference to all its properties and methods.

In System.Xml 1.x there is an XPathDocument class that is identical to the XPathDocument2 class but is a read-only XML store.

In this Technology Preview release, the XPathDocument2 class is an editable XML in-memory store. This is due to be combined with the XPathDocument class in the next release, so that there will be only a single in-memory store based on the XPath data model.

Constructors

The constructors for XPathDocument2 create a new instance of the class (see Table 6.1).

Table 6.1. The Constructors for the XPathDocument2 Class

Constructor

Description

XPathDocument2()

Creates a new empty XPathDocument2 instance.

XPathDocument2(Boolean)

Creates a new empty XPathDocument2 instance with the Boolean value indicating whether the XPathDocument2 should set up a default root document node. This is useful when storing XML fragments and is typically used in advanced scenarios where XML fragments are loaded into memory for querying or manipulation.

XPathDocument2 (XmlNameTable)

Creates a new empty XPathDocument2 instance with an XmlNameTable. The name table is used during edit operations to atomize element and attribute names. The XmlName Table is an optimization. By sharing this across components it allows for faster lookup of attribute and element names. For example, if the XPathDocument2 is loaded from an Xml Reader, the XmlNameTable from the Xml Reader can be set onto the XPathDocument2 for decreased load time, with the performance improvement depending entirely on the type of XML document.

XPathDocument2 XPathDocument2 (XmlNameTable, Boolean)

Creates a new empty instance with a combination of the XmlNameTable and default root options.

Properties

The properties of the XPathDocument2 class are shown in Table 6.2.

Methods

The methods of the XPathDocument2 class are shown in Table 6.3.

Table 6.2. The Properties of the XPathDocument2 Class

Property

Description

AcceptChangesOnLoad

Specifies whether the document automatically calls AcceptChanges after a load operation, i.e., whether the load keeps the pending changes or replaces them with the new values. Setting this to False causes any loaded XML to be treated as one big insert operation.

DefaultRoot

Returns a Boolean value indicating whether the XPathDocument2 was instantiated to have a default root node or not.

NameTable

Returns a reference to the XmlNameTable for the XPathDocument2. If the XPathDocument2 was loaded using the XmlReader, this name table will be the one obtained from the XmlReader. If not, a new XmlNameTable is created by default when the XPathDocument2 is created.

Table 6.3. The Methods of the XPathDocument2 Class

Method

Description

AcceptChanges()

Commits all changes made to all the nodes since the last time AcceptChanges or RejectChanges was called (or the last time AcceptChange or RejectChange was called on each node). Calling AcceptChanges has the following effects on the nodes:

  • Inserted nodes: The node is no longer marked as Inserted; i.e., the XmlNodeChangeType enumeration value for this node is set to Unchanged and the current value of the node is then propagated to the original value of the node.

  • Deleted nodes: The node and its subtree are no longer accessible from the main tree using the XPath Navigator2, XPathEditor, or XPathChange Navigator classes.

  • Updated nodes: The node is no longer marked as Updated; i.e., the XmlNodeChangeType for this node is set to Unchanged, and the current value of the node is propagated to the original value for the node.

CheckValidity(schemas, ValidationEventHandler)

Checks that the XPathDocument2 conforms to the specified schema(s) but does not perform schema PSVI InfoSet augmentation. (PSVI is Post Validation Schema Information—the information that an XML schema adds to a document as a result of the validation process.) This means that schema defaults such as default attributes are not applied to the document. The XPathDocument2 is unchanged by this method. The ValidationEventHandler parameter is the registered callback event handler that receives information about schema validation errors and warnings.

CreateXPathChangeNavigator()

Creates and returns a new XPathChange Navigator instance positioned at the root node. This XPathChangeNavigator can navigate through all the Updated, Inserted, Deleted, and Unchanged nodes and access the current and original values of a node where applicable. If there is no root node, an XmlException is thrown.

CreateXPathEditor()

Returns a new XPathEditor class positioned at the root node. The XPathEditor provides a cursor API for editing the XPathDocument2. The tree navigated is the current view that does not include any deleted nodes. If there is no root node, an XmlException is thrown.

CreateXPathNavigator2()

Returns a new XPathNavigator2 class positioned at the root node. The XPath Navigator2 provides a cursor API for read-only navigation of the XPathDocument2. The tree navigated is the current view that does not include any deleted nodes. If there is no root node, an XmlException is thrown.

HasChanges()

Returns a Boolean value indicating whether here are pending changes on the XPath ocument2. The second overload for this method accepts a parameter and returns True only if there are any pending changes in the XPathDocument2 that match the specified value from the XmlChangeFilters enumeration:

HasChanges (ChangeFilter)

IsDeletedFragment (XPathNavigator)

Used to check whether the specified XPathNavigator is referencing a node that is part of a deleted subtree. Returns a Boolean value that is True if the subtree for the XPathNavigator2 has been deleted from the XPathDocument2. The table below shows the return values. ("Permanently deleted" means that after this subtree was deleted the AcceptChanges method has been called.)

XPathNavigator Pointing to Deleted Node?

Permanently Deleted?

Return Value

No

No

False

Yes

No

True

No

Yes

True

Yes

Yes

True

The second overload of this method also accepts a parameter that defines a node that is permanently deleted:

IsDeletedFragment(XPathNavigator, is-permanent)

The table below shows the return values when is-permanent = true.

XPathNavigator Pointing to Deleted Node?

Permanently Deleted?

Return Value

No

No

False

Yes

No

True

Yes

Yes

True

The table below shows the return values when is-permanent = false.

XPathNavigator Pointing to Deleted Node?

Permanently Deleted?

Return Value

No

No

False

Yes

No

True

Yes

Yes

True

Load(parameter)

Loads XML data into the XPathDocument2. There are four overloads of this method:

  • Load(file-name): Loads the data from the specified file.

  • Load(Stream): Loads a stream containing XML data.

  • Load(reader): Loads the XML from an XmlReader.

  • Load(TextReader): Loads from a TextReader containing XML data.

LoadXml(xml-markup)

Loads the XPathDocument2 with the given xml-markup string that is well-formed XML.

RejectChanges()

Rejects all changes made to all the nodes since the last time AcceptChanges or RejectChanges was called (or the last time AcceptChange or RejectChange was called on each node). Calling RejectChanges has the following effects on the nodes:

  • Inserted nodes: The node and its subtree are no longer accessible from the main tree using the XPath Navigator2, XPathEditor, or XPathChange Navigator classes.

  • Deleted nodes: The node is no longer marked as Deleted; i.e., the XmlNodeChangeType for this node is set to Unchanged; and the current value of the node is changed to the original value of the node.

  • Updated nodes: The node is no longer marked as Updated; i.e., the XmlNodeChangeType for this node is set to Unchanged, and the current value of the node is changed to the original value of the node.

Save(parameter)

Saves the XML data from the XPathDocument2. There are three overloads of this method:

  • Save(file-name): Saves the XML to the specified file.

  • Save(Stream): Saves the XML to the specified stream.

  • Save(TextWriter): Saves the XML to the specified TextWriter.

Events

The events of the XPathDocument2 class are shown in Table 6.4. Generally, edit operations cannot occur inside an event ending in -ing, only in events ending in -ed. The events are useful for applying business rules when the state of the XPathDocument2 changes. For example, if each new ISBN attribute inserted has to be checked against a regular expression, code that handles the ItemInserting event can apply that regular expression to check for validity.

Of course, in this case it could also be enforced by a schema for the ISBN attribute and validated by calling the CheckValidity method on the XPathDocument2. Events can also be used to apply constraints across the document, for example, that you can write a price element for a book only if it has both publicationdate and ISBN attributes.

Also note the XPathDocument2 guarantees that, after an exception is thrown (e.g., to cancel a process), the document is left in the state prior to the operation starting.

Table 6.4. The Events of the XPathDocument2 Class

Event

Description

ItemInserting

Occurs when any node belonging to this document is about to be inserted into another node. This allows for extra checking and the ability to throw an exception if the operation should be canceled. The event handler receives an XPathDocument2 ChangedEventArgs containing information related to this event.

ItemInserted

Occurs after a node has been inserted in this document. This includes all nodes created by this document, whether they were in the document tree or outside the tree, and includes attribute nodes. The event handler receives an XPathDocument2 ChangedEventArgs containing information related to this event.

ItemRemoving

Occurs when any node belonging to this document is about to be removed from its parent. This allows for extra checking and the ability to throw an exception if the operation should be canceled. The event handler receives an XPathDocument2 ChangedEventArgs containing information related to this event.

ItemRemoved

Occurs after a node belonging to this document has been removed from its parent. This includes all nodes that were created by this document, whether they were in the document tree or outside the tree, and includes attribute nodes. The event handler receives an XPathDocument2ChangedEventArgs containing information related to this event.

ItemChanging

Occurs when the value of any item belonging to this document is about to be changed. Applies only to nodes that have a value. This allows for extra checking and the ability to throw an exception if the operation should be canceled. The event handler receives an XPathDocument2ChangedEventArgs containing information related to this event.

ItemChanged

Occurs when the value of any item belonging to this document has been changed. Applies only to nodes that have a value. Includes all nodes that were created by this document, whether they were in the document tree or outside the tree, and includes attribute nodes. The event handler receives an XPath Document2ChangedEventArgs containing information related to this event.

RejectingChange

Occurs when a change to any item belonging to this document is about to be rejected by means of the RejectChanges method. Applies only to nodes that have been changed. This allows for extra checking and the ability to throw an exception if the operation should be canceled. The event handler receives an XPath Document2ChangedEventArgs containing information related to this event.

ChangeRejected

Occurs when a change to any item belonging to this document is rejected by means of the RejectChanges method. Applies only to nodes that have been changed. The event handler receives an XPathDocument2ChangedEventArgs containing information related to this event.

The XPathDocument2ChangedEventArgs Class

The events of the XPathDocument2 class are returned via the registered delegate class XPathDocument2ChangedEventHandler. The signature of this class returns an XPathDocument2ChangedEventArgs instance containing information about the changed item within the tree, as shown in Table 6.5. The properties of this class can be used to create the required business logic around the event.

Table 6.5. The Properties of the XPathDocument2ChangedEventArgs Class

Property

Description

Action

A value from the XPathDocument2ChangedEventAction enumeration indicating the type of node change event. See Table 6.6.

Item

An XPathNavigator2 instance pointing to the item that is being acted on.

OldValue

A String containing the old value of the text or attribute node being updated. Contains Nothing (null in C#) for insert and delete operations.

NewValue

A String containing the new value of the text or attribute node being updated. Contains Nothing (null in C#) for insert and delete operations.

OldParent

An XPathNavigator2 instance pointing to the result of the call to the MoveToParent method that takes place on the node being deleted before the delete operation takes place. Contains Nothing (null in C#) for insert operations if the node does not have a parent beforehand. This is the same as the NewParent property if a change occurred. For attribute nodes this is the owning element.

NewParent

An XPathNavigator2 instance pointing to the result of the MoveToParent method that takes place on the node being changed.

OldPreviousItem

For delete operations this is an XPathNavigator2 instance pointing to the node that was the previous sibling of the node being deleted or updated. For insert operations this is Nothing (null in C#). For update operations this is the same as the NewPreviousItem property.

NewPreviousItem

For insert operations this is an XPathNavigator2 instance pointing to the node that will become the previous sibling of the node being inserted or updated. For delete operations this is Nothing (null in C#). For update operations this is the same as the OldPreviousItem property.

Enumerations

The various properties and methods described in the preceding subsections of this chapter use values from three enumerations:

  1. The XPathDocument2ChangedEventAction enumeration specifies the type of event that is being signaled based on the change that is taking place to the document.

  2. The XmlNodeChangeType enumeration specifies the change status of each node since the last call to the AcceptChanges or Reject Changes method and is returned from the NodeChangeType property of the XPathChangeNavigator class.

  3. The XmlChangeFilters enumeration specifies values that are passed to the SelectChanges method of the XPathChangeNavigator class to retrieve all the nodes that have a value corresponding to the NodeChangeType property. This enumeration allows a bitwise combination of its member values. Hence expressions like Inserted Or Deleted (Inserted | Deleted in C#) can be used to get both the inserted and deleted nodes.

Tables 6.6, 6.7, and 6.8 document these enumerations and their values.

Table 6.6. The XPathDocument2ChangedEventAction Enumeration

Member

Description

Changing

Indicates that the node is being updated.

Changed

Indicates that the node has been updated.

Inserting

Indicates that the node is being inserted.

Inserted

Indicates that the node has been inserted.

Removing

Indicates that the node is being deleted.

Removed

Indicates that the node has been deleted.

Rejecting

Indicates that the node is being rejected.

Rejected

Indicates that the node has been rejected.

Table 6.7. The XmlNodeChangeType Enumeration

Member

Description

Deleted

Indicates that this node has been deleted from the tree. Even though any descendants of this node are no longer part of the current view, their XmlNodeChangeTypes are not automatically set to Deleted but instead are left as they were. The reason for this is to ensure that each delete operation has exactly one node marked as Deleted.

Inserted

Indicates that this node has been inserted. Nodes can be inserted only as leaf nodes, i.e., at the end of a tree. If a node is marked as Inserted, the entire subtree under the node has also been inserted. There is no concept of inserting an entire subtree in one shot. This is equivalent to inserting a node and all its children one by one. Any updates to an inserted node do not affect the XmlNodeChangeType for the current node. If an inserted node is deleted, the node is no longer visible on the XPathChangeNavigator and the node change type is set to Unchanged (i.e., the AcceptChange and RejectChange methods will do nothing if called on this node).

Unchanged

Indicates that this node has not been changed. The AcceptChange and RejectChange methods will do nothing if called on this node.

Updated

Indicates that the value of this node has been changed. This means that it has both original and current values that can be examined as properties of the node.

Table 6.8. The XmlChangeFilters Enumeration

Member

Description

Inserted

Used in the SelectChanges method of the XPathChangeNavigator2 class to retrieve nodes that have been inserted. This filter shows the roots of subtrees that have been inserted. Nodes can be inserted only as leaves, so if a node is marked as inserted, the entire subtree below the node has been inserted.

Deleted

Used in the SelectChanges method of the XPathChangeNavigator2 class to retrieve nodes that have been deleted. This filter shows the roots of subtrees that have been deleted. Deleting a node detaches the subtree from the main tree.

Updated

Used in the SelectChanges method of the XPathChangeNavigator2 class to retrieve nodes that have been updated.

AllChanges

Used in the SelectChanges method of the XPathChangeNavigator2 class to retrieve all nodes that have been changed. This is equivalent to calling SelectChanges with Inserted Or Deleted Or Updated (Inserted | Deleted | Updated in C#).

The values in Tables 6.6 through 6.8 are flags, so you can combine them with filter nodes based on the exact type of change that has occurred. For example, you can specify that you want to see only the inserted and deleted nodes in the tree. Listing 6.3 shows the XPathChangeNavigator class retrieving the inserted and deleted nodes from a document and writing the names of these items to the console.

Listing 6.3 Retrieving Inserted and Deleted Nodes with an XPathChangeNavigator
Dim nav As XPathChangeNavigator = doc.CreateXPathChangeNavigator()

Dim iter As IEnumerable = nav.SelectChanges( _
    XmlChangeFilters.Inserted Or XmlChangeFilters.Deleted)

Dim item As XPathChangeNavigator

For Each item In iter
  If item.NodeChangeType = XmlNodeChangeType.Inserted Then
    Console.WriteLine("Inserted item: " & item.Name)
  ElseIf item.NodeChangeType = XmlNodeChangeType.Deleted Then
    Console.WriteLine("Deleted item: " & item.Name)
  End If
Next


The XPathEditor and XPathChangeNavigator Classes

The XPathEditor and XPathChangeNavigator classes, both derived from the XPathNavigator2 class, add their own particular functionality based on the cursor navigation model of traversing the document. By accepting XPath queries via the Select method, all these XPathNavigator2 classes allow you to quickly find nodes in the tree and then perform specific operations on the nodes depending on the type of navigator created.

In the Technology Preview release you will notice that there is an XPathNavigator2 class. This is due to be combined with the existing System.Xml version 1.x XPathNavigator class in the next release. Hence this chapter does not cover the functionality of the XPathNavigator2 class that is already exposed by the version 1.x XPathNavigator.

The XPathEditor Class

The XPathEditor class is designed to leverage the XPathNavigator2 and the XmlWriter functionality. The XPathEditor provides a cursor-based API for editing an XML tree and provides additional helpers to allow you to perform edit operations with a single method call. The following list shows the features provided by the XPathEditor in relation to the XPathDocument2 class.

  • Multiple XPathNavigator2/XPathEditor instances: The XPath Document2 allows multiple XPathNavigator2 and/or XPathEditor instances to be in use at the same time. These can be positioned on the same node or on different nodes in the same tree.

  • Not multithread write safe: As mentioned above, the XPath Document2 can have multiple XPathEditors in use at the same time. The XPathEditor allows users to obtain an XmlWriter on any node and to start inserting a subtree. However, the XPathEditor is a single-user XML store and does no locking of the XPathDocument2 while editing. If multiple XPathEditors are created on separate threads, the application is responsible for ensuring that locks are taken out and released appropriately.

  • XmlWriter implementation: The XPathEditor uses the XmlWriter for inserting new node trees. The CreateFirstChild, CreateNext Sibling, and CreateAttributes methods return an XmlWriter that can be used to write new nodes at different locations in the XPathDocument2 instance. The XmlWriter exposed by the XPath Document2 allows writing of well-formed XML, otherwise the write operation fails and an XmlException is thrown. For example, it is invalid to write attributes at the root node of the document.

  • Namespace nodes: The XPathDocument2 does not allow creation of namespace nodes nor the updating of namespace nodes on existing elements. New namespace nodes can be created while inserting new elements using the XmlWriter. The scope for these new namespace nodes is the scope of the newly inserted elements.

  • Text nodes: The XPathEditor provides the helpers CreateFirst ChildElement and CreateNextSiblingElement, which take a string value parameter. These methods create new element nodes with appropriate text values. This text value can be updated by calling the SetValue method after positioning the XPathEditor on this text node. Calling SetValue on an element is not legal and results in an XmlException.

  • Delete behavior: The XPathEditor API allows users to navigate to a node in the tree and call DeleteCurrent to delete the node. This node and its subtree are removed from the main tree and are no longer reachable from its parent. DeleteCurrent is not a valid operation when the XPathEditor is positioned on the document root. As a result of a delete operation there are further clarifications on the behavior of the XML store for existing navigators and change tracking.

    • Existing navigators: The delete operation doesn't affect the position of any other navigators. There are potentially other XPath Navigator2 or XPathEditor instances that are positioned on a deleted node or somewhere in the node's subtree. These navigators are valid in the sense that they can move around in the subtree. The XPathNavigator2.MoveTo(XPathNavigator2) method can be used to move these navigators back into the main tree if necessary. Although no further edit operations are allowed on a node marked as deleted, the user can continue to make changes to other nodes in the subtree, and these changes are visible through the XPathChangeNavigator.

    • Change tracking: Even though a node is deleted, edit operations are allowed in the subtree, and these changes are visible through the XPathChangeNavigator. This behavior gets further refined once AcceptChanges is called on a deleted node. Calling AcceptChanges on a deleted node marks the node and its subtree as permanently removed from the XPathDocument2 (i.e., these nodes are no longer considered part of the document). Any further changes to these nodes, even though they are allowed, are not tracked by the XPathChangeNavigator.

  • Select method: The XPathEditor extends the XPathNavigator2 class, with the Select method returning an IEnumerator that can be used to enumerate over the results of the Select operation. The XPathEditor implementation of Select is slightly different in the sense that Enumerator.Current is an XPathEditor pointing to the node. This allows you to edit the results of a Select statement when called on the XPathEditor. For example, the code in Listing 6.4 selects all the price elements and then deletes each one by iterating over the set of nodes that are returned.

Listing 6.4 Selecting and Deleting Nodes with an XPathEditor
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim editor As XPathEditor = doc.CreateXPathEditor()

Dim iter As IEnumerable = editor.Select("//price")

For Each editor As XPathEditor In iter
  ' delete this price node
  editor.DeleteCurrent()
Next

The methods described in Table 6.9 are specific to the XPathEditor class and detail the editing support for this navigator.

Table 6.9. The Methods of the XPathEditor Class

Method

Description

CloneAsNavigator()

Gets a read-only XPathNavigator2 from an XPathEditor. This is useful for handling read-only XPathNavigator2 classes from the XPathEditor.

CreateFirstChild()

Inserts new elements before the child element of the current element. It returns an XmlWriter instance that can be used to construct the tree of nodes at the specified location. The actual insert operations are performed only when the XmlWriter.Close method is called, and until then the nodes are not visible on any XPathNavigator2. This method is valid only when the XPathEditor is positioned on the document root node or an element node; otherwise an XmlException is thrown. The XPathEditor remains positioned on the current element after this call.

A second overload of this method accepts a parameter that is an XML string representing well-formed XML:

CreateFirstChild(xml-content)

This XML content can be a fragment or sequence of fragments that need to be inserted or just text content. The method's implementation is identical to calling CreateFirstChild to get back an XmlWriter, breaking down the XML string, and outputting it through the writer. The xml-content can be a fragment that represents a subtree; in this case the subtree is created and inserted in one shot. This method is valid only when the XPathEditor is positioned on the root node or an element node; otherwise an XmlException is thrown. The XPathEditor remains positioned on the current element after this call. The in-scope namespaces are automatically derived from the insertion context.

CreateNextSibling()

Inserts new elements as the next sibling(s) of the current element. It returns an XmlWriter that can be used to construct the tree of nodes at the specified location. The actual insert operations are performed only when the XmlWriter.Close method is called, and until then the nodes are not visible on any XPathNavigator2. This method is valid only when the XPathEditor is positioned on an Element, Comment, Text, or PI node. Otherwise an XmlException is thrown. Calling this method when the XPathEditor is positioned on the document root node also causes an XmlException to be thrown. The XPathEditor remains positioned on the current element after this call.

A second overload of this method accepts a parameter that is an XML string representing well-formed XML:

CreateNextSibling(xml-content)

This XML content can be a fragment or sequence of fragments that need to be inserted or just text content. The method's implementation is identical to calling CreateNextSibling to get back an XmlWriter, breaking down the XML string, and outputting it through the writer. The xml-content can be a fragment that represents a subtree; in this case the subtree is created and inserted in one shot. This method is valid only when the XPathEditor is positioned on an Element, Comment, Text, or PI node. Otherwise an XmlException is thrown. The XPathEditor remains positioned on the current element after this call. The in-scope namespaces are automatically derived from the insertion context.

CreateFirstChildElement (prefix, local-name, namespace-uri, schema-type, value)

Inserts a new element as a child of the current element. The parameters are:

  • prefix: The prefix of the element to add. If this parameter is Nothing (null in C#) or String.Empty, no prefix is set.

  • local-name: The local name of the element to add. If this parameter is Nothing (null in C#) or String.Empty, an error occurs.

  • namespace-uri: The namespace URI of the element to add. If this parameter is Nothing (null in C#) or String.Empty, no namespace URI is set.

  • schema-type: The XSD type of the element to add. If this parameter is Nothing (null in C#), the element is assumed to be untyped. In the Technology Preview this property is ignored; i.e., there is no schema type support.

  • value: The text content of the element to add. If this parameter is Nothing (null in C#) or String.Empty, it is assumed there is no text content.

CreateFirstSiblingElement (prefix, local-name, namespace-uri, schema-type, value)

Inserts a new element as a sibling of the current element. See CreateFirstChildElement for the parameter descriptions.

CreateAttributeString (prefix, local-name, namespace-uri, schema-type, value)

Creates an attribute node with the specified prefix, name, namespace, schema type, and value. If the XPathEditor is positioned on an element, the attribute is inserted as the first attribute of the element. If the XPathEditor is positioned on an attribute, the new attribute is inserted after the current attribute. This is equivalent to using an XmlWriter obtained through calling the CreateAttributes method and then calling the StartElement and EndElement methods, followed by Close. The XPathEditor remains positioned on the current node afterward. If you try to create an attribute that is actually a namespace node, an XmlException is thrown. In some cases adding new attributes causes new namespace nodes to be added because the attribute belongs to a namespace undeclared in the XML document. In this case an error is thrown only if inserting the namespace node changes the namespace name of other elements or attributes in the document.

CreateAttributes()

Inserts a new attribute into the current element. If the XPathEditor is positioned on an element, the attribute is inserted as the first attribute of the element. If the XPathEditor is positioned on an attribute, the new attribute is inserted after this attribute. The XPathEditor remains positioned on the current node after this call. If you try to create an attribute that is actually a namespace node, an XmlException is thrown. In some cases adding new attributes causes new namespace nodes to be added because the attribute belongs to a namespace undeclared in the XML document. In this case an error is thrown only if inserting the namespace node changes the namespace name of other elements or attributes in the document.

DeleteCurrent()

Removes the current node from the tree. The node is no longer reachable using an XPathNavigator2. The XPathEditor remains positioned on the current node. DeleteCurrent is not valid when the XPathEditor is positioned on the root node; in this case an XmlException is thrown. Deleting a namespace node throws an XmlException if there are other nodes referencing this namespace.

If the node being deleted is a namespace node, a check is performed to see if any elements or attributes would have their namespace name altered by the disappearance of the node. If the deletion of the node changes the namespace name of an element or attribute within the scope of the namespace node, an exception is thrown. For example, attempting to delete the namespace node that represents xmlns:xsl="http://www.w3.org/1999/XSL/Transform" from the document element of an XSLT stylesheet would be an error. On the other hand, deleting an unreferenced namespace node from the same document element would not.

Deleted nodes are still accessible to navigators positioned over them prior to deletion. However, methods that attempt to move from the deleted subtree back to the document will fail. For example, MoveToParent and MoveToPreviousSibling always fail (return False) when positioned over the top-most deleted node because the node is no longer connected to the document. Similarly, MoveToNextSibling always fails if positioned on the bottom-most deleted node.

SetValue(string)

Updates the value in the current node with the supplied String parameter value. Namespace nodes cannot be updated, and calling SetValue when positioned on a namespace node results in an XmlException being thrown. Similarly, calling SetValue on the document root node results in an XmlException being thrown. Otherwise, calling SetValue replaces the value of the node. The description of what constitutes the value of an XPath node is given in the description of the XPathNavigator2.Value property.

The XPathChangeNavigator Class

The properties and methods detailed in Tables 6.10 and 6.11 are specific to the XPathChangeNavigator class and describe the change navigation support available for this XPathNavigator2. The XPathChangeNavigator class is used to identify the changes made to an XPathDocument2 class and is created via a call to the CreateXPathChangeNavigator method on the XPathDocument2 class.

Table 6.10. The Properties of the XPathChangeNavigator Class

Property

Description

NodeChangeType

Returns a value from the XmlNodeChangeType enumeration (Updated, Inserted, Deleted, or Unchanged) that indicates the status of the node. It will be Updated if the value of the node has been changed, Inserted if this node has been inserted explicitly, and Deleted if this node has been deleted explicitly. Otherwise the value is Unchanged.

OriginalValue

Returns a String that is the original value of the node currently pointed to by the XPathChangeNavigator. If the current value of NodeChangeType is Inserted, the property returns Nothing (null in C#). It also returns Nothing (null) for all nodes except text and attribute nodes (e.g., there is no original value for an element or a PI).

Table 6.11. The Methods of the XPathChangeNavigator Class

Method

Description

AcceptChange()

Commits any changes made to this node. The AcceptChange method has the following effects:

  • Inserted nodes: The node is no longer marked as Inserted; i.e., the XmlNodeChangeType value for this node is set to Unchanged, and the current value of the node is propagated to the original value of the node.

  • Deleted nodes: The node and its subtree are no longer accessible from the main tree using an XPathNavigator2, XPathEditor, or XPathChangeNavigator.

  • Updated nodes: The node is no longer marked as Updated; i.e., the XmlNodeChangeType value for this node is set to Unchanged, and the current value of the node is propagated to the original value for the node.

RejectChange()

Rejects any changes made to this node. The RejectChange method has the following effects:

  • Inserted nodes: The node and its subtree are no longer accessible from the main tree using the XPathNavigator2, XPathEditor, or XPathChangeNavigator.

  • Deleted nodes: The node is no longer marked as Deleted; i.e., the XmlNodeChangeType value for this node is set to Unchanged, and the current value of the node is propagated to the original value of the node.

  • Updated nodes: The node is no longer marked as Updated; i.e., the XmlNodeChangeType value for this node is set to Unchanged, and the original value of the node is propagated to the current value for the node.

SelectChanges()

Returns an enumerator over all changed nodes within the subtree of this node. This is the set of nodes where the XmlNodeChangeType property has the value (Inserted Or Deleted Or Updated). This is a static set of nodes (a snapshot), so subsequent changes to the tree do not affect the enumerator. The MoveNext method is used to iterate through the nodes that match the filter criteria, or a For Each statement can be used to iterate through the results.

A second overload of this method accepts a parameter that is a value or combination of values from the XmlChange Filters enumeration, which specifies the types of changed nodes that are selected with the XPath statement:

SelectChanges(XmlChangeFilters)

CreateXmlEditor()

Returns an XPathEditor positioned on the current node. This XPathEditor could potentially be positioned on a deleted node (i.e., a node of type XmlNodeChange Type.Deleted), in which case any changes made through the XPathEditor are disconnected from the original document.

Using the XPathEditor Class with the XPathDocument2

At the start of this chapter we saw how to update an XPathDocument2 instance with an XPathEditor. This section provides more examples of using an XPathEditor with the XPathDocument2 to manipulate the XML. Typically the most common pattern is to select a set of nodes using the Select method, enumerate these nodes, and work with them.

Updating Text Node Values with the SetValue Method

The example in Listing 6.5 shows how you could increase the prices of all the books in the books.xml document. The Select method selects the text values of all the price nodes using the XPath "//price/text()", and then the SetValue method of the XPathEditor is used to increase the price of each book by 50% after it has been converted to decimal value.

Listing 6.5 Updating Text Node Values with the SetValue Method
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim xmledit As XPathEditor = doc.CreateXPathEditor()

Dim iter As IEnumerable = xmledit.Select("//price/text()")

For Each editor As XPathEditor In iter

  ' convert the price to a decimal value
  Dim price As Double = XmlConvert.ToDouble(editor.ReadStringValue())
  price = price * 1.5

  ' set the new value onto the selected price text node
  editor.SetValue(price.ToString())

Next

The screenshot in Figure 6.4 shows the results.

Figure 6.4. Updating price nodes in a document with the XPathEditor

Deleting Nodes with the DeleteCurrent Method

The next example, in Listing 6.6, shows how you can delete all the book prices. Again the Select method selects all the price nodes, this time with the XPath "//price", and then each one is deleted using the DeleteCurrent method.

Listing 6.6 Deleting Nodes with the DeleteCurrent Method
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim editor As XPathEditor = doc.CreateXPathEditor()

Dim iter As IEnumerable = editor.Select("//price")

' delete all the price nodes
For Each editor As XPathEditor In iter
  editor.DeleteCurrent()
Next editor

The screenshot in Figure 6.5 shows the results of this example.

Figure 6.5. Deleting price nodes in a document with the XPathEditor

Inserting Nodes with the CreateNextSibling Method

The next example, in Listing 6.7, shows how you can add a new book to the document, after the last book. This time, the Select method selects the last book node with the XPath "//book[position()=last()]" (i.e., select the book element that is at the last position of the document). The Create NextSibling method is then used to create the new book element.

However, instead of using an XmlWriter to create the node tree, a well-formed XML string is passed to the CreateNextSibling method. Although it is temping to provide strings to build the node tree, this tends to be error prone for anything other than small sections of XML or XML copied from other sources. It is usually preferable to use an XmlWriter. However, since the XmlWriter is used to write the node tree anyway, exceptions are still thrown for any invalid XML.

Listing 6.7 Inserting Nodes with the CreateNextSibling Method
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim xmledit As XPathEditor = doc.CreateXPathEditor()

Dim iter As IEnumerable = xmledit.Select("//book[position()=last()]")

For Each editor As XPathEditor In iter
  editor.CreateNextSibling("<book genre='technology' " _
                         & "publicationdate='10-27-2003' " _
                         & "ISBN='1-861003-11-9'>" _
                         & "<title>ASP.NET V2</title>" _
                         & "<author><first-name>Alex</first-name>" _
                         & "<last-name>Homer</last-name></author>" _
                         & "<author><first-name>Dave</first-name>" _
                         & "<last-name>Sussman</last-name></author>" _
                         & "<price>29.22</price></book>")
Next

The screenshot in Figure 6.6 on the next page shows the newly inserted book node.

Figure 6.6. Inserting a sibling node in a document with the XPathEditor

Inserting Nodes with the CreateFirstChildElement Method

The next example, in Listing 6.8, shows how you can add a new child element. This is useful for creating elements with simple content, for example, an element with just a text value and no child elements. The Select method selects all the <book> elements with a price greater than 9.99 using the XPath "/bookstore/book[price>9.99]". Then the SelectSingleValue method is used to find the single price element that we know exists somewhere in the subtree for the selected book elements.

In this case it is the last child element for the book, but this does not always have to be the case. As long as we stick to using XPath to find elements, the structure of the XML can vary without causing errors. The price is converted to a numeric Double type using the XmlConvert method, and the CreateFirstChildElement method is called to create a <discount price> element that is 80% of the original price.

In all the examples so far we are using String values when the values are either read or written, since we have no schema to tell us what the types are for the XML nodes in the document. This means that to get these into CLR values such as a Double, we have to use the XmlConvert class.

Listing 6.8 Inserting a Node with the CreateFirstChildElement Method
Dim doc As New XPathDocument2()
doc.Load("books.xml")

Dim xmledit As XPathEditor = doc.CreateXPathEditor()
Dim iter As IEnumerable _
    = xmledit.Select("/bookstore/book[price>9.99]")

For Each editor As XPathEditor In iter
  Dim price As Double = XmlConvert.ToDouble(CStr( _
                        editor.SelectSingleValue("price")))
  editor.CreateFirstChildElement("", "discountprice", "", Nothing, _
                                         (price * 0.8).ToString())
Next

The screenshot in Figure 6.7 shows the new discountprice node on the second book element.

Figure 6.7. Inserting child nodes into a document with the XPathEditor

Using the XPathChangeNavigator Class with the XPathDocument2

We can now see how the XPathChangeNavigator can be used to examine the changes made to the XPathDocument2 class. The important aspect of this class to consider is that the XPathChangeNavigator provides a "merged view" onto the XPathDocument2, allowing you to see all the nodes in the tree (including deleted nodes), along with their original values. The XPathEditor class, on the other hand, provides only a "current view" of the nodes—meaning that deleted nodes cannot be moved to, nor the original values of the nodes retrieved. They are hidden from the XPathEditor class. The "merged view" enables the following functionality with the XPathChangeNavigator.

  • It is possible to accept and reject changes at a node level. We will see later that, when using events on the XPathDocument2, this can be used to enforce business rules on the document.

  • It provides an API for navigating through changes made to the document. This API can be used by the XmlAdapter class to generate the appropriate SQL statements to update SQL Server when combined with a single XML view.

The XPathChangeNavigator exposes a value from the XmlNodeChange Type enumeration as the NodeChangeType property, which indicates the status of each node since the last call to the AcceptChanges or Reject Changes methods. A combination of values from the XmlChangeFilters enumeration can be passed to the SelectChanges method to retrieve all the nodes that have a corresponding value for their NodeChangeType property. This enumeration allows a bitwise combination of its member values; hence expressions such as Inserted Or Deleted (Inserted | Deleted in C#) can be used to get both the inserted and deleted nodes.

The next example performs the same actions as the XPathEditor examples shown earlier, applying the changes to the books.xml document. Once these changes have been made, an XPathChangeNavigator is created to iterate over them using the SelectChanges method, and each different type of change made to the book.xml document is reported. The code in Listing 6.9 shows the XPathChangeNavigator code—the code shown in earlier examples is omitted for clarity and to avoid repetition.

Listing 6.9 Iterating over the Changes to the book.xml Document
...
' code here to change the XML document in the XPathDocument2
' as demonstrated in previous code, Listings 6.5 through 6.8
...

' use the XPathChangeNavigator to iterate over the changes
Dim changes As XPathChangeNavigator = doc.CreateXPathChangeNavigator()
Dim iter4 As IEnumerable = changes.SelectChanges( _
                                   XmlChangeFilters.AllChanges)

Dim sType As String
For Each item As XPathChangeNavigator In iter4

  ' get the node type as a String
  Select Case item.ItemType
    Case 0: sType = "Element"
    Case 3: sType = "Text"
    Case Else: sType = "Other"
  End Select

  If item.NodeChangeType = XmlNodeChangeType.Inserted Then
    text1.Items.Add("inserted item - " & sType _
                  & " node:" & item.Name _
                  & " new value:" & item.ReadStringValue())

  ElseIf item.NodeChangeType = XmlNodeChangeType.Deleted Then
    text1.Items.Add("deleted item - " & sType _
                  & " node:" & item.Name _
                  & " value:" & item.ReadStringValue())

  ElseIf item.NodeChangeType = XmlNodeChangeType.Updated Then
    text1.Items.Add("updated item - " & sType _
                  & " node:" & item.Name _
                  & " new value:" & item.ReadStringValue() _
                  & " original value:" & item.OriginalValue)
  End If

Next

The screenshots in Figures 6.8 and 6.9 on the next two pages show the books.xml document before and after the changes made by the XPathEditor. Figure 6.9 is the result of navigating over the changes with an XPathChange Navigator.

Figure 6.8. The books.xml document before changes with the XPathEditor

Figure 6.9. Navigating the changes to the books.xml document with the XPathChangeNavigator

We will see in the next section how the XPathChangeNavigator can be used to accept or reject changes on individual nodes, depending on business rules applied to the document via events, when updates are made through the XPathEditor. And in Chapter 7 we will see how the XPathChangeNavigator interacts with the XmlAdapter to push changes made to a document into SQL Server through an XML view.


Using the XPathDocument2 Class

Having seen how the XPathDocument2 class can be updated with the XPathEditor class, we'll now look at some of the methods on the XPath Document2 class that can be used in conjunction with the XPathEditor.

Validating an XPathDocument2 with an XML Schema

The content of an XPathDocument2 can be validated with an XML schema to check the structure of the XML document, using the CheckValidity method. This is useful because it allows the document to be revalidated when it is loaded or updated with the XPathEditor. Before we look at the code to achieve this, we need to discuss the new XML schema library called the XmlSchemaSet class.

The XmlSchemaSet Class

In System.Xml 1.x, W3C XML schemas can be loaded into an XmlSchema Collection class, which acts as a collection (or library) of schemas. Because of interpretations of the still young W3C XML Schema specification, the XmlSchemaCollection has incorrect behavior with multiple schemas with the same namespace. Schemas and their associated imported schemas were treated as independent entities, meaning that if a schema with namespace A imported a schema with namespace B, and then a schema with namespace C also imported schema B, there would be two separate instances of schema B in the XmlSchemaCollection.

Of course, this can continue n levels deep and cause all sorts of different dependency graphs between the schemas. In most cases schema B is identical, but this need not always be the case. If a second instance of a schema with the same namespace is imported, all schemas referencing the imported schema should be recompiled. The correct implementation is that there is a "set" into which all the schemas are compiled, so that there are no "islands" of schemas. Instead, there is one logical schema into which all references between all schemas are resolved and there are no isolated instances.

The XmlSchemaSet has been introduced into System.Xml 2.0 to fix this deficiency in XmlSchemaCollection—along with some other advantages, including deprecating Microsoft's XDR schema support. The differences between the XmlSchemaCollection and the XmlSchemaSet are summarized in Table 6.12.

Table 6.12. Differences between the XmlSchemaCollection and the XmlSchemaSet

XmlSchemaCollection

XmlSchemaSet

This class supports XDR and XML schemas.

This class supports only XML schemas.

Schemas are compiled when the Add method is called.

Schemas are not compiled when the Add method is called. This provides significant performance improvement during creation of the schema library.

Each schema generates an individual compiled version, which can result in "schema islands."

Compiled schemas generate a single logical schema, a "set" of schemas.

Only one schema for a particular ce target namespace can exist in the collection.

Multiple schemas for the same target namespa can be added as long as there are no conflicts. This is because schemas in the XmlSchemaSet are identified by the combination of their target namespace and schema location.

From an API perspective, the XmlSchemaSet is similar to the Xml SchemaCollection, with the main difference being the Compile method that is added for the XmlSchemaSet. This enables you to add multiple schemas to the XmlSchemaSet and then compile or recompile the entire logical schema set only when required.

Checking the Validity of an XML Document

Now we are ready to validate the XPathDocument2 with the CheckValidity method. First we must associate an XML schema with the books.xml document. To do this we have created a new XML document called bookswithschema.xml and added a default namespace declaration xmlns =http://adoxml/ch6/samples at the start of the document. This applies to all the child element nodes in the document (see Listing 6.10).

Listing 6.10 The bookswithschema.xml Document
<?xml version="1.0" encoding="utf-8"?>
<!-- This file represents a fragment of a bookstore database -->
<bookstore xmlns="http://adoxml/ch6/samples">
  <book genre="technology" publicationdate="10-27-2003"
        ISBN="1-861003-11-1">
    <title>ADO.NET and System.Xml V2</title>
    <author>
      <first-name>Alex</first-name>
      <last-name>Homer</last-name>
    </author>
    <author>
      <first-name>Mark</first-name>
      <last-name>Fussell</last-name>
    </author>
    <price>29.99</price>
  </book>
  <book genre="autobiography" publicationdate="1981"
        ISBN="1-861-11-0">
    <title>The Autobiography of Benjamin Franklin</title>
    <author>
      <first-name>Benjamin</first-name>
      <last-name>Franklin</last-name>
    </author>
    <price>8.99</price>
  </book>
  <book genre="novel" publicationdate="1967" ISBN="0-201-63361-2">
    <title>The Confidence Man</title>
    <author>
      <first-name>Herman</first-name>
      <last-name>Melville</last-name>
    </author>
    <price>11.99</price>
  </book>
  <book genre="philosophy" publicationdate="1991"
        ISBN="1-861001-57-6">
    <title>The Gorgias</title>
    <author>
      <name>Plato</name>
    </author>
    <price>9.99</price>
  </book>
</bookstore>

An XML schema, shown in Listing 6.11, is required to validate this document. This schema is called booksschema.xsd, and the targetNamespace is set to xmlns=http://adoxml/ch6/samples.

This schema was generated using the XML Schema Inference tool on the gotdotnet Web site. See http://apps.gotdotnet.com/xmltools/xsdinference/. The tool can generate schemas for any well-formed XML document and is not bound by any of the limitations of the xsd.exe tool that ships with the version 1.x Framework SDK.

Listing 6.11 The booksschema.xsd Schema
<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified"
           targetNamespace="http://adoxml/ch6/samples"
           elementFormDefault="qualified"
           xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <xs:element name="bookstore">
  <xs:complexType>
   <xs:sequence>
    <xs:element maxOccurs="unbounded" name="book">
     <xs:complexType>
      <xs:sequence>
       <xs:element name="title" type="xs:string" />
       <xs:element maxOccurs="unbounded" name="author">
        <xs:complexType>
         <xs:sequence>
          <xs:element minOccurs="0" name="name" type="xs:string"/>
          <xs:element minOccurs="0" name="first-name"
                      type="xs:string" />
          <xs:element minOccurs="0" name="last-name"
                      type="xs:string"/>
         </xs:sequence>
        </xs:complexType>
       </xs:element>
       <xs:element name="price" type="xs:decimal" />
      </xs:sequence>
      <xs:attribute name="genre" type="xs:string" use="required" />
      <xs:attribute name="publicationdate"
                    type="xs:string" use="required" />
      <xs:attribute name="ISBN" type="xs:string" use="required" />
     </xs:complexType>
    </xs:element>
   </xs:sequence>
  </xs:complexType>
 </xs:element>
</xs:schema>

The code in Listing 6.12, for the next example, shows how to validate the bookswithschema.xml document after it has been loaded. First the booksschema.xsd schema is loaded into an XmlSchemaSet instance, setting the appropriate namespace, and the XmlSchemaSet is compiled to generate the schema set that will be used for validation. The bookswith schema.xml document is then loaded and the CheckValidity method called—passing the XmlSchemaSet and a callback handler with which to report any validation errors. This handler is called only if there are errors or warnings when the document is checked against the schema.

Listing 6.12 Validating the XML Document against Its Schema
Dim schemaSet As New XmlSchemaSet()
schemaSet.Add("http://adoxml/ch6/samples", "booksschema.xsd")
schemaSet.Compile()

Dim doc As New XPathDocument2()
doc.Load("bookswithschema.xml")

doc.CheckValidity(schemaSet, _
    New ValidationEventHandler(AddressOf ValidationCallback))
...

' the validation callback handler for error and warnings
Sub ValidationCallback(sender As Object, e As ValidationEventArgs)
  label1.Text = "Validation failed: " & e.Message
End Sub 'ValidationCallback

Revalidating the Document after Editing

If the document is edited, it can be revalidated against the schema before being saved. The code in Listing 6.13 shows a new book node being added to the bookswithschema.xml document. However, an invalid publisher attribute is also added—which means that when the CheckValidity method is called for a second time, the validation will fail. Notice that, when writing the XML, these nodes all belong to the namespace http://adoxml/ch6 /samples so that they can be validated with the schema.

Listing 6.13 Revalidating a Document after Editing
Dim schemaSet As New XmlSchemaSet()
schemaSet.Add("http://adoxml/ch6/samples", "booksschema.xsd")
schemaSet.Compile()

Dim doc As New XPathDocument2()
doc.Load("bookswithschema.xml")

Dim xmledit As XPathEditor = doc.CreateXPathEditor()
xmledit.MoveToFirstChild()
xmledit.MoveToNextSibling()

Dim writer As XmlWriter = xmledit.CreateFirstChild()

writer.WriteStartElement("book", "http://adoxml/ch6/samples")

'invalid attribute "publisher" written according to the schema
writer.WriteAttributeString("publisher", "Addison-Wesley")

writer.WriteAttributeString("genre", "technology")
writer.WriteAttributeString("publicationdate", "10-27-2003")
writer.WriteAttributeString("ISBN", "1-861003-11-1")
writer.WriteElementString("title", "http://adoxml/ch6/samples", _
                          "ADO.NET and System.Xml V2")
writer.WriteStartElement("author", "http://adoxml/ch6/samples")
writer.WriteElementString("first-name", "http://adoxml/ch6/samples", _
                          "Alex")
writer.WriteElementString("last-name", "http://adoxml/ch6/samples", _
                          "Homer")
writer.WriteEndElement()
writer.WriteStartElement("author", "http://adoxml/ch6/samples")
writer.WriteElementString("first-name", "http://adoxml/ch6/samples", _
                          "Mark")
writer.WriteElementString("last-name", "http://adoxml/ch6/samples", _
                          "Fussell")
writer.WriteEndElement()
writer.WriteElementString("price", "http://adoxml/ch6/samples", _
                          "29.99")
writer.WriteEndElement()
writer.Close()

doc.CheckValidity(schemaSet, _
    New ValidationEventHandler(AddressOf ValidationCallback))

The screenshot in Figure 6.10 shows the results of executing this code. You can see that the page displays a validation error message, which is created by the ValidationCallback event handler shown at the end of Listing 6.12.

Figure 6.10. Validation failure on the document with an XML schema

Events and Event Handling on the XPathDocument2

The XPathDocument2 exposes events that are fired when changes are made to nodes within the document. The various events available, such as ItemInserting and ItemUpdating, were described in the XPathDocument2 methods and properties section in Table 6.4.

Events can be classified into two categories: those that are raised before the node has changed (they end with the suffix –ing, such as Item Inserting) and those that are raised after the node has changed (they end with the suffix –ed, such as ItemInserted). A reference to an XPathDocument2ChangedEventArgs class is passed to all the event handlers, and this contains information about the changed node in the tree.

Events are useful for determining what changed in the document and hence for applying business rules to these changes (e.g., "Before you change the person's state, ensure that the country is set to USA"). Note that some business rules can also be applied via an XML schema, such as using the key/keyref attributes, enforcing minimum and maximum values, and pattern matching.

The ItemInserting Event

The code in Listing 6.14 shows a new book being inserted into the books .xml document but without a publicationdate attribute. An event handler named ItemInsertingEventHandler is added to the ItemInserting event of the XPathDocument2 before the new nodes are inserted. When the Close method is called for the XmlWriter, the new tree is written to the document.

However, ItemInserting events are raised for the new nodes before the values are actually changed in the document, calling the Item InsertingEventHandler method shown at the end of the code listing. Within this handler, the item being inserted is a book element identified via the Item property of the XPathDocument2ChangedEventArgs. An attempt is made to move the XPathNavigator2 to the publicationdate attribute, and if this fails (as it does in this case), an exception is thrown and an error message is displayed.

Listing 6.14 Handling the ItemInserting Event
Try

  Dim doc As New XPathDocument2()
  doc.Load("books.xml")

  AddHandler doc.ItemInserting, AddressOf ItemInsertingEventHandler

  Dim xmledit As XPathEditor = doc.CreateXPathEditor()
  xmledit.MoveToFirstChild()
  xmledit.MoveToNextSibling()

  Dim writer As XmlWriter = xmledit.CreateFirstChild()

  writer.WriteStartElement("book")
  writer.WriteAttributeString("genre", "technology")
  '  writer.WriteAttributeString ("publicationdate", "10-27-2003");
  writer.WriteAttributeString("ISBN", "1-861003-11-1")
  writer.WriteElementString("title", "Essential XML")
  writer.WriteEndElement()
  writer.Close()

Catch ex As Exception

  label1.Text = "Failed creating a book. " + ex.Message

End Try
...
...

' Check to ensure that a publicationdate attribute is supplied
' for an inserted book element. An exception is raised if no
' publicationdate attribute is supplied. This exception can be
' caught. The XPathDocument is guaranteed to be restored to
' its state prior to the insert after the exception is thrown.
Sub ItemInsertingEventHandler(sender As Object, _
                              e As XPathDocument2ChangedEventArgs)
  If e.Item.Name = "book" Then
    Dim nav As XPathNavigator2 = _
               e.Item.MoveToAttribute("publicationdate", "")

    If nav Is Nothing Then
      Throw New Exception("The 'publicationdate' attribute is " _
                        & "missing for the inserted book")
    End If
 End If

End Sub

The screenshot in Figure 6.11 shows the failed business rule for the ItemInserting event. The error message indicates that the node was not inserted into the tree because the publicationdate attribute is missing. Remember that, when an exception is raised, the content of an XPath Document2 is left in the same state as it was before the process started. In other words, the node is not actually inserted into the document.

Figure 6.11. Failure to insert a new book without a publicationdate attribute

The ItemChanged Event

Often it is more useful to invoke a business rule once a node has been changed, rather than before it changes. In these cases the XPathChange Navigator can be used to decide whether to retain or revoke the change. The code in Listing 6.15, taken from an ASP.NET example, shows all the prices in the document being changed—but with one book (currently priced at 8.99) having its new price set to 0.

The document being referenced is held in ASP.NET Session state, and the event handler named ItemChangedEventHandler is attached to the ItemChanged event property of the XPathDocument2. After the SetValue method is called each time, the ItemChangedEventHandler is invoked for each price that changed.

Listing 6.15 Handling the ItemChanged Event
Sub Page_Load()

  Dim doc As New XPathDocument2()
  doc.Load("books.xml")
  Session("doc") = doc

End Sub

Sub UpdatingBtn_click(Src As Object, E As EventArgs)

  Try
    Dim doc As XPathDocument2 = CType(Session("doc"), XPathDocument2)
    AddHandler doc.ItemChanged, AddressOf ItemChangedEventHandler

    Dim xmledit As XPathEditor = doc.CreateXPathEditor()
    Dim iter As IEnumerable = xmledit.Select("//price/text()")

    For Each editor As XPathEditor In iter
      If editor.ReadStringValue() = "8.99" Then
        editor.SetValue("0")
      Else
        editor.SetValue("20")
      End If
    Next

  Catch ex As Exception

    label1.Text = "Failed to update a book. " + ex.Message

  End Try

End Sub

The event handler named ItemChangedEventHandler is shown in Listing 6.16. In this example, in order to get hold of an XPathChangeNavigator, a reference to the original document is maintained in the ASP.NET session and positioned on the changed item by calling the MoveTo method.

Since the event is fired on the text node, we move to the parent and check that this is a price element. If so, we check the NewValue property of the changed node. If this is 0, we treat it as an invalid price and call RejectChange on the XPathChangeNavigator to change the price back to its original value.

Listing 6.16 The Event Handler for the ItemChanged Event
Sub ItemChangedEventHandler(sender As Object, _
                            e As XPathDocument2ChangedEventArgs)
  Dim doc As XPathDocument2 = CType(Session("doc"), XPathDocument2)
  Dim nav As XPathChangeNavigator = doc.CreateXPathChangeNavigator()

  nav.MoveTo(e.Item)
  e.Item.MoveToParent()
  If e.Item.Name = "price" Then
    Dim price As Double = XmlConvert.ToDouble(e.NewValue)
    If price <= 0 Then
      nav.RejectChange()
    End If
  End If
End Sub

The difference between this example and the previous one in Listing 6.14 is that an exception is not thrown (which would cause the application to terminate). Instead, all the logic is handled within the change tracking capabilities of the XPathDocument2 and XPathChangeNavigator classes, which is the preferred approach. The screenshot in Figure 6.12 shows the price still at 8.99 for the book that we tried to set to 0, whereas the others were updated successfully.

Figure 6.12. Failure to set the price of a book to 0

The ItemRemoved Event

Finally, we will look at the ItemRemoved event, which follows the same principles as the previous example. The code in Listing 6.17 shows all the title elements in a document being deleted with the DeleteCurrent method. However, the code has also added an event handler named ItemRemovedEventHandler to the ItemRemoved event property of the XPathDocument2.

Listing 6.17 Handling the ItemRemoved Event
Sub DeletingBtn_click(Src As Object, E As EventArgs)

  Try
    Dim doc As XPathDocument2 = CType(Session("doc"), XPathDocument2)
    AddHandler doc.ItemRemoved, AddressOf ItemRemovedEventHandler

    Dim xmledit As XPathEditor = doc.CreateXPathEditor()
    Dim iter As IEnumerable = xmledit.Select("//title")

    For Each editor As XPathEditor In iter
      editor.DeleteCurrent()
    Next editor

  Catch ex As Exception

    label1.Text = "Failed to update a book. " + ex.Message

  End Try

End Sub

Sub ItemRemovedEventHandler(sender As Object, _
                            e As XPathDocument2ChangedEventArgs)
  Dim doc As XPathDocument2 = CType(Session("doc"), XPathDocument2)
  Dim nav As XPathChangeNavigator = doc.CreateXPathChangeNavigator()

  nav.MoveTo(e.Item)
  Dim nav1 As XPathChangeNavigator = CType(nav.Clone(), _
                                     XPathChangeNavigator)
  nav1.MoveToParent()
  nav1.MoveToAttribute("publicationdate", "")

  If nav1.ReadStringValue() = "1981" Then
    nav.RejectChange()
  End If

End Sub

After the DeleteCurrent method is called for each title element, the ItemRemovedEventHandler is invoked for that node. An XPathChange Navigator is created from the document and positioned on the deleted item by calling the MoveTo method. The important aspect to remember here is that deleted nodes can be seen only via the XPathChangeNavigator, not via the XPathNavigator2.

So that we can call the RejectChange method on the deleted node, a second XPathChangeNavigator is copied to this node using the Clone method, and this is used to walk up to the parent node and see if the publicationdate attribute of the book node is 1981. If it is, the book must not have its title deleted, so we call RejectChange on the first XPath ChangeNavigator to put the deleted title element back into the document. The screenshot in Figure 6.13 shows that the title element is still present for the first book, which was published in 1981, while the title elements have been removed from the other two.

Figure 6.13. Removing the title from each book whose publication date is not 1981

    [ Team LiB ]


SUMMARY

In this chapter, we looked at the XPathDocument2 class members and how these properties and methods can be put to work. We saw how the XPathEditor turns the XPathDocument2 into an editable XML store. The XPathEditor, a descendant of the XPathNavigator2 class, provides a very efficient and usable model by combining XPath Select statements that can select nodes with the update methods of the XPathEditor. This XML cursor-based API approach to editing XML documents is set to replace the current XmlDocument DOM API as the preferred way to update in-memory XML documents.

The XPathDocument2 also provides many features that make it a better XML store than the XmlDocument. Based on the XPath (and hence XQuery) data model, it is the optimum store for fast, efficient XPath, XSLT, and XQuery queries. It supports change tracking at the node level by storing original and current values for each node, which are exposed via the XPathChangeNavigator class. This allows you to iterate over changes that have occurred in your XML document. The XPathChangeNavigator also allows you to select changed nodes and then either accept or reject these node changes as required.

We then looked at the XmlSchemaSet class, which is a replacement for the XmlSchemaCollection. It provides faster and more conformant validation of W3C XML schemas against an XPathDocument2 class. This enables you to edit the document with an XPathEditor and then validate or revalidate it according to a schema. We saw that, from an API perspective, the XmlSchemaSet is similar to the XmlSchemaCollection—with the main difference being an explicit Compile method.

Several examples demonstrated the events that are surfaced from the XPathDocument2 class. When combined with the XPathChangeNavigator, these enable you to apply business rules to changes on an XML document and allow you to roll back changes that do not conform to the business rules.

All in all, you should now be able to see why, because of the significant number of features and the XPath-based selection and editing model, the XPathDocument2 class is set to become the preferred XML store in the .NET Framework.

In the next chapter, we'll dive in even deeper to look at the XmlAdapter and SqlXml classes in version 2.0 of System.Xml and the .NET Framework.


Chapter 7. The XmlAdapter and SqlXml Classes

The previous chapter concentrated on a discussion of the XPathDocument2 as a better XML store. In this chapter we'll look at the XmlAdapter, and XML views that map relational tables from SQL Server to an XML schema, which altogether can provide opportunities for executing bidirectional queries and updates. This offers a comprehensive and powerful middle-tier XML programming API in ADO.NET to access relational data as XML.

Then, near the end of this chapter, we will cover support for the specific SQLXML functionality included in SQL Server 2000 today, which is now incorporated into the .NET Framework. This includes classes such as Xml BulkLoad for streaming XML into SQL Server. Finally, we'll conclude with a brief look at the XML data binding support that has been built into ASP.NET.

So, the topic list for this chapter is:

  • An introduction to the XmlAdapter class

  • XML views and three-part mapping

  • How to update SQL Server using the XmlAdapter class

  • How to migrate from SQLXML 3.0 to the .NET Framework

  • XML data source controls and XML data binding

To start, we'll look at the concepts introduced with the new XmlAdapter class and the use of XML views and the three-part mapping technology we discussed back in Chapter 5.


The XmlAdapter Class and XML Views

The XmlAdapter class provides query and update capabilities via an XML view to a SQL Server database. If you are a SQLXML advocate, you may be familiar with updategrams, which provide a declarative format for pushing changes to SQL Server. The XmlAdapter provides a programmatic way to achieve this in code, and it notably works with any XML document that is mapped.

The XmlAdapter Class

The XmlAdapter class provides the ability to synchronize changes made to an XPathDocument2 instance with SQL Server. Tables 7.1 through 7.4 contain a concise reference to the constructors, properties, methods, and event of this class. Then we'll look at some of the associated classes before examining how we can use the XmlAdapter in code examples.

The XmlUpdateEventArgs Class

The XmlUpdateEventArgs class provides information about failures for updates to SQL Server, the items that were affected, and the underlying error that occurred during the update. Table 7.5 documents the properties of this class.

Table 7.1. The Constructors for the XmlAdapter Class

Constructor

Description

XmlAdapter()

Creates an XmlAdapter used to retrieve and update data from an XPathDocument2 to a SQL Server database via an XML view.

XmlAdapter(datasource)

Creates an XmlAdapter used to retrieve and update data from an XPathDocument2 to a SQL Server database via an XML view. The datasource parameter is an XmlResolver, typically an XmlDataSourceResolver, which stores the database connections used by the Fill and Update methods to synchronize the changes.

Table 7.2. The Properties of the XmlAdapter Class

Property

Description

AcceptChangesDuringUpdate

A Boolean value that indicates whether AcceptChanges will be called on the XPathDocument2 when the changes have been applied to the SQL Server database using the Update method. The default is True.

ContinueUpdateOnError

A Boolean value that determines whether updates will continue after an error has occurred during the Update process until all have been attempted. Any errors that occur are reported after the Update process has completed. To stop updating after the first error, set this property to False. To continue attempting to update the remaining items, set this property to True. The default is False.

DataSources

A value that provides resolution to data sources used by the Fill and Update methods. An XmlResolver instance can be passed into the constructor or set using this property. Typically this is an XmlDataSourceResolver with DbConnections set to specify the SQL Server database.

Table 7.3. The Methods of the XmlAdapter Class

Method

Description

Fill(doc, query)

Executes the XmlCommand query and fills the XPathDocument2 doc with the results of the query. Any errors that occur during the Fill cause the XPathDocument2 to revert to its original state before Fill was called.

Fill(doc, query, arglist)

As above, except the XmlQueryArgumentList provides the ability to pass parameter values to the query before the Fill method is called.

Update(changes, MappingSchema)

Pushes the changes back into the database. The changes parameter is an enumerator of XPathChangeNavigators that point to the changed nodes, and the MappingSchema defines the XML view mapping with which to apply the updates.

Update(doc, MappingSchema)

As above, except that the doc parameter is the XPathDocument2 from which an XPathChange Navigator is retrieved.

Update(navigator, MappingSchema)

As above, but with a single XPathChange Navigator as the first parameter.

Table 7.4. The Event of the XmlAdapter Class

Event

Description

OnUpdateError

An UpdateEventHandler that is called when an update fails. The event handler is called once for each row that fails to update. The number of rows updated depends on the mapping in the XML view. The XmlUpdateEventArgs class contains the names of the items that failed during the update and the appropriate error information.

Table 7.5. The Properties of the XmlUpdateEventArgs Class

Property

Description

ErrorItems

An enumerator for the XPathChangeNavigators, each pointing to an item that was not successfully updated to the SQL Server. The number of items in the array depends on the unit of change. For example, if a Customer node is created in the XPathDocument2 and the insert fails, the ErrorItems property contains an array of XPathChangeNavigators pointing to the items in the XPathDocument2 that map to the Customer row in SQL Server.

Executed

A Boolean value that indicates whether or not the update was attempted. This allows you to distinguish errors that occur because of a problem during the update from errors that occur because the update was never attempted.

InnerException

The underlying exception received from SQL Server.

The XmlDataSourceResolver Class

The XmlDataSourceResolver class provides the information about database connections and XML documents over which queries are performed. The XmlAdapter and XQueryProcessor classes use this to provide a set of named values at query execution time and to determine the data sources. The constructors, properties, and methods of this class are detailed in Tables 7.6, 7.7, and 7.8, respectively.

Table 7.6. The Constructors for the XmlDataSourceResolver Class

Constructor

Description

XmlDataSourceResolver()

Creates an XmlDataSourceResolver that resolves data sources, either to databases through DbConnection instances or to XML documents through an XPathNavigator2. The XmlDataSourceResolver is an XmlResolver that supports enumeration.

XmlDataSourceResolver (nametable)

As above, with the nametable parameter specifying a name table that is shared across the data sources.

Table 7.7. The Properties of the XmlDataSourceResolver Class

Property

Description

NameTable

Returns the XmlNameTable for the XmlDataSourceResolver, which is shared across all the XML documents added to the XmlDataSourceResolver.

Count

Returns the number of data sources for the XmlDataSourceResolver.

Item

Returns the value associated with the given key.

Credentials

Allows the user to provide credentials that are used in Web requests for authentication purposes.

Table 7.8. The Methods of the XmlDataSourceResolver Class

Method

Description

Add()

Allows various XML or database connections to be added to the XmlDataSourceResolver, with a specified name. The XML documents can be XPathNavigator2 or XmlReader instances, or XML from a disk file or URL. The database connections are either IDbConnection or IDbTransaction interfaces. The overloads of this method are:

  • Add(name, DbConnection)

  • Add(name, DbTransaction)

  • Add(name, XPathNavigator2)

  • Add(name, XmlReader)

  • Add(name, source-url)

Clear

Removes all the connections from the XmlDataSourceResolver.

Contains(item-uri)

Returns a Boolean value indicating whether an item associated with the specified URI exists in the XmlDataSourceResolver.

Remove(item-uri)

Removes the item with the specified URI from the XmlDataSourceResolver.

GetEntity(absolute-uri, role, object-to-return)

Returns either XPathNavigator2 or IDbConnection instances, depending on the specified object-to-return for the given absolute-uri. The role parameter is provided for future extensibility.

ResolveUri(base-uri, relative-uri)

Generates an absolute URI from a base URI and a relative URI for use with the GetEntity method.

XML Views Using XSD, RSD, and MSD Schemas (Three-Part Mapping)

To use the XmlAdapter, you need to understand how XML views are created for querying and updating SQL Server. In Chapter 5 we discussed how XML views are a declarative XML provider for translating from the relational domain to the XML domain, by mapping elements and attributes to tables and columns.

Let's look at what it takes to write an XML view using the declarative mapping format. The mapping translates structure and relationships between data models, and in order to achieve this we use a persistent representation of each. To see it in more detail, we'll examine the mapping for just a small section of the Northwind sample database found in SQL Server 2000. We'll map the Customers, Orders, and related Products tables.

Recall that the three-part mapping requires three documents:

  1. A W3C XML schema for the XML domain (XSD)

  2. A Relational Schema Definition (RSD) for the relational domain

  3. A Mapping Schema Definition (MSD) for the mapping transformation between the domains
The XSD View for the Northwind Database

The XSD document (itself an XML schema) defines the shape of the XML view that is exposed and that is available to query against. In this example we create a structure that is a list of customers, their orders, and the products for each order. We have also decided to represent additional information as attributes, such as the customer name. The XSD view is shown in Listing 7.1.

Listing 7.1 The XSD Mapping Document
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
 <xsd:element name="Customers">
 <xsd:complexType>
 <xsd:sequence>
 <xsd:element name="Customer">
  <xsd:complexType>
   <xsd:sequence>
    <xsd:element name="Order">
     <xsd:complexType>
      <xsd:sequence>
       <xsd:element name="Product">
        <xsd:complexType>
         <xsd:attribute name="UnitQuantity" type="xsd:integer"/>
         <xsd:attribute name="ProductName" type="xsd:string"/>
         <xsd:attribute name="UnitPrice"  type="xsd:string" />
         <xsd:attribute name="QuantityPerUnit" type="xsd:string"/>
        </xsd:complexType>
       </xsd:element>
      </xsd:sequence>
      <xsd:attribute name="CustomerID" type="xsd:string" />
      <xsd:attribute name="orderid" type="xsd:string"/>
      <xsd:attribute name="OrderDate" type="xsd:string"/>
      <xsd:attribute name="ShipCity" type="xsd:string"/>
     </xsd:complexType>
    </xsd:element>
   </xsd:sequence>
   <xsd:attribute name="name" type="xsd:string" />
   <xsd:attribute name="CustomerID" type="xsd:string" />
   <xsd:attribute name="Country" type="xsd:string" />
   <xsd:attribute name="City" type="xsd:string" />
  </xsd:complexType>
 </xsd:element>
 </xsd:sequence>
 </xsd:complexType>
 </xsd:element>
</xsd:schema>
The RSD View for the Northwind Database

The RSD document specifies the shape of the relational view that represents the section of the database to which we want to map the XML view. The RSD is a serialized representation of the Northwind database tables, and Listing 7.2 shows the Customers, Orders, Order Details, and Products tables and the named columns for each table.

Listing 7.2 The RSD Mapping Document
<rsd:Database Owner="sa" Name="Northwind"
     xmlns:rsd="http://schemas.microsoft.com/data/2002/09/28/rsd">
 <rsd:Schema Name="dbo">
 <rsd:Tables>
  <rsd:Table Name="Orders">
   <rsd:Columns>
    <rsd:Column Name="OrderID" AutoIncrement="1" IncrementStep="1"
                SqlType="int" IncrementSeed="1" />
    <rsd:Column Name="CustomerID" SqlType="nchar" AllowDbNull="1"
                Length="5" />
    <rsd:Column Name="EmployeeID" SqlType="int" AllowDbNull="1" />
    <rsd:Column Name="OrderDate" SqlType="datetime" AllowDbNull="1" />
    <rsd:Column Name="RequiredDate" SqlType="datetime"
                AllowDbNull="1"/>
    <rsd:Column Name="ShippedDate" SqlType="datetime"
                AllowDbNull="1"/>
    <rsd:Column Name="ShipVia" SqlType="int" AllowDbNull="1" />
    <rsd:Column Name="Freight" SqlType="money" AllowDbNull="1" />
    <rsd:Column Name="ShipName" SqlType="nvarchar" AllowDbNull="1"
                Length="40" />
    <rsd:Column Name="ShipAddress" SqlType="nvarchar" AllowDbNull="1"
                Length="60" />
    <rsd:Column Name="ShipCity" SqlType="nvarchar" AllowDbNull="1"
                Length="15" />
    <rsd:Column Name="ShipRegion" SqlType="nvarchar" AllowDbNull="1"
                Length="15" />
    <rsd:Column Name="ShipPostalCode" SqlType="nvarchar"
                AllowDbNull="1" Length="10" />
    <rsd:Column Name="ShipCountry" SqlType="nvarchar" AllowDbNull="1"
                Length="15" />
   </rsd:Columns>
    <rsd:Constraints>
     <rsd:PrimaryKey Name="PK_Orders">
      <rsd:ColumnRef Name="OrderID" />
     </rsd:PrimaryKey>
     <rsd:ForeignKey Name="FK_Orders_Customers"
                     ForeignTable="Customers"
                     UpdateRule="None" DeleteRule="None">
      <rsd:ColumnMatch Name="CustomerID" ForeignName="CustomerID" />
     </rsd:ForeignKey>
    </rsd:Constraints>
   </rsd:Table>
   <rsd:Table Name="Products">
    <rsd:Columns>
     <rsd:Column Name="ProductID" AutoIncrement="1" IncrementStep="1"
                 SqlType="int" IncrementSeed="1" />
     <rsd:Column Name="ProductName" SqlType="nvarchar" Length="40" />
     <rsd:Column Name="SupplierID" SqlType="int" AllowDbNull="1" />
     <rsd:Column Name="CategoryID" SqlType="int" AllowDbNull="1" />
     <rsd:Column Name="QuantityPerUnit" SqlType="nvarchar"
                 AllowDbNull="1" Length="20" />
     <rsd:Column Name="UnitPrice" SqlType="money" AllowDbNull="1" />
     <rsd:Column Name="UnitsInStock" SqlType="smallint"
                 AllowDbNull="1"/>
     <rsd:Column Name="UnitsOnOrder" SqlType="smallint"
                 AllowDbNull="1"/>
     <rsd:Column Name="ReorderLevel" SqlType="smallint"
                 AllowDbNull="1"/>
     <rsd:Column Name="Discontinued" SqlType="bit" />
    </rsd:Columns>
    <rsd:Constraints>
     <rsd:PrimaryKey Name="PK_Products">
      <rsd:ColumnRef Name="ProductID" />
     </rsd:PrimaryKey>
    </rsd:Constraints>
   </rsd:Table>
   <rsd:Table Name="Order Details">
    <rsd:Columns>
     <rsd:Column Name="OrderID" SqlType="int" />
     <rsd:Column Name="ProductID" SqlType="int" />
     <rsd:Column Name="UnitPrice" SqlType="money" />
     <rsd:Column Name="Quantity" SqlType="smallint" />
     <rsd:Column Name="Discount" SqlType="real" />
    </rsd:Columns>
    <rsd:Constraints>
     <rsd:PrimaryKey Name="PK_Order_Details">
      <rsd:ColumnRef Name="OrderID" />
      <rsd:ColumnRef Name="ProductID" />
     </rsd:PrimaryKey>
     <rsd:ForeignKey Name="FK_Order_Details_Orders"
          ForeignTable="Orders" UpdateRule="None" DeleteRule="None">
      <rsd:ColumnMatch Name="OrderID" ForeignName="OrderID" />
     </rsd:ForeignKey>
     <rsd:ForeignKey Name="FK_Order_Details_Products"
          ForeignTable="Products" UpdateRule="None" DeleteRule="None">
      <rsd:ColumnMatch Name="ProductID" ForeignName="ProductID" />
     </rsd:ForeignKey>
    </rsd:Constraints>
   </rsd:Table>
   <rsd:Table Name="Customers">
    <rsd:Columns>
     <rsd:Column Name="CustomerID" SqlType="nchar" Length="5" />
     <rsd:Column Name="CompanyName" SqlType="nvarchar" Length="40" />
     <rsd:Column Name="ContactName" SqlType="nvarchar" AllowDbNull="1"
                 Length="30" />
     <rsd:Column Name="ContactTitle" SqlType="nvarchar"
                 AllowDbNull="1"
                 Length="30" />
     <rsd:Column Name="Address" SqlType="nvarchar" AllowDbNull="1"
                 Length="60" />
     <rsd:Column Name="City" SqlType="nvarchar" AllowDbNull="1"
                 Length="15" />
     <rsd:Column Name="Region" SqlType="nvarchar" AllowDbNull="1"
                 Length="15" />
     <rsd:Column Name="PostalCode" SqlType="nvarchar" AllowDbNull="1"
                 Length="10" />
     <rsd:Column Name="Country" SqlType="nvarchar" AllowDbNull="1"
                 Length="15" />
     <rsd:Column Name="Phone" SqlType="nvarchar" AllowDbNull="1"
                 Length="24" />
     <rsd:Column Name="Fax" SqlType="nvarchar" AllowDbNull="1"
                 Length="24" />
    </rsd:Columns>
    <rsd:Constraints>
     <rsd:PrimaryKey Name="PK_Customers">
      <rsd:ColumnRef Name="CustomerID" />
     </rsd:PrimaryKey>
    </rsd:Constraints>
   </rsd:Table>
  </rsd:Tables>
 </rsd:Schema>
</rsd:Database>

There is a tool (with source code) available, called RSG (rsg.exe), which allows you to automatically generate RSD file formats from a SQL Server database. This tool is provided along with the sample XSD, RSD, and MSD mapping files you see in this chapter. RSG generates the RSD schema for all the user tables in a given database. The Northwind RSD in Listing 7.2 was generated with the following command:

[View full width]

C:\RSG>rsg.exe "Server=MyServer; database= Northwind; Integrated
   Security=SSPI" C:\RSG

Alternatively you can use an account and password to SQL Server:

[View full width]

C:\RSG>rsg.exe "Server=MyServer; database=Northwind;
   uid=myusername; pwd=mypassword" C:\RSG

The first parameter provides database and connection information. The second parameter provides the file path to the source file for the RSG tool. The resulting RSD file is generated in the current directory.

The MSD Mapping Transformation Document

The MSD file ties together the XSD and the RSD views by mapping structure and relationships between the XML and relational domains. The MSD mapping for our example is shown in Listing 7.3.

Listing 7.3 The MSD Mapping Document
<m:MappingSchema
  xmlns:m="http://schemas.microsoft.com/data/2002/09/28/mapping">
 <m:DataSources>
  <m:DataSource Name="XML" Direction="Target" Type="Xml">
   <m:Schema Location="nwind.xsd" />
  </m:DataSource>
  <m:DataSource Name="Northwind" Direction="Source" Type="SQL Server">
   <m:Schema Location="nwind.rsd" />
   <m:Variable Name="Orders" Select="Orders" />
   <m:Variable Name="Customers" Select="Customers"/>
   <m:Variable Name="Products" Select="Products"/>
   <m:Variable Name="OrderDetails" Select="Order Details"/>
   <m:Relationship Name="rel1" FromVariable="Customers"
                   ToVariable="Orders">
    <m:FieldJoin From="CustomerID" To="CustomerID" />
   </m:Relationship>
   <m:Relationship Name="rel2" FromVariable="Orders"
                   ToVariable="OrderDetails">
    <m:FieldJoin From="OrderID" To="OrderID" />
   </m:Relationship>
   <m:Relationship Name="rel3" FromVariable="OrderDetails"
                   ToVariable="Products">
    <m:FieldJoin From="ProductID" To="ProductID" />
   </m:Relationship>
  </m:DataSource>
 </m:DataSources>

 <m:Mappings>
  <m:Map SourceVariable="Customers"
         TargetSelect="/Customers/Customer">
   <m:FieldMap SourceField="CompanyName" TargetField="@name" />
   <m:FieldMap SourceField="CustomerID" TargetField="@CustomerID" />
   <m:FieldMap SourceField="Country" TargetField="@Country" />
   <m:FieldMap SourceField="City" TargetField="@City" />
  </m:Map
  <m:Map SourceVariable="Orders"
         TargetSelect="/Customers/Customer/Order">
   <m:FieldMap SourceField="orderid" TargetField="@orderid" />
   <m:FieldMap SourceField="CustomerID" TargetField="@CustomerID" />
   <m:FieldMap SourceField="OrderDate" TargetField="@OrderDate" />
   <m:FieldMap SourceField="ShipCity" TargetField="@ShipCity" />
  </m:Map>
 </m:Mappings>
</m:MappingSchema>

Breaking Down the MSD Mapping

To help you understand the nature of these mapping documents, we'll break down the two top-level element definitions in the MSD (the DataSources element and Mappings element) and examine their child elements.

The DataSources Element

The <DataSources> element contains data source references and declarations, as described below.

  • The <DataSource> elements describe data sources. For example, the Northwind mapping schema defines two data sources, nwind.rsd and nwind.xsd. The nwind.rsd is the database source, and the nwind.xsd is the target specified by the Direction attribute:

    <m:DataSource Name="XML" Direction="Target" Type="Xml">
<m:DataSource Name="Northwind" Direction="Source" Type="SQL Server">

  • The <Schema> element provides the schema filename location as a URL:

    <m:Schema Location="nwind.xsd"/>
<m:Schema Location="nwind.rsd"/>

  • The <Variable> element defines a variable that maps to a relational table. For example, the variable named Orders maps to the Orders table and acts as an alias to the table. This alias name is then used in the mappings in the <Mappings> element and can be considered to be a variable or a cursor over the table. You can map a variable to multiple targets, in which case these are populated with the same set of data. In the Northwind mapping schema, the Orders, Customers, Products, and OrderDetails variable names are like cursors on the Orders, Customers, Products, and Order Details tables, respectively:

    <m:Variable Name="Orders" Select="Orders" />
<m:Variable Name="Customers" Select="Customers"/>
<m:Variable Name="Products" Select="Products"/>
<m:Variable Name="OrderDetails" Select="Order Details"/>

  • The <Relationship> element defines relationships between variables. In describing the data source, you should declare the variables first and then define the relationships between these variables (or tables) using the <Relationship> element. You describe the parent/ child relationship between the tables using the <FieldJoin> child element of the <Relationship> element and specify the column names as values in the From and To attributes. Alternatively, you can use the Constraint attribute as shown in the last example in this list (on the next page). In the Northwind mapping schema there are three relationships, defined rel1, rel2, and rel3. The relationship rel1 defines a parent/child relationship between the Customers and Orders tables based on the CustomerID column from each one:

    <m:Relationship Name="rel1" FromVariable="Customers"
                ToVariable="Orders">
 <m:FieldJoin From="CustomerID" To="CustomerID" />
</m:Relationship>

  • Similarly, rel2 defines a parent/child relationship between the Orders and Order Details tables based on the OrderID column:

    <m:Relationship Name="rel2" FromVariable="Orders"
                ToVariable="OrderDetails">
 <m:FieldJoin From="OrderID" To="OrderID" />
</m:Relationship>

  • Finally, rel3 defines a parent/child relationship between the Products and Order Details tables based on the ProductID column:

    <m:Relationship Name="rel3" FromVariable="Products"
                ToVariable="Order Details">
 <m:FieldJoin From="ProductID" To="ProductID" />
</m:Relationship>

  • If there is a PrimaryKey or ForeignKey (PK/FK) constraint defined on the relational tables in the RSD, you can specify a Constraint attribute instead. In the Northwind mapping schema we could define another relationship, rel4, between the Customers and Orders tables as shown below. This is equivalent to the relationship rel1 shown earlier:

    <m:Relationship Name="rel4"
                FromVariable="Customers"
                ToVariable="Orders"
                Constraint="FK_Orders_Customers" >
</m:Relationship>
The Mappings Element

The <Mappings> element maps between the XSD and RSD schemas and is a container for the Map elements. It contains the following element definitions.

  • The <Map> child elements are used to specify the mappings between tables (or variables) and elements. If we consider just the Customer element from our Northwind XSD view (Listing 7.1), it has four attributes:

    <xsd:element name="Customer">
 <xsd:complexType>
  <xsd:attribute name="name" type="xsd:string" />
  <xsd:attribute name="CustomerID" type="xsd:string" />
  <xsd:attribute name="Country" type="xsd:string" />
  <xsd:attribute name="City" type="xsd:string" />
 </xsd:complexType>
</xsd:element>

  • The first <Map> element maps the <Customer> element to the Customers table (variable) and the attributes of the <Customer> element to the columns in the Customers table:

    <m:Map SourceVariable="Customers"
       TargetSelect="/Customers/Customer">
 <m:FieldMap SourceField="CompanyName" TargetField="@name" />
 <m:FieldMap SourceField="CustomerID" TargetField="@CustomerID" />
 <m:FieldMap SourceField="Country" TargetField="@Country" />
 <m:FieldMap SourceField="City" TargetField="@City" />
</m:Map>

  • Likewise, the next <Map> element maps the <Orders> element, which is a child of the <Customer> element, to the Orders table (variable) and the attributes of the <Orders> element to the columns in the Orders table:

    <m:Map SourceVariable="Orders"
       TargetSelect="/Customers/Customer/Order">
 <m:FieldMap SourceField="orderid" TargetField="@orderid" />
 <m:FieldMap SourceField="CustomerID" TargetField="@CustomerID" />
 <m:FieldMap SourceField="OrderDate" TargetField="@OrderDate" />
 <m:FieldMap SourceField="ShipCity" TargetField="@ShipCity" />
</m:Map>

  • The SourceVariable and TargetSelect attributes map the variables defined for the Northwind data source to the elements in the Northwind XSD.

  • The <FieldMap> child elements map relational columns to XML schema attributes using the SourceField and TargetField attributes. For example, the CompanyName column in the Customers table is mapped to the name attribute of the <Customer> element.

  • We can also map columns to elements rather than attributes. To do this we need to alter the XSD and the MSD. The XSD schema below shows the <Customer> element with child elements instead of attributes:

    <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<xsd:element name="Customer">
 <xsd:complexType>
  <xsd:sequence>
   <xsd:element name="name" type="xsd:string" />
   <xsd:element name="CustomerID" type="xsd:string" />
   <xsd:element name="Country" type="xsd:string" />
   <xsd:element name="City" type="xsd:string" />
  </xsd:sequence>
 </xsd:complexType>
</xsd:element>
</xsd:schema>

In order to map this element-centric XSD, the MSD mapping would need to be amended as shown in Listing 7.4.

Listing 7.4 The Amended MSD for Element-Centric Mapping
<m:Mappings>
 <m:Map SourceVariable="Customers"
        TargetSelect="/Customers/Customer"/>
 <m:Map SourceVariable="Customers"
        TargetSelect="/Customers/Customer/name">
  <m:FieldMap SourceField="name" TargetField="text()"/>
 </m:Map>
 <m:Map SourceVariable="Customers"
        TargetSelect="/Customers/Customer/CustomerID">
  <m:FieldMap SourceField="CustomerID" TargetField="text()"/>
 </m:Map>
 <m:Map SourceVariable="Customers"
        TargetSelect="/Customers/Customer/Country">
  <m:FieldMap SourceField="Country" TargetField="text()"/>
 </m:Map>
 <m:Map SourceVariable="Customers"
        TargetSelect="/Customers/Customer/City">
  <m:FieldMap SourceField="City" TargetField="text()"/>
 </m:Map>
</m:Mappings>

Note that the RSD does not have to change, only the structure of the XML view and how it is mapped.

The XmlViewSchema and XmlViewSchemaDictionary Classes

Now that we have shown the mapping format and syntax, there are two classes we should introduce that represent the XML view: the XmlView Schema and the XmlViewSchemaDictionary. Both of these can be found in the System.Xml.Query namespace of the Framework.

The XmlViewSchema class represents a compiled XML view. The mapping allows a non-XML source, such as a relational database, to represent its data in a shape that conforms to an XML schema or XSD. In mapping terminology, the exposed XSD is the "target" or "query" side of the mapping while the underlying data source is the "source" or "persistent" side of the mapping.

The XmlViewSchemaDictionary is a dictionary of XmlViewSchema objects, where the five overloads of the Add method (see Table 7.8 earlier) provide the ability to add new XML views. The views can be used to issue queries and updates to the data source such as SQL Server. You pass in an MSD file reference, and the Add method creates a new XmlViewSchema object, compiles the mapping, and adds this to the dictionary.

The System.Data.Mapping Namespace

The System.Data.Mapping namespace contains object models for the MSD and RSD formats. The one class that is most important here for using an XmlAdapter is the MappingSchema, which represents the compiled state for the MSD file. This is used when we call the Update method on the XmlAdapter.

Meanwhile, the XSD object model classes are in the System.Xml.Schema namespace and were shipped in version 1.x of the .NET Framework. Many classes in this namespace are beyond the scope of this book and for the most part are never needed by applications using XML views—only by tools that manipulate the object models.


Updating SQL Server with the XmlAdapter Class

Developers are already familiar with the Fill method that the ADO.NET DataAdapter uses to retrieve relational data from a DBMS data source. In principle, the Fill method exposed by the XmlAdapter class is identical, except that rather than filling a DataSet it fills an XPathDocument2. There are, however, differences in the way it is used. The query language we use is XQuery, and the three-part mapping that shapes the XML as an XML view is also required. In the examples that follow we use the mapping files that we examined earlier in this chapter for the Customers, Orders, and related Products tables from the Northwind database, using them to retrieve data from and update data to SQL Server.

The map:view Command and XQuery

In Chapter 8, we'll look in more detail at the XQuery language. For now, we can just consider this as an XPath. In order to identify which XML view to use when the query is issued to SQL Server, the map:view extension function has been added to the XQuery. This takes as a parameter the name of the XML view or XSD schema.

Since this is an extension function, it must be identified by the namespace: http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions. For example, the XQuery shown below uses the nwind XML view to return all the customers with their orders as a well-formed XML document that can be displayed in Internet Explorer:

declare namespace
map='http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions'
map:view('nwind')/Customers

Using the Fill Method with an XPathDocument2

To use the Fill method, you just specify the connection string and an XQuery statement when creating the XmlAdapter. The code is shown in Listing 7.5.

Listing 7.5 Using the Fill Method with an XPathDocument2
Dim myConn As New SqlConnection("your-connection-string")
myConn.Open()

' create an XQueryProcessor to compile the XQuery statement
Dim xq As New XQueryProcessor()
xq.XmlViewSchemaDictionary.Add("nwind", New XmlTextReader("msd-file"))

xq.Compile("declare namespace map='http://schemas.microsoft.com/xml" _
   & "/2002/09/28/mappingfunctions' map:view('nwind')/Customers")

Dim datasource As New XmlDataSourceResolver()
datasource.Add("Northwind", myConn)

' instantiate an XmlAdapter
Dim adapter As New XmlAdapter()

' assign the XmlResolver to the adapter's DataSources property
' to resolve the SQL data source when the query is executed
adapter.DataSources = datasource

' instantiate an XPathDocument2 object to fill
Dim doc As New XPathDocument2()

' execute Fill method of XmlAdapter to populate XPathDocument2
' XmlCommand property of the XQueryProcessor is used to specify
' the command representing the compiled XQuery
adapter.Fill(doc, xq.XmlCommand)

' instantiate an XPathNavigator2 object to navigate the results
Dim nav As XPathNavigator2 = doc.CreateXPathNavigator2()

' an XmlDocument is returned to use in the <asp:xml> control
xmldoc.LoadXml(nav.OuterXml)

myConn.Close()   ' close connection

To display the results, we've included an ASP.NET <asp:xml> control on the page, so the final line of code shown above returns an XmlDocument that can be used to populate the control and display the results (see Figure 7.1).

Figure 7.1. Using the XmlAdapter to fill an XPathDocument2

How It Works

First a SqlConnection is opened to the Northwind database. Next an instance of an XQueryProcessor class is created and used to compile the XQuery that will be executed. We will be discussing this class in detail in the next chapter, but it is analogous to a SqlCommand class in that it lets you issue a SQL statement to a database.

Next the Add method of the XmlViewSchemaDictionary is used to load the nwind.msd mapping file, which in turn loads the nwind.rsd and nwind.xsd schema files and compiles these ready to issue queries against SQL Server. The XQuery map:view('nwind')/Customers, which references the nwind XML view, is then compiled. This improves performance, allows reuse, and occurs because it is composed with the XML view. Note that it's important to ensure that the names used in the map:view function are the same as the names added to the XmlViewSchemaDictionary; otherwise the query will fail at runtime with an error because it cannot find the data source.

The XmlDataSourceResolver is then used to associate a connection with a data source, in this case the Northwind data source that represents the nwind.rsd schema. This is the name of the data source in the MSD file, as shown below:

<m:DataSource Name="Northwind" Direction="Source" Type="SQL Server">

This must match the name of the data source that is added to the XmlDataSourceResolver:

datasource.Add("Northwind", myConn)

The approach here is that there can be multiple different connection and schema representations, and these can be combined as needed. The XmlAdapter can work against only a single XML view, but in Chapter 8 we will see how the XQueryProcessor can work over multiple XML views, connections, or databases.

The XmlDataSourceResolver is then set on the XmlAdapter via the DataSources property, and we are ready to issue a query to return results as XML from relational tables. The Fill method is called with an XmlCommand parameter (the XQuery to execute) and the XPathDocument2 to fill with the results.

On return, we have successfully retrieved relational data and built this into our chosen XML structure, via the XML view. The XPathNavigator2 is used to retrieve the XML from the document and display the results. In the screenshot in Figure 7.1, you can see that it returns all the customers and all their orders from the map:view('nwind')/Customers query.

Using Parameters with the Fill Method

You may have noticed that the Fill method can take an XmlQueryArgument List class, which allows parameters to be passed to the query. At the time of this writing, the W3C XQuery language specification does not have a syntax defined for passing external parameters to a query. It will be implemented once the syntax is agreed upon and becomes a recommendation.

Using the Update Method with an XPathDocument2

The Update method of the XmlAdapter makes it easy to push changes made to the XPathDocument2 class back into the SQL Server data source, using an XML view schema to define the mapping:

XmlAdapter.Update(doc, schema)

The Update method automatically generates the SQL updates, based on the XML view schema. This is analogous to the CommandBuilder in ADO.NET, but the update always happens behind the scenes, and you don't actually have to do anything to make it work. OK, so you have to define the XML to relational mapping via the XML view, but since this is explicit it means that it is straightforward for the XmlAdapter to generate all the required SQL statements to translate the XML changes into table and row updates.

As before, let's look at the code for an example that uses the Update method, then see the results and walk through how it works. The code for this example is shown in Listing 7.6. The first part is the same as the example in Listing 7.5, creating an XPathDocument2 and then filling it using the query map:view('nwind')/Customers. We've omitted this repeated code for clarity. Then the code updates the Country column for customers in Germany to Nigeria, using the Fill and Update methods of the XmlAdapter.

Listing 7.6 Using the Update Method with an XPathDocument2
...
...  create and fill an XPathDocument2 here as in Listing 7.5
...

' create XPathEditor object to change orders for Germany to Nigeria
Dim xmleditor As XPathEditor = doc.CreateXPathEditor()
Dim iter As IEnumerable = _
    xmleditor.Select("/Customers/Customer[@Country='Germany']")
For Each editor As XPathEditor In iter
  editor.MoveToAttribute("Country", "")
  editor.SetValue("Nigeria")
Next

' execute the Update method to update SQL Server.
' same MappingSchema can be used as for the Fill method when
' the destination of the updates is the same data source
Dim schema As New MappingSchema("msd-file")
adapter.Update(doc, schema)

The screenshots in Figures 7.2 and 7.3 on the next page show the "before" and "after" views.

Figure 7.2. Using the XmlAdapter to fill an XPathDocument2

Figure 7.3. Using the XmlAdapter to update SQL Server

How It Works

The section of code we omitted in Listing 7.6 is identical to the previous Fill example. After this, the code updates the XPathDocument2 by first using an XPathEditor to find all the customers that have the value Germany for the Country attribute and then changing the value of this attribute to Nigeria with the SetValue method.

The Update method requires a mapping schema to perform the update, but this can be the same one as used for the Fill method (the schema of the XML document and the data source have not changed). We recompile the nwind.msd file and provide this along with the changed document to the Update method that updates the database.

Using the Update Method with an XPathChangeNavigator

We can also use an XPathChangeNavigator to perform the update. This has two advantages. First, we can be selective about exactly what is updated, perhaps by choosing to update only deleted values or changes made to a certain set of nodes. This provides a very flexible approach.

The second advantage is that, like the XPathNavigator, it's possible to implement the XPathChangeNavigator over any store of data that has the ability to track changes and push these changes back into the data source. For example, you may choose to implement an ArrayList that tracks changes. By building an XPathChangeNavigator implementation for the ArrayList, it could be used for updates to SQL Server. The code in Listing 7.7 creates an XPathChangeNavigator and uses it to update SQL Server. In this case, all the document changes are updated because the XPathChange Navigator was created and located at the root of the document.

Listing 7.7 Using the Update Method with an XPathChangeNavigator
Dim changenav As XPathChangeNavigator _
              = doc.CreateXPathChangeNavigator()

' write all the changes to the console
IEnumerable iter = changenav.SelectChanges()
For Each navigator As XPathChangeNavigator in iter
  Console.WriteLine(navigator.InnerXml)
Next
' perform the update with the XPathChangeNavigator
adapter.Update(changenav, schema)

Handling Update Errors

The XmlAdapter can fail to perform updates from an XML document in the same way that an ADO.NET DataAdapter can fail to perform updates from a DataSet, for example, through violation of a primary key constraint by trying to insert a duplicate entry. These failures are reported via the On UpdateError event handler, which is registered with the XmlAdapter before calling the Update method. The code in Visual Basic .NET is:

AddHandler adapter.OnUpdateError, _
          New UpdateEventHandler(AddressOf MyHandler)

Or in C#:

adapter.OnUpdateError += new UpdateEventHandler(MyHandler);

The event handler for UpdateError is a callback method with a typical .NET signature. The event handler used in the next example is shown in Listing 7.8.

Listing 7.8 The Event Handler for the UpdateError Event
' the XmlAdapter.OnUpdateError event handler
Sub OnUpdateError(src As Object, args As XmlUpdateEventArgs)
  lblError.Text &= "Update Error  : " _
                & args.InnerException.Source & "<p />"
  lblError.Text &= "Error Details : " _
                & args.InnerException.Message & "<p />"

  ' iterate the set of ErrorItems that failed the update
  For Each navigator As XPathChangeNavigator In args.ErrorItems
    ' write name and value of each element or attribute in error
    lblError.Text &= "Item in error: " & navigator.LocalName _
                  & ": " & navigator.InnerXml & "<p />"
  Next navigator
End Sub
An Example of Handling the UpdateError Event

The code in Listing 7.9 uses an XPathEditor to add a customer that already exists to the XML document, in this case, the customer with ID ALFKI. When the Update method is called, SQL statements are generated to insert the new customer. The duplicate value for the CustomerID column causes the ADO.NET SqlClient class to fail the insert with a primary key constraint.

This error is propagated back to the XmlAdapter, which calls the Update Error event handler attached to the adapter. As well as reporting the exception message, the code in the error handler displays details of the items that failed the update; these details are reported via an XPathChange Navigator. The precise failure location within the XML document can be identified.

The complete code for the example is shown in Listing 7.9, with the exception of the event handler itself, which appears in Listing 7.8.

Listing 7.9 Demonstrating the UpdateError Event
...
...  create and fill an XPathDocument2 here as in Listing 7.5
...

' create XPathEditor object and use to add illegal new element
' that will fail to update because of key restriction in database
Dim xmleditor As XPathEditor = doc.CreateXPathEditor()
Dim iter As IEnumerable = _
    xmleditor.Select("/Customers/Customer[@Country='Germany']")
Dim editor As XPathEditor = doc.CreateXPathEditor()
editor.MoveToFirstChild()
editor.CreateFirstChild("<Customer name='Alfreds Futterkiste' " _
                       & "CustomerID='ALFKI' />")

' register event handler for the XmlAdapter.OnUpdateError event
AddHandler adapter.OnUpdateError, AddressOf OnUpdateError

' execute the Update method to update SQL Server.
' same MappingSchema can used as for the Fill method when
' the destination of the updates is the same data source
Dim schema As New MappingSchema("msd-file")
adapter.Update(doc, schema)

Executing the example in Listing 7.9 results in an error during the update. The error details are reported from the OnUpdateError event handler, as shown in Figure 7.4.

Figure 7.4. Handling the UpdateError event for the XmlAdapter

Continuing after an Update Error

By default, the first update error that occurs terminates the whole update and rolls back any changes made to the DBMS. Alternatively, the Continue UpdateOnError property can be set to True, which causes the Update method to continue with the rest of the updates. Each subsequent failure is reported via the UpdateError event handler.

This technique is useful if there have been a large number of changes to the document that are independent of one another. For example, when five new customers have been added and fifty orders have been updated, you can choose in the XPathChangeNavigator to apply all the changes at once, just the changes for new customers, or just the updates to the orders, and report failures for each to the user.

The code in Listing 7.10 shows two new customers being added to an XPathDocument2. The first has CustomerID='ALFKI' and, as we saw in the previous example, will result in a primary key constraint violation. However, in this example a second addition, with CustomerID='ACME', succeeds because this customer is a new entry (i.e., there is no existing record with that CustomerID in the Customers table of the Northwind database).

Listing 7.10 Using the ContinueUpdateOnError Property
' create XPathEditor and add 2 records, 1 duplicate and 1 new
Dim editor As XPathEditor = doc.CreateXPathEditor()
editor.MoveToFirstChild()

' add a duplicate record
editor.CreateFirstChild("<Customer name='Alfreds Futterkiste' " _
                       & "CustomerID='ALFKI' />")

' add a new record
editor.CreateFirstChild("<Customer name='Acme Corporation' " _
                       & "CustomerID='ACME' />")

' set the ContinueUpdateOnError property to True
adapter.ContinueUpdateOnError = True

' execute the Update. OnUpdateError event handler will report an
' error for the first insert. The update process will continue
' after error is reported and the second record will be inserted.
Dim schema As New MappingSchema("msd-file")
adapter.Update(doc, schema)

Tips and Tricks When Working with the XmlAdapter

This section provides some useful tips when working with the XmlAdapter and SQL Server.

Use SetValue for Updating Values

When updating the value of an element, use the SetValue method of the XPathEditor on a Text node, rather than deleting the element and inserting a new one. This is because the XmlAdapter understands changes that are based on operations, rather than on snapshots of the data. Hence a deletion and an insertion are treated as two separate operations, and in the Technology Preview release there is no support for multiple operations on the same row or column in the same update.

Call CheckValidity Before Calling Update

XML views are not used to validate an XPathDocument2 when the Update method is called on the XmlAdapter. If you need to validate the XPath Document2 before updating SQL Server, call the CheckValidity method first.

Provide an OnUpdateError Handler

Always provide an OnUpdateError event handler and register it with the XmlAdapter before calling the Update method so that it will report update errors. Typically set the ContinueUpdateOnError property to True so that multiple update errors are reported.

Note the Limitations in the Technology Preview Release

The XmlAdapter does not support updates to multiple SQL Server databases.

Tips and Tricks When Working with XML Views

This section provides some useful tips when working with XML views and mappings.

Use a Top-Level Node in the Mapping

Generally it is desirable to have a top-level node in the mapping in order to generate a well-formed XML document. Although this is not a requirement (e.g., you can choose to have just a list of customers), it is often useful so that you can display and work with the XML. Unlike the XmlDocument, the XPathDocument2 class can load XML fragments and does not require a top-level document element.

Avoid Constructing Nodes in the Query Unless They Fit the Mapping

XQuery enables you to construct nodes in the XML result that is returned. If you construct nodes that are not in the XML view, the XmlAdapter does not know how to map the XPathDocument2 back to the SQL Server DBMS. For example, given the Customers XML view in the previous section, the following XQuery is valid and can be used to wrap the returned Customer nodes:

declare namespace map='http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions'
<Customers>
  {map:view('nwind')/Customers/Customer[@CustomerID='ALFKI']}
</Customers>

However, the following queries would then fail on update because neither the <root> nor <Orders> elements are mapped to the XML view:

declare namespace map='http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions'
<root>
  {map:view('nwind')/Customers/Customer[@City='London']}
</root>

declare namespace map='http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions'
<Orders>
  {map:view('nwind')/Customers/Customer[@CustomerID='ALFKI']/Order} </Orders>
Map Only What Is Needed and Create Multiple Views If Necessary

You need to map only what you want to query over. This applies to each file format of the three-part mapping. You have to create an RSD file only for the tables you want to expose. Your XSD can contain additional elements and attributes in the XSD that are not mapped, but they are there for validation purposes only. Any elements or attributes that are not mapped are simply ignored during the query and update.

For example, if we rework part of the Customer XML view we used earlier to add a new Description element, but we choose not to map the Description element, this does not result in an error. Let's say the XML schema defines Customer as:

<xsd:element name="Customer">
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name="CustomerID" xsd:type="xsd:string" />
      <xsd:element name="CName" xsd:type="xsd:string" />
      <xsd:element name="Description" xsd:type="xsd:string" />
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>

And the element-centric MSD mapping is defined as:

<m:Map SourceVariable="Customer" TargetSelect="/Customer" />
<m:Map SourceVariable="Customer" TargetSelect="/Customer/CustomerID">
  <m:FieldMap SourceField="CustomerID" TargetSelect="text()" />
</m:Map>
<m:Map SourceVariable="Customer" TargetSelect="/Customer/CName">
  <m:FieldMap SourceField="CustomerName" TargetSelect="text()" />
</m:Map>
<m:Map SourceVariable="Customer" TargetSelect="/Customer" />

Then if the new Customer shown below is inserted, the Description element is discarded and does not result in an error.

<Customer>
  <CustomerID>AW</CustomerID >
  <CName>Addison-Wesley</CName>
  <Description>Book Publisher</Description>
</Customer>

Finally, you can define multiple XML views and MSD mappings for any given RSD file, depending on how you want to shape the XML. We will see this in Chapter 8 when we use XQuery to retrieve XML through a number of different XML views.


Migrating from SQLXML 3.0 to the .NET Framework

In the late 1990s, as XML began to gain prominence as a useful way to expose data over the Web, developers focused on creating XML dynamically from a relational database and sending it to the client. Microsoft shipped a technology to enable this called SQLXML. This technology for exposing SQL Server as XML has been integrated into SQL Server 2000 so that any SQL query can append the keywords FOR XML to receive the result as a stream of XML.

SQLXML, now in version 3.0, supports a range of features that allow precise control over the format of the XML that is returned, updates to the data using updategrams, templating (where the data is inserted into an XML template), and more. SQLXML 3.0 is still included in SQL Server "Yukon" and can be used in the same way as with SQL Server 2000.

SQLXML is a native COM-related technology with some managed wrapper classes that can be used from managed (.NET) code, but when considering how this is integrated into the .NET Framework, many of the original requirements have changed.

The XML chapters in this book describe a middle-tier XML programming model that has been built into the .NET Framework, which provides the same functionality as SQLXML and further extends the concept of accessing relational data as XML. The merging of the SQLXML technology into ADO.NET provides a richer set of mapping features through XML views, using XQuery as the query language.

The main advances and requirements of the .NET SQLXML implementation are listed on the next page.

  • The use of XQuery as the primary query language provides a complete XML query technology that is able to sort, perform joins, and do type checking.

  • A shared mapping technology between XML views and ObjectSpaces maps relational data from SQL Server to the middle tier, depending on whether objects or XML are required by the application. This provides additional functionality and is more flexible than the annotated XML schemas used in SQLXML.

  • A programmatic API in ADO.NET called the XmlAdapter provides update capabilities for SQL Server that are easier to use than updategrams in SQLXML. When combined with the XPathDocument2 class, this provides a complete XML API for queries and updates to SQL Server.

  • An equivalent implementation in the .NET Framework of SQLXML technologies within the new System.Data.SqlXml namespace contains classes that offer many of the features of SQLXML but use managed code. Specifically these are:

    • The XmlBulkload class, which allows XML documents to be quickly and easily loaded into SQL Server

    • The DBObject class, which provides cursor-like read/write access to a database using XPath expressions to specify the path and exposes the data through a DataReader

    • The XmlRowsetAdapter, which converts XML into relational data and loads it into a DataSet

Although some of the classes are a direct replacement for parts of SQLXML, this is about integrating SQLXML into the .NET Framework and at the same time further extending the XML data access technology to provide a richer and more flexible application framework. These classes are not 100% backward or forward compatible. Table 7.9 provides a comparative picture between SQLXML and the features introduced in System.Xml version 2.0.

This section of the chapter looks at the XmlBulkLoad, DBObject, and XmlRowsetAdapter classes, and how they can be used. We'll then briefly overview how you can access XML over HTTP using the new data source controls included in ASP.NET and bind other controls to them to display XML data.

Table 7.9. A Comparison of Features in SQLXML 3.0 and the .NET Framework

SQLXML 3.0 Feature

.NET Framework Equivalent

XML bulk loading

The XmlBulkload class provides identical functionality.

XML templates

In the .NET Framework the combination of XQuery and ASP.NET provides the equivalent functionality.

Updategrams

The XPathDocument2 class includes change tracking along with the XmlAdapter to push updates back into the database through an XML view.

SQLXML Web Services

SQL Server "Yukon" exposes stored procedures as Web Services, but there is no direct support for Web Service XML templates. Note that this moves the processing from the middle tier into the database.

HTTP access to SQLXML

ASP.NET includes Web Form controls that can access and expose XML data and documents for other controls to bind to or for direct access in code.

Annotated XSD mapping schemas

XML views use the three-part mapping features throughout the ADO.NET and System.Xml classes and namespaces.

The goals for the XmlBulkLoad, DBObject, and XmlRowsetAdapter classes are to provide:

  • The best possible performance, exceeding that of other XML parsing techniques

  • Ease of use, avoiding the need to instantiate a parser and parse the XML in code

  • Integration with the existing classes and features in the System.Xml namespaces

  • A programming model that is similar to the existing SQLXML features

  • A consistent programming model for both middle-tier and "Yukon" CLR-hosted code

It's interesting to note that the XmlBulkLoad and DBObject classes mirror features in the ADO.NET namespace classes, but for XML data rather than relational data. ADO.NET provides a Sql BulkCopy Operation class to bulk-load data into a database and the Sql ResultSet and SqlRecord features for cursor-based single row access to a relational database table.

The XmlBulkLoad Class

One of the most popular features in SQLXML was the ability to load data quickly and easily from an XML document into SQL Server. To offer the same feature in managed code, the new System.Data.SqlXml namespace includes a class named XmlBulkLoad. The XmlBulkLoad class depends on XML views that, as we have seen in this chapter, describe the relationship between an XML document and the schema of the relational database.

The XmlBulkLoad Class Interface

The XmlBulkLoad class has a relatively simple interface. As shown in Tables 7.10 through 7.12, there are a single constructor, eight properties, and two methods (one of which has various overrides). The properties of the XmlBulkLoad class specify the mapping and the rules to use when performing the bulk load, the buffer and transaction sizes, and the path to the error log. The most important method of the XmlBulkLoad class is Execute, which pushes the data into the database. There is also a method for specifying a nondefault buffer size.

Table 7.10. The Constructor for the XmlBulkLoad Class

Constructor

Description

XmlBulkLoad()

Creates a new instance of the XmlBulkLoad class with default values for all the properties.

Table 7.11. The Properties of the XmlBulkload Class

Property

Description

Mapping

Sets or returns a reference to a MappingSchema instance that maps the elements in the XML to the columns in the database. This is an XML view.

CheckConstraints

Sets or returns a Boolean value that specifies whether foreign key and column constraints are checked when inserting data. When True, a constraint violation results in an error. When False (the default), only primary keys and unique constraints are checked. In the Technology Preview release, the table containing the primary key must be defined before any tables that contain a matching foreign key.

ForceTableLock

Sets or returns a Boolean value that specifies how the target tables should be locked. When True, a lock is applied for the duration of the bulk load process. When False (the default), the default system for the database is used. This may mean that a lock is acquired for each row inserted into a table.

KeepIdentity

Sets or returns a Boolean value that specifies whether values for an identity column are applied to the target table. The default is True. When False, values for identity columns are ignored and SQL Server assigns an appropriate value. The new value is automatically propagated to any child rows that have a foreign key relationship based on the identity column.

KeepNulls

Sets or returns a Boolean value that specifies whether empty columns should retain a null value during the bulk load operation. When False (the default), empty columns are set to the default value if one is defined for the column.

ClientBufferSize

Returns an Integer value indicating the number of rows that will be buffered on the client before sending them over to the server for processing. The value for this property is set with the SetBufferingSizes method. The default is 0 (all rows).

ServerTransactionSize

Returns an Integer value indicating the number of rows that will be inserted before a new transaction is started in the database. Transaction size is a multiple of ClientBufferSize. The value for this property is set with the SetBufferingSizes method. The default is 0 (all rows).

ErrorLogFile

Sets or returns a String that is the path and filename for the log of errors and messages. The default is an empty string, in which case no logging takes place.

Table 7.12. The Methods of the XmlBulkLoad Class

Method

Description

Execute(reader, connection)

Executes the bulk load operation using an existing XmlReader instance that points to the XML data to be loaded and a String value that is the connection string for the database. No return value.

Execute(data-stream, connection)

Executes the bulk load operation using an existing Stream instance that points to the XML data to be loaded and a String value that is the connection string for the database. No return value.

Execute (file-name, connection)

Executes the bulk load operation using the contents of a file containing the XML data to be loaded and a String value that is the connection string for the database. No return value.

SetBufferingSizes (buffer-size, transaction-size)

Sets the values of the ClientBufferSize and ServerTransactionSize properties as two Integer values. The ServerTransactionSize value must be a multiple of the ClientBufferSize value. The default values for both are 0 (all rows). No return value.

Using the XmlBulkLoad Class

As an example of using the XmlBulkLoad feature, the code in Listing 7.11 creates a new connection to a database and a new instance of the XmlBulkLoad class. Then it creates an instance of an XmlMapping object from the mapping file nwind.msd and assigns this to the Mapping property of the XmlBulkLoad object (in this example the default values are used for all the other properties).

Listing 7.11 Using the XmlBulkLoad Class
' get connection string
Dim sConnectString As String = "your-connection-string"

' create an XmlBulkLoad instance
Dim oLoad As New XmlBulkLoad()

' create a MappingSchema instance for the XSD mapping schema and
' assign it to the Mapping property - must use full physical path
oLoad.Mapping = New MappingSchema("c:\mappings\nwind.msd")

' create the connection and open it
Dim oConn As New SqlConnection(sConnectString)
oConn.Open()
...

The data to load into the database is in the XML file named bulkload .xml, and the Execute method of the XmlBulkLoad object is called specifying this file and the database connection (see Listing 7.12). After this completes, a new SqlCommand object fetches the data from the table and displays it in an ASP.NET DataGrid control located within the <body> section of the page; then it deletes the new rows to allow the page to be executed again without incurring primary key violations.

Listing 7.12 Executing The Load Process
...
' specify the source XML document path and execute the bulk load
oLoad.Execute(sMappingPath & "bulkload.xml", oConn)

' extract the values from the table to check that it worked
Dim oCmd As New SqlCommand("SELECT TOP 10 CustomerID, CompanyName, " _
                         & "City, Country FROM Customers", oConn)
Dim oReader As SqlDataReader = oCmd.ExecuteReader()

' display data in a DataGrid elsewhere on the page
Grid1.DataSource = oReader
Grid1.DataBind()
oReader.Close()

' delete the rows just added so page can run again
oCmd.CommandText = "DELETE FROM Customers WHERE CustomerID LIKE 'AA%'"
oCmd.ExecuteNonQuery()

oConn.Close() ' remember to close the connection
The Mapping File and XML Data

The mapping file used in this example, nwind.msd, contains the mappings between the XML schema of the source document and the schema of the database. XML view mapping files were discussed in detail earlier in this chapter. A portion of the MSD mapping document we used earlier is repeated in Listing 7.13 for reference in this XmlBulkload example. This shows that the identical XML view can be used for either the XmlAdapter or the XmlBulkload class.

Listing 7.13 The MSD Mapping Document
<m:MappingSchema
     xmlns:m="http://schemas.microsoft.com/data/2002/09/28/mapping">
 <m:DataSources>
  <m:DataSource Name="XML" Direction="Target" Type="Xml">
   <m:Schema Location="nwind.xsd" />
  </m:DataSource>
  <m:DataSource Name="Northwind" Direction="Source"
                Type="SQL Server">
   <m:Schema Location="nwind.rsd" />
   <m:Variable Name="Customers" Select="Customers" />
  </m:DataSource>
 </m:DataSources>
 <m:Mappings>
  <m:Map SourceVariable="Customers" TargetSelect="/Customer">
   <m:FieldMap SourceField="CompanyName" TargetField="@name" />
   <m:FieldMap SourceField="CustomerID" TargetField="@CustomerID" />
   <m:FieldMap SourceField="Country" TargetField="@Country" />
   <m:FieldMap SourceField="City" TargetField="@City" />
  </m:Map>
 </m:Mappings>
</m:MappingSchema>

The other two files for the three-part mapping are specified in the relevant <m:DataSource> sections: nwind.xsd is the XML schema that describes the XML document, and nwind.rsd is the relational schema for the tables in the Northwind database. The nwind.msd file provides the mapping between these two schemas so that the XmlBulkLoad class knows how to translate the XML file into the correct format for the database tables.

The XML data file, named bulkload.xml, contains the new data that will be loaded into the database and conforms to this XML view. The content of this file is shown in Listing 7.14.

Listing 7.14 The XML Data File for the Bulk Load
<?xml version="1.0" ?>
<Customers>
  <Customer name="New Customer 1" CustomerID="AAAA1"
                  Country="England" City="London" />
  <Customer name="New Customer 2" CustomerID="AAAA2"
                  Country="Australia" City="Perth" />
  <Customer name="New Customer 3" CustomerID="AAAA3"
                  Country="USA" City="Chicago" />
  <Customer name="New Customer 4" CustomerID="AAAA4"
                  Country="Poland" City="Warsaw" />
  <Customer name="New Customer 5" CustomerID="AAAA5"
                  Country="Japan" City="Tokyo" />
</Customers>

The screenshot in Figure 7.5 shows the result of executing the code. You can see the five new rows that are inserted by the Execute method of the XmlBulkLoad object.

Figure 7.5. Using the XmlBulkLoad class to populate a database table

The DBObject Class

The DBObject class provides cursor-like access into SQL Server, selecting a single row and then a single column within that row using XPath-like expressions. The value of the column can then be exposed through a DataReader and updated through the DBObject.

The DBObject Path Syntax

To select a row and column, the DBObject class exposes a property named Path. The value of this property is set to a String that contains the XPath-like expression that specifies the required row and column. The general syntax is:

"dbobject/table-name[row-specifier]/@column-name"

The table-name is the name of the table in the database. It cannot include any spaces or [ or / characters. The column-name is the name of the column within the table. It cannot include any space characters. The row-specifier selects the row within the table. The format is:

@column-name=$value

The column-name is the name of a column within the table and $value is the value in the target row. The common scenario is to use a primary key column. A value is passed into the XPath to replace the $value placeholder using an XmlQueryArgumentList instance populated with the parameter value. Multiple specifiers can be included to select a row based on the values in more than one column by concatenating them with AND, for example:

@columnA-name=$value1 AND @columnB-name=$value2

As usual, if a string value contains a single quote, the string must be delimited with double quotes, or the single quotes within the value must be escaped. Likewise, if a string value contains a double quote, the string must be delimited with single quotes, or the double quotes within the value must be escaped.

Here's an example Path value that selects the Address column from the row in the Customers table that has the value passed in by an XmlQuery ArgumentList for the CustomerID column:

"dbobject/Customers[@CustomerID=$ThisCustID]/@Address"
The DBObject Class Interface

The DBObject interface is remarkably simple, containing a single constructor, a single property, and two methods (see Tables 7.13 through 7.15). Each method provides four overrides.

Table 7.13. The Constructor for the DBObject Class

Constructor

Description

DBObject()

Creates a new instance of the DBObject class, with the Path property set to an empty string.

Table 7.14. The Property of the DBObject Class

Property

Description

Path

Sets or returns a String value that is the XPath to use to select the row and column. See the subsection on DBObject path syntax for more details.

Table 7.15. The Methods of the DBObject Class

Method

Description

Retrieve(connection)

Generates a SELECT statement internally based on the current value of the Path property, and executes it against the database specified in the connection string. Returns a DataReader instance that exposes the column contents.

Retrieve(argument-list, connection)

As above, but also accepts an XmlQueryArgument List instance that contains the values to apply as parameters, as indicated by $ placeholders within the Path property value.

Retrieve(stream, connection)

Generates a SELECT statement internally based on the current value of the Path property, and executes it against the database specified in the connection string. The result is sent directly to the specified Stream instance. No return value.

Retrieve(stream, argument-list, connection)

As above, but also accepts an XmlQueryArgument List instance that contains the values to apply as parameters, as indicated by $ placeholders within the Path property value.

Update(bytes-array, connection)

Generates an UPDATE statement internally based on the current value of the Path property, and executes it against the database specified in the connection string. The value specified as an array of Byte types is inserted into the column specified in the Path property. No return value.

Update(bytes-array, argument-list, connection)

As above, but also accepts an XmlQueryArgument List instance that contains the values to apply as parameters, as indicated by $ placeholders within the Path property value. No return value.

Update(stream, connection)

Generates an UPDATE statement internally based on the current value of the Path property, and executes it against the database specified in the connection string. The value taken from the specified Stream is inserted into the column specified in the Path property. No return value.

Update(stream, argument-list, connection)

As above, but also accepts an XmlQueryArgument List instance that contains the values to apply as parameters, as indicated by $ placeholders within the Path property value. No return value.

The XmlQueryArgumentList Class Interface

In the description of the DBObject class, you saw how an XmlQueryArgument List instance is used to pass values into the XPath expression assigned to the Path property of the DBObject. The main use of the XmlQuery ArgumentList class is to pass arguments into an XQuery process or an XSLT stylesheet. The commonly used methods it exposes are shown in Table 7.16.

Table 7.16. The Methods of the XmlQueryArgumentList Class

Method

Description

AddParam(local-name, namespace-uri, value)

Adds a parameter with the specified local-name (String), namespace-uri (String), and value (Object) to the list. No return value.

GetParam(local-name, namespace-uri)

Returns the value of the parameter with the specified local-name (String) and namespace-uri (String) as an Object.

RemoveParam(local-name, namespace-uri)

Removes the parameter with the specified local-name (String) and namespace-uri (String) from the list. No return value.

Using the DBObject Class

As an example of the DBObject at work, Listing 7.15 shows code that extracts the value of the CompanyName column from the Customers table in the sample Northwind database for the row where the CompanyID primary key value is ANTON. The code creates and opens a connection to the database, then creates a new instance of the DBObject class and specifies the XPath-like expression that selects the row and column.

Listing 7.15 Using the DBObject Class
' get connection string
Dim sConnectString As String = "your-connection-string"

' create connection and open it
Dim oConn As New SqlConnection(sConnectString)
oConn.Open()

' create DBObject instance and set Path property
Dim oDBObject As New DBObject()
oDBObject.Path _
    = "dbobject/Customers[@CustomerID=$CustomerID]/@CompanyName"
...

The Path value assigned to the DBObject instance contains a parameter named CustomerID (prefixed with the $ character), so an XmlQuery ArgumentList is required to pass that parameter to the DBObject at runtime. Listing 7.16 shows how a new XmlQueryArgumentList is created, and a parameter named CustomerID (with the value ANTON and an empty string for the namespace) is added to it. Then the code calls the Retrieve method to get back a SqlDataReader instance pointing to the value in that column. To display the value, the SqlDataReader is bound to an ASP.NET DataGrid control elsewhere on the page.

Listing 7.16 Retrieving a Column Value
...
' create XmlQueryArgumentList and add parameter
Dim oArgs As New XmlQueryArgumentList()
oArgs.AddParam("CustomerID", "", "ANTON")

' retrieve the value from the database
Dim oReader As SqlDataReader = oDBObject.Retrieve(oArgs, oConn)

' display data in a DataGrid elsewhere on the page
Grid1.DataSource = oReader
Grid1.DataBind()

' remember to close the connection
oConn.Close()

The screenshot in Figure 7.6 shows the result.

Figure 7.6. Retrieving a value with an XPath and a DBObject

Updating a Column Value with a DBObject

The code from the previous example in Listing 7.16 can easily be modified to demonstrate how the DBObject can be used to push a value into the column of a database table row. Much of the code is the same as before (see Listing 7.17), but this time it reads a value from a text file on disk as a Stream and inserts this into the specified column before reading it back and displaying the value in the page.

The highlighted code in Listing 7.17 shows the FileStream being created from the text file. This, along with an XmlQueryArgumentList containing the CustomerID parameter and the database connection, is passed to the Update method of the DBObject.

Listing 7.17 Updating a Column Value with a DBObject
' create and open connection as before
...

' create DBObject instance and set Path property
Dim oDBObject As New DBObject()
oDBObject.Path _
    = "dbobject/Customers[@CustomerID=$CustomerID]/@CompanyName"

' create XmlQueryArgumentList and add parameter
Dim oArgs As New XmlQueryArgumentList()
oArgs.AddParam("CustomerID", "", "ALFKI")

' open the data to use from a file as a Stream
Dim oFileInfo As New FileInfo("c:\data\DBObjectNewData.txt")
Dim oStream As FileStream = oFileInfo.OpenRead()

oDBObject.Update(oStream, oArgs, oConn)

' close file stream
oStream.Close()

' code here to retrieve the column value using the same technique
' as in the previous example, and display it in a DataGrid

oConn.Close()  ' remember to close the connection

The code uses exactly the same techniques as the previous example to extract the value again to prove that it worked. You can edit the text file and run this page again to see that it is updated. Note that the file must be in Unicode format; you can select this format when you edit the file in Windows Notepad. The screenshot in Figure 7.7 shows the result when you enter the text "This is the new company name" into the file DBObjectNewData.txt.

Figure 7.7. Updating a value with an XPath and a DBObject

The XmlRowsetAdapter Class

The XmlBulkLoad class loads data into SQL Server through a normal Connection object. But there are times when you want to read complex XML data into an object that can persist it when disconnected but exposes the data in a traditional relational format.

The obvious choice is a DataSet, and you can load XML into a DataSet using its own native methods, such as ReadXml and ReadXmlSchema. However, these methods are not optimally efficient and usually cannot cope with XML that does not follow the specific format required. Instead, a new class called XmlRowsetAdapter has been added in version 2.0 of the Framework. This class is designed to provide a bridge between XML documents in almost any format, including those containing complex hierarchical data, and the DataSet.

In effect, the XmlRowsetAdapter provides a kind of "bulk load" function for use with a DataSet, taking advantage of the new three-part mapping technology. It can "shred" complex XML into multiple tables in a DataSet, in much the same way as the XmlBulkload object does with SQL Server.

The XmlRowsetAdapter Class Interface

The interface of the XmlRowsetAdapter is refreshingly simple. It exposes just one constructor, one property, and two methods (though there are three overloads for one of the methods). All of these interface members are listed in Tables 7.17 through 7.19.

Table 7.17. The Constructor for the XmlRowsetAdapter Class

Constructor

Description

XmlRowsetAdapter()

Creates a new instance of an XmlRowsetAdapter.

Table 7.18. The Property of the XmlRowsetAdapter Class

Property

Description

Mapping

Sets or returns a reference to a MappingSchema instance that maps the elements in the XML to the columns in the DataSet tables.

Table 7.19. The Methods of the XmlRowsetAdapter Class

Method

Description

FillSchema(dataset)

Takes a reference to a DataSet, and adds tables to it according to the schema in the mapping file. Does not load any data, and can be used to create an empty DataSet ready to accept data. No return value.

Fill(dataset, filename)

Fills the specified DataSet with data from the XML document specified by filename (as a String). The XML data is "shredded" according to the schema and mapping file definitions, then loaded into the relevant tables. The Fill method only creates tables for which data is present in the source XML document, even if the mapping defines other tables. To create all the tables defined in the mapping, call the FillSchema method first. No return value.

Fill(dataset, xmlreader)

As above, but takes the XML data from an XmlReader instance.

Fill(dataset, filename, resolver)

As above, but loads the XML through the specified XmlResolver instance.

Using the XmlRowsetAdapter Class

To demonstrate the XmlRowsetAdapter in action, the example in Listing 7.18 uses the same nwind.xsd three-part mapping between XML and the Northwind database. This is used to create the database structure in the DataSet. The code then loads into the DataSet the same XML document used in the XmlBulkLoad example earlier in this chapter.

So, we start by creating a new empty DataSet and a new instance of the XmlRowsetAdapter class. The mapping file named nwind.msd is used to create a new MappingSchema instance, and this is assigned to the Mapping property of the XmlRowsetAdapter.

Listing 7.18 Using an XmlRowsetAdapter to Load a Table in a DataSet
' create empty DataSet and new XmlRowsetAdapter
Dim oDS As New Dataset()
Dim oXRA As New XmlRowsetAdapter()

' set Mapping of XmlRowsetAdapter to three-part mapping file
oXRA.Mapping = New MappingSchema("c:\mapping\nwind.msd")
...

The FillSchema method of the XmlRowsetAdapter is called next (see Listing 7.19). It uses the relational schema (nwind.rsd) that is referenced in the mapping file to create the database structure within the DataSet. Then the Fill method is called to load the data from the XML file named bulkload.xml into the DataSet. Afterward, the contents of the DataSet are displayed in two ASP.NET DataGrid controls located in the <body> section of the page.

Listing 7.19 Filling the Schema and Loading the Data
...
' load schema for XML into DataSet
' adapter knows where to find schema from mapping information
oXRA.FillSchema(oDS)

' load XML data into DataSet
oXRA.Fill(oDS, "c:\data\bulkload.xml")

' display list of tables in DataSet in first DataGrid
Grid1.DataSource = oDS.Tables
Grid1.DataBind()

' display data loaded into Customers table in second DataGrid
Grid2.DataSource = oDS
Grid2.DataMember = "[northwind].[dbo].[Customers]"
Grid2.DataBind()

The screenshot in Figure 7.8 on the next page confirms that both the database structure and the XML data were loaded into the DataSet. The first DataGrid is bound to the Tables collection of the DataSet, and you can see that it contains the same set of tables as found in the Northwind database. Notice that they have fully qualified names, which include the database name and owner, as well as the "local" table name.

Figure 7.8. Filling a DataSet with an XmlRowsetAdapter

This also illustrates why the second DataGrid is bound to the DataMember named [northwind].[dbo].[Customers], rather than just Customers. But it obviously works because you can see the five rows that were inserted into the table from the XML data file.


XML Data Source Controls and Data Binding

ASP.NET developers, and Web developers in general, have used XML in many ways to generate output in a Web page and have been well supplied with tools and components to help with this task. For example, in ASP.NET 1.x, you can use the Xml server control to display an XML document and can optionally apply an XSL or XSLT transformation to it first.

However, ASP.NET is changing the approach to connecting to data sources with the advent of a series of data source controls. The three that apply to XML are listed below.

  1. The XmlDataSource control can expose hierarchical XML documents and XML data for data binding to controls such as a TreeView, as well as exposing the data as an XmlDataDocument.

  2. The DataSetDataSource control exposes nonhierarchical XML as a rowset for data binding, as well as exposing the data as an ADO.NET DataSet and an XmlDataDocument.

  3. The WebServiceDataSource control allows you to work with data exposed from a Web Service.

We'll look briefly at these three controls in the next subsections. For more details on data source controls and their use in data binding with ASP.NET, refer to the companion book to this one, A First Look at ASP.NET v. 2.0 (Boston, MA: Addison-Wesley, 2004, ISBN 0-321-22896-0).

The XmlDataSource Control

To give you some idea of what the data source controls do, the following example uses an XmlDataSource control to load XML data and expose it for data binding. The XML is a list of "extra" packages for cars and has the format shown in Listing 7.20 (abbreviated to save space).

Listing 7.20 An XML List of "Extra" Packages for Well-Known Cars
<Automobiles>
  <Manufacturer Make="Audi">
    <Car Model="A4" Id="02347">
      <Package Trim="Sport"/>
      <Package Trim="Luxury"/>
      <Package Trim="Cold Weather"/>
    </Car>
    <Car Model="A6" Id="02932">
      ... packages ...
    </Car>
    <Car Model="A8"  Id="09381">
      ... packages ...
    </Car>
  </Manufacturer>
  <Manufacturer Make="BMW">
      ... cars and packages ...
  </Manufacturer>
  <Manufacturer Make="Porsche">
      ... cars and packages ...
  </Manufacturer>
</Automobiles>

A TreeView control is designed to display hierarchical data like this XML document and can be used to display it in an ASP.NET page without requiring any code to be written. An XmlDataSource control specifies the path to the XML document. The TreeView control is bound to the XmlDataSource control by specifying its id property value as the DataSourceID property of the TreeView control (see Listing 7.21).

Listing 7.21 Declaring a TreeView Control to Display the XML
<form runat="server">

<asp:XmlDataSource id="ds1" runat="server"
     DataFile="XmlDataSource.xml" />

<asp:TreeView id="tv1" DataSourceID="ds1" runat="server">
  <Bindings>
    <asp:TreeNodeBinding DataMember="Manufacturer" TextField="Make" />
    <asp:TreeNodeBinding DataMember="Car" TextField="Model" />
    <asp:TreeNodeBinding DataMember="Package" TextField="Trim" />
  </Bindings>
</asp:TreeView>

</form>

The XML document contains nested repeated elements, so the bindings must be declared or all that will be displayed are the element names. Within the TreeView element declaration are three bindings that specify that the control should bind to the Manufacturer element and expose the value of the Make attribute as the text of the nodes, then bind to the Car elements and expose the Model attribute of each one as the text of the nodes, and finally bind to the Package elements and display the value of the Trim attribute as the node values. The result can be seen in Figure 7.9.

Figure 7.9. Binding XML data to an ASP.NET TreeView control

Rather than specifying the XML as a file path and name for the DataFile property, the XML can be declared in-line in a <Data> element within the <asp:XmlDataSource> element. And you can also specify the schema and stylesheet as attributes in the declaration of the control or assign the values at runtime.

As well as all these features, the control supports caching to improve performance and can be used to update the XML disk file if required. Also note that an XPath can be specified to filter or select which of the elements in the XML source will be exposed. The outline declaration of the XmlDataSource control is shown in Listing 7.22.

Listing 7.22 The ASP.NET XmlDataSource Control Outline Declaration
<asp:XmlDataSource id="String" runat="server"
  DataFile="String"
  ReadOnly="[True|False]"
  SchemaFile="String"
  TransformFile="String"
  AutoSave="[True|False]"
  XPath="String"
  UniqueID="String"
  EnableCaching="[True|False]"
  CacheDuration="Integer"
  CacheExpirationPolicy="[Absolute|Sliding]"
  OnDataSourceChanged="EventHandler"
  OnTransforming="EventHandler" >

    <Data>
      [...In-line XML Data...]
    </Data>
    <Schema>
      [...In-line XML Schema...]
    </Schema>
    <Transform>
      [...In-line XSL/XSLT Transform...]
    </Transform>

</asp:XmlDataSource>

The DataSetDataSource Control

The DataSetDataSource control is similar to the XmlDataSource control but works only with nonhierarchical XML data, that is, data that contains only one "level" of elements within the root element. The root element is exposed as the "table" name, and the nested elements as the columns of this "table," as in Listing 7.23.

Listing 7.23 Nonhierarchical XML Data for a DataSetDataSource Control
<Root>
  <Column>value</Column>
  <Column>value</Column>
  <Column>value</Column>
  ...
</Root>

The DataSetDataSource control does not provide the XPath and UniqueID attributes. However, it has an extra method named GetDataSet that exposes the XML source data as a standard ADO.NET DataSet, allowing for runtime manipulation of the content in code.

The WebServiceDataSource Control

The WebServiceDataSource control can expose hierarchical data and is designed to interact with a Web Service and expose the XML content it returns. Three properties specific to this control specify how it connects to the Web Service: URL (a required String value), TypeName (an optional String value), and Timeout (an optional Integer value).

For more details, see the companion book mentioned earlier.


SUMMARY

In this chapter we looked at the XmlAdapter and XML views that map relational tables from SQL Server to an XML schema and provide bidirectional queries and updates. XML views provide several advantages, including removing the need for custom mapping code.

We explored how XML views are a type of XML provider, and are implemented via a declarative mapping format. This mapping, known as three-part mapping, consists of three format types: XSD for the XML schema, RSD for the relational schema, and MSD for the mapping schema that maps between the relational and XML domains. These XML views are combined with XQuery commands to allow queries to be issued over different data sources such as (but not limited to) DBMSs like SQL Server. We also saw how the XmlViewSchema class represents a compiled XML view that can be used with an XmlCommand to return results as XML.

Next we further discussed the XmlAdapter class, which is an addition to the ADO.NET DataAdapter programming model—but for XML data rather than relational data. This extremely powerful class enables you to pull relational data from SQL Server, shape it into any XML structure via an XML view, and then fill an XPathDocument2 class that can subsequently be queried and edited. Most importantly, the changes to the XPathDocument2 can then be "pushed back" or updated to SQL Server via the XML view, which automatically generates all the necessary SQL statements to perform the inserts, updates, and deletes. This provides the extraordinary ability to transform relational data into XML; take advantage of the rich, expressive, and hierarchical data model; utilize other XML technologies such as XSLT and XML Web Services; and then synchronize the changes back to the SQL Server DBMS.

We then looked at the System.Data.SqlXml namespaces and at the set of classes that are the .NET Framework equivalents of the existing SQLXML 3.0 technology. In particular we looked at the XmlBulkLoad class, which has the ability to load data quickly and easily from an XML document into SQL Server through an XML view. In effect, this allows streaming updates of XML into SQL Server, whereas the other class we examined, the XmlAdapter, provides in-memory document updates. Both of these components use XML views to perform the updates.

We discussed the DBObject class, which provides cursor-like access into SQL Server, selecting a single row and then a single column within that row using simple XPath-like expressions. We also investigated the XmlRowsetAdapter, which provides a kind of "bulk load" function for use with a DataSet, taking advantage of the XML views technology. It can transform XML into multiple tables in a DataSet, via an XML view, in the same way as the XmlBulkload class updates SQL Server.

Finally we touched on the ASP.NET XmlDataSource control and other data source controls that can load XML data and expose it for data binding. This provides the ability to easily bind parts of an XML document to a Web application to display hierarchical data on a form such as a tree control or menu.

In the next and final chapter, we'll end with a look at XQuery and other XML features that integrate with SQL Server.


Chapter 8. XQuery and SQL Server

The previous two chapters discussed how the XPathDocument2 and XmlAdapter classes can be used for retrieving and updating data as XML from SQL Server, via XML Views that use XQuery as the XML query language. In this chapter, we'll look in more detail at the XQuery language, the XQueryProcessor class, and how they can be used to perform queries over XML documents loaded into an XPathDocument2 and over SQL Server using XML views.

We'll see how XQuery can be used as a data aggregation language for retrieving data from numerous different data sources, such as different SQL Server databases or a combination of SQL Server and XML documents on disk files. We will then discuss and demonstrate the XsltProcessor class used for performing XSLT transformations. It shares a Common Query Architecture with the XQueryProcessor class. Finally, this chapter looks at the XML data type introduced into SQL Server "Yukon," which enables it to store XML as a native type.

So, the topics for this chapter are:

  • A very brief overview of the XQuery language

  • Using the XQueryProcessor class

  • Using the XsltProcessor class

  • The Common Query Architecture

  • Using SQL Server as an XML database

We'll start with a look at what is now the primary way to query XML documents, the XQuery language.


A Very Brief Overview of the XQuery Language

Before diving into an examination of the classes in the .NET Framework, it's necessary that you have an understanding of the XQuery language so you can construct queries against data sources. Covering the full syntax of the XQuery language is well beyond the scope of this book, but we will discuss the minimal essentials here for the queries used in this chapter. Look for the book XQuery by Michael Brundage (Boston, MA: Addison-Wesley, ISBN 0-321-16581-0), due to be published in 2004, for a complete description of the XQuery language.

However, to start you down the road, XQuery is based on XPath 1.0. So if you know XPath, you are well on the way to understanding XQuery. Below we will outline the essential additions beyond XPath for the queries used in this book.

Where's the Data Located?

In all queries, the XML document itself has to be referenced in some way. This is referred to as the document context. There are three ways to achieve this in the XQuery specification, plus a fourth provided by the use of XML Views in the .NET Framework. These are listed below.

  1. Use the input() function, which provides an implicit XML document. For example, the query input()/Customers/Customer can be executed such that the input document can vary, but all the Customer elements are still found within the specified document.

  2. Use the document() function, which references an XML document via a URL. For example, document("CustomerOrders.xml")/Customers/Customer loads the CustomerOrders.xml document and queries for all the Customer elements.

  3. Use the map:view() extension function (see Chapter 7), which allows you to reference an XML view over a SQL Server database from the middle tier. For example, the following query finds all the Customer elements via the CustomerView XML view:

    declare namespace map='http://schemas.microsoft.com/xml /2002/09/28/mappingfunctions'
<root>{map:view('CustomerView')/Customers/Customer }</root>

    Since the map:view() function is an extension that is not part of the XQuery language, you must provide the namespace to indicate this. For the map:view() function, this is http://schemas.microsoft.com/xml/2002/09/28/mappingfunctions.

  4. Use the collection() function. This is not supported in the Technology Preview release but provides a named collection of input documents to query over in a manner similar to the input() function.

FLWOR Expressions

The acronym FLWOR, pronounced "flower," is created from the keywords used in an XQuery expression—for, let, where, order by, and return—which support iteration and binding of variables to results. FLWORs are probably the most common expression construct and can be considered to be the SQL Select * from table where... equivalent in XQuery. The for keyword provides iteration, and the let keyword provides variable binding. Here is an example:

<Customers>
{
  for $i in document('customers.xml')/Customers/Customer
  where $i[@City='London']
  <CustomerName>{return $i/@name}</CustomerName>
}
</Customers>

The first thing to notice is $i, which declares a variable called i. The document function and the path expression /Customers/Customer retrieve all the Customer elements from the customers.xml document, and the for expression starts an iterative loop over the values, which are individually bound to the $i variable. The where clause applies a filter, or predicate, to match only Customer elements whose City attribute has the text value of London.

Finally, we construct a <CustomerName> element and insert into it the value of the name attribute from the current Customer element using the return statement. The ability to construct XML on the output is also shown by the creation of an enclosing <Customers> element in the output. The end result may look something like this:

<Customers>
  <CustomerName>Alex Homer</CustomerName>
  <CustomerName>Mark Fussell</CustomerName>
  <CustomerName>David Sussman</CustomerName>
</Customers>

Another point to note in the query above is the use of curly braces ({}), which indicate this is a query statement that needs to be executed, not a literal piece of XML that is written to the output (e.g., within the constructed <CustomerName> element). Without the curly braces, the XQuery processor would treat this as element content rather than an XQuery expression that needs to be executed, that is:

<CustomerName>return $i/@name</CustomerName>

If some of this syntax seems strange at first, it is worth spending some time working with the different examples to achieve familiarity. Then you will start to gain an understanding of how powerful this query language is and the extensive capabilities you can take advantage of when querying and manipulating XML. Reading the W3C XML Query Use Cases specification also helps clarify the syntax with examples.

To learn more about the XQuery syntax, functions, and operators, check out the following W3C specifications:

  • XQuery 1.0: An XML Query Language— http://www.w3.org/TR/xquery/

  • XQuery 1.0 and XPath 2.0 Functions and Operators— http://www.w3.org/TR/xpath-functions/

  • XQuery 1.0 and XPath 2.0 Data Model— http://www.w3.org/TR/xpath-datamodel/

  • XML Query Use Cases— http://www.w3.org/TR/xquery-use-cases/


The XQueryProcessor Class

The XQueryProcessor class provides a compiler and execution engine for the W3C XQuery language. The .NET Framework documentation has a list of the language features, functions, and operators that are supported for the current release.

The Properties and Methods of the XQueryProcessor Class

Before we look at how we can use the XQueryProcessor, this section contains a concise reference to the properties and methods of this class. There are a single constructor, three properties, and four methods, as documented in Tables 8.1 through 8.3.

Table 8.1. The Constructor for the XQueryProcessor Class

Constructor

Description

XQueryProcessor()

Creates a new empty XQueryProcessor instance.

Table 8.2. The Properties of the XQueryProcessor Class

Property

Description

XmlCommand

A reference to an XmlCommand class instance representing a compiled query that can be executed more than once.

XmlViewSchemaDictionary

A reference to an XmlViewSchemaDictionary instance that is used when XML views are needed during the compilation of the query via the map:view extension function, for example:

[View full width]
Dim xqp As New XQueryProcessor()
xqp.XmlViewSchemaDictionary.Add("Northwind",
   "nwind.xml")
xqp.Compile("map:view('Northwind')//Customers")

SqlQueryOptions

A reference to a SqlQueryOptions instance, which itself exposes a value from the SqlQueryPlan enumeration that specifies the way generation of SQL will take place when queries are performed over XML views. The SqlQueryPlan enumeration values are Mars, Serialized, and UnionAll. By default, the XQueryProcessor will generate Serialized query plans in order to be as compatible as possible between SQL Server 2000 and SQL Server "Yukon." If only working against SQL Server "Yukon," Mars is the option that performs best. If the SqlQueryPlan is changed after the Compile method has been called on the XQueryProcessor, the change is not reflected at execution time. The query must be recompiled in order to execute with the new settings.

Table 8.3. The Methods of the XQueryProcessor Class

Method

Description

Compile(query)

Compiles the specified query into an XmlCommand. The XQueryProcessor then becomes a wrapper for the XmlCommand, which can be continuously executed.

CompileView(query, mapping-location)

Compiles the specified query and the XML view pointed to by the mapping-location parameter. This file helper method is used to query a single database with a single XML view and is used in conjunction with the ExecuteView method. This saves having to create and populate an XmlSchemaDictionary class, unless multiple XML views or database connections are required. For example:

[View full width]
Dim xqp As New XQueryProcessor()
xqp.CompileView("for $i in /Customers/Customer
   return $i@City", "northwind.msd")

Execute(datasources, results)

Executes the compiled XQuery against one or more datasources, represented by an XmlResolver, and outputs the results as a TextWriter provided as the results parameter. Execute is called to run the query after the Compile method has completed.

Execute (context-document-uri, datasources, results)

As above, but if the query uses the context document (i.e., '/' or the XQuery input() function), you can pass the context document via the context -document-uri parameter using this overload of the method.

ExecuteView (connection,results)

Executes the XQuery. This method is called after the CompileView method has been used to compile the XQuery. You pass the connection parameter that you want to execute the query against, and the results are returned via the results parameter as a TextWriter. For example:

[View full width]
Dim xqp As New XQueryProcessor()
xqp.CompileView("for $i in /Customers/Customer
   return $i@City", "northwind.msd")
myConnection.Open()
xqp.ExecuteView(myConnection, Console.Out)
myConnection.Close()

The XmlCommand Class

The XmlCommand class represents an XML query that can be executed at runtime, either for XSLT or XQuery, and is used as a property of the XQueryProcessor and XsltProcessor classes. You provide the Xml Command with the data sources and runtime parameters necessary to execute the query. It can then be used to execute the same query multiple times over any data source. The single method is Execute, described in Table 8.4.

Table 8.4. The Execute Method of the XmlCommand Class

Method

Description

Execute(context-document-uri, datasources, results)

Executes the compiled XQuery against one or more datasources, which are represented by an XmlResolver, and outputs the results to the TextWriter specified as the results parameter. Execute is called to run the query after the Compile method has completed. If the query uses the context document (i.e., '/' or the XQuery input() function), you can pass the context document via the context-document-uri parameter.

Querying an XML Document with the XQueryProcessor Class

The example in Listing 8.1 uses the bookstore XML document introduced in Chapter 6 and queries for all the books using the XQueryProcessor class. First an XmlDataSourceResolver is created to resolve the document() function references in the query, in this case for the books.xml document. Here the XQuery is supplied as a string. A StreamWriter is then created for the output. The query is compiled and then executed over the data sources to produce the result.

Listing 8.1 Querying an XML Document with an XQueryProcessor
' initialize the XmlDataSourceResolver
Dim datasource As New XmlDataSourceResolver()
datasource.Add("bookstore", "books.xml")

Dim writer As New StreamWriter("output.xml")

' initialize the XQueryProcessor
Dim query As String = "<bookstore>" _
  & "{ for $b in document('bookstore')/bookstore/book" _
  & "where $b/@genre='philosophy' and $b/@publicationdate='1991'" _
  & "return $b/title }" _
  & "</bookstore>"
Dim xq As New XQueryProcessor()

' compile query and execute against the books.xml document
xq.Compile(query)
xq.Execute(datasource, writer)

writer.Close()

The XQuery used in this example, shown in Listing 8.2, finds all the books with a value of philosophy for the genre attribute and a publicationdate attribute value of 1991.

Listing 8.2 The XQuery Expression for Listing 8.1
<bookstore>
{ for $b in document('bookstore')/bookstore/book
where $b/@genre='philosophy' and $b/@publicationdate='1991'
return $b/title }
</bookstore>

In the XQuery, the document function is used to specify the context document against which the query will be executed, in this case, bookstore. The screenshot in Figure 8.1 shows the results. The query returns the book titles, though in this case there is only a single title that matches.

Figure 8.1. Using the XQueryProcessor to query an XML document

Querying Multiple XML Documents with the XQueryProcessor Class

The XPathDocument2 class is an XML store designed to work efficiently with XQuery and XSLT. The next example, shown in Listing 8.3, queries over two bookstore documents. This example is identical to the previous one, except that in this case the books2.xml document is loaded into an XPathDocument2 instance and then added to the XmlDataSourceResolver. This kind of situation may arise when you want to edit the document with the XPathEditor before performing XQuery operations over it.

It is also worth noting that the XmlResolver must use a single Xml NameTable for all of its resources. This is generic behavior for all situations that use the XmlResolver class, not just the XmlDataSourceResolver. If the XmlDataSourceResolver is used to reference multiple documents, each document must share the same XmlNameTable (this is why the XPathDocument2 created in Listing 8.3 uses the XmlNameTable from the XmlData SourceResolver). Without a single XmlNameTable shared across documents, comparisons between element and attribute names are not possible without causing severe performance impacts on the query.

Listing 8.3 Querying across Two XML Documents
' initialize the XmlDataSourceResolver
Dim datasource As New XmlDataSourceResolver()
datasource.Add("bookstore", "books.xml")

Dim doc As New XPathDocument2(datasource.NameTable)
doc.Load("books2.xml")
datasource.Add("bookstore2", doc.CreateXPathNavigator2())

Dim writer As New StreamWriter("output.xml")

' initialize the XQueryProcessor
Dim query As String = "<bookstore>" _
  & "{ for $b in document('bookstore')/bookstore/book" _
  & "where $b/@genre='philosophy' and $b/@publicationdate='1991'" _
  & "return $b/title }" _
  & "{ for $b2 in document('bookstore2')/bookstore/book" _
  & where $b2/@genre='philosophy' and $b2/@publicationdate='2002'" _
  & "return $b2/title }" _
  & "</bookstore>"
Dim xq As New XQueryProcessor()

' compile query and execute against documents
xq.Compile(query)
xq.Execute(datasource, writer)

writer.Close()

The XQuery used in this example, shown in Listing 8.4, finds all the books with a value of philosophy for the genre attribute and a publicationdate attribute value of 1991 from the first XML document (bookstore.xml), plus all books with a value of philosophy for the genre attribute and a publicationdate attribute value of 2002 from the second XML document (bookstore2.xml), and returns their title attribute values.

Listing 8.4 The XQuery Expression for Listing 8.3
<bookstore>
{ for $b in document('bookstore')/bookstore/book
where $b/@genre='philosophy' and $b/@publicationdate='1991'
return $b/title }
{ for $b2 in document('bookstore2')/bookstore/book
where $b2/@genre='philosophy' and $b2/@publicationdate='2002'
return $b2/title }
</bookstore>

The screenshot in Figure 8.2 shows the results of executing this query.

Figure 8.2. Using the XQueryProcessor to query multiple XML documents

Using an XQueryProcessor with an XML View

XML views were discussed in the previous chapters. We used them to retrieve data from SQL Server and to push updates back into SQL Server via the XmlAdapter class. The next example, in Listing 8.5, uses the map:view() extension function to query SQL Server via an XML view.

This example is similar to querying over an XML document, except that the data source added to the XmlDataSourceResolver is a SqlConnection class rather than the XPathNavigator2 or XPathDocument2 we used for XML documents. For the XML view, a mapping file referenced with the name nwind is added to the XmlViewSchemaDictionary and then referenced by the map:view() function to execute the XQuery.

Remember that the XmlViewSchemaDictionary is a dictionary of Xml ViewSchema objects, where the Add methods provide the ability to add new XML views that can be used to issue queries and updates to a data source such as SQL Server. The Add method creates new XmlViewSchema instances. You can pass MSD file references to the Add method, whereupon it automatically creates a new XmlViewSchema object, compiles the mapping, and adds this to the dictionary. Also recall that the XmlViewSchema class represents a compiled XML view.

Listing 8.5 Using the XQueryProcessor Class with an XML View
Dim myConn As New SqlConnection("your-connection-string")
myConn.Open()

' create an XQueryProcessor to compile XQuery statement
Dim xq As New XQueryProcessor()

' add the XML view
xq.XmlViewSchemaDictionary.Add("nwind", New XmlTextReader("msd-file"))

' add the database connection
Dim datasource As New XmlDataSourceResolver()
datasource.Add("Northwind", myConn)

Dim writer As New StreamWriter("output.xml")

xq.Compile("your-query")
xq.Execute(datasource, writer)

writer.Close()
myConn.Close()

The XQuery, shown below, returns all of the Customers rows from the Northwind sample database, through the nwind XML view. This view is loaded into the XmlViewSchemaDictionary instance using the connection to SQL Server named Northwind:

map:view('nwind')/Customers

We saw in Chapter 7, when using the XmlAdapter class, how the name of the <DataSource> in the MSD file is used to reference this connection by name. The same applies to the XQueryProcessor class, for example:

<m:DataSource Name="Northwind" Direction="Source" Type="SQL Server">

The screenshot in Figure 8.3 shows the results of executing this query.

Figure 8.3. Using an XQueryProcessor to query an XML view

Using an XQueryProcessor with Multiple XML Views

The combination of XQuery and XML Views provides a powerful technique for aggregating data from a number of different views. The next example uses multiple XML views, again using the map:view()extension function to query SQL Server. The data source added to the XmlData SourceResolver is still a SqlConnection reference named Northwind, which is used by all of the XML views. For each XML view, a mapping file is added to the XmlViewSchemaDictionary and referenced by name in the map:view() function when executing the XQuery.

The Three-Part Mapping Files

In this example, we have taken the nwind XML view from Chapter 7 and split it into two separate XML views: one for customers, called customersView, and one for orders, called ordersView. Listings 8.6 through 8.9 show the XML schemas and MSD mappings for these two XML views. The nwind RSD file remains the same because this represents the relational view—which is unchanged.

Listing 8.6 The XML Schema for the customersView
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:element name="Customer">
    <xsd:complexType>
      <xsd:attribute name="name" type="xsd:string"/>
      <xsd:attribute name="CustomerID" type="xsd:string"/>
      <xsd:attribute name="Country" type="xsd:string"/>
      <xsd:attribute name="City" type="xsd:string"/>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>
Listing 8.7 The MSD Mapping File for the customersView
<m:MappingSchema
    xmlns:m="http://schemas.microsoft.com/data/2002/09/28/mapping">
  <m:DataSources>
    <m:DataSource Name="XML" Direction="Target" Type="Xml">
      <m:Schema Location="customers.xsd"/>
    </m:DataSource>
    <m:DataSource Name="Northwind" Direction="Source"
                  Type="SQL Server">
      <m:Schema Location="nwind.rsd"/>
      <m:Variable Name="Customers" Select="Customers"/>
    </m:DataSource>
  </m:DataSources>
  <m:Mappings>
    <m:Map SourceVariable="Customers" TargetSelect="/Customer">
      <m:FieldMap SourceField="CompanyName" TargetField="@name"/>
      <m:FieldMap SourceField="CustomerID"
                  TargetField="@CustomerID"/>
      <m:FieldMap SourceField="Country" TargetField="@Country"/>
      <m:FieldMap SourceField="City" TargetField="@City"/>
    </m:Map>
  </m:Mappings>
</m:MappingSchema>
Listing 8.8 The XML Schema for the ordersView
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:element name="Order">
    <xsd:complexType>
      <xsd:attribute name="CustomerID" type="xsd:string"/>
      <xsd:attribute name="orderid" type="xsd:string"/>
      <xsd:attribute name="OrderDate" type="xsd:string"/>
      <xsd:attribute name="ShipCity" type="xsd:string"/>
      <xsd:attribute name="ShipDate" type="xsd:string"/>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>
Listing 8.9 The MSD Mapping File for the ordersView
<m:MappingSchema
    xmlns:m="http://schemas.microsoft.com/data/2002/09/28/mapping">
  <m:DataSources>
    <m:DataSource Name="XML" Direction="Target" Type="Xml">
      <m:Schema Location="orders.xsd"/>
    </m:DataSource>
    <m:DataSource Name="Northwind" Direction="Source"
                  Type="SQL Server">
      <m:Schema Location="nwind.rsd"/>
      <m:Variable Name="Orders" Select="Orders"/>
    </m:DataSource>
  </m:DataSources>
  <m:Mappings>
    <m:Map SourceVariable="Orders" TargetSelect="/Order">
      <m:FieldMap SourceField="orderid" TargetField="@orderid"/>
      <m:FieldMap SourceField="CustomerID"
                  TargetField="@CustomerID"/>
      <m:FieldMap SourceField="OrderDate"
                  TargetField="@OrderDate"/>
      <m:FieldMap SourceField="ShipCity" TargetField="@ShipCity"/>
      <m:FieldMap SourceField="ShippedDate"
                  TargetField="@ShipDate"/>
    </m:Map>
  </m:Mappings>
</m:MappingSchema>
The Example Code

The code for this example is shown in Listing 8.10. Multiple XML views are added to the XmlViewSchemaDictionary, which all use a single data source (database) connection called Northwind. A StreamWriter is created for output; then the XQuery is compiled and executed, with the output being sent to this stream.

Listing 8.10 Using the XQueryProcessor Class with Multiple XML Views
Dim myConn As New SqlConnection("your-connection-string")
myConn.Open()

' create an XQueryProcessor to compile XQuery statement
Dim xq As New XQueryProcessor()

' add multiple XML views
xq.XmlViewSchemaDictionary.Add("customersView", _
                           New XmlTextReader("customers-msd-file"))
xq.XmlViewSchemaDictionary.Add("ordersView", _
                           New XmlTextReader("orders-msd-file"))

' add the database connection
Dim datasource As New XmlDataSourceResolver()
datasource.Add("Northwind", myConn)

Dim writer As New StreamWriter("output.xml")

xq.Compile("your-query")
xq.Execute(datasource, writer)

writer.Close()
myConn.Close()

The XQuery used in Listing 8.10 is shown in Listing 8.11. It returns all the Customer elements through the customersView XML view and then loads all the Order elements through the ordersView XML view, but only where the CustomerID value of the Order matches the CustomerID value for the Customer in the query predicate.

Finally, a Customer element is constructed with the element keyword. The /@* expression adds all the attributes for this Customer to the new element, and the $orders variable inserts the list of orders for this particular customer as the element content. The element keyword is another syntax for creating elements in XQuery and is identical in behavior to the XML <Customer> syntax.

Listing 8.11 The XQuery Expression for Listing 8.10
<root>
{
for $customer in map:view('customersView')/Customer
let $orders := map:view('ordersView')/Order[@CustomerID
                                 = $customer/@CustomerID]
return element Customer {$customer/@*, $orders}
}
</root>

The screenshot in Figure 8.4 shows the results of executing this query.

Figure 8.4. Using the XQueryProcessor to query multiple XML views

An important aspect to note here is that this XQuery is translated into a SQL statement that retrieves the orders for each customer and creates the results on the server, rather than pulling all the orders back to the client and filtering them for each customer. This means that execution of the XQuery via the XML view performs very well, irrespective of the number or type of XML views in use.

Using an XQueryProcessor with Multiple XML Views and Multiple Database Connections

The previous example showed XQuery execution with multiple XML views over the same connection to the same database. You can also create XML views over different connections, typically to different physical databases that can be on different machines. This shows the ability to aggregate data from a number of different views that are physically separated.

A new XML view, called productsView, is used in the next example. This view is again a subset of the nwind XML view, mapping the Products table from the Northwind sample database. The XML schema and MSD file for the productsView XML view are shown in Listings 8.12 and 8.13.

Listing 8.12 The XML Schema for the productsView
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:element name="Product">
    <xsd:complexType>
      <xsd:attribute name="UnitQuantity" type="xsd:integer"/>
      <xsd:attribute name="ProductName" type="xsd:string"/>
      <xsd:attribute name="UnitPrice" type="xsd:decimal"/>
      <xsd:attribute name="QuantityPerUnit" type="xsd:string"/>
      <xsd:attribute name="ProductID" type="xsd:integer"/>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>
Listing 8.13 The MSD Mapping File for the productsView
<m:MappingSchema
     xmlns:m="http://schemas.microsoft.com/data/2002/09/28/mapping">
  <m:DataSources>
    <m:DataSource Name="XML" Direction="Target" Type="Xml">
      <m:Schema Location="products.xsd"/>
    </m:DataSource>
    <m:DataSource Name="Northwind" Direction="Source"
                  Type="SQL Server">
      <m:Schema Location="nwind.rsd"/>
      <m:Variable Name="Products" Select="Products"/>
    </m:DataSource>
  </m:DataSources>
  <m:Mappings>
    <m:Map SourceVariable="Products" TargetSelect="/Product">
      <m:FieldMap SourceField="productname"
                  TargetField="@ProductName"/>
      <m:FieldMap SourceField="UnitPrice" TargetField="@UnitPrice"/>
      <m:FieldMap SourceField="QuantityPerUnit"
                  TargetField="@QuantityPerUnit"/>
      <m:FieldMap SourceField="ProductID" TargetField="@ProductID"/>
    </m:Map>
  </m:Mappings>
</m:MappingSchema>

The code itself (shown in Listing 8.14) is similar to that in the previous example, except that the XML view named productsView is added to the XmlViewSchemaDictionary as well as the two views we used in that example. And two data sources are added to the XmlDataSourceResolver—the Northwind data source shown in the previous example and a new one called productsDb. The query can then be executed, as shown earlier in Listing 8.10.

Listing 8.14 Using an XQueryProcessor with Multiple XML Views and Multiple Database Connections
...
... create XQueryProcessor, as in Listing 8.10
...

' add multiple XML views
xq.XmlViewSchemaDictionary.Add("customersView", _
                           New XmlTextReader("customers-msd-file"))
xq.XmlViewSchemaDictionary.Add("ordersView", _
                           New XmlTextReader("orders-msd-file"))
xq.XmlViewSchemaDictionary.Add("productsView", _
                           New XmlTextReader("products-msd-file"))

' add multiple database connections
Dim datasource As New XmlDataSourceResolver()
datasource.Add("Northwind", myConn)
datasource.Add("productsDb", productsConn)

' add OrderDetails.xml
datasource.Add("orderDetails", "orderdetails.xml")

...
... execute query, as in Listing 8.10
...

The XQuery used in this example, shown in Listing 8.15, returns all the Product elements through the productsView XML view. Then it loads all of the OrderDetails elements whose ProductID values match the ProductID value of the current Product. For each of these OrderDetails elements, it selects the matching Order, and from each Order it matches the Customer. This list of customers is returned in the $customers variable.

The output of the query is constructed within the outermost return section, consisting of a Product element for each Product found. We use the element keyword and add all the attributes of the Product (using the /@* expression). The content of each Product element is the list of customers for this particular product as referenced by the $customers variable. In effect, we have navigated the database relationships to create a list of products and the customers who purchased them.

Listing 8.15 The XQuery Expression for Listing 8.14
<root>
{
for $product in map:view('productsView')/Product
 return
  element Product
   {
    $product/@*,
    for $orderdetails in
        document('orderDetails')/details/orderdetail[@ProductID
                                        =  $product/@ProductID]
    for $orders in map:view('ordersView')/Order[@orderid
                               = $orderdetails/@OrderID]
    let $customers := map:view('customersView')/Customer[@CustomerID
                                              = $orders/@CustomerID]
    return $customers
   }
}
</root>

The screenshot in Figure 8.5 shows the results of executing this query.

Figure 8.5. Using an XQueryProcessor to query multiple XML views via multiple database connections

Using an XQueryProcessor with Multiple XML Views and XML Documents

You may have noticed that the previous query also used the document function to load the order details, rather than using an XML view. The order details were in fact stored in an XML document. In effect, a query join was performed between the XML document and the XML view to retrieve all the orders for a particular order detail.

This further shows the flexibility of the data aggregation process in that joins and predicates can be made between XML documents and XML views. The relevant code from the previous example is repeated in Listing 8.16, showing an XML file named orderdetails.xml being added to the XmlDataSourceResolver in addition to the XML views.

Listing 8.16 Adding XML Disk Files and XML Views to an XmlDataSourceResolver
Dim datasource As New XmlDataSourceResolver()
datasource.Add("Northwind", myConn)
datasource.Add("productsDb", productsConn)

' Add orderdetails.xml
datasource.Add("orderDetails", "orderdetails.xml")


The XsltProcessor Class

The first question you might ask about the XsltProcessor class is "Why do we need a second XSLT processor, when there is already the XslTransform class?" This will become more apparent when we look at the Common Query Architecture section later, but for now this class should be considered an introductory prototype whose main aim is to significantly improve XSLT performance without breaking existing applications. You will have noticed that there is a new namespace in the Framework called System.Xml.Query, which provides the XsltProcessor and XQueryProcessor classes—both of which share a common architecture.

In effect XML languages share so much in common that, in reality, only a single XML query engine is needed, with compilers for each different language. This is a similar approach to the .NET languages, where there is a common set of features (the class library) and the choice of language is simply based on user preference. XSLT is just another one of the XML query languages, and the common approach to querying is demonstrated by the XsltProcessor class. The XsltProcessor class provides the ability to compile XSLT stylesheets and execute these against XML documents. However, the map:view extension function cannot be used to perform XSLT over XML views.

Before we look at how we can use the XsltProcessor, this section contains a concise reference to the properties and methods of this class. There are a single constructor, one property, and two methods (each with two overloads). Details are provided in Tables 8.5 through 8.7.

Table 8.5. The Constructor for the XsltProcessor Class

Constructor

Description

XsltProcessor()

Creates a new empty XsltProcessor instance.

Table 8.6. The Property of the XsltProcessor Class

Property

Description

XmlCommand

A reference to an XmlCommand instance that represents a compiled stylesheet.

Table 8.7. The Methods of the XsltProcessor Class

Method

Description

Compile(stylesheet-uri)

Compiles the specified stylesheet into an XmlCommand. The XsltProcessor then becomes a wrapper for the XmlCommand, which can be continuously executed.

Compile(stylesheet-uri, XmlResolver)

As above, except that the XmlResolver is used to resolve the stylesheet URI and any stylesheets referenced within it by <import> and <include> elements.

Execute (context-document-uri, result-document-uri)

Executes the compiled stylesheet. The context-document-uri is the URI of the XML input document, and the result-document-uri is a file to write the results to.

Execute (context-document-uri, datasources, arglist, results)

As above, except that an XmlResolver provided as the datasources parameter is used to resolve external documents via the document function. The arglist provides a list of arguments to the stylesheet, and the final results parameter is a reference to a Stream or TextWriter that the results will be sent to.

Transforming XML Documents with the XsltProcessor Class

The example in Listing 8.17 uses an inventory store XML document named storedb.xml to generate an HTML page containing inventory information using the XsltProcessor class. First, an XsltProcessor instance is created; then the Compile method is called on the storedb.xsl stylesheet ready to prepare it for execution against an XML document.

Next, the Execute method loads the storedb.xml XML document, executes the stylesheet (which does not need to resolve any external documents via the XmlResolver nor provide any parameters via the XmlQueryArgument List). The resulting HTML is written to the Response.OutputStream as part of this ASP.NET page.

Listing 8.17 Transforming an XML Document to HTML with an XsltProcessor
Dim xslt As New XsltProcessor()

' compile the XSLT stylesheet
xslt.Compile(strPath & "data\storedb.xsl")

' execute compiled stylesheet against XML document and output result
xslt.Execute("storedb.xml", New XmlUrlResolver(), Nothing, _
                                        Response.OutputStream)

The screenshot in Figure 8.6 shows the results of the transformation from the example in Listing 8.17.

Figure 8.6. Using the XsltProcessor class to transform an XML document into HTML

Transforming Directly to a File

There is also an Execute overload that allows you to provide an input file and an output file for the transformation, as shown the C# code in Listing 8.18.

Listing 8.18 Transforming an XML Document Directly to a File
XsltProcessor xslt = new XsltProcessor ();

//compile XSLT stylesheet
xslt.Compile("storedb.xsl");

//execute stylesheet against XML document and output result
xslt.Execute("storedb.xml", "outputdocument.xml" );

As you can see, currently the number of method overloads is limited. This reflects the XsltProcessor class prototype status, along with the fact that there are a number of features that have not been implemented (such as C# and Visual Basic scripting support and the xsl:key function). Despite its prototype status, expect this class to evolve into an extremely high-performance XSLT processor.


The Common Query Architecture

The diagram in Figure 8.7 is the best way to describe what the Common Query Architecture is all about. (Note that some of the classes in the diagram are internal in the Technology Preview release.) For each supported XML query language, there is a compiler that generates an intermediate query language representation, as is represented by the XmlExpression class.

Figure 8.7. The Common Query Architecture showing multiple XML query languages

Multiple XmlExpression instances can be composed together, meaning that a query—once compiled—can be used as input to another query. This is epitomized by the XmlViewSchema class, which represents an XML view. In effect the XML view is compiled just like any other XML query language and composed into an XQuery whenever the map:view() extension function is used. The intermediate format provides an abstraction layer for any XML query language.

After compilation and composition, the query engine takes over. Here optimizations of the generated query are applied in an identical fashion to SQL query rewrites in a relational database, in order to generate a better-performing and hence more optimal query. These can vary from simple to extremely complex, depending on the query and the types of data sources.

The query engine then generates executable MSIL code and SQL statements, if the data sources are targeted at SQL Server, as an XmlCommand class. The XmlCommand class is a query plan that can be cached and executed multiple times against the specified data sources by repeatedly calling the Execute method. The output from the execution is a stream of XML (or other format) via a TextWriter class. Having a query architecture that is common across all the XML query languages means that it can take advantage of language-independent optimizations, and hence provide performance improvements.


SQL Server as an XML Database

The beta 1 release of SQL Server, currently code-named "Yukon," has the facility to store XML as a native format, for example, as a dedicated column type, passed as a parameter, or declared as a variable. Until now, most users have stored XML in "character" or "text" columns, as a simple String value. However, now you can actually store it as XML and then access and query it as XML rather than having to extract it as a String and manage the process yourself.

With the new XML data type you can do the following things.

  • Store XML in "typed" form, where a schema is registered with SQL Server and the XML must conform to this schema.

  • Store XML in "untyped" form, where no schema is required and the XML just has to be well formed.

  • Select the value from an XML column in SQL statements and stored procedures, just like any other column type.

  • Query the XML in the column directly, using XPath and XQuery techniques, to find and return specific values.

  • Modify the XML in a column by adding or removing nodes or by changing the values of existing nodes.

  • Bind XML to other SQL Server data types, allowing queries to access values in stored procedure variables and other non-XML columns.

  • Access and return sets of node values, which resemble a single-column rowset.

  • Create indexes on the XML columns to vastly improve performance when querying and updating the data.

All of these topics are covered in depth in the SQL Server Books Online reference documentation that is provided with the "Yukon" beta release. In the remainder of this chapter we'll briefly examine the important features, giving you a feel for what the new data type can do and how you can use it.

The XML Data Type in SQL Server "Yukon"

The XML data type in SQL Server "Yukon" is called xml, and you use it when creating tables, stored procedures, indexes, and so on in exactly the same way as any other native SQL Server data type. For example, the following declaration creates a table containing an int column named MyRowKey and an xml data type column named MyXmlDoc:

CREATE TABLE MyTable(MyRowKey int, MyXmlDoc xml)

However, you cannot use an xml column as the primary key of a table or as a foreign key column, and you cannot specify the UNIQUE or COLLATE properties for it or use it in a GROUP BY clause. There is also a maximum limit of 32 columns in a table that can be of the xml type.

Typed and Untyped XML Data Columns

The CREATE TABLE statement shown above creates a table with an untyped xml data type column. Effectively the XML you insert into this column is stored in character format complete with the element tags and attribute names, using UCS-2 encoding. An untyped column is ideal where you will generally extract the complete XML document as a String when you want to access the values.

However, although you can still perform all the other operations on the column (as described later), this process will be less efficient than using a typed column. And when you insert XML into the column, it is checked for well-formedness and rejected if it is not well formed. This is the same kind of behavior as you would expect when, for example, trying to insert a char (String) value into a column that is defined as being of type int.

The alternative is to create a typed xml data column. This is a little more complicated because you first have to register the schema with SQL Server using a CREATE XMLSCHEMA statement. We look at how you register an XML schema with SQL Server in the following subsection of this chapter.

However, once the schema is registered, you just have to specify the schema for the XML that will be placed in this column. The simple way to do this is to specify the schema (using its registered namespace) with the column data type when you create the table:

CREATE TABLE MyTable(MyRowKey int,
             MyXmlDoc xml ('http://mysite.com/schemas/myschema'))

The advantages with a typed data column are that SQL Server can then perform validity checking on values that are inserted or when the individual nodes of the XML document are updated or deleted. It also changes the way that SQL Server stores the data in the XML document.

Because the schema defines the data types of the values within the XML document, SQL Server can store them as its native data types (e.g., varchar, int, money, datetime, and so on) and then map these values to the query and result when it is executed. This provides better performance when querying the XML and reduces storage overhead for the document.

The XML Schema Repository in SQL Server "Yukon"

When you register schemas with SQL Server, they are stored within the system tables of the Master database to create a schema repository. The schemas define the structure, format, and content of XML documents that will be considered valid, and a schema can therefore be applied to any column that will store instances of this document or to any variable that will hold an instance of the document. You can't remove (drop) the schema while any table still contains a column based on it or any stored procedure still contains a variable that references it.

Registering an XML Schema

To use a schema to define an XML data column, you must first register the schema with SQL Server using the CREATE XMLSCHEMA statement. This statement takes a single string-type parameter that contains or references the schema to be registered. In the simplest case:

CREATE XMLSCHEMA '<xsd:schema xmlns="...">... etc. ...</xsd:schema>'

After registration, the schema is identified by the namespace(s) you specify in the targetNamespace or xmlns attributes of that schema. An alternative way to register a schema is to assign it to a variable and then use the variable in the CREATE XMLSCHEMA statement:

DECLARE @myschema varchar(1000)
SET @myschema = '<xsd:schema xmlns="...">... etc. ...</xsd:schema>'
CREATE XMLSCHEMA @myschema

You can register more than one schema at a time by simply placing them all in the string value you pass to the CREATE XMLSCHEMA statement:

CREATE XMLSCHEMA '<xsd:schema xmlns="...">... etc. ...</xsd:schema>
                  <xsd:schema xmlns="...">... etc. ...</xsd:schema>
                  <xsd:schema xmlns="...">... etc. ...</xsd:schema>'

To manage security when registering schemas, SQL Server now exposes a new permission attribute named CREATE XML SCHEMA. The account being used to register the schema must have this permission. The permission can be assigned and managed in exactly the same way as the other permissions included in SQL Server, for example, by using the GRANT and REVOKE statements.

Dropping an XML Schema

To remove a schema from the SQL Server repository, you use the DROP XMLSCHEMA statement, specifying the namespace of the schema to be removed. For example:

DROP XMLSCHEMA NAMESPACE 'http://mysite.com/schemas/myschema'

This will fail if any table or variable within the database is currently using the specified schema.

Viewing an XML Schema

To view a schema that is registered in SQL Server, you use the xml_schema_namespace function and supply the schema namespace that was defined when it was registered:

SELECT xml_schema_namespace(N'http://mysite.com/schemas/myschema')

One point to note is that the returned schema is built up from the data stored in the system tables and is not the original "character" version you registered. For example, it will not contain any comments that were in the original schema and may differ in the ordering of the schema elements.

Limitations on Schemas Stored in SQL Server

There are some limitations applied to the use of schemas that define xml column types in SQL Server "Yukon."

  • You cannot register a schema that contains a redefine element or unique, key, or keyref expressions.

  • Annotations and comments are not stored in the database.

  • Notations are not stored in the database but are validated when the schema is registered.

  • The XSD id attribute is not supported but is validated when the schema is registered.

  • Default values cannot exceed 4,000 Unicode characters.

  • An xsd:import element causes the target schema to be imported and incorporated into the current schema, and the registered schema will no longer reference the imported schema.

  • You should avoid using schemas that define patterns for values in the data because these may not be suitable for validation if the internal representation of the values differs from the representation in the XML document or fragment.

  • Date and time values are normalized to the current time zone and do not contain the time zone offsets when stored in the database.

Inserting and Selecting against an XML Data Column

You have several options for loading an XML document or fragment into an XML data column. Because the xml data type behaves just like any other data type, it can be used in SQL statements, stored procedures, and T-SQL statements that create and modify tables, indexes, and so on.

XML is, of course, notionally text or character data, and the normal data storage type used to handle it outside SQL Server is a String variable. A String containing XML is automatically and implicitly cast to an xml type when included in a SQL statement or stored procedure, so the following could be used to insert a row into a simple two-column table that contains one int column and one xml column:

INSERT INTO MyTable(MyRowKey, MyXmlDoc)
VALUES (42, '<cars><onecar id="1">... etc. ...</onecar></cars>')

Alternatively, you can explicitly CAST a string value into an xml type or use the CONVERT function:

INSERT INTO MyTable(MyRowKey, MyXmlDoc)
VALUES (42, CAST('<cars>... etc. ...</cars>') AS xml))

or:

INSERT INTO MyTable(MyRowKey, MyXmlDoc)
VALUES (42, CONVERT(xml, '<cars>... etc. ...</cars>'))

When extracting XML from a column of type xml, you can perform the reverse process to convert it to a character-type value:

SELECT CONVERT(nvarchar(500), MyXmlDoc, 'ucs-2')
FROM MyTable WHERE ... etc. ...

However, you can CAST or CONVERT only to char, varchar, nchar, and nvarchar types. You cannot CAST or CONVERT the value in an xml column to text or ntext types.

Querying and Updating the Contents of an XML Data Column

SQL Server provides several techniques for querying and updating the values stored in a column of type xml:

  • The query method with an XQuery statement to return a fragment of untyped XML

  • The value method with an XQuery statement to return a single value from the XML and expose it as a standard (non-xml) SQL data type

  • The exist method to test whether a specific node/value exists in the XML data

  • The modify method to execute an XML Data Modification Language (XML DML) statement

  • The xmlnoderefs function to return a single-column rowset of values

We'll look at each of these techniques in turn next. The syntax and usage of XQuery is covered in detail earlier in this chapter, so the next sections concentrate only on the syntax of the SQL Server XML functions.

The query Method

You use the query method of the xml data type when you want to query an xml column in a table and get back a fragment of untyped XML as the result. The syntax for the query method is:

query('xquery-statement'[, node-ref])

where xquery-statement is the XQuery as a literal string that selects the XML fragment (an element, attribute, and so on), and the optional node-ref parameter specifies a node or a set of nodes against which the XQuery expression is evaluated.

For example, if the XML document contains a <cars> element that itself contains multiple <car id="xxx"> elements, you can select a specific car on its id value using:

SELECT MyXmlDoc::query('/cars/car[@id="42"]') FROM MyTable

The result is an XML fragment such as:

<car id="42"><make>Rover</make><model>Rampage</model></car>

If you use a node-ref parameter, it must specify a set of nodes in the XML document that contain the value you are querying for. See the section on the xmlnoderefs function later in this part of the chapter for more details. If the XML instance itself (the column of type xml that is specified in the SELECT statement) contains null, the query method returns NULL.

The value Method

The value method also uses an XQuery statement to access a node in the XML document, but this time returns it as a SQL Server native data type such as int, char, or datetime (but not an xml type, user-defined type, image, text, ntext, or timestamp). The syntax for the value method is:

value('xquery-statement', sql-type[, node-ref])

where xquery-statement is the XQuery as a literal string that selects a single node from the XML, sql-type is the native data type to return the value as, and the optional node-ref parameter specifies a node or a set of nodes against which the XQuery expression is evaluated.

For example, the following uses an XQuery that specifies the single model node (element) for the car with id="42" from the same document used in the previous example:

SELECT MyXmlDoc::value('/cars/car[@id="42"]/model', nchar(50))
FROM MyTable

If you use a node-ref parameter, it must specify a set of nodes in the XML document that contain the value you are querying for. See the subsection on the xmlnoderefs function later in this chapter for more details. If the XML instance itself (the column of type xml specified in the SELECT statement) contains null, the value method returns NULL.

The exist Method

The exist method is used to test for the presence of a node or value in an XML document. The syntax of the exist method is:

exist('xquery-statement'[, node-ref])

where xquery-statement is the XQuery as a literal string that selects a single node, and the optional node-ref parameter specifies a node or a set of nodes against which the XQuery expression is evaluated. The method returns True (bit value 1) if the XQuery gives a nonempty result, False (bit value 0) if the result is empty. For typed XML documents where the target node contains null, the exist method returns NULL.

For example, you can specify that a SELECT statement should select only the row in a table named MyTable that contains the value 42 for a <car> element in the XML document stored in an xml column named MyXmlDoc like this:

SELECT SomeOtherColumn FROM MyTable
WHERE MyXmlDoc::exist('/cars/car[@id="42"]')

This example returns the value of the column named SomeOtherColumn, which is a native SQL Server data type such as nvarchar or int. However, if the SomeOtherColumn were an xml data type, you could combine the query or the value method with the exist methods in the SELECT statement to get back either untyped XML or a single value:

SELECT SomeOtherColumn::value('/root/enginetype', nchar(50))
FROM MyTable
WHERE MyXmlDoc::exist('/cars/car[@id="42"]')

If you use a node-ref parameter, it must specify a set of nodes in the XML document that contain the value you are querying for. See the subsection on the xmlnoderefs function later in this chapter for more details. If the XML instance itself (the column of type xml specified in the SELECT statement) contains null, the exist method returns NULL.

The modify Method

To modify the contents of an XML document in an xml column, you can use the modify method. It takes as a parameter an XML DML statement, which itself contains an insert, update, or delete statement that manipulates nodes within the xml column data. The syntax of the modify method is:

modify(xml-dml-statement)

The modify method of the xml data type can be used only in the SET clause of a SQL UPDATE statement. Let's look briefly at XML DML.

The XML Data Modification Language

XML Data Modification Language (XML DML) extends XQuery beyond the current W3C recommendations to provide a full-featured query and data modification language that can be used against the xml data type. It adds three keywords to XQuery for this purpose.

  1. The insert keyword is used to insert one or more nodes into an XML document that resides in an xml data column.

  2. The delete keyword is used to remove one or more nodes from an XML document that resides in an xml data column.

  3. The update keyword is used to modify the value of one or more nodes in an XML document that resides in an xml data column.

XML DML statements containing these three keywords are used as parameters to the modify method of the xml data type.

Using the XML DML insert Keyword

The insert keyword specifies that one or more nodes will be inserted into the XML document as either children or siblings of a specified node. The syntax for the insert keyword is:

insert expression1 position expression2

where expression1 is either an XML fragment or an XQuery expression that selects an XML fragment, and expression2 identifies a single target node for the XML fragment to be inserted into. The position value indicates the location of the inserted content (which can be an element or an attribute) relative to the node specified by expression2 and follows this form:

{as first | as last}  into | before | after

In other words, you specify either as first or as last, followed by into, before, or after. For example, to insert the XML fragment <seats>4</seats> into the <car> element whose id value is 42, you could use:

UPDATE MyTable SET MyXmlDoc::modify('insert <seats>4</seats>
                             as last into /cars/car[@id="42"]')

This locates the new element after all the existing elements in the <car id="42"> element. If you use the before or after keyword instead of into, the new element becomes a sibling of the <car> element (i.e., it is located at the same level in the hierarchy of the document) rather than a child of it.

When you use into, and the target node is an element that already contains other child elements, you must specify either as first or as last. However, these keywords are ignored if you are inserting an attribute into an existing element.

If you insert a sequence of values (e.g., a single-column rowset such as is returned from the xmlnoderefs function), they are inserted into the target node as a single text node containing the values delimited by single spaces. If you want to achieve the same result when specifying the XML explicitly as a character string for expression1, the values must be wrapped in parentheses and delimited by commas.

Using the XML DML delete Keyword

The delete keyword specifies that one or more nodes will be removed from the target XML document. The syntax for the delete keyword is simply:

delete expression

where expression is an XQuery expression that selects one or more nodes in the target document. For example, to delete the <car> node with id value 42 you could use:

UPDATE MyTable SET MyXmlDoc::modify('delete /cars/car[@id=42]')

To delete all the <make> elements from the document you could use:

UPDATE MyTable SET MyXmlDoc::modify('delete //make')
Using the XML DML update Keyword

The update keyword specifies that one or more nodes will be modified within the target XML document. The syntax for the update keyword is:

update [value of] expression1 to expression2

where expression1 identifies the target node whose value is to be updated, and expression2 is the new value for that node. The target node specified in expression1 must evaluate to a single node; otherwise, an exception is generated. The new value specified in expression2 can be an untyped XML fragment or a simple value. If it is an XML fragment that specifies multiple values, these values are inserted as a space-delimited list for the new value of the target node. The optional value of keyword indicates that expression2 is a value, not an untyped XML fragment.

As an example, the following will update the value of the <model> element within the <car> element that has id value 42 to a new string value:

UPDATE MyTable SET MyXmlDoc::modify('update value of
                /cars/car[@id=42] to "Wrampage Mark 2"')
The xmlnoderefs Function

The xmlnoderefs function takes an XML document stored in an xml data column and returns a single-column rowset, called a noderef column table. This contains references to a sequence of nodes in the XML document, as specified by an XQuery you supply. The syntax of the xmlnoderefs function is:

xmlnoderefs(xml-column, xquery-statement [, node-ref])

where xml-column is the xml data column to extract the results from, and xquery-statement is an XQuery statement that selects the nodes in that column that will be returned. The optional node-ref parameter specifies a node or a set of nodes against which the XQuery expression is evaluated and allows nested noderef selections to be made. In other words, you can query against an existing set of nodes that are themselves extracted by the xmlnoderefs function.

For example, to create a noderef that references all the <car> elements in the document we've been using so far, you could simply use:

xmlnoderefs(MyXmlDoc, '/cars/car')

This noderef can then be used as the optional third parameter of the value method to select the model name of the car with the id value 42. In this case, the XQuery expression in the first parameter will be evaluated against the noderef list of <car> elements:

SELECT MyXmlDoc::value('car[@id="42"]/model',
                       nchar(50),
                       xmlnoderefs(MyXmlDoc, '/cars/car'))
FROM MyTable

A noderef column table can be used in the optional parameter for the query, value, and exist methods described earlier. This tends to improve performance of queries by preselecting and providing a path to the set of nodes on which the query will operate. However, bear in mind that a noderef cannot be used in situations other than those described above, nor stored for later reuse, because it contains references that are valid only during the current query.

Binding Relational Data inside XML

The new xml data type allows you to use values from other non-xml type columns and from non-xml variables within an XQuery statement. This process is referred to as binding XML to other SQL Server data types. For example, you may want to use an XQuery to select a value in an xml column but include in that XQuery a reference to another column that is not of type xml. To achieve this there are two new "pseudo-functions":

  1. The sql:column function, which allows a non-xml column to be referenced within an XQuery expression

  2. The sql:variable function, which allows a non-xml variable to be referenced within an XQuery expression.

As an example of their use, the following SELECT statement extracts the value of a node from an XML document stored in the MyXmlDoc column, referencing both the xml column named MyXmlDoc and the int column named Seats in the WHERE clause:

SELECT MyXmlDoc::query('car/model', nchar(50))
FROM MyTable
WHERE MyXmlDoc::exist('/cars/car/[make="Rover"]
                      and sql:column("MyTable.Seats")=4)')

Or, if the number of seats were stored in a variable named @seats within a stored procedure, you could use:

SELECT MyXmlDoc::query('car/model', nchar(50))
FROM MyTable
WHERE MyXmlDoc::exist('/cars/car/[make="Rover"]
                      and sql:variable("@seats")=4)')

While you could easily adapt the two SQL statements above to avoid using the new functions, they come in particularly handy when using XQuery statements that return templated fragments of XML—for example:

DECLARE @price money
SET @price=2500.00
DECLARE @x xml
SELECT @x = CatalogDescription::query('
  <Product ProductID="{sql:column("Product.ProductID")}"
    ProductName="{ProductDescription[1]/@ProductModelName}"
    ListPrice="{sql:column("Product.ListPrice")}"
    DiscountPrice="{sql:variable("@price")}"
  />
')
FROM Product, ProductModel
WHERE Product.ProductModelID = ProductModel.ProductModelID
AND ProductID=168

The ProductID and ListPrice values that are inserted into the XQuery template come from non-xml type columns, while the ProductName value is extracted from the xml column. The DiscountPrice value comes from the @price variable (of type money) that is declared and set at the start of the procedure. This produces a result, depending on the data in the database, something like the following:

<Product ProductID="1062"
    ProductName="ML Road Frame"
    ListPrice="594.83"
    DiscountPrice="125.5"
/>

However, note that the sql:column and sql:variable functions cannot be used for columns or variables that are of User-Defined Types, are unique identifiers, or are of type xml, timestamp, text, ntext, or image.

Indexing an XML Data Column

A useful feature of the new xml data type is that you can create an index on a column of this type (but only for a table, not for a view). This can increase performance considerably, but you must bear in mind a few limitations.

  • The table containing the xml column must also contain a clustered primary key, and once an xml column index exists, you cannot modify the clustered primary key unless you first drop all the xml indexes.

  • You can specify only a single column in an xml index—multiple column indexes are not supported for xml columns.

  • You must remove the index on a column before you can change it from "typed" to "untyped" (i.e., register a schema for it).

Note that, while this section covered the basics of creating and using the new xml data type, it is not a comprehensive reference to all the opportunities, techniques, and limitations. The SQL Server Books Online help files provided with "Yukon" contain plenty of useful information on the xml data type and how XML data columns behave.


SUMMARY

In this chapter, we worked with the XQueryProcessor and XsltProcessor classes—looking first at the interface members and then at how they can be used. We started with a very brief introduction to the XQuery language for the queries used in this chapter. It is really necessary to read the XQuery Language specification at http://www.w3.org/TR/xquery/ to fully appreciate the power of this language.

We saw how the XQueryProcessor class can be used to query over XML documents and over XML views of SQL Server tables. There can be multiple XML views across a single database connection, or multiple XML views across multiple database connections, making XQuery perfect for data aggregation scenarios with data stored in SQL Server tables. We also saw how queries can be combined to retrieve XML through XML views and from XML documents. Because they are built on a Common Query Architecture, both the XQueryProcessor and the XsltProcessor classes can take advantage of query optimization to significantly improve the performance of the executed query, in particular by generating MSIL.

Finally, we covered the xml type that has been introduced in the SQL Server "Yukon" beta 1 release, which allows you to use XML as a native type for columns and variables and within stored procedures. The XML you store can be either untyped or typed. (As mentioned, typed means that it is associated with an XML schema that enforces the XML structure of the stored documents.)

The xml type provides several techniques for querying and updating the values stored in a column, including using the query method with an XQuery statement to return a fragment of untyped XML and using the modify method to execute an XML DML statement. And the introduction of XML as a native type to SQL Server enables it to act like an XML database and hence store documents produced from the Microsoft Office 2003 family. The trilogy of XML content generation from Microsoft Office 2003, the XML storage in SQL Server "Yukon," and the query capabilities available through XML Views technology on the middle tier in the .NET Framework provide an impressive new application-building platform.

With this, we come to the end of the book. Although version 2.0 of the .NET Framework is still young and evolving—it will almost definitely change as it progresses toward a final release—we would like to think that you have found the content of this book useful. It should have helped you understand where Microsoft is taking data access as a whole in version 2.0 and what you have to do to be ready when it arrives.


Team LiB

Brought to You by

Like the book? Buy it!