Using XSL and XML to generate dynamic web pages from UNIFACE

Tony Marston - 8th January 2003
amended 1st February 2003

1. Introduction

The method of generating a web page from a UNIFACE application has traditionally been to use the WEBGEN and WEBGET statements from within a UNIFACE Server Page (USP).

• WEBGEN - Generate an HTML skeleton document from a skeleton file (ending in .hts), replacing any field values (marked by <X-SUBST> and <X-OCCURRENCE> tags) with the current values in the form. The generated document is then passed to the Web server.
• WEBGET - Load data sent from a client browser into the current form, reconnect any database occurrences to the database, and activate a <Detail> or <On Error> trigger if required.

This particular method has the following drawbacks:

• It requires that HTML tags be inserted into the UNIFACE component. This means that either your UNIFACE developers need to acquire knowledge of HTML, or your HTML developers need to acquire knowledge of UNIFACE.
• It generates a skeleton file (with a .hts extension) that is unique to UNIFACE.
• It uses <X-SUBST> and <X-OCCURRENCE> tags that are unique to UNIFACE.
• It often needs the use of a 3rd party product (such as Macromedia Dreamweaver) in order to make the resulting web pages look aesthetically pleasant.

I have personally witnessed how much time can be wasted in tracking down a bug in a web page caused by an HTML tag being somehow misplaced between the time it was inserted into the UNIFACE component and the time the actual web page was generated. Too many decisions appeared to have been made 'behind the scenes' about which the developer had no knowledge and over which the developer had no control.

A more modern approach would be to use a combination of XML and XSL in order to generate HTML documents. The advantages are as follows:

• Both XML (eXtensible Markup Language) and XSL (eXtensible Stylesheet Language) are international non-proprietary standards supported by the World Wide Web Consortium.
• Since the Apache Software Foundation started the Cocoon Project this method of generating dynamic web pages has spread around the internet community. It can be used by any language that can generate XML documents and has access to an XSLT processor.
• UNIFACE has the ability to generate XML documents - refer to the XMLSAVE command.
• UNIFACE has access to an XSLT processor - refer to the XMLTRANSFORM operation of the USYSXSLT component.
• It removes the requirement for embedding any HTML tags into any UNIFACE component. This means that UNIFACE developers don't need to meddle with any HTML, and HTML developers don't need to meddle in any UNIFACE.

2. How the internet works

Here is a brief overview of how the internet works. A web browser running on a client device (such as a PC) communicates to a web server on a remote server device via the internet, or World Wide Web, using the Hyper Text Transfer Protocol (HTTP). The client sends a REQUEST, the server receives it, processes it, then sends back a RESPONSE.

Figure 1 - The Internet in a nutshell

2.1. The HTTP Request

The HTTP Request is sent from the client to the server. It consists of 3 parts:

• A Request Line - which contains an HTTP command known as a method such as GET or POST. (There are others, but I shall ignore them for the purposes of this document.)
  • The GET method is a request for information residing at a particular URL. This may also include a limited amount of data in the form of arguments appended to the URL, such as in:

    http://www.stuff.com/product.php?id=123456

    With this method any data is visible in the URL, which means that the request can be bookmarked for retransmission at a later time.
  • With the POST method all data is sent to the server as part of the HTTP body. No data is visible in the URL, which means that the request cannot be bookmarked.
• A Header - which contains information about either the client or the server, information about the data being sent between the client and the server, and information about the client configuration and different types of acceptable documents. This may also contain any data held in a cookie. Note that most parts of an HTTP header are optional, so you cannot guarantee that they will be present.
• A Body - which contains any data that is being sent to the server if the POST method was used in the HTTP request line, otherwise it will be empty.

2.2. The HTTP Response

The HTTP Response is sent from the server to the client. It consists of 3 parts:

• A Response Line - which contains the HTTP version number and an HTTP request code that reports the success or failure of the request.
• A Header - which contains information similar to that found in the request header.
• A Body - which, if the request was successful, contains the HTML code (plus any script that is to be executed by the browser) ready for the browser's interpretation.

As you can see from the HTTP response the language in which your web site is written is totally irrelevant as what is returned to the client is nothing more than a text file containing HTML tags and data. You can use any language, or combination of languages, that takes your fancy provided that the output is an HTML document.

2.3. A sample HTML document

Here is a sample HTML document:

<html>
<head>
   <title>This is the document title</title>
   <link rel="stylesheet" type="text/css" href="style.css">
</head>
<body>
   <p>This is a paragraph...</p>
   <table width="100%">
      <tr>
         <td>This is a table cell</td>
         <td>This is another table cell</td>
      </tr>
   </table>
</body>
</html>

Within an HTML document there are a predefined set of tags which have predefined meanings which should be interpreted in a predefined way by whatever browser is running on the client. Be aware that some browsers may use additional 'extensions' to the HTML standard that may not be rendered correctly, if at all, in a different browser.

An HTML document contains tags, processing instructions and data, such as:

• Each document starts with <html> and ends with </html>.
• Everything between <head> and </head> is part of the header.
• Text between <title> and </title> will be used as the document title.
• The <link rel="stylesheet"...> line points to an external Cascading Style Sheet (CSS) file that will be used for any formatting instructions.
• Everything between <body> and </body> forms the body of the document.
• Everything between <p> and </p> is treated as a paragraph.
• Everything between <table> and </table> is part of a table.
• Everything between <tr> and </tr> is a row within the current table.
• Everything between <td> and </td> is a cell or column within the current row.

There are numerous other tags with other meanings, but if you want to know more I suggest you visit w3schools.com for a brief tutorial.

3. The XML/XSL method

With this method an XML file contains nothing but data, while an XSL file contains processing instructions to transform that file into another document. This could be another XML file, an HTML document, a Microsoft WORD document, an Adobe PDF document, or any type of text-based document that you choose.

3.1. A sample XML document

Here is a sample XML document:

<?xml version='1.0' ?>
<!-- Created by UNIFACE - (C) Compuware Corporation -->
<UNIFACE release="7.2.06.2(u-s602r1l)" xmlengine="1.0">
<TABLE xmlns:USOURCE="USOURCE.DICT">
<OCC>
<USOURCE:UTIMESTAMP>1999-06-19T19:02:53</USOURCE:UTIMESTAMP>
<USOURCE:USUB>M</USOURCE:USUB>
<USOURCE:UVAR>USYS</USOURCE:UVAR>
<USOURCE:ULABEL>M_90001</USOURCE:ULABEL>
<USOURCE:ULAN>USA</USOURCE:ULAN>
<USOURCE:MSGTYPE>M</USOURCE:MSGTYPE>
<USOURCE:UDESCR>STORE</USOURCE:UDESCR>
<USOURCE:UCONFIRM>F</USOURCE:UCONFIRM>
<USOURCE:UAUDIO>0</USOURCE:UAUDIO>
<USOURCE:UTEXT>90001: STORE failed - see Message Frame for more details</USOURCE:UTEXT>
</OCC>
<OCC>
<USOURCE:UTIMESTAMP>1999-06-19T19:07:31</USOURCE:UTIMESTAMP>
<USOURCE:USUB>M</USOURCE:USUB>
<USOURCE:UVAR>USYS</USOURCE:UVAR>
<USOURCE:ULABEL>M_90002</USOURCE:ULABEL>
<USOURCE:ULAN>USA</USOURCE:ULAN>
<USOURCE:MSGTYPE>M</USOURCE:MSGTYPE>
<USOURCE:UDESCR>STORE</USOURCE:UDESCR>
<USOURCE:UCONFIRM>F</USOURCE:UCONFIRM>
<USOURCE:UAUDIO>0</USOURCE:UAUDIO>
<USOURCE:UTEXT>90002: STORE successful</USOURCE:UTEXT>
</OCC>
</TABLE>
</UNIFACE>

The only entry that MUST be contained in an XML document is the <?xml version='1.0' ?> which identifies it as an XML document, with the XML version number. All the others are user-defined, but they must adhere to the following rules:

• An XML element is that which exists between a start tag and an end tag.
• A start tag is a unique name that exists between a '<' and '>' symbol.
• An end tag must have the same name as the start tag, but exists between a '</' and '>' symbol.
• An element name may be qualified in order to make it unique by the addition of a namespace. This is declared with an xmlns:namespace="unique identifier" instruction. The namespace is then prefixed to the relevant elements with a ':' separator. In the above example USOURCE: is used as a namespace.
• A start tag may also contain one or more attributes in the form of attributename="attributevalue".
• An XML element may contain data or another element (known as a child element), but not both.
• The first element in any XML document is known as the root element, and may not contain data, only child elements. In the example above the root element is <UNIFACE>.
• Child elements may contain other child elements down to any depth. This produces what is known as a tree structure where the path to any element can be traced all the way back to he root element. In the above example UNIFACE/TABLE/OCC/USOURCE:TEXT is such a path.
• Child elements may be repeated any number of times. In the above example the element <OCC> appears multiple times, and each occurrence has its own set of child elements and values.

If you want to know more about XML I suggest you visit w3schools.com for a brief tutorial.

3.2. A sample XSL document

Here is a sample XSL document:

<?xml version='1.0'?> (1)
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
                xmlns:USOURCE="USOURCE.DICT"> (2)

<xsl:template match="/"> (3)

  <html>
  <head>
      <title>Contents of Message File</title>  
  </head>

  <body>
    <table border="1">
      <thead>
        <tr><th>Library</th><th>ID</th><th>Message</th></tr>
      </thead>

      <tfoot>
        <tr><th colspan="3">end of messages</th></tr>
      </tfoot>

      <xsl:apply-templates select="UNIFACE/TABLE/OCC[USOURCE:UVAR]"/> (4)

    </table>
  </body>
  </html>

</xsl:template> (5)

<xsl:template match="UNIFACE/TABLE/OCC[USOURCE:UVAR]"> (6)
  <tr>
     <td><xsl:value-of select="USOURCE:UVAR"/></td> (7)
     <td><xsl:value-of select="USOURCE:ULABEL"/></td>
     <td><xsl:value-of select="USOURCE:UTEXT"/></td>
  </tr>
</xsl:template> (8)

</xsl:stylesheet> (9)

Here is an explanation of its components. Note that those tags that are prefixed with xsl: are XSL instructions; the remainder you should recognise as standard HTML tags.

• Line (1) is always required; it identifies the XML version number.
• Line (2) is always required; it identifies this as an XSL stylesheet and provides the version number for those elements within the xsl: namespace. Additional xmlns: statements are optional depending on the contents of the documents being processed.
• Line (3) defines a template for producing output. This is executed for the root node, symbolised by match="/". As there is only one root node the contents of this template are executed only once. This will start creating the HTML output where the body will contain a table with a predefined header and footer, with the contents being obtained by the instruction at line (4).
• Line (4) will be executed for each occurrence of UNIFACE/TABLE/OCC which has a child element named USOURCE:UVAR.
• Line (5) marks the end of the template started at line (3).
• Line (6) marks the start of a new template that will only be processed when the current node being executed is the one indicated in the match=".." part.
• Line (7) (actually, all those lines between line (6) and (8)) will create a new row in the table. Each row will have three cells containing the values from the USOURCE:UVAR, USOURCE:ULABEL and USOURCE:UTEXT elements respectively.
• Line (8) marks the end of the template started at line (6).
• Line (9) marks the end of the stylesheet.

If you want to know more about XSL and XSL Transformations I suggest you visit w3schools.com for a brief tutorial.

3.3. Performing an XSL Transformation

There are actually two ways in which an XML document can be transformed by an XSL document:

• On the CLIENT. This is where the browser is given both the XML and the XSL file and performs the transformation itself. The disadvantage of this method is that you cannot guarantee that the client is running a browser that can perform XSL transformations. For example, different versions of Microsoft's Internet Explorer came with different versions of the MSXML parser. To ensure that you are using the latest version you can visit http://msdn.microsoft.com/en-us/data/bb190600.aspx and download it. As at March 2002 this was v4.0 SP1.
• On the SERVER. This method is more reliable as the transformation is performed on a device where you have total control over the software that is available. What is sent to the client is the output from the transformation process which is a standard HTML document. This is capable of being understood by any web browser (notwithstanding which version of HTML/HTML/CSS you are using, and the version of the browser).

When a transformation takes place the same XSL file will perform different actions depending on what data is available. This can come from 3 sources:

• From elements within the XML file. This is data enclosed by start and end tags, such as:

  <author>Tony Marston</author>

  Elements are referenced using code such as: //author
• From attributes within the XML file. These appear within start tags as one or more name="value" pairs, such as:

  <author age="21" hair="dark brown">

  Attributes are referenced using code such as: //author[@age]
• From parameters included in the activation of the transformation process. These are entirely optional. They must be predefined within the XSL file as in:

  <xsl:param name="title">This is the title</xsl:param>

  Parameters are referenced using code such as: $title.

  If a parameter value is not included in the transformation process then the value defined in the <xsl:param ...> statement will be used as the default.

There is one important factor to note about this method of producing web pages - you do not have to wait until the programs that generate the XML data have actually been written before you can develop and test your XSL scripts. XML files are text files, therefore files with sample data can easily be constructed using any text editor. The only proviso is that your software generates XML files with the same structure and naming conventions as your test samples.

3.4. Transforming within UNIFACE

This is how the XML/XSL transformation process can be incorporated into a UNIFACE Server Page (USP) component:

Figure 2 - XSL Transformation within UNIFACE

The processing flow is as follows:

1. The HTTP request is extracted using the $webinfo("input") command. Note that I do not use the webget command as this automatically reconnects occurrences to the database. You should be using the 3-Tier architecture for your web application, in which case no Presentation Layer component (such as a Server Page) should communicate directly with the database. This should be done indirectly through a service component in the Business Layer.
2. Information is extracted from the $webinfo string and inserted into the component. As this is returned in the form of an associative list this should be a straightforward task.
3. The request is processed. This may mean activating operations on one or more service components in order to read from or write to the database. Data and/or error messages may be returned.
4. The contents of the component is saved to an XML stream by using the XMLSAVE command. This builds the XML stream (a single text string) using the structure defined in a DTD (Document Type Definition). This may also use optional mapping parameters to resolve any differences between entity and field names between the component and the DTD. Note that at present UNIFACE does not support the inclusion of user-defined attributes with values that are supplied at runtime. Such attributes will have to be inserted by editing the XMLSAVE output string.
5. The XMLTRANSFORM operation of the USYSXSLT component is activated. This uses the Xalan XSLT Processor from the Apache Software Foundation. It will take the XML data from step (4), the contents of an external XSL file and produce a string containing an HTML document.
6. The $webinfo("output") command will then pass the HTML document to the client's browser.

Optional 'cookie' data can be retrieved using the $webinfo("cookiesin") command and set by the $webinfo("cookiesout") command.

4. Summary

I hope that in this article I have clearly demonstrated two things:

• Using XML and XSL to generate dynamic web pages is a solution that is gaining popularity in the whole of the internet community. It offers the following advantages:
  • Both XML and XSL are non-proprietary standards supported by the World Wide Web Consortium.
  • It completely separates the task of generating HTML documents from whatever language is used to communicate with the database and apply business rules.
• UNIFACE is able to generate XML files extremely easily, and has access to a well-known XSLT processor, therefore this solution is available to the UNIFACE community without the need for any expensive add-ons. This removes the following disadvantages:
  • It removes the need to deal with proprietary .hts skeleton files and <X-SUBST> and <X-OCCURRENCE> tags.
  • It means that UNIFACE developers do not have to meddle with any HTML code, and your HTML developers do not have to deal with any UNIFACE code.

I am aware that the sample XML and XSL documents used in this article may not convince you that this method is actually capable of producing sophisticated dynamic web pages that are essential in a typical web site, so I have written a follow-up article showing what I have been able to produce in my own tests using PHP and MySQL running under the Apache web server. This article is available at 'Generating dynamic web pages using XSL and XML'.

Tony Marston
8th January 2003

mailto:tony@tonymarston.net
mailto:TonyMarston@hotmail.com
http://www.tonymarston.net

Amendment history: