WSRPtk

Maintaining this Website

Under Construction

This site is under construction. It is generated from source files using Apache Forrest (version 0.7). The source files for this website are stored in the WSRPtk project's CVS repository. The generated site is uploaded to the Source Forge website using SFTP.

Project Roles

Source Forge User Types

Unauthenticated User

A member of the general public who downloads and uses the test kit software.

Authenticated User

A person who has established a Source Forge user id and password. In order to obtain a user id, the person must sign Terms and Conditions, indicating that they promise to abide by the license terms of any project to which them make a contribution.

Developer

An Authenticated User who has obtained commit priveleges on a Source Forge Project.

Release Technicial

A Developer who also has the privelege to use the File Release System to create and post releases.

Project Administrators

The Authenticated User that created the project in the first place. The Project Administrator has all priveleges and can grant Developers Project Administrator priveleges. The Project Administrator also grants commit priveleges to Authenticated Users to enable them to be Developers.

Becoming a Developer

The steps to becoming a developer on the WSRPtk Project are as follows:

1. Obtain a SourceForge account to become an Authenticated User and sign the Terms and Conditions
2. Request Developer priveleges by sending an email to the Project Administrator
   • The OASIS Technical Committee Conformance Sub Committee must vote to grant developer priveleges to the requester.
   • The requester must be an employee of an OASIS TC member company.
3. The Project Administrator will add priveleges to the requester's account.

Conformance Sub Committee

The OASIS Technical Committee Conformance Sub Committee is involved in the management of the WSRPtk project in the following ways:

1. Chair of the Conformance Sub Committee will be granted Project Administrator priveleges.
2. The Conformance Sub Committee votes to grant Project Administrator priveleges to one or more Developers.
3. The Conformance Sub Committee votes to release each version of the Test Kit
4. The Conformance Sub Committee designates a Developer (i.e. Release Technician) to publish the release to the File Release System
5. The Conformance Sub Committee reviews and votes to accept 3rd Party Contributions

Handling 3rd Party Contributions

If a person who is not a Developer wishes to contribute code to the project:

1. The Conformance Sub Committee shall review the code contribution to verify the origin of the code and that it can be released under the Common Public License.
2. The Conformance Sub Committee votes to accept it.
3. A Developer then commits the code to the project.

WSRP V1 Conformance Test Kit

Introduction

The V1 Conformance Test Kit is designed to verify conformance of running WSRP Producers and Consumers to the V1 WSRP Specification. The Test Kit does not provide 100% coverage of the V1 conformance statements. Refer to the WSRPtk project website under Feature Requests for a list of assertions that are outstanding.

This website provides documentation for the V1 Test Kit, but is also a work in progress. Information on how this website is produced can be found here.

Refer to links on left hand side of this page to find information on how to download the Test Kit and its prerequisites and how to configure and execute the Test Kit.

A rudimentary export of the Power Point tutorial is also provided. Most of this site's initial content has come from this tutorial. The documentation of how to write test assertions has not been incorporated into these web pages yet, so refer to the tutorial for help with development.

A brief overview of the architecture is available.

The development content is a work in progress and has not yet been extracted from the tutorial.

Getting Started

Obtaining the Release

Download the release from the website. Unzip the file in any directory.

Obtaining and Installing Prerequisites

Eclipse Web Tools

1. Download Web Tools 1.0.1 from here.
2. Unpack Zip File
3. Copy from: eclipse/plugins/org.eclipse.wst.wsi_1.0.1.v200602030355.jar to: wsrptk/lib
   Note

   Note: the name of this jar file has changed over time. You may have wsi-core.jar and wsi-validate.jar already on your system if you have previously installed Web Tools. These jars *should* work also.

WS-I Testing Tools

1. Download the WS-I Testing Tools 1.1 from here.
2. Unpack Zip File
3. Copy from: wsi-test-tools/common/xsl/*.* to: wsrptk/xsl
4. Copy from: wsi-test-tools/common/schemas/*.* to: wsrptk/schemas

XSL Stylesheet Issues

• The namespace definitions in the XSL stylesheets obtained from WS-I are out of date with respect to the jar files from Eclipse.
• The embedded style directives use ALL CAPS for HTML tag names, and they have to be lower case to get proper formatting.
• The WSRPTK project has patch files that can be applied to make these corrections.

Executing the Test Kit

Summary of Steps

Executing the Test Kit happens in 2 phases. The first phase is the Monitoring phase, during which a message log is created from a running WSRP service. The second phase is the Analysis phase, during which the log file is analyzed for conformance.

Here is an architectural diagram of the components:

In general, these are the steps for running the test kit:

1. Configure the Producer to listen on a particular port (e.g. 8080).
2. Configure the Consumer to write to a particular port (e.g. 8081).
3. Configure the Monitor to listen to the Consumer (e.g. on 8081) and send to the Producer (e.g. on 8080) by setting the port numbers in the config file.
4. Start your Producer (e.g. WSRP4J on Tomcat)
5. Start the Monitor (bin/Monitor.bat)
6. Start your Consumer (e.g. WSRP4J Swing Consumer)
7. Exercise the User Interface
8. Exit the Consumer
9. Stop the Monitor (type "exit")
10. Run the Analyzer (bin/Analyzer.bat)
11. View the Report (logs/AnalyzerLog.xml)

The Monitor Component

The Monitor is a WS-I Testing Tools component that intercepts web service traffic between parties and writes the messages to a log file.

The Functional Specification for the Monitor component can be found here.

The Monitor gets parameters from a configuration file, the path to which is passed in at runtime. The config file is found in wsrptk/config. Here is a sample of configuration file:

The Analyzer Component

The Analyzer is a WS-I Testing Tools component that intercepts web service traffic between parties and writes the messages to a log file.

The Functional Specification for the Analyzer component can be found here.

The Analyzer gets parameters from a configuration file, the path to which is passed in at runtime. The config file is found in wsrptk/config. Here is a sample of configuration file:

Project Directory Structure

The WSRPTK project has the following directories:

Architecture

Introduction

The WSRPtk makes use of the Web Services Interoperability (WS-I) Testing Tools, by using a man-in-the-middle architecture to capture the web service message flow between Producer and Consumer. The Monitor tool is used to log messages between Producer and Consumer. It passes the messages unchanged between them. The messages are saved in the Message Log.

The Message Log file is then analyzed offline by the Analyzer tool to see if the messages meet the conformance requirements. The WSRPtk extends the WS-I Analyzer by implementing test assertions that evaluate the messages with respect to the WSRP V1 Specification. The Profile is an XML file that contains a description of each test assertion, and is used to tell the Analyzer which test assertions to execute. The logic for each test assertion is written in a Java class which the Analyzer loads at runtime.

The Analyzer creates an XML file containing the Report which summarizes the results of the test assertions.

Development

Development

Introduction

This section covers information needed by developers to add new test assertions to the V1 Conformance Test Kit. It is assumed that the reader is familiar with the test kit architecture and WSRP.

Creating a new Test Assertion requires 2 basic steps:

1. Enable the Test Assertion in the Profile, or add it to the Profile if it doesn't already exist.
2. Write the Java code to implement the Test Assertion logic.

Additional steps are required if the Test Assertion requires the use of the Specialized Portlet:

1. Add the JSP file to the webapp.
2. Add the JSP to the flow_script_master.xml and flow_script.xml files.

WSRP Profile

Introduction

The Profile is an XML document that contains a description of each Test Assertion in the test kit. The Profile is found in the profile directory and is called WSRPProfileTestAssertions.xml. If you open the file in a browser window, an XSL stylesheet is loaded which converts the XML to HTML for more convenient viewing.

The Profile was created initially from the conformance statements found in the WSRP Specification. The conformance statements are documented in a spreadsheet on the OASIS WSRP site. The conformance statements were boiled down into Test Assertions and documented in the Profile.

The Profile is read at runtime by the Analyzer to determine which Test Assertions should be executed. Test Assertions which are "enabled" are executed, the rest are skipped.

Types of Test Assertions

There are 3 types of Test Assertions. The primary focus the V1 Conformance Test Kit is to verify the message content of a running WSRP service. As a result, the vast majority of Test Assertions are of the Message type.

• Discovery -- Verifies UDDI entries
  • WSRP Profile has no UDDI assertions
• Description -- Verifies WSDL entries
  • WSRP Profile has some WSDL assertions
• Message -- Verifies message content in the log file
  • WSRP Profile has many message assertions
  • Primary focus of test kit.

If the configuration options for the Analyzer requires WSDL processing, then the Analyzer reads the WSDL and execute the Test Assertions from the Profile that are in the description artifact.

After the WSDL is evaluated, the Analyzer processes the message log file one message at a time. In general, for each message in the log file, the Analyzer executes each enabled Test Assertion in the Profile, although the entryType attribute results in some high level filtering. Test Assertions which have an entryType of requestMessage are executed only for requests, those having an entryType of responseMessage are executed only for responses, and those with an entryType of anyMessage, are executed for all messages.

A Test Asssertion evaluates each artifact passed to it, and generates a result. The result can be Pass, Fail, or Not Applicable.

The Analyzer finds the proper Java class to execute for a Test Assertion based on a naming convention where the Java class has the same name as the assertion id in the Profile.

Sample Test Assertion

Elements and Attributes of a Test Assertion

Many of the elements and attributes in the Test Assertion are not used, but the ones that are important are:

Programming New Test Assertions

Introduction

The following sections provide an overview of the organization of the Java classes which makeup the test kit. The test kit extends the WS-I implementation and relies on the behaviour of existing base classes wherever possible.

Factory

There is an extension point in the WS-I framework which allows us to insert our own factory implementation during initialization. We use these factories to cause our specialized classes to be constructed. The wsi.properties file specifies our factory:

WS-I Base Validator

Refer to the diagram below. The classes in yellow are the WS-I base classes and the classes in white are the WSRP extensions. The WSRPValidatorFactoryImpl creates the WSRP specific extensions when the Analyzer is started. The WSRPMessageValidatorImpl class plays an important role in the execution of Test Assertions because there is only one instance of this class for all message Test Assertions. The WSRPMessageValidatorImpl instance is used to share data between Test Assertions or between messages. This is discussed in more detail in a subsequent section.

In the WSRPTK, there are very few Test Assertions which deal with the WSDL. The vast majority of Test Assertions are focused on examining the messages in the log file. In general, there is a symmetrical treatment of message artifacts and description artifacts. For the next few sections, only the structure and relationships among the classes involved in the message processing side of the system are discussed. Classes which handle description artifacts may be included on the diagrams, but are not distinguished further.

Extending AssertionProcess

Refer to the diagram below. Again, the yellow classes are WS-I base classes, and the white classes are WSRP specific. The small class, RP0090, at the bottom of the diagram is an example of a Test Assertion.

Ultimately, all Test Assertions are derived from AssertionProcess. This class declares the validate() method that all Test Assertions must implement. The AssertionProcess class also declares a member variable to hold an instance of BaseValidatorImpl.

WSRPTK introduces 2 intermediate classes between Test Assertions and AssertionProcess. WSRPBaseAssertionProcess is used to implement convenience methods such as pass(), fail(), etc, for setting the result of a Test Assertion, regardless of whether the Test Assertion is on the message artifact side or the description artifact side.

Constructor Hack

WSRPMessageAssertionProcess extends the WSRPBaseAssertionProcess in order to declare the instance of the specialized WSRPMessageValidatorImpl to which all the Test Assertions need access for data sharing. The BaseValidatorImpl is responsible for parsing and loading the message log file and then creating and invoking the Test Assertions in the Profile. When a Test Assertion Java class, e.g. RP0090, gets initialized, the BaseValidatorImpl passes an instance of BaseMessageValidatorImpl into the constructor. The Test Assertion calls the constructor on it's superclass, WSRPMessageAssertionProcess, passing the BaseMessageValidatorImpl. The superclass forces a cast of the BaseMessageValidatorImpl to WSRPMessageValidatorImpl and saves it in a protected member variable called validator.

This hack is necessary for historical reasons. The original WS-I framework was organized so that all the Test Assertion classes were inner classes of the WSRPMessageValidatorImpl class, and so they had direct access to its methods and state. At some point, the architecture was changed so that the Test Assertions were separated into individual files and this hack became necessary to preserve access to the specialized methods and state of the WSRPMessageValidatorImpl.

WSRPMessageValidatorImpl

As explained above, there is a single instance of this class which is available to all Test Assertions through the use of the protected variable validator. This class offers many convenience methods as well as saving state between messages. Let's look at the following Test Assertion:

This Test Assertion looks at a getMarkupRequest and has to check the value of window state against a set of valid window states which were supplied in the getServiceDescriptionResponse. In the following code fragment, RP0510 uses convenience methods from the WSRPMessageValidatorImpl to locate the offeredPortlet from the saved getServiceDescriptionResponse.

This Test Assertion relies on the fact that another Test Assertion, RP0320 saves the serviceDescriptionResponse. Note that if a log file is analyzed that does not contain the serviceDescriptionResponse message, then any Test Assertion that relies on it will fail.

The WSRPMessageValidatorImpl has many methods which are beyond the scope of this tutorial. The reader is encouraged to examine the source code. The following diagram shows the WSRPMessageValidatorImpl and all the classes it references. Not all the methods are visible.

Choosing a Base Class to Extend

There are several choices as to which class to extend to create a new Test Assertion. The WSRPBaseAssertionProcess class, the BaseRP class, CustomItemChecker, or an existing Test Assertion. The diagram below shows several Test Assertions and the classes they extend. CustomItemChecker is specialized and is used to parse the markup have a listener called when certain things are found. The BaseRP class checks the message type and then delegates handling of the message to the Test Assertion subclass. This design pattern was introduced after many Test Assertions were already implemented, so you may discover Test Assertions that would have benefited from this approach.

Worked Example

Let's look at a simple Test Assertion.

Here is the source code:

This Test Assertion extends WSRPBaseAssertionProcess and implements the validate() method. It also implements a constructor so it can establish a handle to the WSRPMessageValidatorImpl class. It uses a class called NodeUtils which implements various shortcuts to XPATH for finding nodes in messages.

Specialized Portlet

Introduction

The purpose of the Specialized Portlet is to cause certain conditions to occur so that the Analyzer can look a the messages and detemine if the Consumer handled the situation according to the spec. The Specialized Portlet generates markup with particular values that the Analyzer can look for in the message log.

The Specialized Portlet works by reading a file called flow_script.xml which tells it which JSP file to load. The JSP file actually contains the markup that the Specialized Portlet supplies on a getMarkupRequest. The flow_script.xml file allows test cases to be repeated a particular number of times.

Here is a sample of the flow_script.xml file:

flowscript.xml

The SpecializedPortlet reads an XML file, called flow_script.xml, which tells the portlet which tests to execute. The test cases are actually represented by JSP files. Here is a sample of the flow_script.xml file:

A Specialized Portlet JSP typically contains a form and has javascript logic that causes the form to be automatically submitted. This allows the Specialized Portlet to execute all the tests without further human intervention.

Specialized Portlet Example

Let's look at an example:

Here is the JSP file for this test (line feeds have been added to improve readability):

Notice that the rewrite URL contains navigationalState. Here is the Test Assertion code that examines the log:

The Test Assertion looks at the value of an interaction parameter for the value that the navigationalState should contain.

WSRP Conformance Test Kit Tutorial

Click here to start