Home

Try it

HTTP test forms

RMI client

Documentation

Installation

Javadocs

Configuration

Overview

Sample files

Webapp

Middleware

Bucket99 driver

Query translator

RMI proxy driver

APIs

Client API

Collection API

DTDs

Query language

Collection metadata

Bucket report

Browse report

Access report

Download

Complete webapp

JAR only

Python modules

3rd-party software

Source code

Other

Servlet URLs

RMI registry names

Change log

Terms of use

This is the middleware's change log, listed in reverse chronological order and grouped by middleware release. Changes to related files (e.g., XML DTDs) and related components (e.g., the Bucket99 driver) are also noted. More detailed information can be found in the javadoc documentation for class edu.ucsb.adl.middleware.Version.

2.4.2

• Minor bug fix in FileLoader.
• Added query translation paradigms Spatial_OracleSpatial and Textual_OracleText.

2.4.1

• Added servlet FootprintServlet to replace the (now defunct) Xerox PARC map service. The servlet requires a base map in the form of a JPEG image; a suitable one is included in the webapp.
• Added result set sharing. "Sharing" here means that a newly submitted query that is textually identical to a previous query associated with an existing result set is not executed, but instead simply returns the existing result set; the existing result set's age is effectively reset to zero. Note that result set sharing introduces a number of considerations, and for that reason it is optional and disabled by default. See the documentation for the main middleware configuration file's query.result_set_sharing property for more information.
• To support greater control over query translation and optimization, the Python query translator now passes the query ranking method and result limit to the Python script as variables rankingMethod and maximumResults, respectively.
• Added support for bucket report processing: see classes BucketReport and StandardBuckets.
• RMI modifications: when unbinding RMI names of the middleware services, if the middleware created the RMI registry, the registry is destroyed. Also, when the RMI control servlet is destroyed by the servlet container, RMI names are unbound. The effect of these two changes is that when the middleware web application is stopped, the RMI server automatically shuts down as well, and when the application is restarted or reloaded, the RMI server automatically restarts and no longer hangs.
• Summary of configuration file changes:
  • In the Tomcat configuration file, add servlet footprint and configure as indicated in the sample configuration file. Modify the bucket99-status servlet's map_server_url parameter to refer to the footprint servlet.
  • In the main middleware configuration file, modify property status.map_server_url to refer to the footprint servlet. Add property query.result_set_sharing.
• Configuration file patchfiles.

2.3.1

• The middleware now requires Berkeley DB Java Edition. (It's included in the webapp.)
• Added metadata report caching. Caching is implemented as a kind of filter that sits between the middleware and collection metadata drivers (and, indirectly, query drivers). Caching is configurable per collection and per metadata view via new properties in collection-specific middleware configuration files. Cached reports are stored in Berkeley DB databases; collections can use separate cache databases, or they can share the same cache directory. Note that there is currently no mechanism by which cached reports are ever invalidated. Added servlet cache_control (CacheControlServlet), which provides the abilities to preload and empty caches.
• Added the identifiers service (IdentifiersService), which returns the identifiers of all items in a collection. Collections must now implement a fourth interface, IdentifiersDriver. Added Bucket99 driver (IdentifiersDriver) and RMI proxy driver (IdentifiersDriver) implementations of this interface.
• All servlets that formerly returned HTTP status code 204 (no content) now return status code 200 (OK) and return the plain text "DONE". This should keep web browsers a little happier.
• Both the reference service and the sample RMI client now allow a remote middleware server to be referred to by just a hostname and optional port number, in which case the service prefix is assumed to be "/edu/ucsb/adl/middleware/rmi/", which it is by default.
• Minor changes for compatability with Java 1.5 and servlet API 2.4.
• Summary of configuration file changes:
  • In the Tomcat configuration file, add servlets identifiers, cache_control, and error. Add <error-page> elements to enable the error servlet as indicated in the sample configuration file. Also, newer versions of Tomcat seem to ignore the rmi_control servlet's empty <load-on-startup/> element; replace the element with an element whose content is a positive integer.
  • In the main middleware configuration file, add properties identifiers.gatekeeper.class, identifiers.gatekeeper.argument, reference.identifiers.driver.gatekeeper.class, reference.identifiers.driver.gatekeeper.argument, and status.cache_control_servlet_url. Also, replace any absolute URL given for the status.cancel_servlet_url property (e.g., "/middleware/cancel") with the relative URL "cancel", and similarly for the status.results_servlet_url property.
  • In the RMI server configuration file, add property identifiers.name.
  • In collection-specific middleware configuration files, add properties identifiers.driver.class, identifiers.driver.gatekeeper.class, identifiers.driver.gatekeeper.argument, metadata.cache, metadata.cache.directory, and metadata.cache.views.
  • In RMI proxy driver configuration files, add property identifiers.name.
  • In Bucket99 configuration files, add properties identifiers.log_file, identifiers.database_driver_class, identifiers.database_url, identifiers.database_properties, and identifiers.query.

2.2.2

• The middleware now requires Tomcat version 4 or later.
• Added support for GZIP compression of query results. Results can be compressed initially (i.e., when first added to a result set) as well as when they haven't been accessed for a specified period of time. Accessing results through the RMI interface preserves compression; accessing results through the HTTP interface causes the results to be uncompressed (but they may later be re-compressed, of course). Added middleware configuration properties result.initial_compression and result.compression_age.
• In the HTTP client interface, added a get-query operation to the results servlet that returns the query associated with a result set. Similarly, in the RMI interface, added method getQuery to RMI interface class ResultSet.
• Added a general macro substitution capability to all configuration files; see ExtendedProperties.
• Added support for specifying different units of measure in all time duration properties (result_set.lifetime, etc.).
• Added the ability, in both the web application and the standalone RMI server, for the middleware to use a local, middleware-created RMI registry. By default, the middleware webapp distribution automatically creates a registry at startup that listens on the standard RMI registry port; this obviates the need to separately run rmiregistry. Added the create_registry RMI server property.
• Completed the previously partial support for multiple instances of the middleware running in the same Java VM. Quantities such as result sets and daemon threads that were previously allocated VM-wide are now localized to each middleware instance; as a side effect of this change, removing a middleware web application now immediately frees all of its resources, result sets in particular (assuming that the middleware RMI names have been unbound first, of course). The driver interfaces (CollectionDriver, MetadataDriver, and QueryDriver) now require an additional argument identifying the middleware instance. The Bucket99 driver's StatusServlet now requires a middleware_configuration_file initialization parameter.
• All servlets now look for the middleware_configuration_file initialization parameter first as a servlet parameter and second as a context-wide parameter. As a result, the parameter can now be specified in just one place in the Tomcat web.xml configuration file.
• Reorganized the webapp distribution. Removed source code and added license statements. There's now a separate source distribution that includes all source code, build scripts, etc.
• Integrated Jython into the webapp distribution. Added middleware configuration properties jython.home, jython.cachedir, jython.module_path, jython.class_path, and jython.jar_path.
• Added a new bucket type, the relational bucket type, that supports description of arbitrary relationships between collection items; relationships may cross collection boundaries. A corresponding relational constraint type supports searching for items having a named relationship to a target item. Added query translation paradigms Relational_Integer and Relational_String.
• Added the "spatial fallback" capability described in version 2.1.1. Where higher-order geometric types such as spherical polygons and polylines are specified in ADL-bucket-report.dtd and ADL-query.dtd, bounding boxes must now be specified as well. Polygonal spatial constraints are automatically approximated by bounding box searches in paradigms that don't support polygons. (The changes to the above-mentioned DTDs are not upward-compatible, strictly speaking, but they should be in practice.)
• In the Bucket99 driver, added BlobMetadataSubdriver, a metadata subdriver for reports stored in the database in BLOB, CLOB, and text columns.
• In the Bucket99 universal query translator, reworked the handling of field-level constraints with respect to nesting of field adaptors; this is an important but very obscure change, and the only effect it should have on existing configuration files is that an Adaptor_Concatenation paradigm nested within another Adaptor_Concatenation paradigm should in most cases be replaced by an Adaptor_IndivisibleConcatenation paradigm. Added the rewriteSetOpsAsSubqueries option. Added support for Oracle-format dates in paradigms Temporal_BeginEnd and Temporal_SingleDate. Added support for multiple vocabularies in paradigm Hierarchical_Constant. Added paradigms Adaptor_IndivisibleConcatenation, Adaptor_Switch, Adaptor_TemporalIsContainedInRewriter, Adaptor_TermMapping, Hierarchical_StringEquality, Spatial_OGCSimpleFeatures, Spatial_PostgreSQLBoxNoCrossing, Temporal_Constant, Textual_MySQLFulltext, and Textual_SuffixTable.
• Reworked the status-reporting mechanism of the Bucket99 database connection pool. The new mechanism allows the status to be retrieved even if the pool is hung (because, for example, the database is hung, not that that ever happens...).
• In GenericQueryMetadataSubdriver, added two operators. The "variable insertion" operator inserts the value of a variable (such as the collection name) in the report. The "query-based conditional inclusion operator" inserts its contents in the report only if a specified query returns at least one row.
• Added SpatialRanker, a ranker that ranks query results by similarity to the query region.
• Summary of configuration file changes:
  • All configuration files now support a general macro substitution capability. Because at-signs are now treated as substitution operators, at-signs that are to be interpreted literally must now be escaped. See ExtendedProperties.
  • In all configuration files, units of measure can now be given for time duration properties.
  • In the Tomcat configuration file, add a middleware_configuration_file parameter to the bucket99_status servlet's configuration (but then, as noted previously, the parameter can now be specified in just one place in the file).
  • In the main middleware configuration file, add properties jython.home, jython.cachedir, jython.module_path, jython.class_path, jython.jar_path, result.initial_compression, and result.compression_age. To property rankers add declarations for spatial ranking. See the sample middleware.conf for the suggested way of doing this.
  • In the RMI server configuration file, add property create_registry.
  • In Bucket99 configuration files, the translator.python_module_path, translator.python_class_path, and translator.python_jar_path properties can generally now be left blank (assuming that the Jython properties in the main middleware configuration file have been appropriately set).

2.1.1

• The middleware now requires JDK 1.4.
• In the Bucket99 universal query translator, added support for NULL columns (or more accurately, the SQL UNKNOWN value). Clarified and made consistent the programming model used by the translator and paradigms. Added paradigms Adaptor_Relationship, Hierarchical_Constant, Identification_Integer, Identification_String, Temporal_BeginEnd, Temporal_IntegerYear, Temporal_SingleDate, Textual_InformixVerity, and Unsupported.
• Significantly reworked the standard item-level ADL metadata reports/views. The scan view was deprecated in favor of the bucket view (see ADL-bucket-report.dtd), which supports arbitrary bucket mappings. The browse view (ADL-browse-report.dtd) was modified slightly. The access view (ADL-access-report.dtd) was completely rewritten. The full view was deprecated entirely. The new view names, for identification purposes, are adl:bucket, adl:browse, and adl:access. Deprecated, in package edu.ucsb.adl.reports, classes AccessReportConverter, FullReportConverter, FullReportIdentifierRenamer, ScanReportConverter, and ScanReportIdentifierRenamer, and in package edu.ucsb.adl.bucket99, classes BrowseQueryMetadataSubdriver and ScanQueryMetadataSubdriver.
• Added support for arbitrary, per-collection and even per-holding metadata views. In addition, all holdings are now required to support the three standard ADL views mentioned above. In MetadataService, changed the view argument's data type from an enumeration to a string and added method performViewsService, which returns a list of the views supported by a holding. MetadataServlet exposes the performViewsService method as the fictitious view adl:view-list. Reworked the Bucket99 item-level metadata-related configuration properties. Property metadata.views is now a list of property name prefixes; for prefix P, properties P.name and P.class specify the view's name and subdriver class. The various subdriver classes follow the same pattern in retrieving subdriver-specific properties.
• Implemented a high-performance, DTD-based, table-driven validating XML parser. All XML processing is based on the new parser, except for query parsing (which will be in the next release; see below).
• Added support for validation of returned item-level metadata. Validation may be enabled at the middleware level (which it is, by default) and by individual Bucket99-hosted collections (which it is not, by default). The standard metadata views are validated against the syntax and semantics of the appropriate DTDs; non-standard views are simply checked that they appear to be XML. (Validation of collection-level metadata will be added in a future release.) Added a pair of middleware configuration properties metadata.validation and query.validation and a pair of like-named Bucket99 configuration properties.
• Query results are now triplets of standard ADL metadata views as opposed to being single (now deprecated) scan views. Programmatically, query results are now Result objects, which are serialized across RMI connections. There's no change to the results service itself, but in ResultsServlet the get and multiget operations now require a views parameter that indicates which view(s) are to be returned. The format of the returned XML from those operations has changed accordingly.
• Renamed the standard ADL buckets. The standard bucket names are now prefixed with "adl:", e.g., the originators bucket is now adl:originators. ADL-bucket-report.dtd provides significantly more specification of the semantics of buckets, sub-buckets, and metadata mapping.
• Reworked the identifier renamers and report generators so that DTD URLs are no longer necessary (and thus configuration properties access_report_dtd_url, etc., are deprecated).
• Added support for logging to Java Loggers. The interpretation of the Bucket99 collection.log_file, metadata.log_file, and query.log_file configuration properties has been modified so that Java logger L may be referenced using the notation "j2se-logger:L". (This is admittedly a bit of a hack; the Bucket99 driver simply logs all records at the INFO logging level and doesn't really take advantage of Java's logging features.)
• Improved error messages in various places.
• The middleware web application now includes the Bucket99 configuration files for a sample collection. The sample collection directly maps FGDC metadata to the standard ADL buckets and includes a non-ADL-standard FGDC metadata view.
• XML character encodings are now stripped out of all returned metadata, on the theory that they're either redundant or wrong. (Note that the middleware's HTTP interface uniformly specifies and employs the UTF-8 encoding using HTTP headers.)
• Summary of configuration file changes:
  • In the Tomcat configuration file, delete the scan_report_dtd_url parameter from the results servlet's configuration.
  • In the main middleware configuration file, delete properties full_report_dtd_url and access_report_dtd_url. Add properties metadata.validation and query.validation.
  • In Bucket99 configuration files, delete properties browse_report_dtd_url and scan_report_dtd_url. The collection.log_file, metadata.log_file, and query.log_file properties have a new, mostly upward-compatible interpretation. Add properties metadata.validation and query.validation. Property metadata.views has changed to a list of prefixes. The property pattern "metadata.view.property" is replaced by "prefix.property"; property "prefix.name" is new.
  • In RMI driver configuration files, delete properties full_report_dtd_url and access_report_dtd_url.
• This version of the middleware is incomplete. The addition of a spatial constraint fallback capability (which will allow polygonal spatial constraints to be automatically approximated by bounding box searches, and which requires several interface changes) will complete middleware major version 2.

1.3.3

• New Bucket99 query translator, UniversalTranslator.py. The translator features a well-defined and documented paradigm interface, a library of paradigms, and many other new features. The translator configuration file format has changed significantly. This completes a change begun with version 1.2.1. UniversalJoinTranslator.py is deprecated.

1.3.2

• Renamed all configuration files for greater consistency and clarity. The main middleware configuration file (formerly middleware) is now middleware.conf and the collection-specific middleware configuration file (also formerly middleware) is now drivers.conf. Other renames: bucket99 to bucket99.conf; translator.py to query-translator.py; rmi-proxy-driver to rmi-proxy-driver.conf; and rmi-server to rmi-server.conf.

1.3.1

• A subdirectory of the master collection directory is now interpreted as a collection if and only if it does not contain a file named "OFFLINE". Added servlet edu.ucsb.adl.middleware.CollectionAvailabilityServlet which allows collections to be enabled and disabled dynamically.
• In the Bucket99 driver, added the ability to preemptively cancel a query. (This works only if the database's JDBC driver supports it, of course.)
• Also in the Bucket99 driver, a separate Java properties file containing database connection-related properties (e.g., username and password) may now be specified. Added Bucket99 configuration properties metadata.database_properties, query.database_properties, and for each vocabulary V, V.vocabulary_database_properties and V.usage_database_properties.
• Database names are now displayed in the Bucket99 status page. Added property query.database_name.
• Minor upgrade to servlet API 2.3.

1.2.1

• Modified the middleware's query language, ADL-query.dtd. Added support for field-level constraints and identification constraints. These new capabilities will be complete with the completion of the new Bucket99 query translator.
• Minor modifications to status.css and bucket99-status.css.

1.1.4

• Internationalization support. When loading XML documents from URLs, the Bucket99 driver now sets the character encoding based on the URL connection's MIME content type, if available. Encoding declarations in XML headers, if present, are ignored and removed; the middleware's driver interfaces specify that it is the drivers' responsibility to remove the declarations. All XML returned through the middleware's HTTP interface is now encoded in UTF-8.

1.1.3

• Additional fixes (more like hacks) to support the Jython implementation of Python when the middleware is packaged as a JAR file within a web application. Added the (Bucket99) translator.python_class_path and translator.python_jar_path properties. Renamed the translator.python_path property to translator.python_module_path.

1.1.2

• Numerous small changes that allow the middleware to be packaged as a self-contained web application. For example, the locations of configuration files can now be specified relative to the web application directory.