 |
| Titel |
Testing Conformance to Standards: Notes on the OGC CITE Initiative |
| VerfasserIn |
Lorenzo Bigagli, Fabrizio Vitale |
| Konferenz |
EGU General Assembly 2010
|
| Medientyp |
Artikel
|
| Sprache |
Englisch
|
| Digitales Dokument |
PDF |
| Erschienen |
In: GRA - Volume 12 (2010) |
| Datensatznummer |
250041844
|
|
|
|
|
|
| Zusammenfassung |
| In this work, we report on the issues and lessons learnt from our recent experience on assessing service compliance to OGC geospatial standards.
Official conformity is warranted by the OGC Compliance & Interoperability Testing & Evaluation (CITE) initiative, through a centrally managed repository of tests, typically developed via initiatives funded by external sponsors. In particular, we have been involved in the ESA-led Heterogeneous Missions Accessibility Testbed (HMA-T) project.
HMA-T objectives included the definition of specifications (and related compliance tests) for Earth Observation (EO) Product discovery and access. Our activities have focused on the EO and Catalogue of ISO Metadata (CIM) Extension Packages (EPs) of the ebRIM Application Profile (AP) of the Catalogue Service for the Web (CSW) OGC standard.
Our main contributions have regarded the definition of Abstract Test Suites (ATS's) for the above specifications, as well as the development of Reference Implementations (RIs) and concrete Executable Test Suites (ETS's). According to the state-of-the-art, we have implemented the ETS's in Compliance Test Language (CTL), an OGC standard dialect of XML, and deployed the scripts onto the open-source Test Evaluation And Measurement (TEAM) Engine, the official OGC compliance test platform.
A significant challenge was to accommodate legacy services, that can not support data publishing. Hence, we could not assume the presence of control test data, necessary for exhaustive assessment. To cope with this, we have proposed and experimented tests for assessing the internal coherence of a target service instance.
Another issue was to assess the overall behavior of a target service instance.
Although quite obvious, this requirement proved to be hard (if unviable) to implement, since the design of the OGC catalogue specification is multi-layered (i.e. comprised of EP, AP, binding and core functionalities) and, according to the current OGC rationale, ATS/ETS at each layer are supposed to only assess layer-specific functionalities. We argue that a sequence of individually correct "steps" may not result in an overall correct "route".
In general, our experience suggests that an ATS is conveniently modeled as a multi-dimensional structure, in that compliance with a (multi-layered) specification may be partitioned into several orthogonal axes (e.g. operation, binding, data model). Hence, the correspondence between Abstract and Executable Test Cases (ATC, ETC, respectively) is not simply biunivocal, but generally an ATC maps to a number of ETCs (or to a single, "multi-modal" ETC), whose actual run-time behavior depend on the intended point of intersection of the ATS axes.
This suggests possible benefits of an Aspect-Oriented extension of the ATC/ETC/CTL conceptual model.
We identified several other open issues in the current CITE framework: noticeably, the lack of support for the test development phase (the design of the current tools seems more oriented to the certification use-case).
Other results we obtained include: the definition of best practices for improved ETS/CTL documentation and the implementation of functionalities for its extraction and formatting; improvements to the readability of test logs and implementation of appropriate log consolidation functionalities in the TEAM Engine; comments and bug reports on the CSW 2.0.2 ETS. These results have been appropriately contributed to the relevant stakeholders.
Besides, this work has provided us with new insights into the general OGC specification framework, particularly into the rationale for modular specifications. |
|
|
|
|
|
|