 |
| Titel |
Aggregation of Grid and HPC resources for running huge experiments in climate and weather prediction |
| VerfasserIn |
Antonio S. Cofino, Carlos Blanco Real, Valvanuz Fernández Quiruelas |
| Konferenz |
EGU General Assembly 2011
|
| Medientyp |
Artikel
|
| Sprache |
Englisch
|
| Digitales Dokument |
PDF |
| Erschienen |
In: GRA - Volume 13 (2011) |
| Datensatznummer |
250057396
|
|
|
|
|
|
| Zusammenfassung |
| The explosion of computer power during the last two decades has increased the number and types of computational resources a researcher has access to. The heterogeneity of architectures and systems along with the different configuration schemas has led to a situation where users find very complicated taking advantage of all the resources available.
In order to solve this problem, a new computing paradigm called “Grid Computing” appeared as an alternative for flexible access to heterogeneous and geographically distributed computing resources. Typically, creating a Grid infrastructure requires coordination among cluster administrators and the installation of a middleware to unify the access to them.
Nowadays, users have access to some resources, like HPC infrastructures, which are not shared using any Grid middleware. Under this situation, the user has to manage separately the access to each non-Grid infrastructure, which is quite cumbersome.
The challenge of this work is to unify the access to Grid and non-Grid resources, allowing researchers to perform huge experiments making use of all the available resources. To achieve this goal, common LRMs used in HPC infrastructures: PBS/Torque, SGE and SLURM and Grid middleware: gLite, GT2.4 and GT4/5 has been considered.
Although nowadays there are several Grid meta-schedulers, none of them has the capabilities needed. To solve the problem of unified access to Grid and non-Grid resources, the GridWay meta-scheduler and his plugin oriented architecture has been used.
Moreover, as Gridway doesn’t provide access to non Grid resources, new GridWay plugins which grant access to non-Grid resources (PBS/Torque, SGE and SLURM) have been developed. These plugins are based on the raw access to clusters provided to the users; typically a ssh account and LRM’s CLI.
As an application of this tool it has been integrated into the WRF4G framework. WRF is high demanding application of HPC resources. WRF4G is a framework that allows the management of the execution of huge experiments, consisting in thousands of jobs (HTC). With this tool we are providing a homogeneous access to Grid and non-Grid resources to the WRF4G users.
The Earth Sciences community, in particular Climate & Weather community, is high demanding HPC user community, therefore it will benefit from this unification of Grid and non-Grid resources. |
|
|
|
|
|
|