Chapter 1: Introduction
Peter Lynch, Project Leader
The HIRLAM Project is a cooperative project
for development of NWP models. The primary goal is to develop
computer models for analysis and prediction
which are state-of-the-art. These models are used as the basis of
operational short range predicition in the participating institutes.
The HIRLAM 4 Project commenced on 1 January, 1997
and ran for three years, finishing on 31 December, 1999. The framework
for the Project was a Memorandum of Understanding signed by the
participating Meteorological Institutes. Overall responsibility for the
Project rested with the Management Group, and the work organized in
sub-projects, each with a coordinator. The research and development
work was specified in a detailed Scientific Plan, which was revised
annually.
This report summarizes the activities and achievements of the
HIRLAM 4 Project over the period 1997-1999.
Memorandum of Understanding
The central purpose of the HIRLAM 4 Project was to
provide participating weather services with the best possible numerical
analysis and prediction system for short-range, high-resolution
forecasting. To achieve this overall goal, several key requirements
were identified in the MoU:
- Provision of a comprehensive forecast production
system suitable for operational use.
- Achievement of accuracy at
least as high as that of comparable alternative European models.
- System suitable to be run locally, with high spatial resolution.
- Availability of a modelling group with a complete range of NWP expertise.
- Execution of a comprehensive programme of research and development,
including research in the area of meso-scale modelling.
The Members of HIRLAM 4 were the participating
meteorological institutes:
- The Danish Meteorological Institute (DMI),
- The Finnish Meteorological Institute (FMI),
- The Icelandic Meteorological Office (IMO),
- The Irish Meteorological Service (IMS),
- The Royal Netherlands Meteorological Institute (KNMI),
- The Norwegian Meteorological Institute (DNMI),
- The National Meteorological Institute of Spain (INM),
- The Swedish Meteorological and Hydrological Institute (SMHI).
The French Meteorological Service (Météo-France) had a
cooperation with the HIRLAM 4 Project, and
participated in research related to several of the HIRLAM 4 Sub-projects.
The HIRLAM Council comprised the Directors
of the participating meteorological institutes,
including Météo-France. Overall authority for
the Project was vested in the Council. The Council met seven times
during the course of the Project.
The HIRLAM Advisory Committee (HAC)
consisted of a representative from each of the Members, including
Météo-France. HAC held six meetings during the Project. The
Scientific Plan was prepared in accordance with the HIRLAM Advisory Committee guidelines and after
consultation with the participating scientists.
Major Achievements of the Project
The major highlights of progress in research and development are
identified here. Fuller details are given in the following sections.
The development of a variational data assimilation system was a major
focus of research in the Project. A series of parallel data
assimilation and forecast experiments were carried out to compare the
performance of 3D-Var and OI. Forecasts based on 3D-Var assimilation
were consistently better than the forecasts based on the OI
assimilation. Development work in 4-dimensional variational data
assimilation (4D-Var) continued with increased emphasis. The target
for the Project was to demonstrate the technical feasibility of running
4D-Var operationally. The main highlights in data assimilation were:
- The superiority of the 3D-Var system in comparison with OI was
demonstrated through parallel tests run over several months.
- The HIRLAM variational data assimilation
software package, HIRVDA, was extended to process radar and
satellite data, TOVS profiles and ground-based GPS data.
- The technical feasibility of applying 4D-Var operationally was
demonstrated by a series of experiments. Thus, the main 4D-Var
objective was achieved.
- A digital filter initialization was introduced into the reference
system, following extensive testing.
- The development work for one of the critical 4D-Var components,
the adjoint physics, advanced significantly during the Project.
- An interface to the Météo-France physical
parameterization package was developed and tested.
- An improved soil moisture assimilation scheme was developed and is
an integrated part of the ISBA scheme
- Progress was made on assimilation of TOVS retrievals produced by
the 1D-Var package, and in assimilation of scatterometer winds.
Significant improvements to the forecast model were made over a wide range of
areas. A list of the main changes in the reference system is given at the end
of this chapter. The main highlights in this area are:
- Introduction and testing of a higher-order turbulent kinetic
energy scheme, the `CBR' scheme.
- Improvements to the parameterization of surface fluxes over the
sea.
- Recoding and testing of the ISBA scheme for modelling the
interaction with the land surface.
- Introduction of a new condensation scheme, STRACO, in the reference system after comprehensive
parallel runs with satisfactory results.
- Implementation and testing of a mass-flux scheme with a `CAPE'
closure, with encouraging results.
- Testing of the Rasch-Kristjánsson scheme together with Tiedtke
mass flux and Kain-Fritsch convection schemes.
- Improvements to the HIRLAM radiation scheme.
- Development of the climate generation system using the
Hierarchical Data Format. Introduction of new basic global data
sets in the climate system.
- Substantial progress with development of the semi-Lagrangian
scheme. Introduction of an improved method for computing the
departure point of trajectories.
- A systematic study of the treatment of the lateral boundary
conditions.
- Work on development of a non-hydrostatic version of HIRLAM advanced significantly.
- A stretched grid version of HIRLAM was coded
and tested, with encouraing preliminary results.
A large number of improvements to the HIRLAM
System have been implemented, many of them substantial in nature.
These have lead to significant benefits in the HIRLAM reference system. Major system developments
include:
- Code to run the OI scheme on a distributed memory machine was developed.
- The forecast model was recoded to allow parallel execution on
distributed memory systems.
- An extranet system, HEXNET, was introduced and developed
- Utilities to facilitate local installation of the HIRLAM system were developed.
- The HIRLAM System was modified to make it
`Millennium-proof'.
- A `Version Control' mechanism was introduced to facilitate
numerical experimentation.
- A comprehensive diagnostics package was implemented in the HIRLAM reference model.
- A Java-based browser was developed for use with the HIRLAM Fortran code.
Finally, a vast number of minor code corrections were made.
As a result, substantial improvements were achieved in
operational forecasts produced by the HIRLAM
System. A small sample of verification scores appears below.
Meetings Programme
An intensive programme of Meetings and Workshops was organized during the Project.
Once each year there was an All-Staff meeting, where a large
proportion of the HIRLAM staff participated.
A full list of meetings is attached in Annex 1.
The first All-Staff Meeting of the Project was held in Dublin on 19-21
March, 1997. There were 29 participants. A full report of the meeting
appeared in Newsletter No. 27. The 1998 All-Staff Meeting was held at
Lysebu, Oslo, on 4-6 May. It was attended by 39 scientists. A report
can be found on HEXNET,
the HIRLAM extranet, and
in Newsletter No. 31. The 1999 All-Staff Meeting was held on 15-17
March in De Bilt. There were 42 participants, making it the largest
and, arguably, most successful, such meeting so far. A full report can
be found on HEXNET and in
Newsletter 33.
There were twenty-one Management Group meetings during the three-year
period. Reports of Management Group meetings were made available to
all scientists on HEXNET.
The Management Group also carried out a programme of visits to the
member countries during the course of the Project.
A large number of exchange visits took place during the period, in
connection with research cooperation. A complete list is attached in
Annex 2.
Publications
Nine issues of the HIRLAM Newsletter were
published during the Project. The standard of production of the
Newsletter was enhanced considerably. Sixteen HIRLAM Technical Reports and three major Workshop
Proceedings were issued during the period. A good number of other
publications produced within the Project, or directly related to it,
are also listed in Annex 3.
Administration
The Project was managed by the Management Group which, as of January, 1997 was
- Peter Lynch, Project Leader
- Nils Gustafsson, Deputy Leader, Analysis
- Sigbjørn Grønås, Deputy Leader, Modelling
- Gerard Cats, System Manager.
Due to pressure of other work, Sigbjørn Grønås was
obliged to withdraw from the Group in mid-Summer, 1997. Bent Hansen
Sass took over as Deputy Leader (Modelling) in September of that year.
The Project Management Group, as of December, 1999 was as follows:
- Peter Lynch, Project Leader
- Nils Gustafsson, Deputy Leader, Analysis
- Bent Hansen Sass, Deputy Leader, Modelling
- Gerard Cats, System Manager.
The staff complement of the Project is equivalent to 22 scientists.
The total hours worked during the year, as reported from the member institutes,
were monitored by the Project Leader, and were well in excess of
the minimum levels specified in the Memorandum of Understanding.
Access to the ECMWF Computer System for research and development work was
arranged through the provision of resources for a Special Project.
The HIRLAM Secretariat consisted of the Project Leader with
supporting secretarial assistance provided by Met Éireann.
The expenditure on the Project was in accordance with the budget, which was
agreed annually by Council.
Sample Verification
Verification of the operational forecasts, from the seven centres where
the HIRLAM model forms the basis of operations, are
produced routinely. A selection of these scores has been collected each year
for the period January/March, and they have been reviewed in HIRLAM
Newsletters. Here we present a small sample of scores generated by the
HIRLAM model run at Met Éireann in Dublin.
The figure shows the annual average scores for 24-hour forecasts of
wind speed, pressure and temperature for each year of the HIRLAM 4 Project. Left panels are scores using the
EWGLAM list of 55 stations over Europe. Right panels show scores using
12 Irish synoptic stations. It is clear from the figures that there has
been a clear trend towards higher accuracy over the period of the
Project. Over Ireland, the region most critical for local operations,
forecasts of all surface parameter forecasts have improved consistently
over the three-year period.
A wealth of evidence from other operational centres confirms that there
have been significant enhancements to the quality and accuracy of operational
forecasts as a result of assimilation and modelling developments during the
HIRLAM 4 Project.
Figure: Annual Trends in Forecast Skill
Detailed Research Progress Report
The Scientific Plan was divided into three sections, dealing with Data
Assimilation, Modelling, and System Management. Progress in each area is
summarized here. Fuller details are found in the following Chapters,
written by the Management Group members.
A comprehensive report by Nils Gustafsson on analysis appears below in
Chapter 2. A full account of progress
in the area of Modelling, written by Bent Hansen Sass, follows in Chapter 3. The developments in Systems
Management and Embedding are described by Gerard Cats
in Chapter 4.
1. Data Assimilation and Data
Development of a variational data assimilation system was a major focus
of research in the Project. In fact, research on this had already begun
in 1995. One of the principal arguments for variational assimilation
is the feasibility of utilizing a wide range of non-conventional
observations, such as radar and satellite data. Moreover, by applying
4-dimensional variational data assimilation (4D-Var), it is possible to
include the time dimension explicitly.
Most data assimilation efforts during the HIRLAM 4
project were devoted to the development and testing of the HIRLAM variational data assimilation, and substantial
progress was made. The HIRLAM 3D-Var and 4D-Var
software system (HIRVDA) has been developed within a carefully
coordinated research programme involving staff from several
participating institutes. By the end of the Project, the superiority of
the 3-dimensional variational data assimilation (3D-Var), in
comparison with OI, had been demonstrated through parallel tests run
over several months. Pre-operational tests during the first year of
HIRLAM 5 are likely to bring 3D-Var into the
operational HIRLAM reference system.
The HIRLAM variational data assimilation software
package, HIRVDA, was developed to allow for the inclusion of radar and
satellite data, TOVS profiles and ground-based GPS data. In addition,
the technical feasibility of applying 4D-Var operationally was
demonstrated by a series of experiments. Thus, the main 4D-Var
objective was achieved.
Research efforts have also involved progress in soil and surface
parameter data assimilation, development of flow-dependent analysis
structure functions, and the improvement and maintenance of the OI
based reference scheme.
At the commencement of the Project, the data assimilation of the
reference HIRLAM system was based on analysis by
Optimum Interpolation (OI) and initialization by a non-linear normal
mode method. Towards the end of the period, a digital filter
initialization was introduced, following extensive testing.
1.1. Variational data assimilation
1.1.1 Three-dimensional variational data assimilation (3D-Var)
Significant progress was achieved with regard to development of
advanced 3D-Var structure functions, based on generalized linear
balance constraints. The new structure functions have been tested and
compared with the standard version structure functions in a parallel
run. The results indicate a marginal positive impact.
The HIRLAM 3D-Var, and also the 4D-Var, have been
developed with an incremental formulation. This allows for
exploitation of advantages of both the spectral and grid-point model
versions. Parallel assimilation and forecast experiments have shown
that this incremental formulation performs satisfactorily.
The software package for handling observational data (OBSPROC) in the
Comprehensive Memory Array (CMA) format, has been adapted for use in
the HIRLAM 3D-Var. Documentation of the CMA
system was carried out in collaboration with ECMWF. A software package
for observation screening, including quality control and data thinning,
was developed and tested. A general set of observation operator
subroutines for treatment of conventional data has been developed.
Non-linear, tangent-linear and adjoint versions of the observation
operators were written. A general software framework for inclusion of
remote sensing data was developed.
An optimized 3D-Var (and 4D-Var) software package, prepared for
parallel computers with explicit message passing between the
processors, was developed. Tests on both shared memory and distributed
memory machines were carried out successfully.
A series of parallel data assimilation and forecast experiments were
carried out to compare the performance of 3D-Var and OI. The results
of the tests carried out in SMHI and DMI indicated that the forecasts
based on 3D-Var assimilation were consistently better than the
forecasts based on the OI assimilation. A sample of verification scores
from Lindskog et al. (2000) for mean sea level pressure and 2 meter
temperature may be found in Chapter 2 of this Report.
Single observation impact experiments have been done to validate the
efficiency of the background error constraint to spread the observed
information in space. These experiments confirmed the possibility of
applying non-zero lateral boundaries during 3D-Var.
A different approach for handling the background error constraint,
using a simple matrix transform within physical space to avoid the
inversion of the back-ground error covariance matrix, has been
investigated. This has been shown to work satisfactorily in a simple
context.
1.1.2 Four-dimensional variational data assimilation (4D-Var)
Development work in 4-dimensional variational data assimilation
(4D-Var), which started in 1995-96, continued with increased emphasis
during the HIRLAM 4 project. The target for the
Project was to demonstrate the technical feasibility of running 4D-Var
operationally.
The development work for one of the critical 4D-Var components, the
adjoint physics, advanced significantly during the Project. The
strategy of using full physics in the adjoint package was chosen for
the first phase of the 4D-Var development. The tangent linear and
adjoint models of a complete physics package (HIRLAM reference version 4.2) have been coded and a code
verification has been made. Some preliminary sensitivity experiments
using the full adjoint physics were performed and results were
encouraging.
As an alternative to full physics, the regularized and simplified
physical parameterization schemes of Météo-France have
also been considered for inclusion in the HIRLAM
4D-Var. For this purpose, an interface to the
Météo-France physical parameterization package was been
developed and tested.
The adjoints of the lateral boundary relaxation scheme have been coded
and tested. The experiments indicated that there is a possibility to
control lateral boundary conditions within the framework of 4D-Var by
the aid of observations inside the model integration area.
A weak digital filter constraint has been included as an option in the
HIRLAM 4D-Var software. The digital filter
constraint enters as an extra term in the cost function to be
minimized. This constraint was tested and compared with the standard
non-linear normal mode initialization constraint in connection with the
technical demonstration of the HIRLAM 4D-Var. The
weak digital filter constraint appeared to perform satisfactorily.
A technical demonstration of the feasibility of running 4D-Var
operationally was carried out during autumn 1999. The experiment was
run for a 4 day period without any problems. Forecast verification
scores were comparable to the corresponding 3D-Var verification
scores.
1.2. Flow-dependent Analysis Structure Functions
An efficient and simple way to introduce flow-dependent analysis
structure functions is to use flow-dependent coordinate
transformations. This idea has been developed and tested successfully
at DNMI using a successive correction system. A parallel test using
the HIRLAM 50 km system at DNMI was carried out.
A systematic reduction of the RMS errors of geopotential was found. On
the basis of those results, the method was made operational in the 50
km model at DNMI from May 1997.
Possible strategies for performing the coordinate transformation in the
3D-Var scheme were investigated, and in particular the possibility of
performing the transformation in the spectral domain.
1.3. Development and Maintenance of Optimum Interpolation (OI)
Analysis
Substantial efforts were required to ensure that the reference OI
system continued to perform satisfactorily. This is absolutely
essential, as the OI system is used in all the seven centres where
HIRLAM is run operationally. A full description of
OI work is found in Chapter 2.
Initial development of a distributed memory parallel version
of the OI analysis, using MPI for communication, was undertaken by
John van de Vegte at KNMI.
A parallel version using the SHMEM message passing library was
developed at SMHI in collaboration with SGI/Cray. This was modified
by SGI/Cray in collaboration with DWD, to use the MPI library.
Later, preliminary work to adapt this for the HIRLAM
reference OI system was started but, in view of the impending availability of the
3D-Var assimilation system, further work was suspended.
The impact of
the additional FASTEX radiosonde observations on the HIRLAM data assimilation and forecasting system was
tested in a 2 week parallel run. Averaged over the whole period, the
impact is consistently positive, especially on upper model levels.
1.4. Soil moisture analysis
An improved soil moisture assimilation scheme was developed and is an
integrated part of the ISBA scheme that has been put into operational
use at Météo-France.
1.5. Assimilation of moisture and cloudiness information
Results from the MetCast system for cloud forecasting in the very short
range at KNMI have been published in a HIRLAM
Newsletter article. MetCast has been made operational at the KNMI.
Some testing of a nudging technique for nowcasting purposes have been
done at SMHI, using cloud information mainly from satellite
information. The nudged forecast has been digitally filtered in order
to get the initial state for nowcasting. The differences between the
cloud fields in the nudged forecast and the original forecast are small
over most of the area, but significant in some places.
1.6. Assimilation of remote sensing data
Progress has been made on assimilation of TOVS retrievals, produced by
the 1D-Var package, and in assimilation of scatterometer winds. With
hindsight, it is clear that the HIRLAM 4
Scientific plan was over-ambitious with regard to the assimilation of
remote sensing data. However, substantial progress on variational data
assimilation (3D-Var and 4D-Var) will now make developments in the use
of remote sensing data easier to achieve.
1.6.1. Satellite sounding data (TOVS and ATOVS).
Development efforts with regard (A)TOVS data assimilation have been
strengthened during the HIRLAM 4 project and
several countries have initiated projects on this topic, partly with
support from external sources. Coordinated efforts are now being
undertaken at FMI, INM, KNMI, SMHI, IMO and DNMI.
The TOVS 1D-Var sounding retrieval package was implemented and tested
in the HIRLAM system. The temperature and
humidity profiles retrieved in this way are assimilated into the HIRLAM OI assimilation system. A systematic improvement
due to the assimilation was observed in verification scores.
1.6.2. Satellite Scatterometer Winds.
Impact experiments with HIRLAM and tandem ERS-1
and ERS-2 scatterometer wind data have been done. The tandem winds
clearly have a beneficial effect, but single scatterometer data does
not yield significant improvement. The improved scatterometer
processing, PRESCAT, is being included in the HIRLAM system.
1.6.3 GPS Data
At DMI, assimilation tests have been performed using retrieved GPS
temperature profiles within the HIRLAM OI
framework. DMI is also involved in an EU project for near real-time
delivery of Zenith Total Delays derived from ground based GPS
measurements. SMHI is involved in a project to assimilate the total
delay of GPS satellite to earth signals in order to improve moisture
fields. INM is investigating the possible benefit of GPS Zenith Total
Delays in NWP. No real-time space-based GPS data is being received
currently and no data assimilation of these data is performed at this
time.
1.6.4 Other Data
At INM, locally received AVHRR data are used in an optimum
interpolation scheme to generate analysed SST fields for the forecast
model. The verification scores of surface parameters show a
significant benefit of this analysis, mainly over Ireland, England,
Spain and Portugal. NOAA SST data are being evaluated for
implementation in HIRLAM at KNMI.
The NWP SAF (Satellite Application Facility) started on 1 January
1999. Requirements for satellite data processing within HIRLAM were discussed and agreed during a meeting held
in Noordwijk aan zee in November 1999 (see Newsletter 34).
2. Model Developments
Introduction
This is a summary of the report on Modelling written by Bent Hansen
Sass which appears in Chapter 3
below. A comprehensive range of modelling activities have been
undertaken during the Project. These have lead to significant benefits
in the HIRLAM reference system. A full list of the
Model Upgrades is given at the end of this chapter.
2.1. Physics
2.1.1. Turbulence parameterization
During the initial phase of the Project a non-local first order
turbulence scheme was implemented in the HIRLAM
reference system. Extensive evaluation showed that improved overall
performance was achieved with this scheme. However, it also had certain
short-comings. Therefore, attention was directed to the implementation
of a higher-order scheme. Two candidates with turbulent kinetic energy
(TKE) as a prognostic variable, were evaluated, the `CBR' scheme (Calvo
and Cuxart 1998, Cuxart et al., 1999), and the K-epsilon scheme (Perov
and Gollvik 1996).
Extensive experimentation and validation of 3-dimensional case studies
and parallel runs with the turbulence schemes was carried out. On the
basis of the results, it was decided that the CBR scheme should become
the reference turbulence scheme. The new scheme significantly
alleviates an old model problem of filling of old cyclones over
Northern Europe.
2.1.2. Surface processes
The drag coefficients were revised to make the parameterized
fluxes more realistic. After successful 3-D parallel runs it was
decided to implement these modifications along with improved diagnostic
formulae for temperature, humidity and wind speed in the unstable
planetary boundary layer.
The HIRLAM version of the ISBA scheme was recoded
to improve its structure and facilitate collaboration with
Météo-France. Several tests were carried out with the
new scheme. It had not reached a state of readiness for operational
implementation by the end of the Project. The final tests, including
the new climate generation scheme, soil moisture assimilation and the
latest version of the ISBA scheme, and also the improved surface flux
computations over sea, will be carried out in the HIRLAM 5 project prior to implementation as a new HIRLAM reference.
2.1.3. Clouds and Condensation
In early 1998, the Tiedtke mass flux convection scheme,
together with the HIRLAM 3 condensation code.
was implemented in the reference model. However, the performance of
the scheme was not entirely satisfactory. A `CAPE' closure was
implemented and tested, with encouraging results. In parallel, a
recoded condensation scheme was developed, in an effort to alleviate
reported problems. This scheme, called STRACO, was
introduced in the reference system, following comprehensive parallel
testing.
A review on the subject of cumulus parameterization in regional
forecast models (Bister, 1998) recommended the
Kain-Fritsch scheme for parameterization of convection as a good
candidate for a high resolution HIRLAM.
Encouraging results have been reported with the Kain-Fritsch scheme in
a preliminary study of a tropical cyclone (Hurricane `Flo').
At the end of 1999, the Rasch-Kristjánsson scheme had been run
together with Tiedtke mass flux scheme and with the HIRLAM version of the Kain-Fritsch scheme in the
framework of HIRLAM 4.6. More work is needed to
test the Rasch-Kristjánsson scheme and the Kain-Fritsch convection
parameterization, in particular with regard to the cloud
parameterization.
2.1.4. Radiation
An improved version of the HIRLAM radiation scheme
was developed and tested during HIRLAM 4. The
treatment of clouds was improved by parameterizing an effective radius
describing the size of cloud droplets and ice crystals. A parallel run
with the new scheme gave results comparable to the reference model.
Further testing was under way at the end of the Project.
2.1.5. Physiography
A major development of the climate generation system has taken place,
using an efficient, machine-independent format, the Hierarchical Data
Format (HDF). New basic global data sets have been employed. The new
climate generation has been implemented in the HIRLAM reference system. Tests with the ISBA surface
parameterization scheme are under way.
2.1.6. Other Physics Work
In the current HIRLAM system, the momentum drag is
parameterized from an effective roughness length. A gravity wave drag
parameterization (GWD), is under consideration. The disadvantages of
not having a GWD parameterization have apparently decreased with the
implementation of the CBR TKE scheme.
Filtering of orography has been shown to reduce numerical noise
including precipitation features in hilly terrain.
Preliminary experiments with physics computations in a coarse grid have
indicated some possible advantages. Recommendations from the Workshop
on high resolution modelling support further research in this area.
Progress has been made to update the interface to the Arpege/Aladin
Physics Package. This work is important for developing and testing code
for 4D variational data assimilation.
Work on the ECMWF Physics Package started towards the end of the
Project.
2.2. Numerics
2.2.1. Time integration methods
Substantial progress has been made with development of the
semi-Lagrangian scheme. The method for computing the departure point
in trajectories has been improved. This has the effect of reducing the
generation of numerical noise on the smallest scales. Also, some
significant programming errors have been detected and corrected. The
combined effect of the changes is a substantial improvement compared to
the previous version of the scheme. This has been indicated by
extensive testing. Bias problems in geopotential height seen in
earlier versions have been greatly reduced. The new scheme is destined
to become the default HIRLAM code as soon as
remaining coding problems (associated with the MPP code) have been
solved
2.2.2. Lateral Boundary Condition
A systematic approach to study and improve on the treatment of the
lateral boundary conditions began in HIRLAM 4.
A proposal for additional model runs at ECMWF to provide improved
boundary conditions has been approved by the ECMWF Council. It is
expected to become operational during 2000, and should result in significant
improvements in the accuracy of limited-area forecasts based on the
HIRLAM System.
2.2.3. Non-hydrostatic equations
In view of the expected increase in model resolution over the next few
years, the issue of non-hydrostatic modelling will become increasingly
important. Work on development of a non-hydrostatic version of HIRLAM has continued by the Tartu group. The precise
approach to be adopted for future HIRLAM systems
is still open.
2.2.4. Digital filter initialization
A digital filter initialization scheme was implemented and tested in
the HIRLAM reference system in 1999. The filter
is more efficient that NNMI in removing spurious high frequency noise.
In a parallel run using FASTEX data, the scores produced by the digital
filter initialization were significantly better than those of the
reference run.
2.2.5. Other Model developments
A spectral HIRLAM model version has been
maintained, and has been used in the context of 4D variational
data-assimilation.
A stretched grid version of HIRLAM has been coded,
in collaboration with the Hitachi Dublin Laboratory. The preliminary
experiments have been encouraging. The method needs to be further tested
on large model domains.
3. System Management and Embedding
Introduction
This is a summary of the report on the HIRLAM
System, written by Gerard Cats, which appears in Chapter 4 below. A large number of
improvements to the HIRLAM system have been
implemented, many of them substantial in nature. These have lead to
significant benefits in the HIRLAM reference
system.
Meteorological improvements were obtained with the implementation of
the following model upgrades:
- A new condensation scheme (STRACO)
- A vertical diffusion scheme (CBR) based on turbulent kinetic energy
- A digital filtering initialization technique
- A new, HDF based, package to generate climatological fields
- Numerous upgrades to the OI analysis scheme
The technical aspects of the implementation were taken care of by the
System Management group. These included writing the technical
documentation and making that accessible over the
HEXNET.
The extranet system, HEXNET,
has been introduced and developed for use in HIRLAM. It has proved invaluable for communications and
for documentation. Its content and use are growing rapidly. An ftp
server has also been provided for HIRLAM. The
cooperation and support of KNMI in the development and maintenance of
these systems is gratefully acknowledged.
A large number of technical improvements to the reference system were
made during the Project. Utilities to facilitate local installation
were developed. The HIRLAM System was modified to
make it `Millennium-proof'. Performance of the system on a range of
hardware architectures was improved.
A `Version Control' mechanism was introduced to facilitate numerical
experimentation. A diagnostics package was added to the system. The
verification package was extended. A Java-based browser has been
developed for use with the HIRLAM Fortran code.
The system was adapted to allow for the 50 and 60-level ECMWF model.
HIRLAM can now also run from products of the
European Re-Analysis project. A facility has been introduces to ensure
that the system adjusts automatically when a new orography field is
introduced by ECMWF.
Finally, a vast number of minor code corrections have been made. As a
result, substantial improvements have been achieved in operational
forecasts produced by the HIRLAM System.
Comparative objective verification of operational forecast systems confirms
this.
3.1. Enhancement of efficiency of the reference system
In collaboration with EMCWF staff, a system was designed to run the
reference system over an inhomogeneous network of computers. The system
is based on the principle that it should be compatible with SMS, the
Supervisor Monitor Scheduler, used by ECMWF and some HIRLAM member states. To this end, ECMWF staff wrote a
simulator of SMS, with reduced functionality. It is now possible to
run the system distributed over SGI and Fujitsu to improve throughput.
3.2. Post-processing and verification
The current HIRLAM code incorporates a system
for the verification of surface variables and model verification,
following the guidelines of the European Working Group on Limited Area
Models, EWGLAM. This system provides a crude indication of performance
and allows for a broad comparison of HIRLAM with
competing limited area models. A meeting was held in De Bilt in 1999,
to determine the requirements of a new verification system. Work has
begun on implementing the recommendations of that meeting. This work
will be continued. In particular, methods are required for verification
of model behaviour in extreme events.
3.3. Model diagnostics
A comprehensive diagnostics package was implemented in the HIRLAM reference model. Its memory requirements are
big, and hence the package was made optional: it can be switched off in
operational runs, and activated as required. The package has already
proven its use by indicating inconsistencies in the turbulence scheme.
3.4. Parallelization
At the beginning of HIRLAM 4, the OI analysis
scheme and the forecast model were coded for parallel execution on
shared memory machines. During HIRLAM 4, code to
run the OI scheme on a distributed memory machine, using MPI for
communication, was developed.
The forecast model was recoded to allow parallel execution on
distributed memory systems. The current code performs very well on
shared memory machines, and on distributed memory machines using the
`SHMEM' paradigm. MPI code has also been developed. Initial results
indicate that there is considerable room for improvement.
3.5. Documentation
In order to facilitate communication in the HIRLAM
project, an extranet for HIRLAM, the HEXNET, has been put in place.
HEXNET has grown into a
powerful means for communication. Much of the documentation that
exists for HIRLAM has been made available on
HEXNET.
The current version of the graphical user interface (GUI) for HIRLAM allows selection of a geographical areas for
defining a HIRLAM run and for displaying results.
Further development will be required to make the GUI ready for
release.
Version 1 of a code browsing system has been released. Version 2
became ready just before the end of the Project, but some work needs
yet to be done to facilitate its local installation.
Main Forecast Model Upgrades.
Below is an overview of the main model upgrades which were introduced
during the Project. A more comprehensive list is kept on
HEXNET.
HIRLAM 4.1.0; released 16 Feb 1998.
The main model changes between 4.1.0 and 2.7.16 are:
- Holtslag (non-local) vertical diffusion
- Mass-flux scheme in Sundqvist
HIRLAM 4.1.1; released 9 March 1998
Introduce non-dimensional diffusion coefficients,
to result in automatic scaling of
horizontal diffusion with horizontal resolution.
HIRLAM 4.2.0; released 15 June 1998
Use the STRACO condensation scheme by default.
HIRLAM 4.3.1; released 30 November 1998
Revised boundary relaxation of cloud condensate.
HIRLAM 4.3.5; released 9 March 1999
Upgraded semi-Lagrangian and mass-flux schemes
HIRLAM 4.4.0; released 22 April 1999
Massively parallel forecast model code.
HIRLAM 4.4.3; released 8 June 1999
DMI physics upgrade
- improved surface fluxes over sea
- upgrades to STRACO scheme (scale dependent formulations)
- improved diagnostics of near surface parameters in the unstable boundary layer.
HIRLAM 4.5.0; released 31 August 1999
CBR scheme, diagnostics package, options array, code corrections.
HIRLAM 4.6.0; released 13 September 1999
Semi-Lagriangian advection upgrade (option)
HIRLAM 4.6.3; released 25 October 1999
Move to DFI initialization;
HIRLAM 4.7.0; released 13 December 1999
New climate generation package - improved orography.
HIRLAM 4.7.1; ( January 2000 )
Updates to the HIRLAM radiation scheme
(effects of parameterized size of cloud hydrometeors).
Remainder of Report
The following three chapters, written by members of the HIRLAM
Management Group, give a very detailed account of work undertaken in
the areas of assimilation, modelling and system management:
Note: Remaining sections not accessable from here. Please apply to
Peter.Lynch[.AT.]met.ie
Chapter 2:
Data Assimilation and Data, by Nils Gustafsson.
Chapter 3:
Model Developments, by Bent Hansen Sass.
Chapter 4:
System and Embedding, by Gerard Cats.
The report also contains three Annexes:
Annex 1
contains a list of HIRLAM 4 Meetings.
Annex 2
is a selected list of Exchange Visits during the Project.
Annex 3
is a list of HIRLAM and related Publications.
Return to top of Chapter.