\magnification=\magstep1
\normalbaselineskip=21pt
\def\pl{\partial}
\parskip=8pt
\parindent=0em
\centerline{{\bf Collaborative Testing of Turbulence
Models}}
\vskip0.15in
\centerline{{\bf Update of  AFOSR 90-0154,}}
\centerline{{\bf 1 February 1990 -- 30 April 1992}}
\vskip0.3in
\centerline{{\bf P. Bradshaw, Principal Investigator}}
\vskip0.2in
\centerline{{\bf Mechanical Engineering Department}}
\centerline{{\bf  Stanford University Stanford, California
94305-3030}}
\vskip0.5in
{\bf SUMMARY}

This project, supported by
AFOSR,  Army
Research Office,  NASA and  ONR, was administered  by   the
writer with Prof.  Brian E.   Launder,  University  of
Manchester,  England   and  Prof.   John L.   Lumley,
Cornell
University.   Statistical data  on turbulent flows, from
lab. experiments and  simulations, were  circulated to
turbulence modelers all over
the world. This is the first large-scale project of its kind
to use the results of simulations (numerically-exact
solutions of the three-dimensional, time-dependent
 Navier-Stokes equations) and for this reason alone is a landmark in
the testing of turbulence models.
The modelers compared their ``predictions'' with  the data
and returned the results to Stanford, for distribution to
all modelers and to
additional participants (``experimenters''); over 100
participants in all.   The object was to obtain a consensus
on the  capabilities of  present-day turbulence models,  and
to
identify the  types of  model  which  most  deserved support
for  future
development. This has not been achieved, mainly because not
enough modelers could produce results for enough test cases
within the duration of the project (our modest request was,
roughly, 25 test cases in two years). However a clear
picture of the capabilities of various modeling groups has
appeared, and the interaction has also clarified the outlook
of the modelers themselves. The results support the
proposition that Reynolds-stress transport closures
(second-moment closures) are more accurate/adaptable, but no account
has been taken  of their greater cost per calculation.


{\bf 1. INTRODUCTION}

After consideration of a 1992 conference at Stanford, as a
linearly-extrapolated successor to the 1968 and 1980-81
meetings (Refs. 1 and 2), the first formal proposal
envisaged  a  4-year
``mail order'' effort, hopefully long enough for significant
improvements to be made in the models. This was subsequently
cut, at the funding agencies' request, to  a nominal  18
months, with the object of finding ``where we are at in
turbulence modeling'' (without allowing time for
improvements)  and then
extended to  just over  2 years.   Although  a great deal of
useful information  has been  obtained, it  has become sadly
clear that very few turbulence modeling groups are both
able
and willing  to compute  test cases, covering a wide range of
turbulent flows,  within a  reasonable period  of time.
Unsurprisingly, some time was taken in improvement of models
and codes, but this was only part of the reason for the
delay.

In brief, the
quality of the results obtained seems to be much more
closely
correlated with  the  competence  of  the  modeler/modeling
group, the personnel available to do the actual ruunning of test cases,  and  the
adequacy of  the computer  program, than  with the  intrinsic
quality of  the turbulence  model.   It  is  certainly  not
possible to  say that  any one class of turbulence model has
conclusively proved  its superiority  over the  others, even
when cost  of computation is ignored. It does, however,
appear that Reynolds-stress-transport methods are a distinct
improvement over eddy-viscosity methods in complex flows,
though both are wounded by the unsatisfactory state of the
dissipation-transport equation. The treatment of the viscous
wall region, which is not closely linked to the model type,
also influences the results.

  Perhaps the most telling result was the large range of
predictions of flat-plate skin friction  -- even for
different  implementations of  a single model (the popular
``2-equation''($k, \epsilon$)  model based  on  partial
differential
equations for  turbulent energy and dissipation rate):
indeed this range was as
large as the differences between independent models. Even after
considerable pressure from the organizers, low-speed flat-plate
 skin-friction predictions still fill a 7 percent band (ignoring
outliers with serious discrepancies): taking the wetted area
of a civil transport aircraft as five times the wing area,
this corresponds to 35 drag ``counts'', or 35 passengers in
a large aircraft.) Numerical inaccuracy was only partly to
blame: a remarkable cause of discrepancy was the
disagreement over the supposedly-universal ``law of the
wall'', discussed in Appendix 1.

The prediction of
compressible flow was a primary interest of the funding
agencies:   most  flat-plate results closely
followed the  Van Driest  correlation of  experimental  data
for the ratio of compressible to incompressible skin
friction, which is
still believed to be the most reliable -- it allows for the
effects of mean density variations but ignores
compressibility effects (density/pressure fluctuations) as
such. (Note that presenting results as the ratio of
compressible to incompressible skin friction suppresses the
scatter in incompressible $c_f$ discussed above.)  The only
well-documented flow that shows large Mach-number effects is
the mixing layer: ``predictions'' of the decrease in
spreading rate with increasing Mach number either used an
{\sl ad hoc} compressibility correction or gave poor
results.

The decision  to run  this project  via interaction  by
mail,
rather than  as a  conference  like  the  1968  and  1980-81
Stanford meetings, was taken so that participants would have
time to  consider their  early results,  compare  them  with
those of  others and  make minor improvements in
their prediction  methods.   Although this  happened, very
usefully (in some cases, program bugs of embarrassingly
long standing  were uncovered)  it seems  that only  a
``drop
dead'' conference  deadline can  concentrate the minds of
the
turbulence  modeling   community  enough   to  produce
results on demand. Each section of the community --
universities, government establishments, consulting
companies -- has its own difficulties over manpower,
facilities and finance.

Detailed  technical   results  will   be  discussed  in  the
following sections:   administratively, the main conclusion
is that in
spite of the rather large rate at which papers on turbulence
modeling are  being  published,  some  with  quite  detailed
comparisons with  experimental data,  few groups can rise to
the challenge  of producing  comparisons with {\sl
independently-chosen} test  data within  a reasonable time frame.  A
number
of modelers  quoted lack  of resources or budget constraints
as a reason for lack of response, but we have not heard that
any sponsors have  refused  explicit  requests  for
diversion of funds from the turbulence modeling efforts they
support.   The organizers  have --  entirely  unofficially
--
repeatedly  pointed   out  to   modelers  that  the  funding
agencies, and  not necessarily  only  those  supporting  the
present project, will use the outcome as a guide to how much
support to  give turbulence  modeling in the next few years,
and  to  which  groups  that  support  should  be  directed.
Quantity and  quality of  response  has  by  no  means  been
proportional to  the size  of the  group.   U.S.  government
laboratories have  contributed practically no results for
the final group  of test  cases,  while  two  one-man
consulting
companies  were   among  the   most  competent  and  helpful
collaborators.    Undoubtedly  a  number  of  modelers  have
dropped out simply because they were not able to predict the
test cases  to an accuracy which they wished to demonstrate
in
public, but we  have  no  way  of  distinguishing  these  from
modelers who  dropped out through lack of facilities or
simply  through   lack  of   motivation.     Most
collaborators were  from the  United States: on a percentage
basis, the  enthusiasm and  competence of  response  was  no
better, and no worse, from the United States than from other
countries.   It was  one of the overseas modelers who
pointed out  that the  research climate has become very much
less favorable  since  the  time  of  the  1980-81  Stanford
meeting, for  which a large number of groups produced
results
for a larger number of test cases than those employed in the
present  collaborative  effort.    Although  no  spectacular
advances have  been made  in turbulence modeling in the last
decade, it remains a lively subject, with at least 100
 high-quality papers  being published each year, apparently as the
results  of   basic  research  rather  than  deadline-driven
development work.   It is difficult to see that the modeling
community's poor  response can  be attributed simply to lack
of funding.

Details of which modelers attempted which test cases are given in Appendix 4.

{\bf 2. HISTORY}

(The history of the project has been recorded in the five
project Newsletters and their attachments, already
distributed to the funding agencies. The following is an
outline: the sixth Newsletter is an Appendix to this report, and vice versa.)


After discussions  at the ``Whither Turbulence'' meeting
at
Cornell University  in March  1989 (Ref.  3) a  proposal for
international collaboration  on testing of turbulence models
was submitted  to U.S. Air  Force Office  of Scientific
Research, acting as coordinator for  U.S.  Army
Research Office,  NASA and  Office of  Naval Research.
Invitations  to participate were sent out in late
August 1989,  to all  originators of turbulence models known
to the  organizers, to  a number of consulting companies and
other organizations  likely to  have well-developed versions
of models  originated by  others, and  to all  experimenters
identified as  likely to  be  able  to  contribute  data  or
comments.

{\bf 2.1 ``Entry'' test cases (flat-plate boundary layers)}

In order  to calibrate  both the models
and the modelers (specifically, the time of response of the
latter), simple ``entry test cases'' were distributed
in February  1990.   The requirement was to predict the skin
friction in  a turbulent  boundary layer  in  zero  pressure
gradient at a Reynolds number, based on momentum thickness, of
10,000, in as many as possible of the following cases: (i)
low-speed flow, (ii)  a Mach number of 5 on an
adiabatic wall,  (iii)  low-speed flow with an absolute wall
temperature   6   times   the   free   stream   temperature,
corresponding approximately  to the temperature ratio across
the $M=5$  adiabatic boundary  layer.   Stanton  number
(heat
transfer) predictions  were requested  for cases  1  and  3.
This set  of test  cases also had  the organizational
purpose of
identifying  modelers   who  could  produce results for
compressible
flow.

The low-speed high-temperature  test case  was chosen  so
that  the
relative   importance    of   density    changes,   and   of
compressibility  (Mach  number)  effects  as  such,  in  the
various models  could be  clarified.  Many flat-plate skin
friction {\sl formulas} (as distinct from detailed
prediction methods), use explicit Mach number factors and
would not necessarily do well for a low-speed hot wall.
In fact most models performed as well for the low-speed hot
wall as for the $M=5$ adiabatic wall, indicating that the
models allowed adequately for density changes: indeed, true
compressibility effects are probably small in boundary
layers up to $M=5$. It was of course very satisfactory that
this test case turned out to be a non-issue.

The ``entry'' cases proved to be  an
invaluable calibration.
Few  modelers  managed  to  keep  to  the  relatively  tight
deadline imposed  for return  of  results,  even  for  these
almost trivial  test cases: moreover, as the results began
to come
in it  became obvious  that the  range  of  predictions  was
rather wide.   In many cases predictions were, quite simply,
outside
the possible  bounds of  experimental error  for these
simple cases. (In general the organizers have attempted to
keep to the error standards appropriate to the aerospace
industry: a discrepancy of 0.0001 in skin-friction
coefficient -- about 3 percent --is
big enough to worry about.)  A great  deal of  time  and
effort  was  spent  on
interactions  with  individual  modelers  and  requests  for
further information. Probably, a few modelers had calibrated
their methods against pipe or duct flow rather than boundary
layers, but the main explanation of the really large errors seems to be that many models
intended for complex or compressible flows had simply not
been adequately checked in simple low-speed flows. Another  cause of error was inconsistency in choice of logarithmic-law constants or their equivalent (see Appendix 1).    The final results can  be described only as a computational catharsis:
many modelers  submitted revised  results after  cleaning up
empirics, numerical  resolution, and  downright  programming
errors.   It should be remarked here that the modelers whose
results were  questioned by  the organizers  were  uniformly
grateful.

{\bf 2.2 August 1990 test data (Cases 3.1-3.5, 4.1-4.3)}

We felt  it essential to clean up most of the questions over
the ``entry case'' flat-plate  computations before
proceeding to  the next
set of test cases, so that it was not until August 1990 that
the first real test cases were sent out.  They were intended
to cover  a wide  range of flows, keeping as far as possible
to thin  shear layers  and/or simple geometries.  We took it
for granted  that the  turbulence  models  which  were  most
 advanced, or  most up-to-date, would probably be imbedded in simplified codes,
 capable of handling only a limited range of geometries --
perhaps to the point of being restricted to thin shear
layers. For this reason, we have concentrated throughout the
project on test cases which are geometrically simple but
physically general.

The ``August 1990'' data included some
recent experiments and simulations, but also test
cases  from  the  Stanford  1980-81  conference  on  complex
turbulent flows  and from  the AGARDograph compilation of compressible-flow data by Fernholz
and Finley (Ref. 4).   In  the case of free shear layers
(plane jets,
round jets, and mixing layers) detailed experimental results
were not  given, and  modelers were  simply asked to compare
 predicted   growth  rates   with   the   consensus   of
experimental data.    The only flows requiring a full
Navier-Stokes program were the backward-facing steps of
Driver and Seegmiller (Ref. 5). The boundary-layer
simulation of Spalart and the duct simulation of Moin, Kim
and Moser were included, and modelers were  asked to compare
the {\sl highest-order quantities} they modeled (e.g.
dissipation or triple products).

Because of the ongoing discrepancies in the
 incompressible-flow results, the compressible  flows in the August 1990
package  were  deliberately
restricted to one real test case, a boundary layer in strong
adverse pressure  gradient, plus a second set of ``entry''
test
cases, namely  the prediction  of flat-plate  skin  friction
for
Mach numbers of 2, 3, 5 and 8  and temperatures down to 0.2
of the adiabatic-wall temperature. The corresponding
``data'' were simply the predictions of the Van Driest II
skin-friction formula, which experts in the field regard as
being still an acceptable data correlation.

One group of modelers who were disadvantaged by our
concentration on thin shear layer data was those who use
(Reynolds-averaged) Navier-Stokes codes, which do not easily
accept the boundary-layer simplification of specified
velocity at the boundary layer edge. Their polite reproaches
were entirely justified: a boundary-layer calculation is a
solution to only half the problem.

{\bf 2.3 Results for August 1990 test cases}

The speed  of response  to these  test cases  was  extremely
disappointing, with  very few  results being returned by
the  specified deadline.   A number
of modelers  stated that  they  would  be  able  to  produce
results, although  not within  the deadline.   Since  the
object  of  the
collaboration was  to avoid  the ``drop  dead'' deadline  of
a
conference,  we, the   organizers,  decided   to  wait
until  a
representative body  of results had been returned. Obviously
this totally disrupted our plans for handling the data,
which envisaged an intensive effort beginning at the
deadline and accomodating only a few latecomers. In fact,
despite promises, only a very
few sets  of results were returned after Spring 1991.  In
August 1991  the assembled  results were  distributed to all
the known  collaborators, with  a request  for comments.

  To
keep down  the amount  of material to  be
redistributed, we  specified  that  modelers  should  return
plots only  of key quantities (e.g. in the
 backward-facing-step flow,  simply the  surface shear
stress and the maximum
shear stress at each streamwise position), and although
many
of those  modelers who did respond did not complete the full
set  of   ``priority''  test   cases,  the   stack  of
graphs
distributed was about 1.5$''$ thick.  Some useful comments
have
been received,  but it is clear  that not  too many of the
experimenters or  modelers who had undertaken to join the
project were able to devote serious effort to assessing other people's
results. Since this was the main reason for calling the
project a {\sl Collaboration}, the poor response by the experimenters, coming on top of delays in the computations, was unfortunate.

The results for the thin shear layers were mixed, with no
obvious best model. The Wilcox $k, \omega$ and multiscale
models gave good and closely similar results: the multiscale
model has some of the features of an Algebraic Stress Model -- a type
which was otherwise used only for a few test cases -- and
this suggests that the improvement shown by ASM-type models
over a good two-equation eddy-viscosity model in 2-D thin shear
layers may not be significant. The spread of results from
the different versions of the $k, \epsilon$ model was
comparable with, but not closely correlated with, the spread
for the ``entry'' cases. The results for free shear layers
showed the usual round-jet / plane-jet ``anomaly'': few
models can predict both flows without some form of special
correction factor. Launder's group has recently shown (Ref. 6) that
Navier-Stokes calculations for jets produce significantly
different results from parabolic (``boundary layer'')
calculations, partly because of the effect of longitudinal stress gradients, but also partly because of the large effect of longitudinal
diffusion of dissipation rate: the modeled transport
equation for dissipation is so highly empirical that there
may be no physical explanation, but the discrepancy provides
a further opportunity for confusion in testing turbulence
models.

The number of different models was too small to build up a
pattern in the comparisons of ``highest-order'' quantities
with the simulation data. Because of the low Reynolds number
of the simulations this was mainly a check on the wall-layer
treatments, and the ``low-Reynolds-number'' versions of the
stress-transport models produced tolerably close agreement
with the simulations. The simulations show higher
dissipation rate in the viscous wall region than do the experiments, and
the models seem to have been tuned for the latter. Both experiments and simulations can suffer from errors due to inadequate spatial resolution, most severe near the wall, but on balance the simulations are likely to be more accurate.

Predictions of the backward-facing step flow were
surprisingly scattered (even discounting the differences
among the $k, \epsilon$ models), and most models
considerably overestimated the maximum negative skin
friction in the recirculating flow, whether they used wall
functions or low-Reynolds-number treatments. The simplest
explanation is excessive diffusion of momentum into the
recirculating flow. Unfortunately no stress-transport models
were integrated for the full length of the test flow (32
step heights -- expensive in a Navier-Stokes calculation) so that their potential advantage in
representing ``history'' effects has not been demonstrated
in this flow.

Predictions of compressible flat plate skin friction up to
$M=8$ and of heat transfer on cold walls at $M=5$ were
mixed. Results for the adiabatic cases were good, with a few
exceptions (thought to be models developed for transonic
flow and not previously tested at hypersonic speeds).
 As in the case of low-speed flow, the results depended
strongly on the treatment of the wall region: many models
reproduce the mixing-length formula with a constant
turbulent Prandtl number, and this is of course the basis of
the Van Driest transformation.  Recent work, independent of
the present project (Ref. 7), has shown that the $k,
\epsilon$ model does {\sl not} reproduce the Van Driest
transformation because of the presence of density gradients
in the diffusion terms: however, many users of $k, \epsilon$
models obtained results in fair agreement with the Van Driest ``predictions''.

Results for cold walls showed a wide spread. Confusion
occured when several modelers did not realise that Stanton
number has to be based on the adiabatic wall temperature
actually predicted by the model, {\sl not} that given by a
selected value of recovery factor (if the latter is used,
$St$ goes to $\pm \infty$ as the predicted adiabatic wall
temperature is approached).

 Only a few modelers reported results for the compressible
boundary layer in strong pressure gradient near $M=3$, and
these results were generally satisfactory (this particular
flow happens to have almost constant skin friction
coefficient and therefore looks uneventful, but it is a
reasonably severe medium-Mach-number test case). The user of
one of the only ``integral'' methods presented pointed out,
in connection with this case, that a simple model which has
been carefully calibrated may out-perform more advanced
models on its home ground. This may be the First Law of Turbulence Modeling.

A few modelers argued that their only concern was with
compressible flows and they therefore did not wish to bother
with incompressible test cases. This seems a shortsighted
attitude: obviously if a model is found to be inaccurate in
incompressible flow it cannot be relied on in compressible
flow. (Modelers with codes that do not run exactly at $M=0$
were encouraged to run at, say, $M=0.4$ and $M=0.3$ and
extrapolate to $M=0$: most compressibility effects vary as
$M^2$ at low $M$ so there is no difficulty of principle
here.)


{\bf 2.4 August 1991 test data (cases 5.1-5.9)}


A second  set of  ``real life'' test cases was sent out at
the
same time  as the  results for  the August 1990 set.
These test cases were chosen to explore various complex-flow
effects,  such   as reverse transition, streamline
curvature,    3-dimensionality   and unsteadiness.   Again,
most of  the test cases were
thin shear  layers; the 3-dimensional flows in fact had only
two independent  variables; and  the time-dependent flow was
homogeneous in the horizontal plane (and therefore
computable by
trivial  adaptation   of  a   2-dimensional   space-marching
program).

Two cases were specifically intended to test
treatments of the viscous wall region. The first, a
simulation of sink-flow boundary layers, gives a simple
performance index: does the model predict reverse transition
at the same value of pressure-gradient parameter as the
simulation? The second, a sinusoidally-oscillating
 time-dependent  flow, in principle causes grief to a ``wall
function'' which uses the friction velocity $\sqrt {\mathstrut}
(\tau_w/\rho)$.

Unfortunately, no modeler did the parametric check we
requested for the sink flow in sufficient detail to bracket
the critical pressure-gradient parameter. To our surprise,
two modelers successfully predicted the oscillating flow
with wall functions (undoubtedly using $\sqrt{\mathstrut}
{|\tau_w/\rho|}$): presumably their finite time steps did
not land them too close to the phase angle at which $\tau_w$
changes sign. Amazingly, only one modeler reported results
for the curved boundary layers, although the curved jet
flows generated the best response.

Despite our  attempts  to  maintain  geometrical
simplicity  and   avoid   excluding modelers without
curvilinear Navier-Stokes codes, the response to the second
set of
test cases  has  been  extremely  disappointing. Many
modelers have stated an inability to devote more effort to
the project: undoubtedly some prompt responders have become
impatient of the delays caused by the slow responders.   A
final
deadline of  31 May  1992 was  imposed, but a few later
results were accepted for good reason -- and were still arriving in December 1992!  The outcome is that
the only complex flow for which a worthwhile number of
predictions has been received is the backward-facing step
flow, which is the complex flow most likely to be
used for testing a model during development and is therefore not an entirely independent test case.

{\bf 3. CONCLUSIONS}

The Collaboration has not clearly revealed a ``best model'':
in particular, stress-transport models have not demonstrated a large superiority over eddy-viscosity-transport
(``two-equation'') models although other reviews have come
to the conclusion that they do perform consistently better.  The main reason for this
lack of clear conclusions is a lack of response from the
modelers, particularly the developers of stress-transport
models.

To minimize numerical difficulties and the time taken to
prepare input data, and to avoid excluding models which were
implemented only in simple codes, the organizers tried to
ensure that the test cases had simple geometries; as far as
possible, the cases were thin shear layers accessible to
parabolic ``marching'' programs. Nevertheless, the
performance of any given prediction method, and the fraction
of the test cases for which results were reported, seemed to
depend much less on the model than on the state of
development of the code and the care devoted to checks of
grid independence and other numerical issues.

The Collaboration raised a number of general questions about
turbulence modeling as well as queries about numerical or
other shortcomings of particular methods. It has resulted in
correction of errors in several models as well as acting as
a clearing-house for facts and opinions.

On balance, the Collaboration has been a success, although
it would have been much more fruitful if more modelers had
been able to devote effort to producing results for more of
the 25 test cases, and if more of the ``experimenters'' had
made a serious effort to return comments on the results.

The reader may have detected a tone of irritation in this
report. It should therefore be recorded that modelers'
replies to the organizers' stream of requests and reminders
were always courteous: on the one occasion when a tardy
modeler wrote claiming that ``the results are in the mail''
-- they {\sl were}! The project has indeed done a lot to
bring the modeling community together, to point out
discrepancies in generally-satisfactory models and, perhaps,
to establish the principle that a basic requirement of a
model of turbulence (or anything else) is good performance
in test cases chosen by someone other than the modeler.

{\bf Future plans}

This Final Report discharges contractual obligations, but
will be considerably augmented later as an unpublished but
publicly-available report: in addition, a journal paper will
be prepared as an ``executive summary'' and an advertisement
for the report. The hierarchy of available material will
then be:--

\hskip2.0truein Journal paper

\hskip2.0truein Final Report (augmented)

\hskip2.0truein Newsletters (including graphs)

\hskip2.0truein Data disks and documentation (used and unused data).

Stanford University will need to make a handling charge for
this material, but it is hoped that one or more of the data
banks now being planned will also archive the disks.

The 1980-81 Stanford meeting on Computation of Complex
Turbulent Flows was hampered by lack of time for
consideration of the results presented at the meeting. The
present ``mail order'' effort was hampered by the failure of
modelers to keep to the deadlines. On the assumption that
there is an ongoing need for public comparisons of
turbulence models, a possible compromise for the mid-1990s
would be to have two meetings, with an interval of six
months or a year: at the first, initial comparisons with
test cases would be presented and briefly discussed (as at
the 1980-81 meeting); modelers would then reconsider their
results and present updated versions for final discussion at
the (longer) second meeting. The volume of simulation data
is already quite large and an increasing range of complex
flows is being covered, so that simulations (direct or
large-eddy) would probably provide the largest part of the
data sets in a mid-nineties project.


{\bf REFERENCES}

1. S.K. Kline, M.V. Morkovin, G. Sovran and D.G. Cockrell (Eds., vol. 1); D. Coles and E.A. Hirst (Eds., vol 2). Computation of Turbulent Boundary Layers -- 1968 AFOSR-IFP-Stanford Conference. Mech. Engg Dept. Stanford University, 1969.

2. S. J. Kline, B.J. Cantwell and G.M. Lilley (Eds.). 1980-81 AFOSR-HTTM-Stanford Conference on Complex Turbulent Flows. Mech. Engg Dept., Stanford University, 1981.

3. J.L. Lumley (Ed.). Whither Turbulence? Turbulence at the Crossroads. Springer, 1990.

4. H.H. Fernholz and P.J. Finley. A further compilation of compressible boundary layer data with a survey of turbulence data. AGARD-AG-263, 1981.

5. D.M. Driver and H.L. Seegmiller. Features of a reattaching turbulent shear layer in divergent channel flow. AIAA J. 23, 163, 1985.

6. A. El Baz, T.J. Craft, N.Z. Ince and B.E. Launder.
On the adequacy of the thin-shear-flow equations for computing turbulent jets in stagnant surroundings.
UMIST, Manchester, TFD/92/1, 1992. `

7. P.G. Huang, P. Bradshaw and T.J. Coakley. Assessment of closure coefficients for compressible flow turbulence models. NASA TM 103882, 1992.



\vfill\eject
APPENDIX 1

THE ``FLAT PLATE'' BOUNDARY LAYER TEST CASES

Flat  plate   boundary  layers,   that  is,  those  in  zero
longitudinal pressure  gradient, are  one of  the most basic
test cases  for turbulent  flows (the  others being  the
2-dimensional duct  or ``channel'', which is an idealization
for which reliable experimental data are  scarce, and the circular pipe, for
which  computations  are  complicated  by  the  axisymmetric
geometry).   Because the  law of  the wall  extends from the
surface to  $y \approx 0.15 \delta$, where the velocity has
typically risen to 70 percent of
the free-stream value,  model  predictions  of  flat  plate
boundary layers  are dominated by the assumptions made about
the law  of the  wall.   Virtually all turbulence models are
compatible  with   law-of-the-wall  scaling,  and  can  thus
reproduce a  logarithmic velocity  profile.    This  applies
whether the  region between the surface and the start of the
logarithmic law  ($y^+ \approx 30$) is predicted by
integrating a ``low
Reynolds number''  version of  the model down to the wall,
or
imposed as  a ``wall function'' boundary  condition at
$y^+=30$ (say).   A data
analysis done  by  Prof.  Donald  Coles  of  the  California
Institute  of  Technology  for  the  1968  Stanford  meeting
(Ref. 1) recommended  $K=0.41$ and  $C=5.0$ in the standard form
of the logarithmic law,
and  we  are  not  aware  of  any  later  review  which  has
specifically challenged  Coles' conclusions.   A rather wide
range of  values for $K$ and $C$ is quoted in textbooks, but this
scatter is attributable  mainly to  the difficulty of
finding,
separately, the  slope and  intercept of  a  line  which  is
defined over  only a  relatively short range (of log $y^+$): 
most of  the published values of $K$ and $C$ lead to very nearly
the same value of $U^+$ at, say, $y^+=100$, which is
somewhere near
the middle  of the logarithmic range in a typical laboratory
boundary layer.

When the  scattered predictions  of flat  plate skin
friction
from the  modelers started  to come in, we requested them to
supply details  of their  treatment of  the universal law of
the wall,  including their predictions (or assumptions) for
$U^+$
at $y^+=100$, hereafter referred to as $I_{100}$.  Figure 1
shows the
values of  skin friction  (modelers' names  not  identified)
plotted against  the  reported  value  of  $I_{100}$.    Now
if  a
turbulence model  as used  in the outer part of the boundary
layer is  left fixed,  but the  value of  $I_{100}$ is
changed  by
changing the  wall treatment,  then $U_e^+-I_{100}$ will
remain very
nearly fixed, so that $U_e^+ \equiv \sqrt{c_f/2}$ changes.
The line on
Figure 1  was produced by Rodi and Scheuerer at
our
request, using  their standard $k, \epsilon$ model with different values of $C$. If the
outer-layer
predictions of  all  the  models  shown  in  Figure  1  were
identical, the  results would  lie along the curved line, or
at least on at line parallel to it.  In fact, there seems to
be very  little correlation between the values of $c_f$ and
the
values of  $I_{100}$, in  spite of the fact that the version
of the
plot shown  in Figure  1 is  the latest,  after considerable
exhortation of the modelers to clean up their assumptions or
predictions of  the logarithmic  law.   Specific requests to
modelers to  justify using  logarithmic law  constants other
than those  recommended by  Coles have  not produced  a very
satisfactory response.   If  Figure 1  is divided  into four
regions by  the  Rodi-Scheuerer line and a vertical  line
$I_{100}=16.24$,
predictions in  the first  and third quadrants would be made
{\sl worse} by imposing the Coles values for the log-law
constants,
while predictions  in the  second and fourth quadrants would
be made {\sl better}.

It is,  of course,  impossible to compare outer-layer models
logically in the face of these differing assumptions/results
for the law of the wall.  Many modelers used the popular
$k,\epsilon$
turbulence model.   When  the collaboration  began  we  were
warned that there would be no point in having a large number
of modelers  all using the same model, but in fact comparisons
of their results have proved instructive, if not heartening.
Several  $k,\epsilon$  users  have modified  the
empirical
constants in  the model, but those whose results in Figure 1
are marked with a tail used the ``standard'' coefficients
with
-- obviously -- different wall treatments.  The failure of
the
tailed symbols  to lie on a line {\sl parallel} to the curved line
in Figure 1 can only be contributed to numerical error in the outer layer
(grid dependence  or programming mistakes).  (Note that grid
resolution near  the wall  is not  an issue  here, since the
law-of-the-wall results are being taken for granted.)

Figure 1,  therefore, presents a gloomy picture.  The common
correlation formulas  for flat-plate  skin friction agree to
within about  2 percent at  a momentum thickness Reynolds
number of
10,000, and  it is  clear that  many models  have simply not
been optimized  for the  flat plate boundary layer.  Popular
values for  the law-of-the-wall constants give values of $I_{100}$ that agree to within 3
or 4 percent, at the outside, but many models use or give
results well
outside this range.  Several models with nominally identical
assumptions in the outer layer show discrepancies of several
percent in  skin friction,  even in  this numerically-simple
flow.
\vskip0.2in
APPENDIX 2

THE MODELS

A2.1 The Process of Turbulence Modeling

The object is always to predict the Reynolds (turbulent)
stresses: the Reynolds stress in the $x_i x_j$ plane is $-
\rho \overline{u_i u_j}$, where the overbar denotes an
average (usually a time average). Exact partial-differential
``transport'' equations for the stresses can be derived from the Navier-Stokes equations, but their right-hand sides
include unknown higher-order statistical
quantities. The transport equation for a given Reynolds  stress contains a source term which is more-or-less proportional to the
mean rate of strain in the plane of that stress,
 which implies that the ratio of the  Reynolds stress to the
mean rate of strain varies {\sl less} than the Reynolds
stress itself and may therefore be easier to correlate
empirically: this ratio is of course the ``eddy viscosity'', which may be different in different planes.
The terms in the Reynolds-stress transport equations all
have dimensions [velocity$^3$/length], and almost all models, of
whatever order, assume that the turbulence can be described
by one velocity scale and one length scale. This is a sweeping assumption, even when one is
concerned only with the larger, Reynolds-stress-producing,
eddies.

A2.2 Review Of The Models

Except for one ``integral'' method, all the methods used
partial differential equations for the mean velocity.
(Integral methods can in principle be derived by applying
the Galerkin technique to PDE models: this is not often done
in practice, but it shows that integral methods are not a class
by themselves and are not restricted to crude turbulence
models.)

Naturally, modelers were asked to use the same model for all
test cases, and to repeat the entry test cases if they
made any changes to their models. Many of the major modelers
have done exactly this and we have no reason to suppose that
the results have been significantly confused by unreported
changes. Several multi-person groups, and even some individual
workers, used entirely different models at different times during the
collaboration. The test cases got progressively harder so
there was a tendency for simple models to be replaced, or for their users to drop out entirely. The results for simple models (algebraic eddy viscosity or mixing length, and the one integral method) reinforce
the conclusion of the 1980-81 meeting that such models, when carefully tuned,  are
useful in a restricted range of flows.

The ``zonal modeling'' technique, in which
coefficients are altered from flow to flow  by logic in the
computer program, was explicitly permitted, on condition of
full disclosure, but seems not to have been used. We did
specify that the same coefficients should be used in the
compressible mixing layer and the compressible boundary
layer, mainly because a correction to the coefficients which
reproduces the observed decrease in spreading rate of a
mixing layer also tends to produce an undesired reduction of
skin friction in a boundary layer. (Compressibility
corrections  developed rapidly during the course of the
Collaboration and we have not tried to reach a consensus: the present situation is that all the corrections include adjustable constants multiplying quantities of order $M^2$, and it is difficult to judge the plausibility of the physics.)

Models can be divided into those which assume a direct
relation between the Reynolds stresses and the mean velocity
field (``eddy-viscosity methods'') and those which solve
explicit equations for the stresses (``stress-equation
methods''). There is a deep hierarchy of eddy-viscosity
methods, but they share the feature that if the mean
velocity gradients change suddenly, so also do the Reynolds
stresses. The exact transport equations for the Reynolds
stresses show that a sudden change in mean-velocity gradient
merely produces a sudden change in the rate of growth of
Reynolds stresses. That is, in reality the mean-velocity
gradient occurs as a source term in a differential equation
for the stress, rather than as a factor in an eddy viscosity
formula.

A2.2.1 Eddy-viscosity models

Since an eddy viscosity is always the ratio of a turbulence
quantity (Reynolds stress) to a mean-flow quantity (mean
rate of strain), it is obviously determined by a combination
of mean-flow scales and turbulence scales and will be
 well-behaved only when the two sets of scales are proportional (a
definition of ``local equilibrium'' flow). It is clear in
practice that relating the eddy viscosity to the turbulence
scales gives better results when the mean-flow scales are
strongly perturbed, by pressure gradients or otherwise. 

The advantage of eddy-viscosity models is that they will
usually give smoothly-varying predictions of Reynolds
stresses -- obviously, like a laminar flow with a
 smoothly-varying viscosity. Their disadvantage is identical: they
will not reproduce the dependence of Reynolds stresses on
mean-flow history and the slow response of Reynolds stresses
to sudden changes in mean flow.


Eddy-viscosity
models are often classified by the number of partial differential equations used to describe the turbulence:--

$\bullet$ ``Zero-equation'' or ``algebraic'' models  relate the eddy viscosity to the velocity and
length scales of the mean flow (typically free-stream velocity
and boundary layer thickness); those used in the present Collaboration included mixing length, eddy
viscosity, and one integral method. ``Zero-equation'' methods can have some success if they
conform to the law of the wall (or the
 skin-friction laws derived from it) and the corresponding density
correlation for compressible flow (amounting to an
assumption of constant total temperature in an
 adiabatic-wall boundary layer). They can be expected to perform well
in boundary layers in mild pressure gradients -- and,
paradoxically, in short regions of very strong pressure
gradient where the skin friction is determined by the
response of the inner layer and the total pressure in the
outer layer changes little (so that prediction of those
changes by the turbulence model is not critical).
 The outer-layer model, for example the assumption that
mixing length is proportional to shear layer thickness,
necessarily relies on the shear-layer thickness being
 well-defined. Although eddy-viscosity models such as the
 Baldwin-Lomax model have been used in quite complicated flows, no
predictions  other than for boundary
layers have been submitted to the present Collaboration.

$\bullet$ ``Two-equation'' or ``eddy-viscosity-transport'' methods relate the eddy viscosity to the
velocity and length scale of the turbulence. Both make the
gross assumption that only {\sl one} scalar velocity scale and {\sl
one} scalar length scale suffice to fix the eddy viscosity.
Current two-equation'' models all use the turbulent kinetic energy $k$ as one variable. They
can be classified by the variable $k^m \epsilon^n$ used in the second
equation to provide a length scale $k^{3/2} / \epsilon \equiv L$ and thus an eddy viscosity proportional to $k^{1/2}L$.  The six Reynolds stresses are then given by
$$-\overline{u_i u_j}= c_{\mu}k^{1/2}L(\pl U_i/ \pl x_j +
\pl U_j /\pl x_i)$$
which is nominally a {\sl definition} of $c_{\mu}$ as a
dimensionless tensor with indices $i$ and $j$. In reality
$c_{\mu}$ is assumed to be a scalar and in all the methods
used in the Collaboration it appears to have been taken as a
constant (except for ``low Reynolds number'' modifications
in the viscous wall region).
The most popular of the
``two-equation'' eddy-viscosity models is the $k,\epsilon$
model ($m=0,\ n=1$ in the above classification. Other
models in the $(m, n)$ family which have actually been implemented
are $k, kL$ ($m=5/2,\  n=-1$), $k, \omega$ ($m=-1,\  n=1$)
and $k, \tau$ ($m=1, \ n=-1$). It is straightforward to
convert from one $(m, n)$ pair to another but the diffusion
term in the first model converts to a diffusion term {\sl
plus a source/sink term} in the second, because the
diffusivity is a function of the dependent variables. Since
the models are usually formulated without a source/sink term
of this sort, the implication is that there are real
differences between the different $(m, n)$ combinations,
with the further implication that there must be a best (and
worst) choice: $m$ and $n$ do not have to be integers, though the physics may become obscure if they are not. This
point was not addressed during the Collaboration but
deserves future consideration.

$\bullet$ ``One-equation'' models either use a partial differential
equation for a velocity scale and relate the length scale to
the shear-layer thickness, or use a single PDE for eddy
viscosity. In the latter case the necessary length scale comes, in effect, from
the ratio of the velocity scale to a typical mean velocity
gradient. One-equation models (with a PDE for turbulent
energy $k$) were sometimes used in the wall region where a
two-equation model was used in the outer layer. This is
primarily a numerical simplification, but comparison of
results in the Collaboration suggested that performance of
two-equation methods in boundary layers in  adverse pressure
gradient can be improved by using the one-equation model as
far out as possible! (The limit is set by the need to match
the models, and corresponds, in principle, to the onset of significant
transport terms in the length-scale equation.) The recent one-equation models of Baldwin \& Barth and of Spalart \& Allmaras were not represented in the Collaboration.


A2.2.2 Stress-equation models

These are  (usually) term-by-term models of
the exact Reynolds-stress transport equations. The object is to avoid using an eddy viscosity for the Reynolds stresses, but gradient-diffusion assumptions are commonly
used for the turbulent transport (``diffusion'') terms. The
Reynolds stresses themselves can yield velocity scales --
$k$, being a scalar, is the natural choice for most purposes
-- but a length (or time) scale is also needed: most
 main-stream models use the dissipation, obtained from essentially
the same equation as in the $k, \epsilon$ model. The key
parts of the model are the dissipation-transport equation,
and the pressure-strain ``redistribution'' terms in the
stress-transport equations themselves. Nearly all the
current models are recognizable descendants of the
 Launder-Reece-Rodi model of 1975, in turn a generalization of the Hanjali\'c-Launder thin-shear-layer model of 1972. The main improvements in stress-equation models since the
1980-81 meeting are the enforcement of ``realizability'' (no
physically-impossible negative values, or correlation
coefficients outside the range $\pm1$) and of correct
behavior in the two-component limit (e.g. at a solid
surface, where $v$ goes to zero faster than $u$ and $w$). Most of the work has gone into improved modeling of the
pressure-strain term, with comparatively little attention to
the dissipation equation. There is considerable current interest in the transport equation for $\omega$, a.k.a. $\epsilon / k$, as an alternative to the transport equation for $\epsilon$ as such, both in two-equation models and in transport-equation models: this is partly a result of the good performance of Wilcox's models in the current collaboration.

``Algebraic stress models'' (ASM), of which one example was used for a few test cases, are stress-transport models with severe simplifications of the mean and turbulent transport terms, resulting in an eddy viscosity model with different values of eddy viscosity for the different Reynolds stresses. A specific advantage over standard two-equation models is that the ASM will at least qualitatively predict stress-induced secondary flow in non-circular ducts. Wilcox's multiscale model falls in the transport-equation family but has some features of the Algebraic Stress Model. 

A2.3 NUMERICAL ERRORS

Early in the Collaboration, a paper by
J.H. Ferziger was  circulated,  offering  simple
advice  for
testing numerical  resolution.   Although some
of the  collaborators themselves urged us to impose
any  explicit  numerical  checks,  we decided  against it,
believing that  the  more
serious  errors   would  be  spotted  by  the  collaborators
themselves  when  comparing  their  results  with  those  of
others.   In particular,  we hoped  that the flat-plate test
cases would  be sufficient  to identify  serious failures of
grid independence  close to  a solid  surface, probably  the
most critical area in most turbulent flows.  Apart from this
the only test case which grid independence is likely to have
been a  serious issue is the flow over backward-facing step,
where the  singularity in  geometry at  the top  face of the
step requires  step lengths  in the $x$- and $y$-directions
which
are considerably  smaller than  one wall  unit.  Again, some
modelers submitted revised results after private querying of
their initial computations.  The curved jets (test cases 5.4 and 5.5) are  also  likely  to  cause  difficulty  in
numerics, because  of the  large angle  between the velocity
vector and the axis, so that rectangular meshes could lead
to large false diffusion.
\vskip 0.2in
\vfill\eject
\magnification=\magstep1
\normalbaselineskip=21pt
\def\pl{\partial}
\parskip=8pt
\parindent=0em
\vskip2.5in
\centerline{{\bf FINAL REPORT ON AFOSR 90-0154,}}
\vskip0.3in
\centerline{{\bf ``Collaborative Testing of Turbulence
Models''}}
\vskip0.3in
\centerline{{\bf 1 February 1990 -- 30 April 1992}}
\vskip1.0in
\centerline{{\bf P. Bradshaw, Principal Investigator}}
\vskip0.2in
\centerline{{\bf Mechanical Engineering Department}}
\centerline{{\bf  Stanford University Stanford, California
94305-3030}}
\vskip0.5in
December 1992\vfill\eject
\centerline{{\bf CONTENTS}}
\vskip0.5in
{\bf 1. INTRODUCTION} \hfill 1

{\bf 2. HISTORY} \hfill  3

{\bf 2.1 ``Entry'' test cases (flat-plate boundary layers)}

{\bf 2.2 August 1990 test data (Cases 3.1-3.5, 4.1-4.3)}

{\bf 2.3 Results for August 1990 test cases}

{\bf 2.4 August 1991 test data (cases 5.1-5.9)}

{\bf 3. CONCLUSIONS} \hfill 8

{\bf REFERENCES} \hfill 9

{\bf APPENDIX 1 THE ``FLAT PLATE'' BOUNDARY LAYER TEST CASES} \hfill 11

{\bf APPENDIX 2 THE MODELS} \hfill 12 

A2.1 The Process of Turbulence Modeling

A2.2 Review Of The Models

A2.2.1 Eddy-viscosity models

A2.2.2 Stress-equation models

A2.3 Numerical Errors

{\bf APPENDIX 3 TEST CASE SUMMARY AND TABULATED RESULTS} \hfill 17

{\bf APPENDIX 4 MODELERS' RESPONSES} \hfill 28

PROJECT Newsletter no. 6 follows page 40