%%------------------------------------------------------------------------
\section{Conclusion}
%%------------------------------------------------------------------------
\label{sec:conclusion}
%%------------------------------------------------------------------------

In this paper, we have presented the design of a \ccdsm/ system
layered on top of a message passing machine.  The design relies on a
simple address capture device and uses one of the processors in each
SMP for protocol processing.  Based on the problems we have
encountered, we draw the following conclusions:

%During the design, we ran into several sets of challenges which are
%described in this paper.  

\vspace{.25in}
\header{Processors and their Snoopy Buses:}

By necessity, the implementation of a \ccdsm/ system is layered on top
of some commercial processor's snoopy bus protocol.  The \ccdsm/
implementations can benefit from the snoopy bus having the following
properties.
\begin{itemize}
\item 
Invalidations should not be blocked by outstanding operations to
the same cache-line. (Section~\ref{sec:inv-vs-rd} and
Section~\ref{sec:cmp-mp})
\item
Cache-to-cache transfer of \ss/ as well as \ms/ data should be
supported. (Section~\ref{sec:cmp-mp})
\item
{\tt Shared Modified} state\cite{Thacker87}, which allows cache-to-cache transfers for
Reads of modified cache-lines without write-backs to memory should be
supported. (Section~\ref{sec:dp-sp-sharing}) 
\item
The snoopy bus should be a split-phase bus (Section~\ref{sec:sp-bus}).  
\end{itemize}
Though it is possible to work around snoopy bus protocol and processor
oversights with software running on the \sP/, it would be preferable
if we did not have to waste \sP/ cycles to do so.

\vspace{.25in}
\header{Buffering:}

To avoid deadlock, buffers should be partitioned into different
classes according to functionality.  For example, for invalidation
based protocol, Read Requests and Write-back Requests should not share
a common buffer pool.  (Section~\ref{sec:sep-bufs}).  To avoid
deadlocks, the service processor should not issue Invalidations.
(Section~\ref{sec:flush}) Static dependence graphs are a good way to
detect potential deadlocks due to buffer sharing.


\vspace{.25in}
\header{Network:}

Protocols are generally much more complicated for a non-FIFO network
because messages can get out of order.  The increased number of
messages may negate the performance advantage of non-FIFO networks.
It is also possible to build a layer of abstraction on top of non-FIFO
networks so that they appear to be FIFO.  While this simplifies the
protocol, it may take even more messages than dealing with the
non-FIFO problems directly in the protocol.

\vspace{.3in}

Our preliminary estimates indicate that \start/'s \ccdsm/ will perform
well for programs with good hit rates.  Like all \ccdsm/ machines, the
performance of coherent shared memory on \start/ depends mostly on the
global memory miss ratios.  Thus, a miss penalty that is three or four
times worse than that of an aggressive implementation can still give
good performance for a range of programs.  This is very encouraging,
considering the cost and experimental nature of our implementation.
Of course the actual performance can only be determined after we get
the machine built, and up and running.  We are looking forward to a
real \start/ machine on which to experiment and measure performance.

%%------------------------------------------------------------------------
%%                             Section End
%%------------------------------------------------------------------------

