\documentstyle[11pt]{article}
\input setup
\input macros

%%% Illustrations --from ILLUSTRATIONS.TEX
\def\illustrationfileprefix{/bk/motomura/pics/}
\def\illfileprefix{/bk/motomura/pics/}

\input{psadobe}
\def\illustration#1{\begin{center}
{\psadobe{\illustrationfileprefix#1.ps}}
\end{center}}
\def\illustrationpercent#1#2{\begin{center}
{\psadobepercent{#2}{\illustrationfileprefix#1.ps}}
\end{center}}

\title{An Analysis of Memory Reference Behavior in A Tightly Coupled
	Multithreaded Architecture}

\author{Masato Motomura\thanks{On leave from System ULSI Research
	Laboratory, Microelectronics Research Laboratories, NEC
Corporation. Phone: 81-427-71-xxxx, Fax: 81-427-71-xxxx, Email:
motomura@mel.cl.nec.co.jp}\\Gregory M.Papadopoulos\thanks{Phone:
1-617-253-2623, Fax: 1-617-253-6652, email: greg@abp.lcs.mit.edu}
\\ \\Laboratory for Computer Science 	\\Masachusetts Institute of
Technology\\Cambridge, MA 02139}



\date {\today}

\begin{document}
\maketitle

\begin{abstract} 
Multithreaded(MT) architecture has been drawing attention because of
its capability of hiding long memory latency by switching among
threads. In a tightly coupled MT architecture, each thread comes from
one of code blocks in a single program. Tightly coupled MT is quite
attractive because it can fully utilize fine grained parallelism that
exists in a single program.  However, possible impact of this
architecture on programs' memory reference locality has not been
studied well.  In this paper, we present the results of trace
simulation study on the locality shown in a tightly coupled MT
architecture. Results shows LRU stack locality model found in single
thread arhitectures also holds for the architecure. We discuss the
relation between parallelism and locality based on this model. The
influence of thread scheduling is also discussed. Furthermore, We
build an analitical cache model based on the analysis of the locality
behavior. The model quantitatively predicts simulated cache miss ratio
quite well. We also propose as adequate cache organization and thread
scheduling strategy to obtain a better hit ratio in a tightly coupled
MT architecture.



\end{abstract}  

%\bibliographystyle{plain}
%\pagestyle{empty}
%\markright{DRAFT: \today}

\newpage
\pagestyle{plain}

%1
\section{Introduction}
\label{Introduction}
How to utilize parallelism and locality are the two major concerns
when one tries to build a high performance computer. In general, no
need to say, computation can be get done faster either by using
several processing units in parallel or by keeping data to be
processed near the processing unit.  Unfortunately, however, these two
aspects of computation often contradict each other.

One good exmaple of this contradiction is a long communication latency
problem encountered in massively parallel computer
architecture\cite{*t1}.  In massively parallel computer, many
processor nodes are utilized to get high performance, resulting in
distribution of the data to be calculated among processing nodes. This
lack of locality causes long communication latency, which prevents the
desired performance improvement.

The multithreaded(MT) architecture\cite{*t2}\cite{*t3} tries to hide
this penalty of localty degradance by utilizing ``parallel
slackness'', i.e., do whatever available now, rather than waiting the
response. There has been plenty of research activity in MT
architecture field recently, because of this simple policy seems
promising\cite{*t4}. Here we categorize MT architecture into two
groups, loosely coupled MT and tightly coupled MT. In the former
architecture, threads from several programs are ecxecuted to avoid
processor idle state during a long latency\cite{*t5}\cite{*t6}. In
some sense, in this category of architecture, multithreading is added
afterwords to hide communication ratency rather than its original
nature.

On the other hand, in a tightly coupled MT architecture,
multithreading is a direct consequence of its execution model. A
program is compiled into a set of threads which is defined as a stream
of instructions that are garanteed at a compile time to be executed
without suspendtion\cite{*t6}. A Thread is executed whenever it is
ready and the scheduling mechanism chooses this thread. In This
tightly coupled MT architecture, long communication latency is not a
problem provided that there are many other threads that are ready.
This architecture, which is clearly a direct descendant of dataflow
architecture, has an advantage of utilizing fine grained parallelism
in a single program and torelating communication latency.

Whatever the origin and grain size of MT architectures, the simple
idea which characteries them seems to work well for solving the
latency tolerance issue.  Interplay between parallelism and locality,
however, is not completely solved yet. Their contradiction do exist in
one level below, i.e., in each processing node. Contexts each thread
may compete each other in a local cache, that may cause high cache
miss ratio. As is well articulated, todays high performance
microprocessors, which are to be used as a processing node in a
massively parallel computer, high cache miss ratio has sivere
performance effect.

In a field of loosely coupled MT architecture, there is a study of
cache miss ratio\cite{*t8}, which treats MT as an extreme case of
multiprograming. Contexts change is treated as process switch in a
short interval where each process has not completely build its own
working set. The result shows linear degradance against number of
context exist in a single processing node. In a tightly coupled MT
architecture, however, this treatment is not adequite. First of all,
as we will see in Section 2, number of contexts are not directly
controllable, and second, given that threads come from a same program,
switching threads may share working set and may not cause high cache
miss ratio. Actually, the definition of working set in this
architecture is rather ambiguous. Each competing threads are sharing
working set of a same program, or compeling each other's local working
set. We have to build up clear notion of working set for deciding
adequate thread sheduling policy or cache organization.

\subsection {Paper Overview}
In this paper, we will closely investigate this issue by utilizing
both simulation and analytical method. In section 2 we present the
tightly coupled MT architecture and its execution model which lies a
basis of our study.  Section 3 describes trace simulation set up. The
result of the simulation is presented in Section 4 with an analisys of
relation between parallelism and locality behavior. Section 5 gives an
analitical cache model that explains the simulation result
quantitatively very well. Section 6 gives a thought for an adequate
cache organization and thread a execution scheduling in tightly
coupled MT architecture.



%2
\section{Tightly coupled MT execution model}



%2.1	
\subsection{*T Ecexution Model}
\label{*T Execution Model}

\section{Trace Driven Simulation}
Previous simulation studies of MT architecture have always been using
traces obtained by artificially interleaving reference streams from
several contexts\cite{Weber}\cite{Agarwal-2} simply because real trace
was not available . Though this kind of approach has its meaning for
loosely coupled MT architectures, it is impossible to capture memory
reference behavior of a titghtly coupled MT model in this way.  As
explained in Section~\ref{Introduction}, context switch in the
architecuture is deeply involved in its fine grained parallel
execution model: it does not make sense to treat number of contexts or
context switch intervals as ad hoc parameters. Thus it is really
important to get real multithreaded memory reference trace for the
study of tightly coupled MT architectures. Our study, as far as the
authours know, is the first try to explore general behavior of memory
references in this architecture by utilizing real traces.

High level *T simulator has been developed at Motorola Cambridge
Research Center\cite{Andrea} for the purpose of verifing several
aspects of *T architecture in an early stage of its design. We used
this simulator for getting memory traces, and ran our own simulator on
them to collect statistics. We first explain the *T simulator briefly.
This simulator uses Id language and TAM compiler\cite{Culler} as its
substrate.  A program written in Id is compiled by TAM compiler into a
threaded code. TL0, an high level machine language for TAM, is used
for the genterated code. The simulator simulates dynamic scheduling of
this code in multiprocessor environment by applying *T execution model
explained in Section~\ref{*T Execution Model}. Memory reference traces
for each processor is obtained by substituting RISC type instruction
for each of scheduled TL0 instructions. This simulator is farely high
level and is not designed for predicting absolute *T performane. Since
the purpose of our study, however, is to explore the general feature
found in a tightly coupled MT architecture, this drawback of our
simulation environment has little impact for the quality of results.

\subsection{Charactersitics of Traces}
We ran following two benchmark programs for our study.

\begin{description}
\item[Gamteb:] An Monte carlo simulation program of photons
traversing through a carbon rod. It has one outermost loop, and the
code inside consists of banch of code blocks, whose calling path
depends on a number randomly generated for each iteration of the loop.

\item[Matrix Multiply(MMP):] A triply nested loop program calculating
multiplication of two matrixes. No blocking is used in this program.
\end{description}

We ran above programs on various system configurations, data size, and
loop bounds. We report here simulation results 8 node configuration,
20,000 photons for Gamteb and 600 $\times$ 600 matrix for MMP. Three
kinds of loop bounds are experimented for each program: 50, 200 and
800 for Gamteb, and 25, 50 and 100 for MMP. Note that loop bounds in
MMP is applied for all of the three loops. Furthermore, to see how
scheduling infuluence a program' smemory reference behavior, We
conducted experiments on thread scheduling policy: We used two
policies, LIFO and FIFO, for a scheduling of local continuations.  A
scheduling of global continuations is fixed to FIFO, in order to
maintain FIFO ordering of a network. The two programs have opposite
characteristics in a sense that Gamteb has only one loop, and MMP is
heavily looped. It is quite interesting to see how behavior changes in
accordance with variations of a loop bound and/or a local continuation
scheduling policy. In total, the number of run we report here is 12.


(NOTE: It should be declared in Introduction that we will do simulation
by changing loop bound and scheduling policy)

Table~\ref{static-tab} summarizes static information for these two
benchmark programs.  Program sizes and thread lengths are measured in
RISC type instruction count.  Aparently, MMP's program size is too
small for the purpose of instruction cache simulation. For data cache,
however, this program is a good test of locality behavior, because it
is well studied that to get high data cache hit ratio for the program
is quite difficult\cite{Monica}. In both programs, the average length
of syncronization threads, around 5 instructions, are considerably
shorter than those of Data threads, well above 10 instructions.This is
quite expectable result from natures of each type of threads. The
average number of syncronization threads, however, is large than one
may imagine. A compiler could be improved to decrease this number by
compiling away unnecessary syncronization event. 


\begin{table}
\begin{center}
\begin{tabular}{||c|l|r|r||}
\hline
\multicolumn{2}{||c|}{Benchmark Name}   &Gamteb   &MMP \\\hline
\hline
\multicolumn{2}{||c|}{Program size (Instruction)}     &24,921   &4,238 \\ \hline
\multicolumn{2}{||c|}{Number of Code Blocks} &51   &13 \\ \hline \hline
Data    &Avg. Number (/CB)   &16.76  &14.69 \\ \cline{2-4}
Threads &Avg. Length (Instruction)  &21.03  &14.63 \\ \hline
Sync.   &Avg. Number    &28.31  &19.69 \\ \cline{2-4}
Threads &Avg. Length  &4.81   &5.64 \\ \hline \hline
\multicolumn{2}{||c|}{Average Thread Length}  &10.84   &9.48 \\ \hline
\end{tabular}
\end{center}
\caption{Static Information of Benchmark Programs.}\label{static-tab}
\end{table}

Table~\ref{trace-length-tab} shows trace length for two runs out of 12
runs that we explore in this paper.  Other runs for each benchmark has
essencialy the same size. The traces are not the ones for all nodes
from the begining of the execution to the end of execution, mainly
because the lack of time and disk space to accomodate full traces.
Instead, We recoreded memory reference trace for one of 8 nodes until
the statistics We report later has reached stable and the number of
memory references is long enough for simulation of caches less than
64KB. For justifing using traces for one node, We verified that
statistics variations between nodes are negligible.  We will discuss
other aspects of possible drawbacks caused by this trace collection
method wherever necessary. About the length of traces, all refereces
except I_structure references are long enough to simulate desired size
of caches. Since I_structure references are not a primary concern for
our local cache study, though we will report cache miss rate result
later, this lack of sufficient trace lengh is not a problem. The
reason why is that I_structure references experience long network
latency, which makes caching behavior less important.

\begin{table}
\begin{center}
\begin{tabular}{||c|c|c||c|c|c|c||}
\hline
Benchmark &Scheduling &Loop  &Number of &Instruction &Frame &I\_structure \\
Name      &Policy     &Bound &Threads &References &References
&References \\ \hline \hline
Gamteb &LIFO &200 &0.69M &7.95M &3.66M &0.18M\\ \hline
MMP    &LIFO &50  &0.81M &9.40M &4.60M &0.34M\\ \hline
\end{tabular}
\end{center}
\caption{Trace Lengh Examples}\label{trace-length-tab}
\end{table}

Figure~\ref{proc-util-fig} shows processor utilization of each run by
dividing processor time into Data thread execution time,
Syncronization thread execution time and Idle time. This graph takes
into account only network latency for calculating Idle time. The large
fraction of idle time in MMP can be explained by the fact that only a
small amout of calculation is done per one I-structure reference in
this program. There isn't significant difference in processor
utilization between LIDO and FIFO scheduling, though slight
improvement can be found for Gamteb. There is a draqback from not
using full trace here. As is wel studied, amount of parallelism in a
program, which strongly affects idle time, changes along execution. So
one should not think Figer~\ref{proc-util-fig} as predicting
performance of these programs. Rather, the important consequence of
Figure~\ref{proc-util-fig} is that choice of loop bounds are just
around the saturation point of processor utilization~\cite{Saavedra},
where resource usage is optimized, for the duration of program
execution we study. 


\begin{figure}
\illustrationpercent{proc-util-fig}{80}
\caption{Processor utilization for 12 runs. The processor time is
devided into Idle time, Syncronization thread execution time and Data
thread execution time. Note that y-axis minimum is not 0.}\label{proc-util-fig}
\end{figure}

\section{Local Memory Reference Behabior Analysis}
Since We are seeing memory references which no one have seen, it is
important to investigate into it closely before trying to understand
cache memory behavior. We analyze what is happening for frame and
instruction memory references in tighly coupled MT architecture. 

\begin{table}
\begin{center}
\begin{tabular}{||c|c|r||r|r|r|r||}
\hline
Benchmark &Scheduling &\multicolumn{1}{|c||}{Loop}  &\multicolumn{1}{|c|}{Frame}
&\multicolumn{1}{|c|}{Frame Life} &\multicolumn{1}{|c|}{Frame Exec.} 
&\multicolumn{1}{|c||}{Extant} \\
Name      &Policy     &\multicolumn{1}{|c||}{Bound} &\multicolumn{1}{|c|}{Size (W)}
&\multicolumn{1}{|c|}{Time (T)}   &\multicolumn{1}{|c|}{Time (T)}
&\multicolumn{1}{|c||}{Frames} \\
\hline \hline
       &      &50  &80.25 &2975.81 &62.92  &63.38\\ \cline{3-7}
       &STACK &200 &80.86 &5294.50 &62.94  &161.49\\ \cline{3-7}
Gamteb &      &800 &81.61 &6559.92 &64.06  &372.96\\ \cline{2-7} \cline{2-7}
       &      &50  &81.21 &1846.34 &63.91  &39.51\\ \cline{3-7}
       &FIFO  &200 &79.06 &8044.61 &62.25  &169.95\\ \cline{3-7}
       &      &800 &80.47 &15445.56 &62.27 &374.81\\ \hline \hline
       &      &25  &17.45 &11306.96 &2671.69 &18.21\\ \cline{3-7}
       &STACK &50  &19.14 &20018.77 &2469.50 &43.01\\ \cline{3-7}
MMP    &      &100 &20.30 &32025.45 &2339.11 &92.25\\ \cline{2-7} \cline{2-7}
       &      &25  &17.24 &12311.56 &2512.49 &21.86\\ \cline{3-7}
       &FIFO  &50  &18.45 &22778.21 &2461.87 &43.12\\ \cline{3-7}
       &      &100 &19.53 &46450.58 &2368.63 &89.93\\ \hline
\end{tabular}
\end{center}
\caption{Frame Statistics}\label{frame-tab}
\end{table}


\subsection{Frame Reference}
Table~\ref{frame-tab} shwos Frame statistics for each run. It is
interesting to see how frame life time and the number of extant frames
varies against loop bound and schduling. In both Gamteb and MMP, there
are strong positive correlation between loop bound and Frame life
time, which is quite natural. Furthermore, STACK scheduling generally
gives shorter life time, which means frame locality is better kept in
this scheduling This is an expected result. The number of extant
frames, which is a number of allocated time at some time, does not
show any significant difference between these two scheduling. This
means what ??? One thing to note here is that MMP's frame execution
time is more than one order longer than Gamteb's one, despite the fact
that the frame size is about 1/4. This means good frame locality of
MMP, i.e., hit ratio should be high for MMP.\\

Probably the most important question about frame reference behavior is
that, what kind of correlation there is between a frame for a thread
and a frame for a next thread. To figure out this, We measure a
probability distribution of based on LRU stack reference
model\cite{???}. 

\begin{figure}
\illustrationpercent{fram-lru-gam-fig}{60}
\caption{Frame LRU jump probabilities for Gamteb runs.}
\end{figure}

\begin{figure}
\illustrationpercent{fram-lru-mmp-fig}{60}
\caption{Frame LRU jump probabilities for MMP runs.}
\end{figure}

\begin{figure}
\illustrationpercent{fram-sd-lru-gam-fig}{60}
\caption{Frame LRU jump probabilities for Gamteb runs.}
\end{figure}

\begin{figure}
\illustrationpercent{fram-sd-lru-mmp-fig}{60}
\caption{Frame LRU jump probabilities for MMP runs.}
\end{figure}

\begin{figure}
\illustrationpercent{fram-wind-gam-fig}{60}
\caption{Frame Wind}
\end{figure}

\begin{figure}
\illustrationpercent{fram-wind-mmp-fig}{60}
\caption{Frame Wind}
\end{figure}



\subsection{Locality in Instruction access}


\section{Tightly coupled Multithread Cache Model}

We will buid up analytical cache model in this section and compare it
to trace simulation data. Before starting analysis, we briefly look
into previous work in cache modeling.


Agarwal,,,,




\subsection{Frame Cache}
We calculate miss rate of a frame cache by counting how many accesses
from a thread miss the cache on average. Let $T_{F}$ be an average
number of total frame reference from a thread, and $M_{F}$ be an
average number of missed reference in a thread, miss rate $m_{F}$ is
defined as
\[m_{F}(L, A, S)=\frac{M_{F}(L, A, S)}{T}.\]
Here, We use $L$ for Line length measured in Byte, $A$ for
associativity in a set, and $S$ for number of set, so that cache
capacity is expressed as $LAS$. In the following discussion, we omit
L, A, S dependency wherever approapriate for notational simplicity.

$T_{F}$ can be divided into three components: $T_{Fc}$, first
reference to cache lines in that frame allocation, $T_{Ft}$, reference
to cache lines which were referenced by other instructions in a same
thread, and $T_{Fi}$, reference to cache lines which were not
referenced in a same thread.Generally, each of $T_{Fc}$ reference
directly gives a miss, which is known as compulsory miss, because it
is the first access to a line any way. It doesn't hold for our case,
however, because $T_{Fc}$ is a first request to a frame location within
one allocation of a frame, and same frame pointer can be reused after
deallocation of that frame. So we can define compulsory miss rate
$m_{Fc}$, so that $m_{Fc}T_{Fc}$ is number of compulsory miss
references.$T_{Ft}$ is a measure of temporal and spatial locality
within one thread.  It is independent of dynamic execution of threads,
because a thread is guaranteed to execute until its completion. Each
of $T_{Ft}$ references gives a hit. To get $T_{Ft}$ from a static code
is quite straitforward, and we are not interested in this
number.$T_{Fi}$ is a most important source of miss, which is known as
interference miss.  We define $m_{Fi}$ as an interference miss rate,
so that $m_{Fi}T_{Fi}$ gives a number of interference miss
references.In totl, the miss rate is given as
\[m_{F}=\frac{T_{Fc}}{T_{F}}m_{Fc}+\frac{T_{F}-T_{Fc}-T_{Ft}}{T_{F}}m_{Fi}.\]

We first explain how to calculate compulsory miss rate component. We
use $F$ to denote average frame size measured in Byte. The number of
cache lines required to hold a frame size of $F$ is given by
\[C(F)=1+\frac{F-1}{L}\]
as shown in \cite{Agarwal-1}. Compulsory miss rate $m_{Fc}$ is given by
\[\frac{R_{c}}{C(F)}\],
where $R_{c}$ is an average number of cache lines which hold a frame
when that frame is deallocated. Suppose the probability of reuse of a
same frame pointer after its deallocation is $p_{r}$, 
\[p_{r}c(F)\left(1-\frac{R_{c}}{C(F)}\right)=(1-p_{r})C(F)\left(1-\frac{R_{c}}{S}\right)
	+(N_{c}-1)C(F)\left(1-\frac{R_{c}}{S}\right)\],
thus 
\[R_{c}=\frac{C(F)}{1+\frac{N_{c}-p_{r}}{p_r}\frac{C(F)}{S}}\]

	




$F$: Averageframe size,\\ $C$: Average code block size,\\ $I$: Average
thread length,\\ 	 	= Average number of instruction access
from a thread\\ $T$: Average number of frame access from a thread,\\

$M_{F}$: Frame cache miss reference\\
$M_{I}$: Instruction cache miss reference\\

$m_{F}$: Frame cache miss ratio\\
$m_{I}$: Instruction cache miss ratio\\


$I$ and $T$ can be further divided into three components, $I_{c}$ and
$T_{c}$, first access to a cache line, $I_{t}$ and $T_{t}$, access that
are for the same line(s) accessed in a same thread, and $I_{i}$ and
$T_{i}$, which is an access to a line not accessd in that thread. Thus,

\[T=T_{c}+T_{t}+T_{i}\] and,
\[F=F_{c}+F_{t}+F_{i}\].  

Correspondingly, cache miss reference $M_{F}$ and $M_{I}$ can be
devided into three components:

\[M_{F}=M_{Fc}+M_{Fi}\] and,
\[M_{I}=M_{Ic}+M_{Ii}\].  

Thus miss rate is obtained as
\[m_{F}=\frac{M_{Fc}}{F}+frac{M_{Fi}}{F}\]

We will look at each component of equation. 


$T_{c}$ causes miss, which is known as compulsory miss. In our case,
however, there is a chance


$W_{F}(t)$: Working set of frame measured at time interval t,\\
$W_{CB}(t)$: Working set of code block measured at time interval t,\\
$W_{t}(t)$: Working set of thread measured at time interval t,\\
$p_{F}(t)$: Working set of frame measured at time interval t,\\
$p_{CB}(t)$: Working set of code block measured at time interval t,\\
$p_{t}(t)$: Working set of thread measured at time interval t,\\

$p_{F}$:
$p_{R}$:
$N_{F}$:
$p_{T}$:
$p_{CB}$:

$\tau(S)$:





\subsection{Validation}



\begin{enumerate}
\item Move $V$ and $FP$ of a message into registers, and store $V$ into
memory, 
\item Add $r_{IP}$ and $L_D$ (immediate) to calculate a target
instruction pointer, 
\item Load a counter for the join operation (called join counter),
and branch if it is not 0 (join fail),
\item If the join has succeeded, jump to the target instruction calculated
in step 2,
\item If the join has failed, subtract 1 from the join counter and store it
back to memory, 
\item Move $IP$ of a next message into a register and jump to the
instruction pointed.  
\end{enumerate}


\newpage
\begin{thebibliography}{99}

\bibitem {Weber} Wolf-Dietrich Weber and Anoop Gupta.  Exploaring the
Benefits of Multiple Hardware Contexts in a Multiprocessor
Architecture: Preliminary Results. In Proceedings of 16th Annual
International Symposium on Computer Architecture, IEEE, June 1989, pp.
273-280

\bibitem {Agarwal-1} Anant Agarwal, Mark Horowitz and John Hennessy.
An Analytical Cache Model. ACM Transactions on Coumputer Systems, Vol.
7, No. 2, May 1989, Pages 184-215

\bibitem {Agarwal-2} Anant Agarwal. Performance Tradeoffs in
Multithreaded Processors. IEEE Transactions on Parallel and
Distributed Systems, Vol. 3, No. 5, September 1992, Pages 525-539.

\bibitem {Monica} Monica S. Lam, Edward E. Rothberg and Michael E.
Wolf. The Cache Performance and Optimizations of Blocked Algorithms.
In Proceedings of 19th Annual International Symposium on Computer
Architecture, IEEE, June 1991, pp. 63-74

\bibitem {Culler} David E. Culler, Anurag Sah, Klaus Erik Shauser,
Thorsten von Eicken and Jhen Wawrzynek. Fine-grain Parallelism with
Minimal Hardware Suppot: A Compiler-Controlled Threadsd Abstract
Machine. In Proceedings of 19th Annual International Symposium on
Computer Architecture, IEEE, June 1991, pp. 164-175.

\bibitem {Saavedra} Rafael H. Saavedra-Barrera, David E. Culler and
Thorstem von Eicken. Analysis of multithreaded architectures for
parallel computing. In proceeding of 2nd Annual ACM Symposium on
Parallel Algorithms and Architectures, IEEE, July, 1990, pp.169-177.

\bibitem {*t5} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275

\bibitem{*t6} Rishiyur S.Nikhil, Gregory M. Papadopoulos and Arvind. *T: a Killer
Micro for a Brave New World. CSG Memo 325

\bibitem {*t7} Rishiyur S.Nikhil, Gregory M. Papadopoulos and Arvind. *T: A
Multithreaded Massively parallel Architecture. CSG Memo 325-1

\bibitem {*t8} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275
\bibitem{*t1} Rishiyur S.Nikhil, Gregory M. Papadopoulos and Arvind. *T: a Killer
Micro for a Brave New World. CSG Memo 325

\bibitem {*t9} Rishiyur S.Nikhil, Gregory M. Papadopoulos and Arvind. *T: A
Multithreaded Massively parallel Architecture. CSG Memo 325-1

\bibitem {*t10} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275

\bibitem {*t11} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275

\bibitem {*t12} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275

\bibitem {*t13} Robert A. Iannucci. Toward a Dataflow/von Neumann Hybrid Architecture.
CSG Memo 275

\end{thebibliography}


\end{document}
