Date: Thu, 3 Apr 2003 11:04:52 +0200
From: Jesper Larsson Traeff <traff@XXXXXXXXXXXX>
To: mpi-21@XXXXXXXXXXXXX
Subject: Re: Correction to One-sided communications, Section 6.7: Semantics and Correctness
References: <20030402091600.GB22715@XXXXXXXXXXXXXXXXXXXX> <PEEAKHMBIHHOHMELDDJKIEPGCCAA.thakur@XXXXXXXXXXX>Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <PEEAKHMBIHHOHMELDDJKIEPGCCAA.thakur@XXXXXXXXXXX>
User-Agent: Mutt/1.4i
X-Virus-Scanned: by amavisd-milter (http://amavis.org/
X-Spam-Status: No, hits=-3.0 required=5.0 tests=IN_REP_TO,DEAR_SOMEBODY,GUARANTEE version=2.21
X-Spam-Level: 
Sender: owner-mpi-21@XXXXXXXXXXXXX
Precedence: bulk
Reply-To: mpi-21@XXXXXXXXXXXXX


Dear Rajeev,

Here are two explicit (correct, I hope) examples illustrating the point

EXAMPLE 1:
----------

Process A:                              Process B:

MPI_Win_post(A,B,...)                   MPI_Win_start(A)
MPI_Win_start(A)

local updates to A's private window     MPI_Get/MPI_Put ...
(explicitly or with MPI_Put)            (it doesn't matter whether B has 
                                         access to A or not in this epoch)
MPI_Win_complete
MPI_Win_Wait                            MPI_Win_complete


                      <one-sided limbo>

                                        MPI_Win_lock(A)
                                        MPI_Get(something A locally updated in
                                                previous epoch)
                                        MPI_Win_unlock

For this to work, process A's updates to its private window must be publicly
visible after MPI_Win_Wait

EXAMPLE 2: (same idea...)

Process A:                              Process B:

MPI_Win_Lock(B)
MPI_Put(something to B)
MPI_Win_unlock

               MPI_Barrier or other means of sync 

                                        MPI_Win_post(B,...)
                                        MPI_Win_start(B)
                                        
                                        access to stuff in the private window

                                        MPI_Win_complete
                                        MPI_Win_wait

MPI_Win_post at the latest must make the update of process A to the public 
window copy visible in B's private copy 

Jesper

On Wed, Apr 02, 2003 at 12:05:01PM -0600, Rajeev Thakur wrote:
> Jesper,
> 
> > In post-wait epoch i process A makes private updates; the epoch
> > is ended with
> > MPI_WIN_WAIT. In epoch i+1 another process accesses memory of
> > process A, using
> > MPI_WIN_LOCK to open epcoh i+1.
> > As per rule 5 (as it is now) IT IS NOT GUARANTEED THAT THE PRIVATE UPDATES
> > ARE VISIBLE IN THE PUBLIC WINDOW
> >
> > I therefore think that MPI_WIN_WAIT (the one that ends epoch i)
> > should enforce
> > that the private update becomes publicly visible
> 
> I think the standard tries to not encourage local updates to a window
> between post and wait. It explicitly forbids local updates in the case of
> simultaneous puts or accumulates from remote proceses (pg 139), but doesn't
> say anything about gets though. I think that is why rule is the way it is.
> In your example, the user would have to do, say, a local lock and unlock
> across the local update before the remote process can access. It would be
> easier if he just did the local update before the post.
> 
> > Another problem with rule 5 (as it is now, regarding only MPI_WIN_POST),
> > is that it somewhat contradicts rule 3, which says that WIN_WAIT completes
> > updates on the target. The private updates could have been done
> > by MPI_PUT's,
> > and thus, according to rule 3, should be completed by the WAIT
> 
> If the user wants to do a local MPI_Put, he would also have to call
> win_start and win_complete locally in addition to the post and wait. So it
> goes through the entire RMA apparatus. But I see what you are saying:
> MPI_Win_wait could do what needs to be done. I think the reason may be the
> one I gave above, that local accesses within an exposure epoch are
> discouraged.
> 
> > > Similarly, Rule 6 specifies when the update made by a remote
> > process becomes
> > > visible to the local process. In the case of post-wait synchronization,
> > > clearly, it is MPI_Win_wait that "completes" the operation, and
> > therefore
> > > only after win_wait is called can one expect the update made by
> > the remote
> > > process to be visible to the local process.
> > >
> >
> > By the same reasoning as above, I think MPI_WIN_WAIT is too late. Imagine
> > a process doing an update on process A's private window in lock-epoch i.
> > Epoch i is ended by MPI_WIN_UNLOCK. If process A now opens epoch i+1 (and
> > exposes its window to itself) with MPI_WIN_POST, rule 6 as it is now, does
> > not guarantee that the update on A's private window is indeed visible to A
> > in epcoh i+1.
> >
> > It seems that MPI_WIN_POST must enforce that the public update in epoch i
> > is privately visible in epcoh i+1
> 
> I see what you are saying here, but if you assume that a process calls
> win_post to expose its window to RMA operations from others (including
> itself), that is if you discount the direct local memory read case, then A
> can call MPI_Get to read what was written in the previous epoch because
> MPI_Get accesses the public copy of the window. Or else, if A wants to do a
> direct memory read, it can call a quick lock-read-unlock locally. But I see
> your point.
> 
> Rajeev