Date: Fri, 4 Apr 2003 13:49:49 +0200 From: Jesper Larsson Traeff To: mpi-21@XXXXXXXXXXXXX Subject: Re: Correction to One-sided communications, Section 6.7: Semantics and Correctness References: <20030403090452.GB24638@XXXXXXXXXXXXXXXXXXXX> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Virus-Scanned: by amavisd-milter (http://amavis.org/ X-Spam-Status: No, hits=-3.0 required=5.0 tests=IN_REP_TO,DEAR_SOMEBODY,GUARANTEE version=2.21 X-Spam-Level: Sender: owner-mpi-21@XXXXXXXXXXXXX Precedence: bulk Reply-To: mpi-21@XXXXXXXXXXXXX Dear Rajeev, As far as I remember, the standard does try to enforce that local load/store accesses within epochs behave sensibly; for instance on p.132 it says that an MPI_Win_lock used to lock the local window must block, "since the lock may protect local load/store accesses". I propose to make the correction to the standard I suggested, that is, swap POST and WAIT in rules 5 and 6 (as it is now, is just a mistake, I think) Jesper On Thu, Apr 03, 2003 at 09:44:37PM -0600, Rajeev Thakur wrote: > Jesper, > In Example 1, if A does a local update with MPI_Put, I think it will > work. That is because MPI_Put updates the public copy of A's window (I > think) and the public version will be visible after win_complete and > win_wait, and B can read it during the next lock/unlock. I agree that if A > does a local update with a memcpy, B is not guaranteed to read it in the > next lock/unlock. Whether we should change the standard to allow it... I > don't know. > > Also, in Example 1, it is illegal for B to do an MPI_Put between the start > and complete if A is doing a local update. B can do an MPI_Get though. > > Similarly, in Example 2, if B does a local access using MPI_Get, I think it > will work because MPI_Get should access the public version of B's window. If > it does a local access directly, I agree it won't work. > > Rajeev > > > > -----Original Message----- > > From: owner-mpi-21@mpi-forum.org [mailto:owner-mpi-21@mpi-forum.org]On > > Behalf Of Jesper Larsson Traeff > > Sent: Thursday, April 03, 2003 3:05 AM > > To: mpi-21@mpi-forum.org > > Subject: Re: Correction to One-sided communications, Section 6.7: > > Semantics and Correctness > > > > > > > > Dear Rajeev, > > > > Here are two explicit (correct, I hope) examples illustrating the point > > > > EXAMPLE 1: > > ---------- > > > > Process A: Process B: > > > > MPI_Win_post(A,B,...) MPI_Win_start(A) > > MPI_Win_start(A) > > > > local updates to A's private window MPI_Get/MPI_Put ... > > (explicitly or with MPI_Put) (it doesn't matter whether B has > > access to A or not in this epoch) > > MPI_Win_complete > > MPI_Win_Wait MPI_Win_complete > > > > > > > > > > MPI_Win_lock(A) > > MPI_Get(something A > > locally updated in > > previous epoch) > > MPI_Win_unlock > > > > For this to work, process A's updates to its private window must > > be publicly > > visible after MPI_Win_Wait > > > > EXAMPLE 2: (same idea...) > > > > Process A: Process B: > > > > MPI_Win_Lock(B) > > MPI_Put(something to B) > > MPI_Win_unlock > > > > MPI_Barrier or other means of sync > > > > MPI_Win_post(B,...) > > MPI_Win_start(B) > > > > access to stuff in the > > private window > > > > MPI_Win_complete > > MPI_Win_wait > > > > MPI_Win_post at the latest must make the update of process A to > > the public > > window copy visible in B's private copy > > > > Jesper > > > > On Wed, Apr 02, 2003 at 12:05:01PM -0600, Rajeev Thakur wrote: > > > Jesper, > > > > > > > In post-wait epoch i process A makes private updates; the epoch > > > > is ended with > > > > MPI_WIN_WAIT. In epoch i+1 another process accesses memory of > > > > process A, using > > > > MPI_WIN_LOCK to open epcoh i+1. > > > > As per rule 5 (as it is now) IT IS NOT GUARANTEED THAT THE > > PRIVATE UPDATES > > > > ARE VISIBLE IN THE PUBLIC WINDOW > > > > > > > > I therefore think that MPI_WIN_WAIT (the one that ends epoch i) > > > > should enforce > > > > that the private update becomes publicly visible > > > > > > I think the standard tries to not encourage local updates to a window > > > between post and wait. It explicitly forbids local updates in > > the case of > > > simultaneous puts or accumulates from remote proceses (pg 139), > > but doesn't > > > say anything about gets though. I think that is why rule is the > > way it is. > > > In your example, the user would have to do, say, a local lock and unlock > > > across the local update before the remote process can access. > > It would be > > > easier if he just did the local update before the post. > > > > > > > Another problem with rule 5 (as it is now, regarding only > > MPI_WIN_POST), > > > > is that it somewhat contradicts rule 3, which says that > > WIN_WAIT completes > > > > updates on the target. The private updates could have been done > > > > by MPI_PUT's, > > > > and thus, according to rule 3, should be completed by the WAIT > > > > > > If the user wants to do a local MPI_Put, he would also have to call > > > win_start and win_complete locally in addition to the post and > > wait. So it > > > goes through the entire RMA apparatus. But I see what you are saying: > > > MPI_Win_wait could do what needs to be done. I think the reason > > may be the > > > one I gave above, that local accesses within an exposure epoch are > > > discouraged. > > > > > > > > Similarly, Rule 6 specifies when the update made by a remote > > > > process becomes > > > > > visible to the local process. In the case of post-wait > > synchronization, > > > > > clearly, it is MPI_Win_wait that "completes" the operation, and > > > > therefore > > > > > only after win_wait is called can one expect the update made by > > > > the remote > > > > > process to be visible to the local process. > > > > > > > > > > > > > By the same reasoning as above, I think MPI_WIN_WAIT is too > > late. Imagine > > > > a process doing an update on process A's private window in > > lock-epoch i. > > > > Epoch i is ended by MPI_WIN_UNLOCK. If process A now opens > > epoch i+1 (and > > > > exposes its window to itself) with MPI_WIN_POST, rule 6 as it > > is now, does > > > > not guarantee that the update on A's private window is indeed > > visible to A > > > > in epcoh i+1. > > > > > > > > It seems that MPI_WIN_POST must enforce that the public > > update in epoch i > > > > is privately visible in epcoh i+1 > > > > > > I see what you are saying here, but if you assume that a process calls > > > win_post to expose its window to RMA operations from others (including > > > itself), that is if you discount the direct local memory read > > case, then A > > > can call MPI_Get to read what was written in the previous epoch because > > > MPI_Get accesses the public copy of the window. Or else, if A > > wants to do a > > > direct memory read, it can call a quick lock-read-unlock > > locally. But I see > > > your point. > > > > > > Rajeev > > > >