From treumann@XXXXXXXXXX Wed Feb 6 12:28:00 2008 In-Reply-To: To: "Mailing list for discussion of MPI 2.1" X-Mailer: Lotus Notes Release 7.0 HF277 June 21, 2006 From: Richard Treumann Date: Wed, 6 Feb 2008 13:22:51 -0500 X-MIMETrack: Serialize by Router on D01ML064/01/M/IBM(Release 7.0.2FP2 IGS702FP2HF5|November 8, 2007) at 02/06/2008 13:22:52 MIME-Version: 1.0 X-Spam-Details: rule=tag_notspam policy=tag score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=3.1.0-0801230000 definitions=main-0802060055 X-Spam-OrigSender: treumann@XXXXXXXXXX X-Spam-Bar: Subject: Re: [mpi-21] Ballot 4 - Re: [mpi-forum] MPI_Startall / MPI_Waitall / MPI_Testall clarification? X-BeenThere: mpi-21@XXXXXXXXXXX X-Mailman-Version: 2.1.8 Precedence: list Reply-To: "Mailing list for discussion of MPI 2.1" List-Id: "Mailing list for discussion of MPI 2.1" List-Archive: List-Post: List-Help: Content-Type: multipart/mixed; boundary="===============0773290967==" Sender: mpi-21-bounces@XXXXXXXXXXX Errors-To: mpi-21-bounces@XXXXXXXXXXX X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at mailgw.mcs.anl.gov --===============0773290967== Content-type: multipart/alternative; Boundary="0__=0ABBF974DFF00B538f9e8a93df938690918c0ABBF974DFF00B53" Content-Disposition: inline --0__=0ABBF974DFF00B538f9e8a93df938690918c0ABBF974DFF00B53 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: quoted-printable One comment - the clarification should not mention the progress engine.= We have a general requirement in the standard that MPI make progress and there is debate about what that requires. (Must there be an asynchronou= s progress engine). In general we do not attach a progress semantic to specific MPI calls. Even for something like MPI_Wait, there is no statement that MPI_Wait drives a progress engine. Only that MPI_Wait returns when the request = is complete. It is not a part of MPI_Wait semantic to say that the MPI_Wa= it call runs a progress engine or does not run one. I assume almost every MPI implementation does run its progress engine within a blocked MPI_Wait and probably no MPI gives the progress engine= a time limited kick in a MPI_Comm_rank call. Either way it is an impementation decision, not part of the standard. (Of course somethin= g like MPI_Comm_rank could not block but if an MPI implementor wanted to = run the progress engine for a max of 50 microseconds on every single MPI ca= ll, the standard would not forbid it.) Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-21-bounces@XXXXXXXXXXX wrote on 02/06/2008 12:37:20 PM: > Dries, > > Sounds fine. Nobody raised problems with this clarification up to now= . > > Please can you write a full proposal of exactly what to change/add/..= . > > I would put it to MPI 2.1 Ballot 4. > > Best regards > Rolf > > On Mon, 4 Feb 2008 16:24:03 +0100 > Dries Kimpe wrote:> >>From the standard: (same for MPI_Startall, MPI_Testall) > > > >The error-free execution of MPI_WAITALL(count, array_of_requests, > >array_of_statuses) has the same effect as the execution of > >MPI_WAIT(&array_of_request[i], &array_of_statuses[i]), for i=3D0 ,..= ., > >count-1, in some arbitrary order. > > > >What about count=3D=3D0? > > > >1) Is it allowed? > > > >2) If it is allowed, should a valid pointer be provided as the > >array_of_requests argument? > > > >Looking at mpich-1.0.6p1, I see the following: > > > > > >Function Allows count=3D=3D0 Allows bufptr=3D0 when count=3D=3D= 0 > >MPI_Testall Y Y > >MPI_Testsome Y Y > >MPI_Testany Y Y > >MPI_Waitsome Y Y > >MPI_Waitall Y Y > >MPI_Waitany Y Y > >MPI_Startall Y N > > > >For MPI_Startall: > > > >Fatal error in MPI_Startall: Invalid argument, error stack: > >MPI_Startall(147): MPI_Startall(count=3D0, req_array=3D(nil)) failed= > >MPI_Startall(80).: Null pointer in parameter array_of_requests[0]0: > Return code =3D 0, signaled with Interrupt > > > >Considering the other ...all functions allow count=3D=3D0 and ignore= the array > >in this case, the behaviour of MPI_Startall is probably just an oversight > >in mpich. > > > >For some reason, OpenMPI has exactly the same behavior; > >MPI_Startall doesn't allow req_array=3D=3D0 even if count=3D=3D0 > >(MPI_ERR_REQUEST is raised/returned) > > > >There is also an issue: progress. For example, in OpenMPI, calling > >MPI_Testall with count=3D=3D0 doesn't do anything at all. More speci= fic, it > >doesn't call the progress engine. > > > >Mpich however, does call the progress engine, even if count=3D=3D0. > >(On the other hand, MPI_Waitall does NOT try to make progress if count=3D=3D0) > > > >I propose to clarify where needed, and explicitly allow count=3D=3D0= ; > >Also, if count=3D=3D0, then 0 should be accepted as pointer for the = request > >array, and no guarantee of progress should be made. > > > >If count=3D=3D0, the returned error code should be MPI_SUCCESS (unle= ss of > >course, an asynchronous error is detected). > > > >It would suffice to add "When count is zero, this call has no effect= ." > >(For MPI_Startall, MPI_Testall, MPI_Waitall this can be right after = the > >text quoted on top of this message; For MPI_Waitany, MPI_Testany, > >MPI_Waitsome, MPI_Testsome, the addition should be at the end of the= > >paragraph.) > > > > > > Greetings, > > Dries > > > >_______________________________________________ > >mpi-forum mailing list > >mpi-forum@XXXXXXXXXXX > >http://lists.cs.uiuc.edu/mailman/listinfo/mpi-forum > > > > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner@XXXXXXXX > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530= > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832= > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner= > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > _______________________________________________ > mpi-21 mailing list > mpi-21@XXXXXXXXXXX > http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21= --0__=0ABBF974DFF00B538f9e8a93df938690918c0ABBF974DFF00B53 Content-type: text/html; charset=US-ASCII Content-Disposition: inline Content-transfer-encoding: quoted-printable

One comment - the clarification should not mention the progress engi= ne. We have a general requirement in the standard that MPI make progr= ess and there is debate about what that requires. (Must there be an asy= nchronous progress engine). In general we do not attach a progress sema= ntic to specific MPI calls.

Even for something like MPI_Wait, there is no statement that MPI_Wait d= rives a progress engine. Only that MPI_Wait returns when the request i= s complete. It is not a part of MPI_Wait semantic to say that the MPI_= Wait call runs a progress engine or does not run one.

I assume almost every MPI implementation does run its progress engine w= ithin a blocked MPI_Wait and probably no MPI gives the progress engine = a time limited kick in a MPI_Comm_rank call. Either way it is an impem= entation decision, not part of the standard. (Of course something lik= e MPI_Comm_rank could not block but if an MPI implementor wanted to run= the progress engine for a max of 50 microseconds on every single MPI c= all, the standard would not forbid it.)

Dick


Dick Treumann - MPI Team/TCEM
IBM Systems & Technology Group
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363


mpi-21-bounces@XXXXXXXXXXX wrote on 02/06/2008 12:37:20 PM:

> Dries,
>
> Sounds fine. Nobody raised problems with this clarification up to = now.
>  
> Please can you write a full proposal of exactly what to change/add= /...
>
> I would put it to MPI 2.1 Ballot 4.
>
> Best regards
> Rolf
>
> On Mon, 4 Feb 2008 16:24:03 +0100
>  Dries Kimpe <Dries.Kimpe@XXXXXXXXXXXXXXXXXX wrote:
> >>From the standard: (same for MPI_Startall, MPI_Testall) > >
> >The error-free execution of MPI_WAITALL(count, array_of_reques= ts,
> >array_of_statuses) has the same effect as the execution of
= > >MPI_WAIT(&array_of_request[i], &array_of_statuses[i]),= for i=3D0 ,...,
> >count-1, in some arbitrary order.
> >
> >What about count=3D=3D0?
> >
> >1) Is it allowed?  
> >
> >2) If it is allowed, should a valid pointer be provided as the=
> >array_of_requests argument?
> >
> >Looking at mpich-1.0.6p1, I see the following:
> >
> >
> >Function     Allows count=3D=3D0    Allows= bufptr=3D0 when count=3D=3D0
> >MPI_Testall     Y          =             Y
> >MPI_Testsome    Y          =             Y
> >MPI_Testany     Y          =             Y
> >MPI_Waitsome    Y          =             Y
> >MPI_Waitall     Y          =             Y
> >MPI_Waitany     Y          =             Y
> >MPI_Startall    Y          =             N
> >
> >For MPI_Startall:
> >
> >Fatal error in MPI_Startall: Invalid argument, error stack: > >MPI_Startall(147): MPI_Startall(count=3D0, req_array=3D(nil)) = failed
> >MPI_Startall(80).: Null pointer in parameter array_of_requests= [0]0:
> Return code =3D 0, signaled with Interrupt
> >
> >Considering the other ...all functions allow count=3D=3D0 and = ignore the array
> >in this case, the behaviour of MPI_Startall  is probably = just an oversight
> >in mpich.
> >
> >For some reason, OpenMPI has exactly the same behavior;
> >MPI_Startall doesn't allow req_array=3D=3D0 even if count=3D=3D= 0
> >(MPI_ERR_REQUEST is raised/returned)
> >
> >There is also an issue: progress.  For example, in OpenMP= I, calling
> >MPI_Testall with count=3D=3D0 doesn't do anything at all. More= specific, it
> >doesn't call the progress engine.
> >
> >Mpich however, does call the progress engine, even if count=3D= =3D0.
> >(On the other hand, MPI_Waitall does NOT try to make progress = if count=3D=3D0)
> >
> >I propose to clarify where needed, and explicitly allow count=3D= =3D0;
> >Also, if count=3D=3D0, then 0 should be accepted as pointer fo= r the request
> >array, and no guarantee of progress should be made.
> >
> >If count=3D=3D0, the returned error code should be MPI_SUCCESS= (unless of
> >course, an asynchronous error is detected).
> >
> >It would suffice to add "When count is zero, this call ha= s no effect."
> >(For MPI_Startall, MPI_Testall, MPI_Waitall this can be right = after the
> >text quoted on top of this message; For MPI_Waitany, MPI_Testa= ny,
> >MPI_Waitsome, MPI_Testsome, the addition should be at the end = of the
> >paragraph.)
> >
> >
> >  Greetings,
> >  Dries
> >
> >_______________________________________________
> >mpi-forum mailing list
> >mpi-forum@XXXXXXXXXXXXXXX > >http://lists.cs.uiuc.edu/mailman/listinfo/mpi-forum
>
>
>
> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner@hlrs= .de
> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65= 530
> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65= 832
> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseif= ner
> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30= )
> _______________________________________________
> mpi-21 mailing list
> mpi-21@XXXXXXXXXXXXXXX > http:= //lists.cs.uiuc.edu/mailman/listinfo/mpi-21
= --0__=0ABBF974DFF00B538f9e8a93df938690918c0ABBF974DFF00B53-- --===============0773290967== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ mpi-21 mailing list mpi-21@XXXXXXXXXXX http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21 --===============0773290967==--