From: Jeff Squyres Date: June 24, 2005 9:22:41 PM GMT+02:00 To: Ewing (Rusty) Lusk ,William Gropp Cc: Brian Barrett ,David Daniel Subject: MPI-2 attributes question Bill/Rusty -- I'm asking you guys this question because: 1. We can't figure this out 2. I don't remember (and can't find) the address of the MPI implementer's list ;-) I'm the unlucky slob who was tagged to write the attributes code for Open MPI. I wrote it about a year ago, but some recent test failures made me go back and look at it again about a week ago. I've been puzzling over MPI-2:4.12.7 for quite some time and asked a whole bunch of other OMPI guys and none of us can figure this out. In particular, I'm referring to the paragraph that starts on MPI-2 p58, line 12: "MPI supports two types of attributes..." I have several problems with this text and the examples that follow it, but perhaps I'm just missing something fundamental. Starting on line 16, it says: "When an address valued attribute is accessed from Fortran, the MPI_xxx_GET_ATTR will convert the address into an integer and return the result of this conversion. This conversion is lossless if new style (MPI-2) attribute functions are used, and an integer of kind MPI_ADDRESS_KIND is returned. This conversion may cause truncation if old style (MPI-1) attribute functions are used." From that text, it sounds like if, in C, I write an attribute (that will be a pointer, such as "int i = 5;", I'll put &i), if I use the Fortran MPI-2 functions, I should get the value of the pointer back (!). However, Example 4.13 directly shows something different -- in C, it COMM_PUT_ATTR's a pointer to an int. Then, in Fortran, it COMM_GET_ATTR's that attribute and gets the value *pointed to by the C pointer*. This, to me, seems like a contradiction. But this raises other questions: 1. If Example 4.13 is showing the spirit of what is intended (i.e., put a pointer in C, get the dereferenced value back in Fortran), then the language about lossless / truncation is meaningless -- and there was no point in extending the size of the MPI-2 Fortran functions to be INTEGER(KIND=MPI_ADDRESS_KIND). 2. If the text is showing the spirit of what is intended (i.e., put a pointer in C, get the pointer value back in Fortran), what is the point of that? I.e., what would one do with an address in Fortran? (other than for MPI_WIN_BASE) 3. I think that Example 4.13 is not always right. Specifically, when running on a 64 bit platform (e.g., an Opteron, where sizeof(int) == 4 and sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)) == 8), the only way that the fortran val variable could get the value of 5 is if the C pointer is dereferenced as an int, which a) is not consistent with the text above it, and b) defeats the point of having the INTEGER be of size MPI_ADDRESS_KIND (also making the lossless / truncation language irrelevant). What do you think? Am I just missing something? Sidenote: Was all of this put in place solely for MPI_WIN_BASE? (this attribute seems to contradict the [confusing] established rules established -- it seems to be the one attribute that is *not* dereferenced in Fortran when read) More generally, I see 9 cases for attribute handling: 1. C writes, C reads 2. C writes, Fortran MPI-1 reads 3. C writes, Fortran MPI-2 reads 4. Fortran MPI-1 writes, C reads 5. Fortran MPI-1 writes, Fortran MPI-1 reads 6. Fortran MPI-1 writes, Fortran MPI-2 reads 7. Fortran MPI-2 writes, C reads 8. Fortran MPI-2 writes, Fortran MPI-1 reads 9. Fortran MPI-2 writes, Fortran MPI-2 reads Here's an extended comment from my code where I explain what I think should happen with each of them. Does this all sound right to you? * Cases where C writes an attribute value: * ---------------------------------------- * * In all of these cases, a pointer was written by C (e.g., a pointer * to an int -- but it could have been a pointer to anything, such as * a struct). These scenarios each have 2 examples: * * Example A: int foo = 3; * MPI_Attr_put(..., &foo); * Example B: struct foo bar; * MPI_Attr_put(..., &bar); * * 1. C reads the attribute value. Clearly, this is a "unity" case, * and no translation occurs. A pointer is written, and that same * pointer is returned. * * Example A: int *ret; * MPI_Attr_get(..., &ret); * --> *ret will equal 3 * Example B: struct foo *ret; * MPI_Attr_get(..., &ret); * --> *ret will point to the instance bar that was written * * 2. Fortran MPI-1 reads the attribute value. The C pointer is * dereferenced as an int and cast to a fortran INTEGER (i.e., * MPI_Fint). * * Example A: INTEGER ret * CALL MPI_ATTR_GET(..., ret, ierr) * --> ret will equal 3 * Example B: INTEGER ret * CALL MPI_ATTR_GET(..., ret, ierr) * --> ret will be the INTEGER representation of the first * sizeof(int) bytes from the struct foo instance * (bar). * * 3. Fortran MPI-2 reads the attribute value. The C pointer is * dereferenced as an int and cast to a fortran * INTEGER(KIND=MPI_ADDRESS_KIND) (i.e., a (MPI_Aint)). * * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) ret * CALL MPI_COMM_GET_ATTR(..., ret, ierr) * --> ret will equal 3 * Example B: INTEGER(KIND=MPI_ADDRESS_KIND) ret * CALL MPI_COMM_GET_ATTR(..., ret, ierr) * --> ret will be the INTEGER representation of the first * sizeof(int) bytes from the struct foo instance (bar). * * Cases where Fortran MPI-1 writes an attribute value: * ---------------------------------------------------- * * In all of these cases, an INTEGER is written by Fortran. * * Example: INTEGER FOO = 7 * CALL MPI_ATTR_PUT(..., foo, ierr) * * 4. C reads the attribute value. The value returned is a pointer * that points to an INTEGER (i.e., an MPI_Fint) that has a value * of 7. * --> NOTE: The external MPI interface does not distinguish between * this case and case 7. It is the programer's responsibility * to code accordingly. * * Example: MPI_Fint *ret; * MPI_Attr_get(..., &ret); * -> *ret will equal 7. * * 5. Fortran MPI-1 reads the attribute value. This is the unity * case; the same value is returned. * * Example: INTEGER ret * CALL MPI_ATTR_GET(..., ret, ierr) * --> ret will equal 7 * * 6. Fortran MPI-2 reads the attribute value. The same value is * returned, but potentially sign-extended if sizeof(INTEGER) < * sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)). * * Example: INTEGER(KIND=MPI_ADDRESS_KIND) ret * CALL MPI_COMM_GET_ATTR(..., ret, ierr) * --> ret will equal 7 * * Cases where Fortran MPI-2 writes an attribute value: * ---------------------------------------------------- * * In all of these cases, an INTEGER(KIND=MPI_ADDRESS_KIND) is written * by Fortran. * * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) FOO = 12 * CALL MPI_COMM_PUT_ATTR(..., foo, ierr) * Example B: // Assume a platform where sizeof(void*) = 8 and * // sizeof(INTEGER) = 4. * INTEGER(KIND=MPI_ADDRESS_KIND) FOO = pow(2, 40) * CALL MPI_COMM_PUT_ATTR(..., foo, ierr) * * 7. C reads the attribute value. The value returned is a pointer * that points to an INTEGER(KIND=MPI_ADDRESS_KIND) (i.e., a void*) * that has a value of 12. * --> NOTE: The external MPI interface does not distinguish between * this case and case 4. It is the programer's responsibility * to code accordingly. * * Example A: MPI_Aint *ret; * MPI_Attr_get(..., &ret); * -> *ret will equal 12 * Example B: MPI_Aint *ret; * MPI_Attr_get(..., &ret); * -> *ret will equal 2^40 * * 8. Fortran MPI-1 reads the attribute value. The same value is * returned, but potentially truncated if sizeof(INTEGER) < * sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)). * * Example A: INTEGER ret * CALL MPI_ATTR_GET(..., ret, ierr) * --> ret will equal 12 * Example B: INTEGER ret * CALL MPI_ATTR_GET(..., ret, ierr) * --> ret will equal 0 * * 9. Fortran MPI-2 reads the attribute value. This is the unity * case; the same value is returned. * * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) ret * CALL MPI_COMM_GET_ATTR(..., ret, ierr) * --> ret will equal 7 * Example B: INTEGER(KIND=MPI_ADDRESS_KIND) ret * CALL MPI_COMM_GET_ATTR(..., ret, ierr) * --> ret will equal 2^40 If you're still reading this, many thanks for your time! :-) -- {+} Jeff Squyres {+} The Open MPI Project {+} http://www.open-mpi.org/