Intel Fortran Compiler: Difference between revisions
No edit summary |
|||
(17 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
'''''[[GNU Fortran compiler|Previous]] | [[ | '''''[[GNU Fortran compiler|Previous]] | [[Fortran language resources|Next]] | [[Guide to compilers for GEOS-Chem]]''''' | ||
#[[Supported compilers for GEOS-Chem]] | #[[Supported compilers for GEOS-Chem]] | ||
#[[GNU Fortran compiler|The GNU Fortran compiler ( | #[[GNU Fortran compiler|The GNU Fortran compiler (gfortran)]] | ||
#<span style="color:blue">'''The Intel Fortran compiler ( | #<span style="color:blue">'''The Intel Fortran compiler (ifort, ifx)'''</span> | ||
#[[Fortran language resources]] | #[[Fortran language resources]] | ||
This page contains information about the Intel Fortran compiler | This page contains information about the Intel Fortran compiler, which has recently been renamed from <tt>ifort</tt> to <tt>ifx</tt>. | ||
'''''The Intel Fortran compiler is our recommended proprietary compiler for GEOS-Chem.''''' | '''''The Intel Fortran compiler is our recommended proprietary compiler for GEOS-Chem.''''' | ||
== Overview == | == Overview == | ||
=== Documentation === | === Documentation === | ||
Line 31: | Line 26: | ||
=== Intel Fortran Compiler versions that have been tested with GEOS-Chem === | === Intel Fortran Compiler versions that have been tested with GEOS-Chem === | ||
The | The [https://geoschem.github.io/support-team GEOS-Chem Support Team] has tested GEOS-Chem with the compiler versions listed below. But you should be able to use other Intel Fortran Compiler versions as well. | ||
{| border=1 cellspacing=0 cellpadding=5 | {| border=1 cellspacing=0 cellpadding=5 | ||
Line 38: | Line 33: | ||
!width="200px"|Compiler | !width="200px"|Compiler | ||
!width="450px"|Status | !width="450px"|Status | ||
|-valign="top" | |-valign="top" | ||
|Linux | |Linux | ||
|ifort 17.0.4 | |ifort 23.0.0 | ||
|Supported | |||
|-valign="top" | |||
|Linux | |||
|ifort 19.0.5.281 | |||
|Supported | |||
|-valign="top" | |||
|Linux | |||
|ifort 18.0.5 | |||
|Supported | |||
|-valign="top" | |||
|Linux | |||
|ifort 17.0.4 | |||
|Supported | |Supported | ||
|-valign="top" | |-valign="top" | ||
Line 51: | Line 58: | ||
|ifort 15.0.0 and similar builds | |ifort 15.0.0 and similar builds | ||
|Supported | |Supported | ||
*NOTE: | *NOTE: IFORT 15 has a compiler bug that causes errors when turning on array-out-of-bounds checking and optimization. | ||
|-valign="top" | |-valign="top" | ||
Line 58: | Line 64: | ||
|ifort 13.0.079 and similar builds | |ifort 13.0.079 and similar builds | ||
|Supported | |Supported | ||
|-valign="top" | |-valign="top" | ||
Line 64: | Line 69: | ||
|ifort 12 | |ifort 12 | ||
|Supported | |Supported | ||
*NOTE: | *NOTE: A|compiler bug in ifort 12 and higher versions]] has forced us to add a workaround to HEMCO in v11-01. | ||
|-valign="top" | |-valign="top" | ||
Line 71: | Line 75: | ||
|ifort 11.1.069 and similar builds | |ifort 11.1.069 and similar builds | ||
|Supported | |Supported | ||
|} | |} | ||
== Environment settings for Intel Fortran == | == Environment settings for Intel Fortran == | ||
Line 81: | Line 82: | ||
Here is some information about how you can customize your Unix environment to use the Intel Fortran compiler. | Here is some information about how you can customize your Unix environment to use the Intel Fortran compiler. | ||
Here is some information about how you can customize your Linux environment to use the GNU Fortran compiler. This information was recently migrated to our [https://geos-chem.readthedocs.io geos-chem.readthedocs.io] manual. | |||
* [https://geos-chem.readthedocs.io/en/latest/getting-started/login-env-files-intel.html Create an environment file for Intel compilers] | |||
* [https://geos-chem.readthedocs.io/en/latest/getting-started/login-env-compilers.html Set environment variables for compilers] | |||
* [https://geos-chem.readthedocs.io/en/latest/getting-started/login-env-parallel.html Set environment variables for parallelization] | |||
* | |||
== Optimization == | == Optimization == | ||
Line 518: | Line 437: | ||
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 14:50, 28 March 2017 (UTC) | --[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 14:50, 28 March 2017 (UTC) | ||
Latest revision as of 19:28, 21 May 2024
Previous | Next | Guide to compilers for GEOS-Chem
- Supported compilers for GEOS-Chem
- The GNU Fortran compiler (gfortran)
- The Intel Fortran compiler (ifort, ifx)
- Fortran language resources
This page contains information about the Intel Fortran compiler, which has recently been renamed from ifort to ifx.
The Intel Fortran compiler is our recommended proprietary compiler for GEOS-Chem.
Overview
Documentation
You can find more information about the Intel Fortran Compiler here:
Also, normally when you installs the Intel Fortran compilers, you also will install the C and C++ compilers.
--Bob Yantosca (talk) 19:44, 10 January 2019 (UTC)
Intel Fortran Compiler versions that have been tested with GEOS-Chem
The GEOS-Chem Support Team has tested GEOS-Chem with the compiler versions listed below. But you should be able to use other Intel Fortran Compiler versions as well.
Platform | Compiler | Status |
---|---|---|
Linux | ifort 23.0.0 | Supported |
Linux | ifort 19.0.5.281 | Supported |
Linux | ifort 18.0.5 | Supported |
Linux | ifort 17.0.4 | Supported |
Linux | ifort 15.0.0 and similar builds | Supported
|
Linux | ifort 13.0.079 and similar builds | Supported |
Linux | ifort 12 | Supported
|
Linux | ifort 11.1.069 and similar builds | Supported |
Environment settings for Intel Fortran
Here is some information about how you can customize your Unix environment to use the Intel Fortran compiler.
Here is some information about how you can customize your Linux environment to use the GNU Fortran compiler. This information was recently migrated to our geos-chem.readthedocs.io manual.
- Create an environment file for Intel compilers
- Set environment variables for compilers
- Set environment variables for parallelization
Optimization
In this section we present information about the various optimization options available in the Intel Fortran Compiler.
Optimization options
Here is a quick reference table of optimization options (taken from the online Intel Fortran Compiler User and Reference Guides.
Option | Description | How invoked in GEOS-Chem? |
---|---|---|
-O0 | Turns off all optimizations. Math expressions will be evaluated in the same order in which they are written, which is necessary for debugging. If you are using a debugger (such as Totalview), compile with -g -O0. | DEBUG=yes or OPT=-O0 |
-O1 | Enables optimizations for speed and disables some optimizations that increase code size and affect speed. The -O1 option may improve performance for applications with very large code size, many branches, and execution time not dominated by code within loops.
Setting -O1 automatically sets the following options:
|
OPT=-O1 |
-O2 (aka -O) | Enables optimizations for speed. This is the generally recommended optimization level.
This option also enables:
On Linux and Mac OS X systems, if -g is specified, -O2 is turned off and -O0 is the default unless -O2 (or -O1 or -O3) is explicitly specified in the command line together with -g. |
Default setting |
-O3 | Enables -O2 optimizations plus more aggressive optimizations, such as prefetching, scalar replacement, and loop and memory access transformations.
Enables optimizations for maximum speed, such as:
On Linux and Mac OS X systems, the -O3 option sets option -fomitframe-pointer. The -O3 optimizations may not cause higher performance unless loop and memory access transformations take place. The optimizations may slow down code in some cases compared to -O2 optimizations. The -O3 option is recommended for applications that have loops that heavily use floating-point calculations and process large data sets. |
OPT=-O3 |
--Bob Y. 11:14, 3 October 2013 (EDT)
Recommended compilation and optimization options for GEOS-Chem
In this section, we present information about the compilation and optimization options that are invoked when you compile a GEOS-Chem simulation.
List of commonly-used compilation options
Here are the IFORT compilation options currently used by GEOS-Chem:
Option | Description | How invoked in GEOS-Chem? |
---|---|---|
Normal compiler settings | ||
-cpp | Turns on the C-preprocessor, to evaluate #if and #define statements in the source code.
|
Default setting |
-w | Suppresses all compiler warnings. This is mainly a convenience to prevent excessive output to the screen or log file.
NOTE: Most compiler warnings are harmless. Execution does not stop when a warning is displayed, unlike an error message, which causes program execution to halt at the point where the error occurred. |
Default setting |
-O2 | Optimizes the source code for speed, without taking too many liberties with numerical precision. For more information, please see the optimization options section above. | Default setting |
-auto | This option places local variables (scalars and arrays of all types), except those declared as SAVE , on the run-time stack. It is as if the variables were declared with the AUTOMATIC attribute. It does not affect variables that have the SAVE attribute or ALLOCATABLE attribute, or variables that appear in an EQUIVALENCE statement or in a common block.
|
Default setting |
-noalign | Prevents the compiler from padding bytes anywhere in common blocks and structures. Padding can affect numerical precision. | Default setting |
-convert big_endian | Specifies that the format will be big endian for integer data and big endian IEEE floating-point for real and complex data. This only affects file I/O to/from binary files (such as binary punch files) but not ASCII, netCDF, or other file formats. | Default setting |
-vec-report0 | Tells the compiler to suppress printing "LOOP HAS BEEN VECTORIZED" messages. This reduces the amount of output that is sent to the screen and/or GEOS-Chem log file. | Default setting |
-fp-model source | Rounds intermediate results to source-defined precision and enables value-safe optimizations. Basically, this tells the compiler not to take too many liberties with how numerical expressions are evaluated. For more information about this option, please see our precision-safe optimization section below. This option can be disabled by compiling GEOS-Chem with the PRECISE=no Makefile option. | Default setting |
-traceback | This option tells the compiler to generate extra information in the object file to provide source file traceback information when a severe error occurs at run time. When the severe error occurs, source file, routine name, and line number correlation information is displayed along with call stack hexadecimal addresses (program counter trace). This option increases the size of the executable program, but has no impact on run-time execution speeds. It functions independently of the debug option. |
|
Special compiler settings | ||
-r8 | This option tells the compiler to treat variables that are declared as REAL as REAL*8 (as opposed to REAL*4 .
NOTE: This option is not used globally, but is only applied to certain indidvidual files (mostly from third-party codes like ISORROPIA. Current GEOS-Chem programming practice is to use either |
Used as needed |
-mcmodel=medium | This option is used to tell IFORT to use more than 2GB of static memory. This avoids a specific type of memory error that can occur if you compile GEOS-Chem for use with an extremely high-resolution grid (e.g. 0.25° x 0.3125° nested grid). | Default setting |
-shared-intel (formerly -i-dynamic) |
This option needs to be used in conjunction with -mcmodel=medium. It causes Intel-provided libraries to be linked in dynamically instead of statically (which is the default). | Default setting |
-ipo | This option enables interprocedural optimization between files. This is also called multifile interprocedural optimization (multifile IPO) or Whole Program Optimization (WPO). When you specify this option, the compiler performs inline function expansion for calls to functions defined in separate files.
NOTE: Yuxuan Wang found that this option was useful for certain nested-grid simulations. See the this wiki post below for more information. |
IPO=yes |
-static | This option prevents linking with shared libraries. It causes the executable to link all libraries statically.
NOTE: Yuxuan Wang found that this option was useful for certain nested-grid simulations. See the this wiki post below for more information. |
IPO=yes |
Settings only used for debugging | ||
-debug all | Tells the compiler turn on all debug error output. | DEBUG=yes |
-g | Tells the compiler to generate full debugging information in the object file. This will cause a debugger (like Totalview) to display the actual lines of source code, instead of hexadecimal addresses (which is gibberish to anyone except hardware engineers). | DEBUG=yes |
-O0 | Turns off all optmization. Source code instructions (e.g. DO loops, IF blocks) and numerical expressions are evaluated in precisely the order in which they are listed, without being internally rewritten by the optimizer. This is necessary for using a debugger (like Totalview). | DEBUG=yes |
-check bounds (aka -CB) | Check for array-out-of-bounds errors. This is invoked when you compile GEOS-Chem with the BOUNDS=yes Makefile option. NOTE: Only use -CB for debugging, as this option will cause GEOS-Chem to execute more slowly! | DEBUG=yes |
-check arg_temp_created | Checks to see if any array temporaries are created. Depending on how you write your subroutine and function calls, the compiler may need to create a temporary array to hold the values in the array before it passes them to the subroutine. For detailed information, please see our Passing array arguments efficiently in GEOS-Chem wiki page. | DEBUG=yes |
-fpe0 | This option will cause GEOS-Chem to halt if any type of floating-point error is encountered. This can happen if an equation results in a denormal value, e.g. NaN, or +/-Infinity. Common causes of floating-point errors are divisions where the denominator becomes zero. NOTE: The default compiler setting is -fpe3, which will convert many of these denormal values to zeros and then continue execution. |
FPE=yes |
-ftrapuv | This option will assign a large numeric value to all local automatic variables. This makes it easier to identify numerical errors caused by improper initialization. | FPE=yes |
--Bob Y. 11:21, 3 October 2013 (EDT)
Typical settings for a GEOS-Chem simulation
The normal GEOS-Chem build uses the following IFORT compiler flags:
-cpp -w -O2 -auto -noalign -convert big_endian -vec-report0 -fp-model source -openmp
whereas a debugging run (meant to execute in a debugger such as TotalView) will typically use these flags:
-cpp -w -O0 -auto -noalign -convert big_endian -g -DDEBUG -check arg_temp_created -debug all -fp-model source -fpe0 -ftrapuv -check bounds
NOTE: In order to avoid running out of memory if you compiling GEOS-Chem at extremely high resolution (e.g. the 0.25° x 0.3125° nested grids), we recommend adding the following flags:
-mcmodel=medium -shared-intel
These are automatically set when you compile with the NETCDF=yes or HDF=yes compiler options (in GEOS-Chem v9-01-03 and higher).
--Bob Y. 17:34, 29 February 2012 (EST)
Precision-safe optimization
You can use the following Intel Fortran Compiler options to select how aggressively you would like to optimize floating-point operations.
Default behavior
-fp-model fast
Example source code:
REAL T0, T1, T2; ... T0 = 4.0E + 0.1E + T1 + T2;
When this option is specified, the compiler applies the following semantics:
- Additions may be performed in any order
- Intermediate expressions may use single, double, or extended precision
- The constant addition may be pre-computed, assuming the default rounding mode
Using these semantics, the following shows some possible ways the compiler may interpret the original code:
REAL T0, T1, T2; ... T0 = (T1 + T2) + 4.1E;
or
REAL T0, T1, T2; ... T0 = (T1 + 4.1E) + T2;
Preferred alternative
-fp-model source (aka -fp-model precise)
Example source code:
REAL T0, T1, T2; ... T0 = 4.0E + 0.1E + T1 + T2;
When this option is specified, the compiler applies the following semantics:
- Additions are performed in program order, taking into account any parentheses
- Intermediate expressions use the precision specified in the source code
- The constant addition may be pre-computed, assuming the default rounding mode
Using these semantics, the following shows a possible way the compiler may interpret the original code:
REAL T0, T1, T2; ... T0 = ((4.1E + T1) + T2);
Summary
If you do not select any -fp-model option, the Intel Fortran Compiler will default to -fp-model fast. As you can see from the examples above, this may not optimize the code in the same way each time. This can lead to minor numerical noise in the output, as was seen in ISORROPIA II.
To avoid this situation, we recommend compiling all source code files with -fp-model source. This will be the new default in GEOS-Chem v9-01-02.
Reference: Intel® Fortran Floating-point Operations; Document Number: 315892-003US
--Bob Y. 17:01, 25 August 2011 (EDT)
Optimization options for faster runs
Yuxuan Wang told us about the optimization options: -ipo and -static and said these options would speed up the simulations. I've tested these options on our system at Harvard. The run with the new options show very tiny differences (much less than 1% over 1 month) compared to a run with the old options only. For a full-chemistry run (43 tracers) on 4x5 resolution and 4 processors, the run time is about 10% shorter than previously.
These options are especially efficient to handle the transport. So in simulations with a faster chemistry (like tagged tracers simulations), we expect to see a higher gain in time. For example, the time for a methane run is shorten by about 30 %.
To use these options, compile GEOS-Chem with the IPO=yes Makefile option, e.g.
make -j4 IPO=yes
--Ccarouge 15:54, 8 September 2009 (EDT)
--Bob Y. 17:50, 29 February 2012 (EST)
Optimization level for debugging
If you would like to run your code in a debugger, such as Totalview, you must use the following compiler switches:
-g -O0
Using -O0 will ensure that the source code gets executed in the same order in which it is written (i.e. this disables all compiler optimizations). The -g switch will tell the debugger to display lines of source code instead of hexadecimal memory addresses (which are more or less gibberish unless you are a hardware engineer).
GEOS-Chem will add these switches automatically for you if you compile with the DEBUG=yes option.
--Bob Y. 15:28, 22 February 2012 (EST)
Caveat about optimizing for specific chipsets
The standard GEOS-Chem build sequence does not include any optimization flags that are specific to a certain type of CPU. If you are interested, you can certainly experiment for yourself. But be aware that this may invoke certain chip-level optimizations that could potentially change the simulation output.
Jenny Fisher wrote:
I have tested the new chips & compiler option. I found that there are small differences [in difference test output]...if I use exactly the same compile commands and number of processors between our old cores and our new Broadwell cores (E5-2690 v4). The differences are very small and I think nothing to worry about.
However, adding the preferred compiler flag -xCORE-AVX2 led to much bigger differences (e.g., up to 5% difference or 10 ppb in ozone…). I haven’t investigated the differences in detail. I did run a one month benchmark comparison, and see that the differences can be consequential after a month (i.e. not just differences in regions where values are low.
I have no idea what is causing these differences. So I guess for the moment, I would recommend *not* using the specific optimisation for Broadwell/Haswell cores. However, I think it probably is ok to use the Broadwell cores without this flag. I am not sure what impact this choice will have on performance.
--Bob Yantosca (talk) 14:50, 28 March 2017 (UTC)