User talk:Salvatore Farina: Difference between revisions

From Geos-chem
Jump to navigation Jump to search
(moved the page to a new home.)
 
(29 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page describes how to acquire the latest source code, data, and libraries required to build and run GEOS-Chem with TOMAS microphysics on the ace-net glooscap cluster.
If you're here, you're probably looking for the [[TOMAS setup guide]]!
 
== Overview ==
 
The latest public release of GEOS-Chem with TOMAS does not include many of the recent developments in aerosol science.  It also cannot take advantage of parallel computing technologies.
Lately I've been working on the "Bleeding Edge" code to address these issues. Here's a guide that should help you get started if you're using the glooscap cluster.
 
== Getting Set Up ==
 
=== Code ===
You can grab the absolute latest code from my source directory on glooscap:
cp -r /home/sfarina/source/GC_Bleeding_Edge/ ~
 
or, (safer) you can grab my latest "snapshot"
cp /home/sfarina/source/GC_BE_snapshot-latest.tgz .
 
=== Libraries ===
'''geos-chem-libraries-intel11''' is a bundle of software required to build and run the latest version of GEOS-Chem.
Included in this package:
* Intel Ifort Fortran compiler - v11.1 - required to build geoschem
* NetCDF - Network Common Data Format libraries - required to read and write certain datasets
* HDF5 - Hierarchical Data Format - required to read and write certain datasets
* other dependencies - required for netcdf and hdf5
 
You can copy this folder as a tarball from /home/sfarina/gclibs.tgz or simply extract it directly to your home directory:
cd ~
tar zxvf /home/sfarina/gclibs.tgz
 
This will extract the libraries folder to your home directory.
 
=== Environment ===
In order to get the compiler to run and recognize the libraries described above, some environment variables must be set.  Below is an excerpt from my ''.bashrc''.
 
ROOT_LIBRARY_DIR="/home/sfarina/geos-chem-libraries-intel11"
GC_BIN=$ROOT_LIBRARY_DIR/bin
GC_INCLUDE=$ROOT_LIBRARY_DIR/include
GC_LIB=$ROOT_LIBRARY_DIR/lib
export GC_BIN
export GC_INCLUDE
export GC_LIB
export FC="ifort"
export LD_LIBRARY_PATH="/home/sfarina/geos-chem-libraries-intel11/lib"
export PATH="/home/sfarina/geos-chem-libraries-intel11/Compiler/11.1/080/bin/intel64:/home/sfarina/opt/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/gnu/lib64:/usr/local/gnu/lib:/home/sfarina/geos-chem-libraries-intel11/lib:/home/sfarina/geos-chem-libraries-intel11/Compiler/11.1/080/lib/intel64/:/home/sfarina/geos-chem-libraries-intel11/Compiler/11.1/080/idb/lib/intel64"
export INTEL_LICENSE_FILE="/home/sfarina/geos-chem-libraries-intel11/software/intel/Compiler/11.1/080/Licenses"
source /home/sfarina/geos-chem-libraries-intel11/Compiler/11.1/080/bin/ifortvars.sh intel64
 
If you are using bash, you can copy/paste this to your ''.bashrc''. Once the compiler and libraries are installed in ''~/geos-chem-libraries-intel11'' change instances of ''sfarina'' to your username.
source ~/.bashrc
ifort --version
 
If ifort returns
ifort (IFORT) 11.1 20101201
you should be all set to start compiling
 
=== Data ===
I have my data directory set up slightly differently than the last versions of geoschem to fix some of the issues that popped up as they added datasets, etc. you can probably just link to it.
 
== Compiling ==
 
=== Compiler ===
Please note that the '''ONLY VERSION''' of the intel compiler which reliably compiles a working executable of geos-chem with TOMAS is version 11.1.
Installation is described above in the libraries section.
 
=== Make ===
cd GeosCore and do
make tomas40
 
and always do make realclean when changing tomas versions.
 
== Running GEOS-Chem with TOMAS ==
 
=== Run Directories ===
There are run directories for each of the tomas versions at:
/net/samqfs/pierce/sfarina/standard_run_directories/
 
Copy the tarballs (named 40.tgz, 30.tgz, etc.) to a standard location. You can then do
tar zxvf YOUR_STANDARD_LOCATION/40.tgz
to extract the appropriate run directory to your current working directory. the folder will be named ''run.TOMASXX'', where ''XX'' is 12, 15,30, or 40 depending on the version you would like to run.
 
Once you have the appropriate version of geostomas compiled and your run directory extracted, copy the executable to your run directory.
 
=== Submitting Jobs to the Parallel Queue ===
In each folder is a file called ''parallel.sh''. Below is a description of some of the parameters:
#!/bin/bash
# $ -S /bin/bash
./etc/profile
#$ -o job_output
#$ -l h_rt=100:00:00                            #wall clock time requested from grid engine. Lower request times will have higher priority in the queue
#$ -l h_vmem=2.0G                              #vmem requested from grid engine. 2.0 is sufficient for all versions at 4x5 and TOMAS15 at 2x2.5 on 16 cores
#$ -l h_stack=12.5G                            #stack memory requested from grid engine
#$ -N RUN_NAM                                  #a name for your run
#$ -pe openmp 16                                #number of cores you are requesting from grid engine
#$ -cwd                                        #inherit properties from your current shell
export OMP_NUM_THREADS=16                      #number of openMP threads
export KMP_STACKSIZE=500000000                  #stacksize memory limit for each thread
ulimit -t unlimited              # cputime
ulimit -f unlimited              # filesize
ulimit -c unlimited              # coredumpsize
ulimit -m unlimited              # memoryuse
ulimit -l unlimited              # memorylocked
cd YOUR_RUN DIRECTORY
./geostomas > log
 
You'll need to edit it slightly (run name and working directory), then run:
qsub parallel.sh
 
You can check on the status in the queue with
qstat
 
You can watch the logfile output of your simulation with
tail -f log
 
== Developing ==
 
=== Version Control ===
Git! You should definitely use git to track your changes. I have a copy built/installed at /home/sfarina/opt/bin/git that you can probably either copy or just use. Make a separate branch for yourself as soon as you make a copy of the code, this way we can easily trade/track updates / advances / bugfixes.
git checkout -b my_new_branch
 
There are many useful resources for git on the web. Here are some I found useful:
* [http://git-scm.com/book/en/Git-Branching-Basic-Branching-and-Merging Branching and Merging]
* [http://ariejan.net/2009/10/26/how-to-create-and-apply-a-patch-with-git/ Creating and Applying Patches]
* [http://lostechies.com/joshuaflanagan/2010/09/03/use-gitk-to-understand-git/ Understanding git through gitk]
 
== Other Advice ==
I use the GNU bourne again shell (bash).  I suggest you do the same.  The csh is fine, but I have written all of my scripts using bash. Your life will probably be easier if you use bash.
 
if you have any questions or you are running into trouble, please ask.  I am usually able to respond to emails within a day, and am willing to use gchat or skype if need be.
 
 
 
--[[User:Salvatore Farina|Salvatore Farina]] 14:51, 25 July 2013 (EDT)

Latest revision as of 15:57, 26 July 2013

If you're here, you're probably looking for the TOMAS setup guide!

Retrieved from "https://wiki.seas.harvard.edu/geos-chem/index.php?title=User_talk:Salvatore_Farina&oldid=13738"