Building a GCGrid node

From Geos-chem
Jump to: navigation, search

SUPPORT FOR NEW GCGRID NODES IS ENDING. Existing nodes will continue to receive updates until a new distribution system is in place.

Deprecated:

Here are instructions for building a GCGrid node at your local institution. The GCGrid is a tool for maintaining synchronization between nodes and distributing GEOS-Chem data over a wide area network. The data is divided into volumes, and a GCGrid node can support some or all volumes depending on your research needs. A node at your institution provides high speed access to the data volumes you request. Please contact help@as.harvard.edu for more information.

System Requirements

  1. Dedicated Linux system running Centos 6, Fedora 16, or higher.
  2. 8GB physical memory (12GB recommended)
  3. 4 cores (8 recommended)
  4. 14TB block storage formatted as an ext4 filesystem (16TB recommended) for all data except 2x2.5 MERRA. 24TB for all data including 2x2.5 MERRA. This could be one or mode NFS mounts.
  5. 100Mbps ethernet connection (1Gbps recommended)
  6. Static IP (routable IPv4) address and name
  7. Stable mount point for the storage

Systems with fewer resources can act as GCGrid nodes if they do not replicate all of the data volumes. Storage less than 2TB is not supported.

Firewall requirements

To support replication, a GCGrid node needs outgoing network access to gcgrid.as.harvard.edu on port tcp/8140 and needs to receive incoming connections from 140.247.104.224/27 (netmask 255.255.255.224) on ports tcp/24007-24200.

To support NFS from your institution's network, a GCGrid node needs to receive incoming connections on ports tcp/2049 and tcp/38465-38600. To support glusterfs from your institution's network, a GCGrid node needs to receive incoming connections on ports tcp/24007-24200.

Installation

You are responsible for installing and starting puppet on your node.

Fedora

If your node runs Fedora, you can install puppet by issuing the following command as root from a shell prompt:

yum install puppet

CentOS

If your node runs CentOS 6, you must first install access to the EPEL repository, which contains software supported by the Fedora project that can run under CentOS. You can download the current EPEL repository rpm file from

http://mirrors.servercentral.net/fedora/epel/6/i386/repoview/epel-release.html

by clicking on the link for epel-release-X-Y.noarch, then transferring the resulting rpm file (e.g. epel-release-6-7.noarch.rpm) to your node, and issuing the following command as root from the directory containing the file:

rpm -ihv epel-release*.rpm

To install puppet, issue the following command from a shell prompt:

yum install puppet

Configuration

Edit the file /etc/hosts and add the following line:

140.247.104.253 puppet

You will need to provide the full path to the storage and the name and IP address of your system, which can be done using email. This is required to customize the configuration for your node and insure that firewall permissions are correct at Harvard.

Note: After the system is configured, a change in hostname, IP address, or path to the storage will break the configuration and your node will effectively cease to be a GCGrid node. If you need to change these after the system is configured, it will be necessary to coordinate the change on both your node and on gcgrid.as.harvard.edu to avoid an interruption in service.

Start

To start the rest of the installation, issue the following commands from a shell prompt as root:

chkconfig puppet on

service puppet start

This completes the installation and configuration of puppet on your node. The rest of the installation and configuration is automated. Puppet will connect to gcgrid.as.harvard.edu at 140.247.104.253 and deposit a certificate signing request. Once the certificate is signed, your node will automatically download and execute a catalogue of instructions to complete the installation and configuration, including creation of the gcgrid volumes using the storage path you provided.

Once the volumes are in place and time is synchronized, replication can be started from gcgrid.as.harvard.edu. Initial replication of the gcgrid volumes will take much longer than updates, which begin almost immediately and complete in the time required to transfer the changes. On a high speed network updates occur in near real time.

Security

Network ports other than those mentioned above can be blocked without impacting operations, and access on open ports can be restricted to specific systems. You'll probably want to maintain access on ssh (tcp port 22) from your own administrative node. If you need to support ftp or http for file transfers, you'll need to mount the data on your ftp/http server or open the appropriate ports for those protocols to the systems that need access.

The installation software uses a key exchange that encrypts communications and establishes a trust relationship between the GCGrid node and the grid master node (currently gcgrid.as.harvard.edu). The GCGrid software does not require privileged account access. An unprivileged account is used for the initial data transfer from the source node and can be used to diagnose problems. Some information about the GCGrid node is passed to the grid master node, and installation information and data are passed back. The grid master node receives connections from authenticated GCGrid nodes on ports 8140 and 8649, it is protected by network and system firewalls, and it is closely monitored.

Please contact us if you have security questions or you would like advice about hardening your node.