Introduction

The ADAM API was originally intended to be used by the ADAM Website as the ‘server side code’, responding to requests for graphs and calculations.

During the course of the ADAM project it became apparent that there was a need for expert users to have a greater degree of control over the inputs, outputs and parameters of the ADAM functions than would be practical via the website. For this reason the API has been made available for such users, allowing them to download the entire codebase behind the website in order to execute calculations locally on their own hardware according to their specific requirements.

Note

There exists the ability for users to use the ADAM website via the command line using HTTP clients such as wget or cURL. This may meet the requirements of certain users who need to frequently repeat tasks which would be possible but time consuming using the website graphical user interface. The help documentation on the ADAM website provides examples of how this can be achieved, it does not involve the use of the ADAM API at all and is therefore not discussed in this documentation.

Installing the API

To use the API the user must first:

Note

The ADAM database is around 2.4 Gigabytes uncompressed, so you will need to have about twice that space available on your disk for the download and decompression.

Installing the ADAM Python code is performed in the same way as any other Python module. The easiest method is by using easy_install:

# easy_install -U ADAM-Tools

The PyPi page has further installation instructions for the Python code.

After the installation of the code the user should uncompress the entire ADAM database archive to a folder, keeping the directry structure intact.

When writing scripts that use the API you will initialise the ADAM API with a reference to the directory containing the unzipped database like this:

import adam
import adam_config

# define config:
my_config = adam_config.adam_config(path_data='/tmp/data', output_root_dir='/tmp/output/')
# path syntax for windows:
# my_config = adam_config.adam_config(path_data='c://adam-data/input/', output_root_dir='c://adam-data/output/')

# initialise the job
job = adam.adam_job( cfg=my_config )

In the example above the ADAM database was unzipped to the directory /tmp/data/ and output data will be written to /tmp/output/

That’s it ! If you can run the above example without an error message then the API is installed and working on your computer.

Getting Started

After installing the API, database and verifying that the installation is happy with the above code, we recommend that you try out some of the scripts found in the test/ directory, particularly spectrum_graph.py and brdf_graph_transectplane.py .

These scripts provide a good starting point for further customisation to provide the results you require from the API.

Looking at the examples you can see the general structure of an ADAM calculation:
  • initialise config, suitable for the local machine
  • initialise an instance of an ‘ADAM Job’ with that config
  • verify & pass the calculation parameters (region / illumination / viewing angles) to the job
  • load input data into the job
  • calculate reflectance
  • optionally: calculate BRDF
  • optionally: output graphs
  • optionally: output netCDF
  • optionally: access the raw calculated reflectance/BRDF data and do something different

The ADAM Job Class

Central to the use of the ADAM API is the concept of the ADAM Job. A single calculation performed by the API is termed a ‘Job’. A job has specific parameters including a geographic extent, an array of view zenth angles and phi (relative azimuth) angles, a sun zenith angle and a month of the year. A job has a collection of methods that can be called including:

Initialisation

The ADAM Job class can be initialised in the normal pythonic way:

job = adam.adam_job( job_output_dir='/tmp/', sza=45, vza=np.arange(0,100) )

However the preferred method is to use the adam.adam_job.validate_input() method which performs a verification of the input fields and performs some automatic preparation depending on the type of job you would like to perform.

adam.adam_job.validate_input() takes a dictionary as an argument, containing the names and values of the various keywords for the new job. This dictionary is the same format as that used by the website, which allows you to use the ADAM website to generate a typical input dictionary for use and later customisation in the API.

For example, after performing a Spectrum Graph via the website, the ‘Debug Log’ in the results window is presented as follows:

_images/help-cline-debug.png

Highlighted in the above example is the ‘request dictionary’. This text can be directly copy and pasted as an argument for the adam.adam_job.validate_input() method as follows:

request_dict = {
    'fieldOperationType': 'spectrum',
    'fieldMonth'        : 'jan',
    'fieldSunZenith'    : 45,
    'fieldViewZenith'   : 0,
    'fieldRelAzimuth'   : 0,
    'fieldSpectralDomain': '858-858 nm',

    'fieldLonMin': '-6',
    'fieldLonMax': '-5.3',
    'fieldLatMax': '35.6',
    'fieldLatMin': '36.3'
}

if not job.validate_input(request_dict):
    raise Exception('request input invalid !')

The adam.adam_job.validate_input() method returns a True or False depending on the success of the validation.

The list of keywords requested for parameterising an API calculation is provided in the table below:

  Value Comment
fieldOperationType ‘spectrum’ [string] Perform a reflectance spectrum calculation
‘brdf’ [string] Perform a reflectance spectrum calculation
‘time’-series [string] Perform a time series calculation
‘download’ [string] Download the data
fieldSpectralDomains optional spectral domain of interest [string] The string consists in the lower and upper bound of the range of variation (in nm) of each spectral domain considered. over which the reflectance values will be averaged or sampled. For averaging, the bounds are separated by a ‘-‘; each spectral domain is separated by a ‘;’ (ex: ‘300 – 2500 nm ; 2600-3000’) For sampling: provide an Python array containing the spectral bands (ex: np.array([400,500,600,700])
fieldBRDFType ‘plane’ [string] Calculate the BRDF in a transect plane (depending on the value of the relative azimuth angle)
‘polar’ [string] Calculate the BRDF over the upper hemisphere. Will generate a polar plot and a 3D representation
fieldCompute_Error optional True or False Enable/Disable the computation of the reflectance uncertainty (spectrally and directionally) for land pixels (True by Default).
fieldSunZenith value of the sun zenith angle (in degrees) [scalar or array] By default the range of variation of the sun zenith angle is set to [-70° ; 70°]. This can be changed with the keyword fieldSunZenithMaxLimit.
fieldViewZenith value of the view zenith angle (in degrees) [scalar or array] By default the range of variation of the view zenith angle is set to [-70° ; 70°]. This can be changed with the keyword fieldViewZenithMaxLimit.
fieldRelAzimuth value of the relative azimuth angle (in degrees) [scalar or array] The range of variation for the relative azimuth angle is [0°; 360°]. The backward scattering direction corresponds to 0° while the forward scattering direction corresponds to 180°.
fieldSunZenithMaxLimit maximum value of the sun zenith angle (in degrees) The behaviour of the BRDF calculation is controlled by fieldBeyondZALimit when the sun zenith angle exceeds fieldSunZenithMaxLimit.
fieldViewZenithMaxLimit maximum value of the view zenith angle (in degrees) The behaviour of the BRDF calculation is controlled by fieldBeyondZALimit when the view zenith angle exceeds fieldViewZenithMaxLimit.
fieldBeyondZALimit ‘0’ Set the corresponding BRDF values to 0
‘nan’ Set the corresponding BRDF values to NaN
‘constant’ Set the corresponding BRDF values to the values it would have for sza/vza = max limit
fieldMonth three first letterq of the month of interest (ex: ‘jan’) [string] Three first letters of the month
fieldLatMin minimum value of the latitude (degrees) [float] Comprised beween -90 and 90
fieldLonMin minimum value of the longitude (degrees) [float] Comprised beween -180 and 180
fieldLatMax maximum value of the latitude (degrees) [float] Comprised beween -90 and 90
fieldLonMax maximum value of the longitude (degrees) [float] Comprised beween -180 and 180
fieldInstrument optional name of the predefined instrument (ex: ‘avhrr’) [string] Choice among the recognized instruments: ‘aster’, ‘avhrr’, ‘gome’, ‘calipso’, ‘landsat_7’, ‘landsat_8’, ‘meris’, ‘misr’, ‘modis’, ‘msg_seviri’, ‘omi’, ‘orbview_2’, polder_parasol’, ‘seawifs’, ‘sentinel2’, ‘sentinel3_olci’, ‘spot_5’, ‘spot_67’, ‘vgt’ The list of all recognized instruments is also available on the web site (scrolling menu ‘Instrument’ once that ‘Spectrum Graph’ has been selected)
fiedlDefineInstrument Characteristics of an instrument defined by the user [list] User defined instrument defined as a Python list containing the instrument name as well as the instrument bands. Example: [‘NewInst’, [400,500,600,700]]
fieldGenerateGraphs True or False Disable the generation of the graphs and of the corresponding .png files
fieldNetCDF_main name of an input ADAM file containing the gridded dataset but the variance-covariance matrix over land to be used by the API only. These keywords have to be used when giving as input to the API non standard input data files (as for instance a raw data file downloaded from the ADAM site, with all data over one month gathered in a single file).
fieldNetCDF_covarland name of an input ADAM file containing the variance-covariance matrix over land

Accessing Job Data

The ADAM Job object has a property called ‘data’ which itself is a dictionary. The dictionary contains all the various arrays of data that have been loaded and calculated for that job, for example, after running job.process_brdf() we can perform the following:

>>> for key in sorted(job.data.keys()):
...     print key
...
NDVI
SDR
SDR_err_land
chloro_conc
idx_land
idx_ocean
idx_snow
latitude
longitude
mask_land
mask_ocean
mask_snow
month_index
qf
ref_land
ref_land_covar
reflectance
reflectance_err_land
reflectance_full
reflectance_full_err_land
wind_speed
>>>

The contents of each item in the dictionary is access like this:

>>> job.data['SDR'].shape
(1, 2, 1, 47, 1)
>>> job.data['SDR'].mean()
0.26286457954569065
>>> job.data['SDR']
array([[[[[ 0.1592543 ],
          [ 0.1608945 ],
          [ 0.16142392],
          [ 0.16178204],
          [ 0.16240701],
          [ 0.16350621],
          [ 0.16517392],
          [ 0.16744705],
          [ 0.17033319],
.......
          [ 0.28698364],
          [ 0.28480038]]]]], dtype=float32)
>>>

Many of the arrays in the ADAM Job data dictionary are multidimensional. For instance in the above example the dimensions of the SDR variable are, after BRDF Transect Plane calculation: pixels_x, pixels_y, wavelength, vza, phi (corresponding to the output of job.data[‘SDR’].shape: (1, 2, 1, 47, 1) )

For details about the dimensions of the various arrays you can refer to the documentation of the functions that generate them in the ADAM Job.

The variables contained in the ADAM output NetCDF4/HDF5 file (produced after a calculation) are described in the table below.

  Variable Description Format Dimensions
ADAM inputs latitude Array of latitude values float (pixels_y)
longitude Array of longitude values float (pixels_x)
ref_land Input normalized reflectances over land short
(month, band,
pixels_y, pixels_x)
chloro_conc Input chlorophyll content over ocean short (pixels_y, pixels_x)
wind_speed Input wind speed over ocean short (pixels_y, pixels_x)
ref_land_covar Input variance-covariance matrix between the land surface reflectance in the 7 MODIS bands (28 elements of the upper diagonal) float
(28,
pixels_y, pixels_x)
qf Quality flag byte (pixels_y, pixels_x)
Configuraton parameters sza SZA values of this calculation float (sza)
vza VZA values of this calculation float (vza)
phi PHI values of this calculation float (phi)
wavelengths Wavelengths over the full spectral domain (240-4000 nm, 1 nm resolution) float (wavelengths)
wavebands Wavebands of this calculation float (wavebands)
spectral_domains Used to calculate averaged reflectance values over broad spectral domains (optional) string (spectral_domains)
ADAM outputs mask Integer flags indicating land/ocean/snow mask int64 (pixels_x, pixels_y)
reflectance_full Calculated reflectance in the standard observation geometry and for all wavelengths (240-4000 nm) float
(pixels_x, pixels_y,
wavelengths)
reflectance_full_err_land Calculated reflectance error in the standard observation geometry over land pixels (transport of the input variance covariance error matrix in the 7 MODIS bands through the spectral interpolation procedure), over the full spectral domain (240-4000 nm) float
(pixels_x, pixels_y,
wavelengths)
reflectance Reflectance averaged over the spectral domains / or sampled in the instrument wavebands float
(pixels_x, pixels_y,
wavebands)
NDVI Calculated NDVI used for scaling the V and R directional parameters for the land BRDF model float (pixels_x, pixels_y)
SDR Spectro-directional reflectance for this calculation, reflectance in the user defined observation geometry and in the output spectral bands (averaged over spectral domains or in the instrument wavebands) float (pixels_x, pixels_y, wavebands, vza, phi)
SDR_err_land BRDF Error over land pixels (error of the BRDF model) float (pixels_x, pixels_y, wavebands, vza, phi)

Table Of Contents

Previous topic

ADAM Reflectance Database API Documentation

Next topic

Default configuration and behaviour

This Page