ADAM Job

The main class performing request validation and various convenience utilities.

Quick synopsis:

import adam
import adam_config

# define config
my_config = adam_config.adam_config(path_data = '/tmp/data', output_root_dir = '/tmp/output/')

# define calculation parameters
request_dict = { 'fieldOperationType': 'spectrum', 'fieldMonth': 'jan',
                 'fieldSunZenith': 40, 'fieldViewZenith': 30, 'fieldRelAzimuth': 0,

                 'fieldLonMin': '-4.1', 'fieldLonMax': '-2.6',
                 'fieldLatMax': '37.5', 'fieldLatMin': '36.8'   }

# initialise the job
job = adam.adam_job( cfg=my_config )

# validate parameters
if not job.validate_input(request_dict):
    raise Exception('request input invalid !')

# load data, according to the parameters supplied
job.load_data()

# calculate the reflectance
job.process_reflectance()

# profit !
print job.data['reflectance'].mean()
0.102804763996

print job.data.keys()
['wind_speed', 'reflectance', 'idx_ocean', 'chloro_conc', 'idx_snow', 'snow', 'longitude', \
'month', 'mask_ocean', 'ref_land_covar', 'ref_land', 'reflectance_err_land', 'latitude', \
'mask_snow', 'mask_land', 'idx_land']

adam.py

ADAM Job object for managing the processing chain of an ADAM calculation.

class adam.adam_job(extent=None, month_index=0, job_output_dir=None, cfg=None, sza=None, vza=None, phi=None, lmbd=None, sza_max_limit=None, vza_max_limit=None, beyond_za_limit=None, brdf_graph_type=None, job_type=None, create_working_dir=True, spectral_domains=None, instrument_name=None, netcdf_datafile=None, netcdf_file_covarland_template=None)[source]
add_output(output_type, filename, case=None, title=None)[source]

Method to add a reference to an output (e.g. a graph / netcdf output) to this job for recuperation later.

This is used by the web client to present a list of outputs (in the form of Tabs in the user interface) resulting from the job.

Args:
output_type (str): The type (‘graph’/’download’) filename (str): the name of the file to be found in the job output directory
Kwargs:
case (str): ‘pixels’ or ‘stats’, used by the web client title (str): title of output, used by the web client
Returns:
None
build_filename(output_type, extension, month_index=None, lats=None, lons=None, rand='')[source]

Method to build the filename of an output according to a predefined standard.

Responsible for the bizarre name given to ADAM downloads. Includes a random component to avoid duplicate filenames in the same job.

Args:
output_type (str): The type (‘netcdf’/’BRDF_polar’..) extension (str): the filename extension
Kwargs:
month_indexes (list): list of integer month indexes contained in the output lats (numpy.array): list of latitudes represented in the output lons (numpy.array): list of longitudes represented in the output rand (string): random text used to make a unique filename if there is already a filename with the desired name in the working directory.
Returns:
str: the new filename to be used
calculate_ndvi()[source]

Populates the NDVI data array for this job.

The NDVI array is used by the process_brdf() method on an ADAM job.

Actual work is performed by the process_reflectance.calculate_ndvi() function, its parameters are detailed there.

calculate_stats(input_array)[source]

Calculate various statistics (min/max/mean) for the array passed in.

Results are inserted into the ‘stats’ key of the data dictionary.

Actual work is performed by the analysis.get_stats() function, its parameters are detailed there.

Args:
input_array (numpy.array): The input array to be analysed
Returns:
None
define_vza_hotspot()[source]

Increase vza sampling depending on sza (increase resolution near sza).

Actual work is performed by the process_brdf.define_vza_hotspot() function, its parameters are detailed there.

get_month_name(month_index=None)[source]

Utility method to return the name of the month for this job, or optionally from an aribtary index.

Used by the graphing functions.

Kwargs:
month_index (int): The month index (from 0 to 11)
Returns:
str. The name of the month.
get_spectral_domains(spectral_domains)[source]

Determine the spectral domains over which the output BRDF is averaged based on inputs

graph_3Dpolar_plot(case='stats', indices=None, title=None, three_d=False, interactive_graph=False, waveband_index=0)[source]

Writes a 3D or polar plot graph to the correct output dir.

Actual work is performed by show.main_brdf_polar_plot(), more documentation can be obtained there.

If pixel graphs are requested for multiple pixels then multiple graphs will be generated. The graph(s) will be added to the ‘outputs’ array on the job.

Kwargs:
case (str): ‘stats’ or ‘pixels’ indicating the type of graph to draw indices (numpy.array): array of pixel indexes to be graphed title (str): graph title three_d (bool): whether to use 3d instead of 2D (polar) mode
Returns:
None
graph_main_ref_spectra(case='stats', indices=None, title=None)[source]

Writes a spectrum graph to the correct output dir.

Actual work is performed by show.main_ref_spectra(), more documentation can be obtained there.

If pixel graphs are requested for multiple pixels then multiple graphs will be generated. The graph(s) will be added to the ‘outputs’ array on the job.

Kwargs:
case (str): ‘stats’ or ‘pixels’ indicating the type of graph to draw indices (numpy.array): array of pixel indexes to be graphed title (str): graph title
Returns:
None
graph_time_series(waveband_index=0)[source]

Method to output time series (1 per month) of BRDF reflectance.

This method drives the chain of calculating the BRDF for each month before finally calling the show.main_time_series() function to do the actual graphing work.

graph_transect_plane(case='stats', indices=None, title=None, graph_error=False, waveband_index=0)[source]

Writes a transect_plane graph to the correct output dir.

Actual work is performed by show.main_brdf_transect_plane(), more documentation can be obtained there.

If pixel graphs are requested for multiple pixels then multiple graphs will be generated. The graph(s) will be added to the ‘outputs’ array on the job.

Kwargs:
case (str): ‘stats’ or ‘pixels’ indicating the type of graph to draw indices (numpy.array): array of pixel indexes to be graphed title (str): graph title graph_error (bool): whether to draw the error bars on the graph spectral_domain_index: indice of the band to be displayed
Returns:
None
is_pixel_request()[source]

Boolean method dictating whether we allow this job to be performed at the pixel level or not

load_data()[source]

Loads the data for this job into the job.data dictionary.

The mask indicies for distinguishing land from ocean from snow are also populated.

map_raw_values(domain=None, width=None, output_dir=None)[source]

Writes a tif map with raw (non coloured) data values.

Actual work is performed by show.map_raw_values(), more documentation can be obtained there.

This is used to create the TIFs that can later be used for colour composite mapping, i.e. those containing raw reflectance values as pixel values as opposed to stretched RGB values.

Kwargs:
domain (list): Upper and lower bound of the spectral domain (in nm). width (int): width of map in pixels output_dir (str): absolute path to directory to write map to
Returns:
None
map_ref_averaged_spectra(domain=None, width=None, output_dir=None)[source]

Writes a map (png or jpg) and a world file providing georeferencing info for the job.

Actual work is performed by show.map_pil(), more documentation can be obtained there.

The method uses PIL rather than Python graphing to ensure no margin for more accurate georeferencing.

Kwargs:
domain (list): Upper and lower bound of the spectral domain (in nm). width (int): width of map in pixels output_dir (str): path to directory to write map to
Returns:
None
process_brdf()[source]

Convenience method to calculate the BRDF and populate the standard variable names.

Further information is available in the process_brdf.main() function, which performs the work for this method.

Kwargs:
do_compute_error (bool): Whether to include the error computation or not do_spectral_averaging (bool): Whether to average the results across all the input spectral bands
Returns:
tuple: Tuple of 2 numpy arrays: (brdf, brdf error)
process_reflectance()[source]

Convenience method to calculate the reflectance and populate the standard variable names.

Kwargs:
do_compute_error (bool): Whether to include the error computation or not do_spectral_averaging (bool): Whether to calculate the results across all the input spectral bands
Returns:
tuple: Tuple of 2 numpy arrays: (reflectance, reflectance error)
save_netcdf(output_filename=None)[source]

Writes a netcdf file to the job working directory containing all the extracted and calculated data.

The netcdf is added to the ‘outputs’ array on the job.

Kwargs:
output_filename (str): absolute name of filename to write to output_dirname (str) : absolute path of the output directory (if provided, replace that of output_filename)
Returns:
None
set_status(status_text)[source]

Method to update the status log for a potentially long-running job, provides for job-specific logging.

Args:
status_text (str): the incoming status text
Returns:
None
validate_input(request_dict)[source]

Method to validate and populate critical properties of this job instance.

This is the preferred way to prepare an ADAM Job instance for a calculation. The input dictionary parameter is normally the CGI dictionary passed by an HTTP client.

Args:
request_dict(dict): The request dictonary
Returns:
bool. The result of the validation.

An example request_dict follows:

request_dict = {
    'fieldOperationType': 'brdf',
    'fieldSpectralDomains': '850-860 nm; 900-1300',
    'fieldBRDFType': 'polar',
    'fieldSunZenith': 45,
    'fieldViewZenith': 0,
    'fieldRelAzimuth': 0,
    'fieldMonth': 'jan',

    'fieldLatMin'   : 63.2,
    'fieldLonMin'   : 27.2,
    'fieldLatMax'   : 63.3,
    'fieldLonMax'   : 27.4
}

Further examples can be obtained by using the ADAM Web Interface and inspecting the ‘Debug’ tab, where the request_dict is displayed for each request.

Table Of Contents

This Page