NPK: Process2D.py Wed Oct 14 15:25:36 2009
    This module contains all the routines needed to process 2D NMR spectra
    
    The following functions realize a set of operations, needed for NMR 2D processing,
    You will find operation for FT analysis, MaxEnt Analysis,
    Inverse Fourier, etc..
    
    Processing are divided in pre - FT - Post phases.
    For instance a typical 2D procesing, would be

    pre_2d()
    pre_ft_2df2()
    ft_2df2()
    post_ft_2df2()
    pre_ft_2df2()
    ft_2df2()
    post_ft_2df2()
    post_2d()

    but other combinations are possible.

    Most functions take the following arguments :
    arguments :
    audit
        the opened audit file, if empty or set to none, audit will go to the stdout
    filein      the file name of the input file, 
        will loaded before operation, if name is "memory", then processing takes place on the current 2D kernel buffer
    fileout     the file name of the input file, can be "memory"
        will written after operation, if name is "memory", then processing results is left in 2D kernel buffer
    p_in        the dictionary containing all the processing parameters
        most entries are optionnal as entries are protected with try / except
    f_in        the dictionary containing all the details on the spectrum "filein"
    f_out       details on the spectrum "fileout" will be put in this dictionary
    location    the directory where the output files will be written

    Most operation are performed in-memory, and filein and fileout are not a requisite.
    It is possible to just realize the processing from the 2D data-set in memory, leaving the result in memory.
    
    To do this, simple use "memory" instead of a real filename.
 
FT2D(audit,p_in_arg_,f_in,f_out,inputfilename,outputfilename)
        
    FT processing of a 2D FID

    uses  :
        pre_2d()
        pre_ft_2df2()
        ft_2df2()
        post_ft_2df2()
        pre_ft_2df1()
        ft_2df1()
        post_ft_2df1()
        post_2d()

 
FT_F2_2D(audit,p_in_arg,f_in,f_out,inputfilename,outputfilename)
        
    processing of the F2 axis of a 2D FID
    used by 3D processing

    uses  :
        pre_2d()
        pre_ft_2df2()
        ft_2df2()
        post_ft_2df2()

 
FT_F1_2D(audit,p_in_arg,f_in,f_out,inputfilename,outputfilename)
        
    processing of the F1 axis of a 2D FID
    used by 3D processing

    uses  :
        pre_2d()
        pre_ft_2df1()
        ft_2df1()
        post_ft_2df1()
        post_2d()

 
MaxEnt2D(audit,p_in_arg,f_in,f_out,inputfilename,outputfilename)
        
    MaxEnt processing of a 2D FID

    uses  :
        pre_2d()
        pre_ft_2df2()
        pre_ft_2df1()
        maxent_2d()
        post_maxent_2d()
        post_2d()

 
ft_2df1( audit, filein, fileout, p_in, f_in, f_out)
        
This macro realizes the FT operation on the F1 FID of the 2D

it implements the spectral analysis step :
- truncate          : remove points at the end of the FID
- lp_extend         : extend the FID by LP analysis
- apodisation       : multiply the FID by some windowing function 
- Fourier_transform : performs FFT
- reverse           : reverses spectral axis

Input Domain in F1:   time
Output Domain in F1:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    truncate
    truncates along the F1 dimension by removing the last points

  • f1_truncate:   boolean / default 0


  • f1_trunc_size:   integer > 0 <= get_si1_2d() / default get_si1_2d()

    • FID size after truncation

  • conditions:   [ (f1_trunc_size > 0 ) and (f1_trunc_size <= get_si1_2d() ) ]

    f1_lp_extend
    extend FID with LP algorithm

  • f1_lp_extend:   boolean / default 0


  • f1_lp_ext_size:   integer > si1_2d / default 2*get_si1_2d()

    • final size of FID

  • conditions:   [ f1_lp_ext_size > get_si1_2d() ]


  • f1_lp_ext_algo:   enum burg mirror lpsvd lpsvd_stable / default burg

    • algorithm used
    burg and miror are much faster, svd_stable is much slower
    miror is to be used when the phase of the spectrum is known before hand (see lp_ext_off)

  • f1_lp_ext_off:   integer / default 0

    • offset determines the position of the t=0 point, used by mirror algo
    0 no shift : in-phase data set.
    -1 acquisition started exactly half a dwell after t=0 - (will need phase 0 180)
    n>0 acquisition started exactly n dwell before t=0

  • f1_lp_ext_order:   integer / default 20

    • the size of the prediction polynomial used for LP, a rough estimate of the complexity

  • conditions:   [ f1_lp_ext_order < get_si1_2d()/2 ]


  • f1_lp_ext_apod:   boolean / default 1

    • apply a sine bell apodisation after LP extenstion.

    apodize
    standard apodisation by a window

  • f1_apodize:   boolean / default 1


  • f1_apodisation:   string / default "sin(0.1)"

    • the string describing the apodisation to apply
    "apodisation" should be a suite of window apodisation commands
    ie : 'sin(0.5)' or 'exbroad(0.2);sin(0.5)'
    the following predefined functions are implemnted
    sin (sine bell) sqsin (squared sine bell) expbroad (exponential)
    gaussbroad (gaussian) gaussenh (gaussian enhancement)

    Fourier_transform
    performs the Fourier transform

  • f1_fourier_transform:   boolean / default 1


  • f1_ask_for_ftsize:   boolean / default 0

    • if set, the final size is determined by f1_ft_size, automatically determined otherwise
    size for FT is determined by size or lp_ext_size or trunc_size
    it will be extended to the next 2^n

  • f1_ft_size:   integer / default 2*power2(get_si1_2d())

    • the size for FT if f1_ask_for_ftsize is true

  • conditions:   (f1_ft_size == power2(n)


  • f1_ft_type:   enum none ft_sh ft_tppi ft_sh_tppi ft_n_p ft_phase_modu ft rft / default ft_sh

    • the Fourier transform algorithm, depends on the acquisition scheme

  • returned value:   f1_size_for_ft

    • contains the data size right after FT (will usually be different from final size)

    f1_reverse

  • f1_reverse:   boolean / default 0

    • value depends on spectrometer

 
ft_2df2( audit, filein, fileout, p_in, f_in, f_out)
        
This macro realizes the FT operation on the F2 FID of the 2D

it implements the spectral analysis step :
- truncate          : remove points at the end of the FID
- lp_extend         : extend the FID by LP analysis
- apodisation       : multiply the FID by some windowing function 
- Fourier_transform : performs FFT
- causal_corr       : perform causal correction
- reverse           : reverses spectral axis

Input Domain in F2:   time
Output Domain in F2:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    truncate
    truncates the FID by removing the last points

  • f2_truncate:   boolean / default 0


  • f2_trunc_size:   integer / default get_si2_2d()

    • new FID size after truncation

  • conditions:   [ (f2_trunc_size > 0 ) and (f2_trunc_size <= get_si2_2d() ) ]

    lp_extend
    extend FID with Linear Prediction algorithm

  • f2_lp_extend:   boolean / default 0


  • f2_lp_ext_size:   integer / default 2*get_si2_2d()

    • final size of FID

  • conditions:   [ f2_lp_ext_size > get_si2_2d() ]


  • f2_lp_ext_algo:   enum burg mirror lpsvd lpsvd_stable / default burg

    • algorithm used
    burg and mirror are much faster, svd_stable is much slower
    mirror is to be used when the phase of the spectrum is known before hand (see f2_lp_ext_off)

  • f2_lp_ext_off:   integer / default 0

    • offset determines the position of the t=0 point, used by mirror algo
    0 no shift : in-phase data set.
    -1 acquisition started exactly half a dwell after t=0 - (will need phase 0 180)
    f2_lp_ext_off>0 acquisition started exactly n dwell before t=0

  • conditions:   [ f2_lp_ext_off > -1 ]


  • f2_lp_ext_order:   integer / default 40

    • the size of the prediction polynomial used for LP, a rough estimate of the complexity

  • conditions:   [ (f2_lp_ext_order < get_si2_2d()/4 ) and (f2_lp_ext_order > 2) ]


  • f2_lp_ext_apod:   boolean / default 1

    • apply a sine bell apodisation after LP extenstion.

    apodize
    standard apodisation

  • f2_apodize:   boolean / default 1


  • f2_apodisation:   string / default "sin(0.1)"

    • the string describing the apodisation to apply
    "apodisation" should be a suite of window apodisation commands
    ie : 'sin(0.5)' or 'exbroad(0.2); sin(0.5)'
    the following predefined functions are implemnted
    sin (sine bell) sqsin (squared sine bell) expbroad (exponential) gaussbroad (gaussian)
    gaussenh (gaussian enhancement)

    Fourier_transform
    performs the Fourier transform

  • f2_fourier_transform:   boolean / default 1


  • f2_ask_for_ftsize:   boolean / default 0

    • if set, the final size is determined by f2_ft_size, automatically determined otherwise
    size for FT is determined by size or lp_ext_size or trunc_size
    it will be extended to the next 2^n

  • f2_ft_size:   integer / default 2*power2(get_si2_2d())

    • the size for FT if f2_ask_for_ftsize is true

  • conditions:   (f2_ft_size == power2(n)


  • f2_ft_type:   enum none ft rft ft_seq ft_sim / default ft_sim

    • the Fourier transform algorithm, depends on spectrometer and acquisition scheme

  • returned value:   f2_size_for_ft

    • contains the data size right after FT (will usually be different from final size)

    causal_corr
    performs causal correction of the spectrum if not done on the FID

  • causal_corr:   boolean / default 1


  • input value:   AXISF2_ZEROTIMEPOSITION


  • returned value:   causalize

    • contains the correction applied by causalize

    f2_reverse
    reverses the spectral axis after FT

  • f2_reverse:   boolean / default 0

    • depends on spectrometer

 
pre_ft_2df1( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the pre FT operation on each F1 FID of the 2D

it implements FID massaging before FT:
- left_shift       : drops first points of the FID 
- right_shift      : adds empty points on the beginning of the FID
- back_extend      : reconstructs missing points in the beginning of the FID by LP analysis

Input Domain in F1:   time
Output Domain in F1:   time
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    f1_left_shift
    shifts the FID to the left by dropping data points, probably always useless !

  • f1_left_shift:   boolean / default 0


  • f1_left_shift_size:   integer / default 2

    • number of points to shift

  • conditions:   [ (f1_left_shift_size > 0) and (f1_left_shift_size < get_si1_2d()) ]

    f1_right_shift
    shifts the FID to the right by adding zero data points

  • f1_right_shift:   boolean / default 0


  • f1_right_shift_size:   integer / default 2

    • number of points to shift

  • conditions:   [ (f1_right_shift_size > 0) ]

    f1_back_extend
    extends the FID backward for reconstruction of initial missing points

  • f1_b_extend:   boolean / default 0


  • f1_b_extend_size:   integer > 0 / default 2*get_si1_2d()

    • number of missing points to reconstruct

  • conditions:   [ f1_b_extend_size > 0 ]


  • f1_b_extend_algo:   enum burg / default burg

    • algorithm used

  • returned value:   f1_b_extend_order

    • the order used during back_extend

  • returned value:   f1_b_extend_size

    • the number of points added by back_extend

 
pre_ft_2df2( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the pre FT operation on each F2 FID of the 2D

it implements FID massaging before FT:
- dc_offset        : corrects for constant offset in FID
- causalize        : changes DSP processed FID (Bruker) to causal FID's by Hilbert transform
- flatten_solvent  : removes solvent signal by FID analysis
- left_shift       : drops first points of the FID 
- right_shift      : adds empty points on the beginning of the FID
- back_extend      : reconstructs missing points in the beginning of the FID by LP analysis

Input Domain in F2:   time
Output Domain in F2:   time
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    dc_offset
    removes a constant level in the fid, estimated from the last points

  • dc_offset:   boolean / default 0


  • fid_noise_f2zone:   float / default 0.2

    • constant offset of the FID, estimated on the last points of the dataset

    causalize
    removes the non causal header found in certain (Bruker) implementation of the digital filter.

  • causalize:   boolean / default 0


  • input value:   axisf2_zerotimeposition


  • returned value:   causalize

    • contains the correction applied by causalize

    flatten_solvent
    removes water from fit of the FID in the time domain

  • flatten_solvent:   boolean / default 0


  • flat_solv_mode:   enum polynomial moving_average polynomial+moving_average moving_average+polynomial / default polynomial

    • algorithm used

    f2_left_shift
    shifts the FID to the left by dropping data points

  • f2_left_shift:   boolean / default 0


  • f2_left_shift_size:   integer / default 2

    • number of points to shift

  • conditions:   [ (f2_left_shift_size > 0) and (f2_left_shift_size < get_si2_2d()) ]

    f2_right_shift
    shifts the FID to the right by adding zero data points

  • f2_right_shift:   boolean / default 0


  • f2_right_shift_size:   integer / default 2

    • number of points to shift

  • conditions:   [ (f2_right_shift_size > 0) ]

    back_extend
    extends the FID backward for reconstruction of initial missing points

  • f2_b_extend:   boolean / default 0


  • f2_b_extend_size:   integer / default 2

    • number of missing points to reconstruct

  • conditions:   [ f2_b_extend_size > 0 ]


  • f2_b_extend_algo:   enum burg / default burg

    • algorithm used

  • returned value:   f2_b_extend_order

    • the order used during back_extend

  • returned value:   f2_b_extend_size

    • the number of points added by back_extend

 
post_ft_2df1( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the post FT operation on each F1 spectrum of the 2D

it implements the massaging of the spectrum after FT :
- phase         : apply phase correction on the complex spectrum
- autophase     : compute and apply phase correction 
- invHibert     : apply inverse Hilbert transform
- calibration   : apply ppm calibration
- autocalibrate : autocalibration of the F1 axis based on the F2 calibration
- spectral_zone : extract a spectral zone

Input Domain in F2:   frequency
Output Domain in F2:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    phase
    applies a F1 phase correction to the spectrum

  • f1_phase:   boolean / default 0


  • f1_phase_0:   float / default 0.0

    • global order phase correction

  • conditions:   ( (f1_phase_0 >= -180 ) and (f1_phase_0 <= 180) )


  • f1_phase_1:   float / default 0.0

    • 1st order phase correction

    autophase
    automatic phase correction of the spectrum in F1
    phase is computed from a set

  • f1_autophase:   boolean / default 0


  • f1_phase_algo:   enum apsl apmin / default apmin

    • algorithm used

  • returned value:   f1_autophase_0

    • the value of the global order phase correction applied by autophase

  • returned value:   f1_autophase_1

    • the value of the 1st order phase correction applied by autophase

    invHilbert
    apply an inverse Hilbert transform.
    This operation takes a complex spectrm, and generate a real spectrum in-place
    i.e. with twice the number of real points.

  • f1_invHilbert:   boolean / default 0

    autocalibrate
    calibrates the F1 axis based on the F2 calibration, using IUPAC / IUPAB conventions

  • f1_autocalibrate:   boolean / default 1


  • autocalibrate_mode:   enum standard biomolecule / default standard

    • if autocalibrate_mode is standard, IUPAC rules are followed,
    if autocalibrate_mode is biomolecule then
    DSS is used instead of TMS for 2H and 13C
    NH3 is used instead of CH3NO2 for 15N
    (CH3O)3PO is used instead of H3PO4 for 31P

    calibration
    calibrate the ppm scale on the spectrum

  • f1_calibration:   float / default 0

    • the coordinate in ppm of the lower-most point in the spectrum

    spectral_zone
    extract one spectral zone of the spectrum

  • f1_spectral_zone:   boolean / default 0


  • f1_spec_zone_left:   float / default 10.0

    • the left border of the extract zone, in unit

  • f1_spec_zone_left_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_left is given

  • f1_spec_zone_right:   float / default 0.0

    • the right border of the extract zone, in unit

  • f1_spec_zone_right_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_right is given

  • returned value:   f1_spec_zone_left

    • the left coordinate of the extracted spectral zone in index

  • returned value:   f1_spec_zone_right

    • the right coordinate of the extracted spectral zone in index

 
post_ft_2df2( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the post FT operation on each F2 spectrum of the 2D

it implements the massaging of the spectrum after FT :
- phase         : apply phase correction on the complex spectrum
- autophase     : compute and apply phase correction 
- invHibert     : apply inverse Hilbert transform
- calibration   : apply ppm calibration
- spectral_zone : extract a spectral zone

Input Domain in F2:   frequency
Output Domain in F2:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    phase
    applies a F2 phase correction to the spectrum

  • f2_phase:   boolean / default 0


  • f2_phase_0:   float / default 0.0

    • global order phase correction

  • conditions:   ( (f2_phase_0 >= -180 ) and (f2_phase_0 <= 180) )


  • f2_phase_1:   float / default 0.0

    • 1st order phase correction

    autophase
    automatic phase correction of the spectrum in F2
    phase is computed on the first experiment

  • f2_autophase:   boolean / default 0


  • f2_phase_algo:   enum apsl apmin / default apmin

    • algorithm used

  • returned value:   f2_autophase_0

    • the value of the global order phase correction applied by autophase

  • returned value:   f2_autophase_1

    • the value of the 1st order phase correction applied by autophase

    invHilbert
    apply an inverse Hilbert transform.
    This operation takes a complex spectrum, and generates a real spectrum in-place
    i.e. with twice the number of real points.

  • f2_invHilbert:   boolean / default 0

    calibration
    calibrate the ppm scale on the spectrum

  • f2_calibration:   float / default 0

    • the coordinate in ppm of the right-most point in the spectrum

    spectral_zone
    extract one spectral zone of the spectrum

  • f2_spectral_zone:   boolean / default 0


  • f2_spec_zone_left:   float / default 10.0

    • the left border of the extract zone, in unit

  • f2_spec_zone_left_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_left is given

  • f2_spec_zone_right:   float / default 0.0

    • the right border of the extract zone, in unit

  • f2_spec_zone_right_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_right is given

  • returned value:   f2_spec_zone_left

    • the left coordinate of the extracted spectral zone in index

  • returned value:   f2_spec_zone_right

    • the right coordinate of the extracted spectral zone in index

 
pp_2d( audit, filein, filepeak, p_in, f_in, f_out )
        
This macro realizes the peak picking of a 2D dataset

it implements
- spec_noise        : estimates noise
- prefilter               : apply a filtering to the data-set to remove features below a given separation in Hz
- restrict          : defines the zone in whch the peak-picking is to be done
- peakpick                : do the peak picking
- postfilter              : apply a post filtering on the peak list

Input Domain in F1:   frequency
Input Domain in F2:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    spec_noise
    evaluate noise, estimated by finding an empty zone

  • spec_noise:   boolean / default 0


  • spec_noise_n:   integer / default 30

    • number of different zones where noise is evaluated

  • returned value:   NoiseInFrequencyDomain

    prefilter
    smooths the spectrum before peak-picking, modification is not stored permanently

  • prefilter:   boolean / default 0


  • f1_prefilter_value:   float / default 10.0

    • pre-smoothing parameter for F1 axis in Hz

  • f2_prefilter_value:   float / default 10.0

    • pre-smoothing parameter for F2 axis in Hz

    restrict
    restricts the peak picking to a certain spectral zone

  • restrict:   boolean / default 0


  • f1_restrict_left:   float / default 10.0

    • the left border of the F1 restrict zone, in unit

  • f1_restrict_left_unit:   enum ppm hz index / default ppm

    • the unit in which F1 restrict_left is given

  • f1_restrict_right:   float / default 0.0

    • the right border of the F1 restrict zone, in unit

  • f1_restrict_right_unit:   enum ppm hz index / default ppm

    • the unit in which F1 restrict_right is given

  • conditions:   (f1_restrict_left{index} < f1_restrict_right{index})


  • returned value:   f1_restrict_left

    • the left coordinate of the F1 restrict zone in index

  • returned value:   f1_restrict_right

    • the right coordinate of the F1 restrict zone in index

  • f2_restrict_left:   float / default 10.0

    • the left border of the F2 restrict zone, in unit

  • f2_restrict_left_unit:   enum ppm hz index / default ppm

    • the unit in which F2 restrict_left is given

  • f2_restrict_right:   float / default 0.0

    • the right border of the F2 restrict zone, in unit

  • f2_restrict_right_unit:   enum ppm hz index / default ppm

    • the unit in which f2_restrict_right is given

  • conditions:   (f2_restrict_left{index} < f2_restrict_right{index})


  • returned value:   f2_restrict_left

    • the left coordinate of the F2 restrict zone in index

  • returned value:   f2_restrict_right

    • the right coordinate of the F2 restrict zone in index

    peakpick
    do the peak picking, by detecting local extrema

  • peakpick:   boolean / default 1


  • noise_thresh:   float / default 10.0

    • minimum peak/noise ratio for peak detection

  • ratio_thresh:   float / default 30.0

    • maximum tallest peak / peak ratio for peak detection

  • peak_sign:   enum positive negative both / default positive

    • sign of the peaks to detect

  • returned value:   nb_detect_peaks

    • the number of detected peaks

  • returned value:   low_limit

    • low limit for peak detection

    aggregate
    sorts peak list to aggregate peaks close from each other

  • aggregate:   boolean / default 0


  • aggregate_value:   float / default 10.0

    • distance parameter in Hz

    symetrize_peak
    symetrizes the peak list to peaks relative to the diagonal

  • symetrize_peak:   boolean / default 0


  • symetrize_peak_tol:   float / default 10.0

    • tolerance for symetrisation, in Hz

  • symetrize_peak_algo:   enum remove add / default add

    • correction to apply,
    either add missing peaks, are remove non symetrical peaks

 
maxent_2d( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the MaxEnt spectral analysis of a 2D FID

 it implements the following steps :
 - freq_massage      : do some preparation in the frequency domain
    - causal_corr    : perform causal correction
    - phase                      : apply phase correction on the complex spectrum before analysis
    - spectral_zone  : extract a spectral zone before analysis
    - reverse        : reverses spectral axis
 - truncate          : remove points at the end of the FID
 - preconvoluate     : apply a preconcolution function of the FID
 - partialsampling   : implements analysis on partially sampled FID
 - positive_negative : implements two channel processing for positive and negative lines
 - deconvoluate      : deconvoluate spectrum from a given time domain response
 - maxent            : apply Maximum Entropy analysis of the dataset

arguments :
    audit       the opened audit file
    filein      the file name of the input file, can be "memory"
    fileout     the file name of the input file, can be "memory"
    p_in        the dictionary containing all the processing parameters
    f_in        the dictionary containing all the details on the spectrum "filein"
    f_out       details on the spectrum "fileout" will be put in this dictionary

 Input Domain in F1:   time
 Output Domain in F1:   frequency
 Input Domain in F2:   time
 Output Domain in F2:   frequency
 Dimensionality:   2

 Author:   Marc-Andre Delsuc
 Version:   1.0
Actions

    f2_freq_massage
    computes a temporary Fourier transform in F2

  • f2_freq_massage:   boolean / default 1


  • f2_ft_type:   enum none ft_seq ft_sim ft rft / default ft

    • the Fourier transform algorithm, depends on spectrometer and acquisition scheme

  • f2_reverse:   boolean / default 0

    • reverses the spectral axis after FT

  • causal_corr:   boolean / default 1

    • performs causal correction of the spectrum if not done on the FID

  • exclusive parameters:   causal [ causalize causal_corr ]


  • f2_phase:   boolean / default 0


  • f2_phase_0:   float / default 0.0

    • global order phase correction

  • f2_phase_1:   float / default 0.0

    • 1st order phase correction

  • input value:   AXISF2_ZEROTIMEPOSITION


  • returned value:   causalize contains the correction applied by causalize


  • :   

    spectral_zone
    extract one spectral zone of the spectrum

  • f2_spectral_zone:   boolean / default 0


  • f2_spec_zone_left:   float / default 10.0

    • the left border of the extract zone, in unit

  • f2_spec_zone_left_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_left is given

  • f2_spec_zone_right:   float / default 0.0

    • the right border of the extract zone, in unit

  • f2_spec_zone_right_unit:   enum ppm hz index / default ppm

    • the unit in which spec_zone_right is given

  • returned value:   f2_spec_zone_left

    • the left coordinate of the extracted spectral zone in index

  • returned value:   f2_spec_zone_right

    • the right coordinate of the extracted spectral zone in index

    f1_freq_massage
    computes a temporary Fourier transform in F1

  • f1_freq_massage:   boolean / default 0


  • f1_ft_type:   enum none ft_sh ft_tppi ft_sh_tppi ft_n_p ft_phase_modu ft rft / default ft_sh

    • the Fourier transform algorithm, depends on spectrometer and acquisition scheme

  • f1_reverse:   boolean / default 0

    • reverses the spectral axis after FT

  • f1_phase:   boolean / default 0


  • f1_phase_0:   float / default 0.0

    • global order phase correction

  • f1_phase_1:   float / default 0.0

    • 1st order phase correction

    F2 truncate
    removes last points of FID

  • f2_truncate:   boolean / default 0


  • f2_trunc_size:   integer / default get_si2_2d()

    • new FID size after truncation

  • conditions:   [ (f2_trunc_size > 0 ) and (f2_trunc_size <= get_si2_2d() ) ]

    F1 truncate

  • f1_truncate:   boolean / default 0


  • f1_trunc_size:   integer > 0 <= get_si1_2d() / default get_si1_2d()

    • new FID size after truncation

    preconvoluate
    apply a preconvolution before analysis, this may help stability of the algorithm and enhance noise rejection

  • preconvoluate:   boolean / default 0


  • f1_exponential_preconv:   float / default 1.0

    • exponential preconvolution along F1 axis

  • f2_exponential_preconv:   float / default 1.0

    • exponential preconvolution along F2 axis

    partialsampling
    set-up for processing data partially sampled in the time domain

  • partialsampling:   boolean / default 0


  • partialsampling_mode:   enum F1 F2 F1+F2 F12 / default F1+F2

    • defines the sampling mod used on the 2D plane
    F1 : sampling along the F1 spectral axis only, -- use f1_samplingfile definition
    F2 : sampling along one F2 spectral axis only, -- use f2_samplingfile definition
    F1+F2 : sampling along both spectral axis, defined independentely
    F12 : global sampling of the F1xF2 plane -- use F12_samplingfile definition

  • f1_samplingfile:   string / default "F1vplist"

    • the filename of the file containing the list of sample to use along the F1 axis

  • f2_samplingfile:   string / default "F2vplist"

    • the filename of the file containing the list of sample to use along the F2 axis
    see sampling.g for details

  • f12_samplingfile:   string / default "F12vplist"

    • the filename of the file containing the list of sample to use on the F1xF2plane

    positive_negative
    set-up for positive_negative analysis

  • positive_negative:   boolean / default 0

    deconvoluate
    describes the deconvolution to apply during analysis,

  • deconvoluate:   boolean / default 1

    • this part is not finished yet, only suport for exponential deconvolution so far

  • f1_exponential_deconv:   float / default 1.0

    • exponential deconvolution along F1 axis

  • f2_exponential_deconv:   float / default 1.0

    • exponential deconvolution along F2 axis

    me_preset
    presets the MaxEnt parameters to default values

  • me_preset_value:   enum 0 1 2 3 4 5 / default 3

    • sets the parameter for a balance between speed (1) and quality (5), 0 is for fit

    me_details
    if this flag is on, default parameters can be set.

  • me_details:   boolean / default 0


  • me_size:   integer


  • me_iteration:   integer


  • me_int_iteration:   


  • me_ncheck:   integer


  • me_lambda_control:   integer


  • me_lambda_speed:   


  • me_algo:   enum 0 1 2 / default 1

    maxent
    apply MaxEnt analysis,

  • maxent:   boolean / default 1


  • me_preset:   boolean / default 1

    • determines convergence parameters from a given preset or gives the details

  • preset_value:   enum 1 2 3 4 5 / default 3

    • preset for maxent# 1 is faster, 5 is more accurate

  • iteration:   integer / default 100

    • global number of iterations - set by preset

  • control:   integer / default 10

    • number of iterations between control step - set by preset

  • lambsp:   float / default 5.0

    • the value of the lambda controlling parameter LAMBSP in the Gifa algorithm

  • lambcont:   enum 0 1 2 / default 1

    • the value of the lambda controlling parameter LAMBCONT in the Gifa algorithm

  • f1_me_size:   integer / default 2*power2(get_si1_2d())

    • F1 size for MaxEnt reconstruction

  • f2_me_size:   integer / default 2*power2(get_si2_2d())

    • F2 size for MaxEnt reconstruction

  • noise_weight:   float / default 1.0

    • the noise used during the MaxEnt iteration is weigthed with this scalar,
    this permits to compensate for over- or under-estimate of the noise level

  • conditions:   [ noise_weight > 0.0 ]


  • conditions:   [ ispower2(f1_me_size) ]


  • conditions:   [ ispower2(f2_me_size) ]

 
post_2d( audit, filein, fileout, p_in, f_in, f_out, location=".")
        
This macro realizes the post FT operation on a 2D spectrum

This macro implements the massaging of the spectrum after complete Spectral analysis :
- autophase2D   : computes and apply global phase correction
- modulus       : take the modulus of the complex spectrum
- baseline      : baseline correction
- shear         : apply the shear correction
- tilt          : apply the tilt correction
- symetrize     : apply symetrisation algorithms
- smoothing     : apply a smoothing filter
- median        : apply a media filter
- projection    : compute F1 and F2 projections
- select_state  : enforce the final state of the data
- spec_noise    : estimates noise in final spectrum

Input Domain in F1:   frequency
Output Domain in F1:   frequency
Input Domain in F2:   frequency
Output Domain in F2:   frequency

Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    autophase_2d
    automatic phase correction of the spectrum in 2D
    phase is computed from a set of lines extracted fro the dataset

  • autophase_2d:   boolean / default 1


  • phase_2d_algo:   enum apsl2d apmin2d / default apmin2d

    • algorithm used for the phase correction

  • phase_2d_axis:   enum F1 F2 F12 / default F12

    • Axis along which the autophasing is to be applied.

  • returned value:   f1_autophase_0

    • the value of the global order phase correction applied by autophase

  • returned value:   f1_autophase_1

    • the value of the 1st order phase correction applied by autophase

  • returned value:   f2_autophase_0

    • the value of the global order phase correction applied by autophase

  • returned value:   f2_autophase_1

    • the value of the 1st order phase correction applied by autophase

    modulus
    modulus of the spectrum

  • modulus:   boolean / default 0

    throw_imaginary
    at this stage, data should be real

  • throw_imaginary:   boolean / default 1

    • throw_imaginary set to false will keep hypercomplex data
    Note that most of the following procesing will fail

    baseline correction
    baseline correction of the spectrum

  • f2_baseline:   boolean / default 1


  • f2_bcorr_algo:   enum offset linear spline quest polynomial moving_average / default moving_average

    • offset : removes a automatically determined offset
    linear - spline : uses the list of user determined pivot points to define the baseline,
    : fit with a straight line or a spline
    quest : reconstruct the beginning of the FID using Linear Prediction technics,
    should usually be followed by offset
    polynomial - moving_average : uses statistics to separate signal from baseline,
    : apply a polynomial or moveing average correction.
    in more than one term is given (blank separated string!), the correction are applied in sequence.

  • spec_noise_n:   integer / default 10

    • used by the offset algorithm to determine the offset to correct

  • f2_bcorr_pivots:   Integerlist / default (int(0.02*get_si2_2d()),int(0.98*get_si2_2d()))

    • pivot points used by the linear or spline algorithm

  • conditions:   (f2_bcorr_pivots > 0 and bcorr_pivots <= get_si2_2d())


  • f2_bcorr_radius:   integer / default 1

    • radius around pivot points used by the linear or spline algorithm

  • f2_bcorr_order:   integer / default 10


  • conditions:   (f2_bcorr_order > 0 and f2_bcorr_order < get_si2_2d() and f2_bcorr_order == 2*n)

    • order (number of points corrected in the time domain) of the quest algorithm

    baseline correction
    baseline correction of the spectrum

  • f1_baseline:   boolean / default 1


  • f1_bcorr_algo:   enum offset linear spline quest polynomial moving_average / default moving_average

    • offset : removes a automatically determined offset
    linear - spline : uses the list of user determined pivot points to define the baseline,
    : fit with a straight line or a spline
    quest : reconstruct the beginning of the FID using Linear Prediction technics,
    should usually be followed by offset
    polynomial - moving_average : uses statistics to separate signal from baseline,
    : apply a polynomial or moveing average correction.
    in more than one term is given (blank separated string!), the correction are applied in sequence.

  • spec_noise_n:   integer / default 10

    • used by the offset algorithm to determine the offset to correct

  • f1_bcorr_pivots:   Integerlist / default (int(0.05*get_si1_2d()),int(0.95*get_si2_2d()))

    • pivot points used by the linear or spline algorithm

  • conditions:   (f1_bcorr_pivots > 0 and bcorr_pivots <= get_si1_2d())


  • f1_bcorr_radius:   integer / default 1

    • radius around pivot points used by the linear or spline algorithm

  • f1_bcorr_order:   integer / default 10


  • conditions:   (f1_bcorr_order > 0 and f1_bcorr_order < get_si1_2d() and f1_bcorr_order == 2*n)

    • order (number of points corrected in the time domain) of the quest algorithm

    shear
    apply the shear correction

  • shear:   boolean / default 0


  • shear_slope:   float / default -1

    • the slope of the shear operation, -1 corresponds to the INADEQUATE to COSY correction

  • shear_pivot:   float / default 0.5

    • the location of the invariant point

  • conditions:   [ (shear_pivot>=0) and (shear_pivot<=1) ]

    tilt
    apply the tilt correction

  • tilt:   boolean / default 0


  • tilt_slope:   float / default 1

    • the slope of the tilt operation, 1 corresponds to the J-Res correction

  • tilt_pivot:   float / default 0.5

    • the location of the invariant point

  • conditions:   [ (tilt_pivot>=0) and (tilt_pivot<=1) ]

    symetrize
    requires the data-set to be square, will reduce to the smallest of both axes

  • symetrize:   boolean / default 0


  • symetrize_algo:   enum mean smallest / default smallest

    smoothing
    apply a smoothing filter to the data-set

  • smoothing:   boolean / default 0


  • smooth_f1_w:   integer / default 2

    • size of the smoothing window in f1

  • smooth_f2_w:   integer / default 2

    • size of the smoothing window in f2

  • smooth_iteration:   / default 1

    • number of loop

  • conditions:   [ smooth_f1_w > 0 ]


  • conditions:   [ smooth_f2_w > 0 ]

    median
    apply a median filter to the data-set

  • conditions:   [ [ median == ((f1_extract_real and f2_extract_real) | modulus) ]


  • median:   boolean / default 0


  • median_f1_w:   integer / default 2

    • size of the median window in f1

  • median_f2_w:   integer / default 2

    • size of the median window in f2

  • median_i:   integer <= median_w / default 2

    • index of point to keep in the median filtering window

  • conditions:   [ median_f2_w > 0 ]


  • conditions:   [ median_f1_w > 0 ]


  • conditions:   [ (median_i > 0) and (median_i <= (median_f1_w*median_f2_w)) ]

    projection
    creates and store in files the projections of the 2D
    2 files are created per axis :
    using a mean algorithm (_M suffix) and using a skyline algorithm (_S suffix)

  • projection:   boolean / default 1


  • returned value:   f1_projection_M


  • returned value:   f2_projection_M


  • returned value:   f1_projection_S


  • returned value:   f2_projection_S

    spec_noise
    evaluate noise, estimated by finding an empty zone

  • spec_noise_n:   integer / default 10

    • number of different zones where noise is evaluated

  • returned value:   spec_std_noise

    • estimate of the noise in the spectrum

    select_state
    permit to choose the state complex / real of the output file
    complex data are changed to real by dropping the imaginary part
    real data are changed to complex by computing the Hilbert transform with tocomplex()
    this is usually not required as all processing commands prepare the state themselves

  • select_state:   boolean / default 0

    • actuallly does the selection

  • f1_state:   enum ignore complex real / default ignore

    • force the f1 axis to real or complex. ignore will let the axis unchanged

  • f2_state:   enum ignore complex real / default ignore

    • force the f2 axis to real or complex. ignore will let the axis unchanged

 
post_maxent_2d( audit, filein, fileout, p_in, f_in, f_out,location="." )
        
This macro realizes the post MaxEnt operation on a 2D spectrum
 
it implements the massaging of the spectrum after FT :
- level_correction      : correct the slight level offseting by doing a simple baseline correction
- calibration           : apply ppm calibration
- positive_negative_cor : reconstruct a composite 2D from the two channel processing
- smoothing             : apply a smoothing filter
- median                : apply a media filter
- projection            : compute F1 and F2 projections
- select_state          : enforce the final state of the data

arguments :
    audit       the opened audit file
    filein      the file name of the input file, can be "memory"
    fileout     the file name of the input file, can be "memory"
    p_in        the dictionary containing all the processing parameters
    f_in        the dictionary containing all the details on the spectrum "filein"
    f_out       details on the spectrum "fileout" will be put in this dictionary

Input Domain in F1:   frequency
Output Domain in F1:   frequency
Input Domain in F2:   frequency
Output Domain in F2:   frequency
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    level_correction
    correct the slight level offseting by doing a simple baseline correction

  • level_correction:   boolean / default 1


  • f2_level_correction_axis:   boolean / default 1

    • apply level on F2 axis

  • f2_level_correction_pivots:   list / default (int(0.01*get_si2_2d()),int(0.99*get_si2_2d()))

    • where level will be computed

  • f1_level_correction_axis:   boolean / default 0

    • apply level on F1 axis

  • f1_level_correction_pivots:   list / default (int(0.01*get_si1_2d()),int(0.99*get_si1_2d()))

    • where level will be computed

    positive_negative_cor
    reconstruct a composite 2D from the two channel processing

  • positive_negative_cor:   boolean / default 0

    calibration
    calibrate 0ppm on the spectrum

  • f2_calibration:   float / default 0

    • the coordinate in ppm of the right-most point in the spectrum

  • f1_autocalibrate:   boolean / default 1

    • calibrate the F1 axis based on the F2 calibration.
    overwrite the F1 calibration.

  • f1_calibration:   float / default 0

    • the coordinate in ppm of the lower-most point in the spectrum

    smoothing
    apply a smoothing filter to the data-set

  • smoothing:   boolean / default 0


  • smooth_f1_w:   integer / default 2

    • size of the smoothing window in f1

  • smooth_f2_w:   integer / default 2

    • size of the smoothing window in f2

  • smooth_iteration:   / default 1

    • number of loop

  • conditions:   [ smooth_f1_w > 0 ]


  • conditions:   [ smooth_f2_w > 0 ]

    median
    apply a median filter to the data-set

  • conditions:   [ [ median == ((f1_extract_real and f2_extract_real) | modulus) ]


  • median:   boolean / default 0


  • median_f1_w:   integer / default 2

    • size of the median window in f1

  • median_f2_w:   integer / default 2

    • size of the median window in f2

  • median_i:   integer <= median_w / default 2

    • index of point to keep in the median filtering window

  • conditions:   [ median_f2_w > 0 ]


  • conditions:   [ median_f1_w > 0 ]


  • conditions:   [ (median_i > 0) and (median_i <= (median_f1_w*median_f2_w)) ]

    projection
    creates and store in files the projections of the 2D
    2 files are created per axis :
    using a mean algorithm (_M suffix) and using a skyline algorithm (_S suffix)

  • projection:   boolean / default 1


  • returned value:   f1_projection_M


  • returned value:   f2_projection_M


  • returned value:   f1_projection_S


  • returned value:   f2_projection_S

    spec_noise
    evaluate noise, estimated by finding an empty zone

  • spec_noise_n:   integer / default 10

    • number of different zones where noise is evaluated

  • returned value:   spec_std_noise

    • estimate of the noise in the spectrum

    select_state
    permit to choose the state complex / real of the output file
    complex data are changed to real by dropping the imaginary part
    real data are changed to complex by computing the Hilbert transform with tocomplex()
    this is usually not required as all processing commands prepare the state themselves

  • select_state:   boolean / default 0

    • actuallly does the selection

  • f1_state:   enum ignore complex real / default ignore

    • force the f1 axis to real or complex. ignore will let the axis unchanged

  • f2_state:   enum ignore complex real / default ignore

    • force the f2 axis to real or complex. ignore will let the axis unchanged

 
pre_2d( audit, filein, fileout, p_in, f_in, f_out )
        
This macro realizes the preparation operation on a 2D FID

it implements global FID evaluation before 2D FT:
- data_check       : performs minimum data checking 
- fid_noise        : measure of noise and offset levels in FID of noise and offset levels in FID
- conv_n_p         : conversion of N+P acquisition (echo / antiecho)

arguments :
    audit       the opened audit file
    filein      the file name of the input file, can be "memory"
    fileout     the file name of the input file, can be "memory"
    p_in        the dictionary containing all the processing parameters
    f_in        the dictionary containing all the details on the spectrum "filein"
    f_out       details on the spectrum "fileout" will be put in this dictionary

Input Domain in F1:   time
Input Domain in F2:   time
Output Domain in F1:   time
Output Domain in F2:   time
Dimensionality:   2

Author:   Marc-Andre Delsuc
Version:   6.0
Actions

    data_check
    performs minimum data integrity

  • data_check:   boolean / default 1

    • checks points which can not be checked by the kernel
    for the moment, only check data-sets with an odd number of points

    fid_noise
    evaluate noise and offset, estimated from the last points

  • fid_noise_F2zone:   float / default 0.2

    • zone of the 2D FID (expressed in % from the end) where to compute noise

  • conditions:   [ (fid_noise_F2zone > 0.0) and (fid_noise_F2zone<=1)]


  • fid_noise_f1zone:   float / default 0.1

    • zone of the 2D FID (expressed in % from the end) where to compute noise

  • conditions:   [ (fid_noise_f1zone > 0.0) and (fid_noise_f1zone<=1)]


  • returned value:   fid_offset

    • constant offset of the FID, estimated on the last points of the dataset

  • returned value:   NOISEINTIMEDOMAIN

    • estimate of the fid noise, from the standard deviation of the last points of the dataset

    conv_n_p
    convert experiment acquired in n+p (echo/antiecho mode) to hypercomplex

  • conv_n_p:   boolean / default 0


  • exclusive parameters:   [ conv_n_p ft_phase_modu ]

 
read_file_2d( audit, filein)
        
    used to read the input binary file, while maintaining the audittrail

 
write_file_2d( audit, fileout)
        
    used to write the output binary file, while maintaining the audittrail