Latest revision as of 14:02, 29 June 2016

NEEMP features three modes: calculation, parametrization and validation. Each mode can be selected specifying one of the following keywords after the option --mode or -m:

Calculation mode : charges (calculate EEM charges)

Parametrization mode : params (calculate EEM parameters)

Validation mode : cover (display how many molecules are covered by the given parameters set) and quality (perform EEM parameters quality assay)

Additional functionality: NEEMP enables the user to display information about the provided training set, upon calling the keywords -m info or --mode info. Specifically, number of molecules, the total number of atoms and number of atoms for each atomic type are printed; the latter being defined by default as the chemical element and the maximal bond order. For details on the usage syntax, see the examples or option list paragraph.

Calculation mode

BASIC USAGE: ./neemp -m charges --sdf-file examples/set01.sdf --par-file examples/ElemBond.par --chg-out-file eem_charges

To compute EEM charges with NEEMP, it is necessary to provide the SDF file with molecules for which the charges will be computed (option --sdf-file) and the PAR file with EEM parameters (option --par-file). EEM charges will be written into the file specified by --chg-out-file option.

The user should keep in mind that, because of the EEM method itself, the EEM charges cannot be computed for atoms or atomic types for which the parameters are missing. This situation is well depicted by Figure 1. On the left side the parameter set contains values for all the atomic types in set01.sdf, on the other hand in the right side image it is evident how the same parameter set is lacking values for several atomic types present in set02.sdf, leading to the discard of many molecules (for details on the syntax, see the examples or option list section).


Figure 1: The lack of parameter values in the parameter set file ElemBond.par for the atomic types highlighted in the right side image causes the discarding of several molecules from set02.sdf, emphasized by the drop in the covered molecules percentage.

Parametrization mode

This mode serves for calculation of EEM parameters. Three main components make up the input for this mode:

a training set of molecules (argument of --sdf-file option)

a set of QM charges for each molecule in the sdf file (argument of --chg-file option)

the specification of the parametrization approach (by -p or --params-method) and its specific options.

Linear Regression (LR)

BASIC USAGE: ./neemp -m params -p lr-full --sdf-file examples/set01.sdf --chg-file examples/set01.chg --par-out-file new_parameters.par

LR (--params-method lr-full or -p lr-full) is the default parametrization approach in NEEMP, since its application in the parametrization process is well documented and described in literature. The main extension in respect to previous version is the ability for the user to define by which metrics the best performing parameter set will be selected (see figure):

-s R2 or --sort-by R2 to select the parameter set with the highest squared Pearson coefficient (R²)

-s RMSD or --sort-by RMSD to select the parameter set with the lowest atomic type root mean square difference (avg(RMSDa))


Figure 2: Detailed view of the LR parametrization settings for two distinct NEEMP executions differing in the best-performance selection metrics (R² in left side image and RMSD in the right side image).

Additionally, the quality of the parameter set may benefit from the usage of a discarding procedure (-d simple or --discard simple). In fact it has been proved that using a subset of the original training set often leads to EEM charges that better agree with the reference QM charges. For useful examples on the LR parametrization approach and the discarding procedure see respectively examples 3/4 and 5. In particular the latter introduces the --limit-iters and --limit-time options, which modulate the behaviour of the discarding procedure, setting respectively an upper bound in the number of iterations and/or in the execution time.

Differential Evolution + Minimization (DE-MIN)

BASIC USAGE: ./neemp -m params -p de --sdf-file examples/set01.sdf --chg-file examples/charges.chg --par-out-file new_parameter.par

In parallel we developed an advanced EEM parametrization approach specifically tailored to cope with heterogeneous data, which in the past presented quite a challenge in the EEM parametrization framework. It consists in the combination of differential evolution method with a local minimization technique (NEWUOA), hence the acronym DE-MIN. To call this approach in NEEMP use the option --params-method de or -p de.

**Figure 3:** Abridged **DE-MIN** approach output for structural file *examples/set01.sdf* and charge file *examples/set01.chg*. In the upper block are displayed the specifics for the **de-min** parametrization procedure. Note as the bottom block presents the same structure as for other parametrization approaches and options (compare here).

The behaviour of the DE-MIN approach can be modulated by several options, but to make life easier we provide the user with reasonable default values that have been tuned to output optimal parameter set (for a complete view of the options refer to the option list and the examples section).

NB: the default quality evaluating metrics for DE-MIN approach is (avg(RMSDa)).

Validation mode

This mode allows us to perform two types of EEM parameter set validation - coverage validation and quality validation.

Coverage validation

BASIC USAGE: ./neemp -m cover -sdf-file examples/set01.sdf --par-file examples/Element.par

**Figure 4:** Percentage of molecules in set *examples/set01.sdf* covered by the displayed parameters set *examples/Element.par*

To see how many molecules from the set are covered by the provided parameters, simply pass NEEMP those two files (options --sdf-file and --par-file). Figure 4 shows a close-up from the cover mode output for set01.sdf and the parameters set Element.par, both present in the examples directory (for details on the syntax and the complete output, see the examples or option list paragraph)

Quality validation

BASIC USAGE: ./neemp -m quality --sdf-file examples/set01.sdf --chg-file examples/set01.chg --par-file examples/Element.par

To assess the quality of the EEM parameters on a different set than one they were based on, NEEMP's quality mode can be called.

Required input files are: EEM parameters (passed as argument of option --par-file) and a new set of molecular structures for which ab-initio charges have been computed (arguments of respectively --sdf-file and --chg-file).

To redirect to a file helpful information about the per-atom QM vs EEM charge comparison, utilize --chg-stats-out-file filename option. As describe in the next section this file is required for the generation of the charge correlation graphs and the quality assessment reports (see here).

For details on the syntax and additional output, see the examples or option list section.

@@ Line 5: / Line 5: @@
 '''Parametrization mode''' : <code>params</code> (calculate EEM parameters)
-'''Validation mode''' : <code>cover</code> (display how many molecules are covered by the given parameters set) and <code>cross</code> (perform EEM parameters quality assay)
+'''Validation mode''' : <code>cover</code> (display how many molecules are covered by the given parameters set) and <code>quality</code> (perform EEM parameters quality assay)
-'''NB''': '''NEEMP''' provides an additional functionality (<code>-m info</code> or <code>--mode info</code>) which displays information about the training set provided as argument to the <code>--sdf-file</code> option. Specifically, number of molecules, the total number of atoms and number of atoms for each type are printed; the latter being defined by default as the chemical element and the maximal bond order. To override this behaviour in all '''NEEMP''''s modes and group the atoms only by chemical element, use the option <code>--atom-types-by Element</code> (for details on the syntax, see the [[NEEMP:Examples#Example 1 - Training set information | examples]] or [[NEEMP:Options | option list]] paragraph).
+'''''Additional functionality''''': '''NEEMP''' enables the user to display information about the provided training set, upon calling the keywords <code>-m info</code> or <code>--mode info</code>. Specifically, number of molecules, the total number of atoms and number of atoms for each atomic type are printed; the latter being defined by default as the chemical element and the maximal bond order. For details on the usage syntax, see the [[NEEMP:Examples#Example 1 - Training set information | examples]] or [[NEEMP:Options | option list]] paragraph.
 =Calculation mode=
-[[File:charge_mode.png | thumb | 500px | right | '''''Figure 1:''''' The percentage of covered molecules in the set is extremely low because of the absence of parameters for C1. All the molecules containing this atom types are automatically discarded.]]
+'''''<u>BASIC USAGE:</u>''''' ''./neemp -m charges --sdf-file examples/set01.sdf --par-file examples/ElemBond.par --chg-out-file eem_charges''
 To compute EEM charges with '''NEEMP''', it is necessary to provide the SDF file with molecules for which the charges will be computed (option <code>--sdf-file</code>) and the PAR file with EEM parameters (option <code>--par-file</code>). EEM charges will be written into the file specified by <code>--chg-out-file</code> option.
-The user should keep in mind that, because of the EEM method itself, the EEM charges cannot be computed for elements or element types for which the parameters are missing. This situation is well depicted by '''''Figure 1''''', where a modified version of the parameters set ''examples/ElemBond.par'' lacking the parameters for '''C1''' (carbon presenting only single bonds), has been employed to compute the EEM charges for the set ''examples/set01.sdf'' (for details on the syntax, see the [[NEEMP:Examples#Example 3 - Calculation mode | examples]] or [[NEEMP:Options | option list]] section).
+The user should keep in mind that, because of the EEM method itself, the EEM charges cannot be computed for atoms or atomic types for which the parameters are missing. This situation is well depicted by '''''Figure 1'''''. On the left side the parameter set contains values for all the atomic types in ''set01.sdf'', on the other hand in the right side image it is evident how the same parameter set is lacking values for several atomic types present in ''set02.sdf'', leading to the discard of many molecules (for details on the syntax, see the [[NEEMP:Examples#Example 2 - Calculation mode | examples]] or [[NEEMP:Options | option list]] section).
 <br style="clear:both" />
+{| class="wikitable" border="1" style="margin-left: auto; margin-right: auto;" width="650px"
+ |-
+ | [[File:chg_new1.png| 450px]]
+ | [[File:chg_new2.png| 450px]]
+ |-
+ |colspan=2 |'''''Figure 1:''''' The lack of parameter values in the parameter set file ''ElemBond.par'' for the atomic types highlighted in the right side image causes the discarding of several molecules from ''set02.sdf'', emphasized by the drop in the covered molecules percentage.
+ |}
 =Parametrization mode=
@@ Line 29: / Line 37: @@
 ===Linear Regression (LR)===
-'''LR''' (<code>--params-method</code> <code>lr-full</code> or <code>-p</code> <code>lr-full</code>) is the default parametrization approach in '''NEEMP''', since its application in the parametrization process is well documented and described in literature (<span style="color:#FF0000"> maybe add a reference </span>). The main extension in respect to previous version is the ability for the user to define by which metrics the best performing parameter set will be selected ('''''see figure'''''):
+'''''<u>BASIC USAGE:</u>''''' ''./neemp -m params -p lr-full --sdf-file examples/set01.sdf --chg-file examples/set01.chg --par-out-file new_parameters.par''
+'''LR''' (<code>--params-method</code> <code>lr-full</code> or <code>-p</code> <code>lr-full</code>) is the default parametrization approach in '''NEEMP''', since its application in the parametrization process is well documented and described in literature. The main extension in respect to previous version is the ability for the user to define by which metrics the best performing parameter set will be selected ('''''see figure'''''):
 :<code>-s</code> <code>R2</code> or <code>--sort-by</code> <code>R2</code> to select the parameter set with the highest squared Pearson coefficient (R<sup>2</sup>)
-:<code>-s</code> <code>RMSD</code> or <code>--sort-by</code> <code>RMSD</code> to select the parameter set with the lowest atom type root mean square difference (avg(RMSD))
+:<code>-s</code> <code>RMSD</code> or <code>--sort-by</code> <code>RMSD</code> to select the parameter set with the lowest atomic type root mean square difference (avg(RMSDa))
-{| class="wikitable" border="1" style="margin: 1em 1em 1em 1em;" width="650px"
+{| class="wikitable" border="1" style="margin-left: auto; margin-right: auto;" width="650px"
   |-
-  | [[File:LR_full_R2.png| 620px]]
+  | [[File:LR_full_R2.png| 600px]]
-  | [[File:LR_full_rmsd.png| 620px]]
+  | [[File:LR_full_RMSD.png| 600px]]
   |-
   |colspan=2 |'''''Figure 2:''''' Detailed view of the '''LR''' parametrization settings for two distinct '''NEEMP''' executions differing in the best-performance selection metrics (''R<sup>2</sup>'' in left side image and ''RMSD'' in the right side image).
   |}
-Additionally, the quality of the parameter set may benefit from the usage of a discarding procedure (<code>-d</code> <code>simple</code> or <code>--discard</code> <code>simple</code>). In fact it has been proved that using a subset of the original training set often leads to EEM charges that better agree with the reference QM charges (<span style="color:#FF0000"> maybe add a reference also if any</span>). For useful examples on the '''LR''' parametrization approach and the discarding procedure see respectively [[NEEMP:Examples#Example 5 - Parametrization mode | 5]]/[[NEEMP:Examples#Example 6 - Parametrization mode k search |6]] and [[NEEMP:Examples#Example 7 - Parametrization mode simple discard|7]]. In particular the latter introduces the <code>--limit-iters</code> and <code>--limit-time</code> options, which modulate the behaviour of the discarding procedure, setting respectively an upper bound in the number of iterations and/or in the execution time.
+Additionally, the quality of the parameter set may benefit from the usage of a discarding procedure (<code>-d</code> <code>simple</code> or <code>--discard</code> <code>simple</code>). In fact it has been proved that using a subset of the original training set often leads to EEM charges that better agree with the reference QM charges. For useful examples on the '''LR''' parametrization approach and the discarding procedure see respectively examples [[NEEMP:Examples#Example 3 - Parametrization mode | 3]]/[[NEEMP:Examples#Example 4 - Parametrization mode k search |4]] and [[NEEMP:Examples#Example 5 - Parametrization mode simple discard|5]]. In particular the latter introduces the <code>--limit-iters</code> and <code>--limit-time</code> options, which modulate the behaviour of the discarding procedure, setting respectively an upper bound in the number of iterations and/or in the execution time.
-===Guided Minimization (GM)===
+===Differential Evolution + Minimization (DE-MIN)===
-In parallel we developed an advanced EEM parametrization approach specifically tailored to cope with heterogeneous data, which in the past presented quite a challenge in the EEM parametrization framework. It consists in the combination of a global minimization method (''simple guidance'') with a local minimization technique (''NEWUOA'', <span style="color:#FF0000"> maybe add a reference </span>), hence the name '''Guided Minimization'''. To call this approach in '''NEEMP''' use the option <code>--params-method</code> <code>gm</code> or <code>-p</code> <code>gm</code>.
+'''''<u>BASIC USAGE:</u>''''' ''./neemp -m params -p de --sdf-file examples/set01.sdf --chg-file examples/charges.chg --par-out-file new_parameter.par''
+In parallel we developed an advanced EEM parametrization approach specifically tailored to cope with heterogeneous data, which in the past presented quite a challenge in the EEM parametrization framework. It consists in the combination of ''differential evolution'' method with a local minimization technique (''NEWUOA''), hence the acronym '''DE-MIN'''. To call this approach in '''NEEMP''' use the option <code>--params-method</code> <code>de</code> or <code>-p</code> <code>de</code>.
 <br style="clear:both" />
-[[File:gm1.png | thumb | 700px | center | '''''Figure 3:''''' Abridged '''GM''' approach output for structures file ''examples/set01.sdf'' and charge file ''examples/charges.chg''. In the upper block are displayed the specifics for the '''gm''' parametrization procedure. Note as the bottom block presents the same structure as for other parametrization approaches and options.]]
+[[File:gm1.png | thumb | 700px | center | '''''Figure 3:''''' Abridged '''DE-MIN''' approach output for structural file ''examples/set01.sdf'' and charge file ''examples/set01.chg''. In the upper block are displayed the specifics for the '''de-min''' parametrization procedure. Note as the bottom block presents the same structure as for other parametrization approaches and options (compare [[NEEMP:Examples#Example 5 - Parametrization mode simple discard | here]]).]]
-The behaviour of the '''GM''' approach can be modulated by several options, but to make life easier we provide the user with reasonable default values that have been tuned to output optimal parameter set (for a complete view of the options refer to the [[NEEMP:Options | option list]] and the [[NEEMP:Examples#Example 8 - Parametrization mode GM | examples]] section).
+The behaviour of the '''DE-MIN''' approach can be modulated by several options, but to make life easier we provide the user with reasonable default values that have been tuned to output optimal parameter set (for a complete view of the options refer to the [[NEEMP:Options | option list]] and the [[NEEMP:Examples#Example 6 - Parametrization mode DE-MIN | examples]] section).
-'''NB''': the default quality evaluating metrics for '''GM''' approach is the ''average RMSD'', but this behaviour can be overridden using the <code>--sort-by</code> option as in the '''LR''' method.
+'''NB''': the default quality evaluating metrics for '''DE-MIN''' approach is ''(avg(RMSDa))''.
 =Validation mode=
@@ Line 61: / Line 73: @@
 ===Coverage validation===
-[[File:cover_mode.png | thumb | 500px | right | '''''Figure 2:''''' Percentage of molecules in set ''examples/set02.sdf'' covered by the displayed parameters set ''examples/Element.par'']]
-To see how many molecules from the set are covered by the provided parameters, simply pass '''NEEMP''' those two files (options <code>--sdf-file</code> and <code>--par-file</code>). '''''Figure 2''''' shows a close-up from the <code>cover</code> mode output for ''set02.sdf'' and the parameters set ''Element.par'', both present in the ''examples'' directory (for details on the syntax and the complete output, see the [[NEEMP:Examples#Example 2 - Cover mode | examples]] or [[NEEMP:Options | option list]] paragraph)
+'''''<u>BASIC USAGE:</u>''''' ''./neemp -m cover -sdf-file examples/set01.sdf --par-file examples/Element.par''
+[[File:cover_mode.png | thumb | 500px | right | '''''Figure 4:''''' Percentage of molecules in set ''examples/set01.sdf'' covered by the displayed parameters set ''examples/Element.par'']]
+To see how many molecules from the set are covered by the provided parameters, simply pass '''NEEMP''' those two files (options <code>--sdf-file</code> and <code>--par-file</code>). '''''Figure 4''''' shows a close-up from the <code>cover</code> mode output for ''set01.sdf'' and the parameters set ''Element.par'', both present in the ''examples'' directory (for details on the syntax and the complete output, see the [[NEEMP:Examples#Example 7 - Coverage validation | examples]] or [[NEEMP:Options | option list]] paragraph)
 <br style="clear:both" />
 ===Quality validation===
-To assess the quality of the EEM parameters on a different set than one they were based on, '''NEEMP''''s <code>cross</code> mode can be used.
+'''''<u>BASIC USAGE:</u>''''' ''./neemp -m quality --sdf-file examples/set01.sdf --chg-file examples/set01.chg --par-file examples/Element.par
-Required input files are: ''EEM parameters'' (passed as argument of option <code>--par-file</code>) and a ''new set of molecular structures'' (argument of <code>--sdf-file</code>) for which ''ab-initio charges'' have been computed (argument of <code>--chg-file</code>).
+To assess the quality of the EEM parameters on a different set than one they were based on, '''NEEMP''''s <code>quality</code> mode can be called.
-For details on the syntax and additional output, see the [[NEEMP:Examples#Example 4 - Cross mode | examples]] or [[NEEMP:Options | option list]] section.
+Required input files are: ''EEM parameters'' (passed as argument of option <code>--par-file</code>) and a ''new set of molecular structures'' for which ''ab-initio charges'' have been computed (arguments of respectively <code>--sdf-file</code> and <code>--chg-file</code>).
-<br style="clear:both" />
+To redirect to a file helpful information about the per-atom ''QM vs EEM'' charge comparison, utilize <code>--chg-stats-out-file filename</code> option. As describe in the next section this file is required for the generation of the charge correlation graphs and the quality assessment reports (see [[NEEMP:Reports | here]]).
-[[File:cross_mode.png | thumb | 700px | center | '''''Figure 4:''''' ''Cross mode'' output for structures file ''examples/set02.sdf'', charge file ''examples/charges.chg'' and parameters set ''examples/Element.par'' . Note the block at the bottom where the statistics descriptors for the whole set and for each chemical element are displayed.]]
+For details on the syntax and additional output, see the [[NEEMP:Examples#Example 8 - Quality validation | examples]] or [[NEEMP:Options | option list]] section.