Final preparation for ESAPI 2.2.0.0 release (ESAPI#501)

* Change release version to 2.2.0.0 for official release.

* Update / correct ESAPI release steps.

* Fix ironic spelling typo.

* Fix ironic spelling typo.

* Changes to suppress most of the noise, but also fixes to all for .XML and .PROPERTIES suffixes and to fix and return null when property is null or empty string.

* Changes to suppress most of the noise and to actually handle the exceptions that we should have been all along (e.g., IOExceptions).

* Comment out the crude benchmark related assertion that sometimes was failing because of JIT-related issues. Needs to be eventually replaced by JMH.

* Close ESAPI#499 by resetting 'parent' when Windows is detected to root of drive where Windows is installed.

* Close issue ESAPI#488. These are slight enhancements to PR ESAPI#489 by @JoergAdler that I rejected because Eclipse did something to cause every line to differ. But shout out to Jörg Adler for originally finding this issue and patching it.

* Close issue ESAPI#488. These are slight enhancements to PR ESAPI#489 by @JoergAdler that I rejected because Eclipse did something to cause every line to differ. But shout out to Jörg Adler for originally finding this issue and patching it.
I also added an additional test or 2, so don't blame @JoergAdler if I messed that up.

* Close issue ESAPI#488. These are slight enhancements to PR ESAPI#489 by @JoergAdler that I rejected because Eclipse did something to cause every line to differ. But shout out to Jörg Adler for originally finding this issue and patching it.
I tremendously stripped down the contents of this file because really all it needed was a single property referencing the validation.properties file.

* Add 3 additional issues that were closed and fixes to some minor formatting.

* Final updates for the ESAPI 2.2.0.0 official release.

* Figure I probably ought to add my name instead of assuming people knew it and my email address.

* Changed schema from allowing unbounded number of properties to allow only 10000. That really should NOT be a problem.
However, it should help silence some of the SAST engines.

* Additional changes to fix GitHub Issue ESAPI#500

Author: kwwall

CONTRIBUTING-TO-ESAPI.txt

@@ -17,7 +17,7 @@ Finding Something Interesting to Work on:
     what it is. Then if you want to work on a particular issue, we can assign
     it to you so someone else won't take it.
 
-    If you have questions, email me or Matt Seil ([email protected]).
+    If you have questions, email Kevin Wall ([email protected]) or Matt Seil ([email protected]).
 
 Overview:
     We are following the branching model described in

documentation/ESAPI-release-steps.odt

@@ -1,5 +1,5 @@
 Release notes for ESAPI 2.2.0.0
-    Release date: 2019-MMM-DD
+    Release date: 2019-June-23
     Project leaders:
         -Kevin W. Wall <[email protected]>
         -Matt Seil <[email protected]>
@@ -26,14 +26,14 @@ It was mainly because of these first two bullet items above that we bumped the r
                 Basic ESAPI facts
 
 ESAPI 2.1.0.1 release:
-    177 source files
-    1547 Junit tests
+    177 Java source files
+    1547 Junit tests in 88 Java source files
 
 ESAPI 2.2.0.0 release:
-    194 source files
-    4145 JUnit tests!!!!!
+    194 Java source files
+    4150 JUnit tests in 118 Java source files
 
-That's 2598 NEW tests since the 2.1.0.1 release!!!
+That's 2603 NEW tests since the 2.1.0.1 release!!!
 
                 GitHub Issues fixed in this release
             [i.e., since 2.1.0.1 release on 2016-Feb-05]
@@ -149,22 +149,26 @@ Issue #			GitHub Issue Title
 462		Allow configurable init parameter in ESAPIFilter for unauthorized requests
 463		Create release notes for next ESAPI release
 465		Update both ESAPI.properties files to show comment for ESAPI logger support for SLF4J
-471     Bump ESAPI release # to 2.2.0.0
-476     DefaultValidator.getValidInput implementation ignores 'canonicalize' method parameter
-478     Remove obsolete references to Google Code in pom.xml and any other release prep
+471		Bump ESAPI release # to 2.2.0.0
+476		DefaultValidator.getValidInput implementation ignores 'canonicalize' method parameter
+478		Remove obsolete references to Google Code in pom.xml and any other release prep
 482		ESAPI 2.2.0.0 release date?
-483     More miscellaneous prep work for ESAPI 2.2.0.0 release
-485     Update Maven dependency check plugin to 5.0.0-M2
+483		More miscellaneous prep work for ESAPI 2.2.0.0 release
+485		Update Maven dependency check plugin to 5.0.0-M2
+488		Missed a legal input case in DefaultSecurityConfiguration.java
 492		Release candidates on maven central
 493		wrong regex validation
+499		ValidatorTest.isValidDirectoryPath() has tests that fail under Windows if ESAPI tests run from different drive where Windows installed
+500		Suppress noise from ESAPI searching for properties and stop ignoring important IOExceptions
+
 
 -----------------------------------------------------------------------------
 
         Changes requiring special attention
 
 * Various deprecated methods were _actually_ deleted! This could break existing application code.
 
-    442		Remove deprecated fields in Encoder interface
+    Issue 442		Remove deprecated fields in Encoder interface
 
         Specifically, if you are using any of these previously deprecated fields from the Encoder interface, you need to update your application code to refer insteat to the constances from org.owasp.esapi.EncoderConstants:
 
@@ -180,19 +184,19 @@ Issue #			GitHub Issue Title
                 public final static char[] CHAR_PASSWORD_SPECIALS = EncoderConstants.CHAR_PASSWORD_SPECIALS;
                 public final static char[] CHAR_PASSWORD_LETTERS = EncoderConstants.CHAR_PASSWORD_LETTERS;
 
-    444		Delete deprecated method Base64.decodeToObject() and related methods
+    Issue 444		Delete deprecated method Base64.decodeToObject() and related methods
 
         Specifically, the following methods were removed from the org.owasp.esapi.codecs.Base64 class. If you will using any of these methods, you likely already had vulnerabilities in your application code. If any of these methods were being used, you will need to rewrite your application code:
 
                 public static String encodeObject( java.io.Serializable serializableObject )
                 public static String encodeObject( java.io.Serializable serializableObject, int options )
                 public static Object decodeToObject( String encodedObject )
 
-    483     More miscellaneous prep work for ESAPI 2.2.0.0 release
+    Issue 483     More miscellaneous prep work for ESAPI 2.2.0.0 release
         Specifically, CipherText.getSerialVersionUID() and DefaultSecurityConfiguration.MAX_FILE_NAME_LENGTH have actually been deleted from the ESAPI code base. For the former, use CipherText.cipherTextVersion() instead. For the latter, there is no replacement. (This wasn't being used, but it was set to 1000 in case you're wondering.)
 
 * Various properties in ESAPI.properties were changed in a way that might affect your application:
-    439		Tighten ESAPI defaults to disallow dubious file suffixes
+    Issue 439		Tighten ESAPI defaults to disallow dubious file suffixes
 
         Specifically, the property HttpUtilities.ApprovedUploadExtensions changed from
             HttpUtilities.ApprovedUploadExtensions=.zip,.pdf,.doc,.docx,.ppt,.pptx,.tar,.gz,.tgz,.rar,.war,.jar,.ear,.xls,.rtf,.properties,.java,.class,.txt,.xml,.jsp,.jsf,.exe,.dll
@@ -220,6 +224,18 @@ Issue #			GitHub Issue Title
                                                 to:
                                                     Validator.HTTPURI=^/([a-zA-Z0-9.\\-_]*/?)*$
 
+* Other changes:
+    Issue 500		Suppress noise from ESAPI searching for properties and stop ignoring important IOExceptions
+
+        Fixing this required changes to the CTORs of the following classes:
+
+                org.owasp.esapi.configuration.EsapiPropertyManager
+                org.owasp.esapi.configuration.AbstractPrioritizedPropertyLoader
+                org.owasp.esapi.configuration.EsapiPropertyLoaderFactory
+                org.owasp.esapi.configuration.StandardEsapiPropertyLoader
+                org.owasp.esapi.configuration.XmlEsapiPropertyLoader
+
+        These CTORs now explicitly throw IOException if the specified ESAPI property file is not found or not readable. Note that this should not affect most people as most use DefaultSecurityCOnfigurator and it still only throws ConfigurationException. (IOExceptions from these other classes are caught and rethrow as ConfigurationException.) Use of these classes directly should be very rare.
 
 -----------------------------------------------------------------------------
 

documentation/esapi4java-core-2.2.0.0-release-notes.txt

@@ -3,7 +3,7 @@
     <modelVersion>4.0.0</modelVersion>
     <groupId>org.owasp.esapi</groupId>
     <artifactId>esapi</artifactId>
-    <version>2.2.0.0-RC3</version>
+    <version>2.2.0.0</version>
     <packaging>jar</packaging>
 
     <distributionManagement>

pom.xml

@@ -2,6 +2,8 @@
 
 
 import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
 import java.util.Properties;
 
 /**
@@ -32,7 +34,7 @@ public abstract class AbstractPrioritizedPropertyLoader implements EsapiProperty
 
     private final int priority;
 
-    public AbstractPrioritizedPropertyLoader(String filename, int priority) {
+    public AbstractPrioritizedPropertyLoader(String filename, int priority) throws IOException {
         this.priority = priority;
         this.filename = filename;
         initProperties();
@@ -64,13 +66,17 @@ public String name() {
     /**
      * Initializes properties object and fills it with data from configuration file.
      */
-    private void initProperties() {
+    private void initProperties() throws IOException {
         properties = new Properties();
         File file = new File(filename);
         if (file.exists() && file.isFile()) {
-            loadPropertiesFromFile(file);
+            if ( file.canRead() ) {
+                loadPropertiesFromFile(file);
+            } else {
+                throw new IOException("Can't read specificied configuration file: " + filename);
+            }
         } else {
-            logSpecial("Configuration file " + filename + " does not exist");
+            throw new FileNotFoundException("Specified configuration file " + filename + " does not exist or not regular file");
         }
     }
 

src/main/java/org/owasp/esapi/configuration/AbstractPrioritizedPropertyLoader.java

@@ -3,7 +3,7 @@
 import org.owasp.esapi.configuration.consts.EsapiConfiguration;
 import org.owasp.esapi.errors.ConfigurationException;
 
-import java.io.FileNotFoundException;
+import java.io.IOException;
 
 import static org.owasp.esapi.configuration.consts.EsapiConfigurationType.PROPERTIES;
 import static org.owasp.esapi.configuration.consts.EsapiConfigurationType.XML;
@@ -17,17 +17,38 @@
 public class EsapiPropertyLoaderFactory {
 
     public static AbstractPrioritizedPropertyLoader createPropertyLoader(EsapiConfiguration cfg)
-            throws ConfigurationException, FileNotFoundException {
+            throws ConfigurationException, IOException {
         String cfgPath = System.getProperty(cfg.getConfigName());
-        if (cfgPath == null) {
-            throw new ConfigurationException("System property [" + cfg.getConfigName() + "] is not set");
+        if ( cfgPath == null || cfgPath.equals("") ) {
+            // TODO / FIXME:
+            // This case was previously a warning, but it should NOT have been
+            // since these system properties are optional. Most people just use
+            // the traditional ESAPI.properties file and not these prioritized ones.
+            // A warning gets logged in EsapiPropertyManager if logSpecial output
+            // has not been discarded.
+            //
+            // Note also there were a LOT of cases in our JUnit tests where the
+            // file extension was empty, causing the ConfigurationException to
+            // be thrown with the error message:
+            //      "Configuration storage type [] is not supported"
+            // I don't think that was intentional, but because prior to the
+            // changes for this commit, these were all ConfigurationExceptions
+            // and they all were just being caught and not re-thrown by
+            // DefaultSecurityConfigurator. I think that is an error, probably
+            // in the tests, but I don't have timed to chase it down right now
+            // because of the pending 2.2.0.0 release.
+            //
+            // Also, I made several fixes in DefaultSecurityConfiguration
+            // related to this clean-up where IOExceptions were being silently
+            // caught when they should not have been.  -kwwall
+            return null;
         }
         String fileExtension = cfgPath.substring(cfgPath.lastIndexOf('.') + 1);
 
-        if (XML.getTypeName().equals(fileExtension)) {
+        if (XML.getTypeName().equalsIgnoreCase(fileExtension)) {
             return new XmlEsapiPropertyLoader(cfgPath, cfg.getPriority());
         }
-        if (PROPERTIES.getTypeName().equals(fileExtension)) {
+        if (PROPERTIES.getTypeName().equalsIgnoreCase(fileExtension)) {
             return new StandardEsapiPropertyLoader(cfgPath, cfg.getPriority());
         } else {
             throw new ConfigurationException("Configuration storage type [" + fileExtension + "] is not " +

src/main/java/org/owasp/esapi/configuration/EsapiPropertyLoaderFactory.java

@@ -4,9 +4,15 @@
 import org.owasp.esapi.errors.ConfigurationException;
 
 import java.util.TreeSet;
+import java.io.IOException;
 
 import static org.owasp.esapi.configuration.EsapiPropertyLoaderFactory.createPropertyLoader;
 
+// Have dependency like this on a reference implmentation is majorly ugly, I know, but I
+// don't want to refactor code and delay the 2.2.0.0 release further and this class
+// is WAY too noisy. - kwwall
+import static org.owasp.esapi.reference.DefaultSecurityConfiguration.logToStdout;
+
 /**
  * Manager used for loading security configuration properties. Does all the logic to obtain the correct property from
  * correct source. Uses following system properties to find configuration files:
@@ -21,7 +27,7 @@ public class EsapiPropertyManager implements EsapiPropertyLoader {
 
     protected TreeSet<AbstractPrioritizedPropertyLoader> loaders;
 
-    public EsapiPropertyManager() {
+    public EsapiPropertyManager() throws IOException {
         initLoaders();
     }
 
@@ -34,7 +40,7 @@ public int getIntProp(String propertyName) throws ConfigurationException {
             try {
                 return loader.getIntProp(propertyName);
             } catch (ConfigurationException e) {
-                System.err.println("Property not found in " + loader.name());
+                logToStdout("Integer property '" + propertyName + "' not found in " + loader.name(), e);
             }
         }
         throw new ConfigurationException("Could not find property " + propertyName + " in configuration");
@@ -49,7 +55,7 @@ public byte[] getByteArrayProp(String propertyName) throws ConfigurationExceptio
             try {
                 return loader.getByteArrayProp(propertyName);
             } catch (ConfigurationException e) {
-                System.err.println("Property not found in " + loader.name());
+                logToStdout("Byte array property '" + propertyName + "' not found in " + loader.name(), e);
             }
         }
         throw new ConfigurationException("Could not find property " + propertyName + " in configuration");
@@ -64,7 +70,7 @@ public Boolean getBooleanProp(String propertyName) throws ConfigurationException
             try {
                 return loader.getBooleanProp(propertyName);
             } catch (ConfigurationException e) {
-                System.err.println("Property not found in " + loader.name());
+                logToStdout("Boolean property '" + propertyName + "' not found in " + loader.name(), e);
             }
         }
         throw new ConfigurationException("Could not find property " + propertyName + " in configuration");
@@ -79,25 +85,37 @@ public String getStringProp(String propertyName) throws ConfigurationException {
             try {
                 return loader.getStringProp(propertyName);
             } catch (ConfigurationException e) {
-                System.err.println("Property : " + propertyName + " not found in " + loader.name());
+                logToStdout("Property '" + propertyName + "' not found in " + loader.name(), e);
             }
         }
         throw new ConfigurationException("Could not find property " + propertyName + " in configuration");
     }
 
-    private void initLoaders() {
+    private void initLoaders() throws IOException {
         loaders = new TreeSet<AbstractPrioritizedPropertyLoader>();
         try {
-            loaders.add(createPropertyLoader(EsapiConfiguration.OPSTEAM_ESAPI_CFG));
-        } catch (Exception e) {
-            System.err.println(e.getMessage());
+            AbstractPrioritizedPropertyLoader appl = createPropertyLoader(EsapiConfiguration.OPSTEAM_ESAPI_CFG);
+            if ( appl == null ) {
+                String msg = "WARNING: System property [" + EsapiConfiguration.OPSTEAM_ESAPI_CFG.getConfigName() + "] is not set";
+                logToStdout(msg, null);
+            } else {
+                loaders.add( appl );
+            }
+        } catch (IOException e) {
+            logToStdout("WARNING: Exception encountered while setting up ESAPI configuration manager for OPS team", e);
+            throw e;
         }
         try {
-            loaders.add(createPropertyLoader(EsapiConfiguration.DEVTEAM_ESAPI_CFG));
-        } catch (Exception e) {
-            System.err.println(e.getMessage());
+            AbstractPrioritizedPropertyLoader appl = createPropertyLoader(EsapiConfiguration.DEVTEAM_ESAPI_CFG);
+            if ( appl == null ) {
+                String msg = "WARNING: System property [" + EsapiConfiguration.DEVTEAM_ESAPI_CFG.getConfigName() + "] is not set";
+                logToStdout(msg, null);
+            } else {
+                loaders.add( appl );
+            }
+        } catch (IOException e) {
+            logToStdout("WARNING: Exception encountered while setting up ESAPI configuration manager for DEV team", e);
+            throw e;
         }
     }
-
-
 }

src/main/java/org/owasp/esapi/configuration/EsapiPropertyManager.java

@@ -12,7 +12,7 @@
  */
 public class StandardEsapiPropertyLoader extends AbstractPrioritizedPropertyLoader {
 
-    public StandardEsapiPropertyLoader(String filename, int priority) {
+    public StandardEsapiPropertyLoader(String filename, int priority) throws IOException {
         super(filename, priority);
     }
 

src/main/java/org/owasp/esapi/configuration/StandardEsapiPropertyLoader.java

@@ -26,7 +26,7 @@
  */
 public class XmlEsapiPropertyLoader extends AbstractPrioritizedPropertyLoader {
 
-    public XmlEsapiPropertyLoader(String filename, int priority) {
+    public XmlEsapiPropertyLoader(String filename, int priority) throws IOException {
         super(filename, priority);
     }
 
@@ -80,7 +80,7 @@ public Boolean getBooleanProp(String propertyName) throws ConfigurationException
             return false;
         } else {
             throw new ConfigurationException("Incorrect type of : " + propertyName + ". Value " + property +
-                    "cannot be converted to boolean");
+                    "cannot be converted to boolean; legal values are: true, false, yes, no");
         }
     }
 
@@ -120,6 +120,7 @@ protected void loadPropertiesFromFile(File file) throws ConfigurationException {
                 }
             }
         } catch (Exception e) {
+            logSpecial("XML config file " + filename + " has invalid schema", e);
             throw new ConfigurationException("Configuration file : " + filename + " has invalid schema." + e.getMessage(), e);
         }
     }

src/main/java/org/owasp/esapi/configuration/XmlEsapiPropertyLoader.java

@@ -235,9 +235,9 @@ public static SecurityConfiguration getInstance() {
      */
     DefaultSecurityConfiguration(String resourceFile) {
     	this.resourceFile = resourceFile;
-        this.esapiPropertyManager = new EsapiPropertyManager();
     	// load security configuration
     	try {
+            this.esapiPropertyManager = new EsapiPropertyManager();
         	loadConfiguration();
         	this.setCipherXProperties();
         } catch( IOException e ) {
@@ -646,7 +646,13 @@ private Properties loadConfigurationFromClasspath(String fileName) throws Illega
 				try {
 					// try root
 					String currentClasspathSearchLocation = "/ (root)";
-					in = loaders[i].getResourceAsStream(DefaultSearchPath.ROOT.toString());
+                        // Note: do NOT add '/' anywhere here even though root value is empty string!
+                        // Note that since DefaultSearchPath.ROOT.value() is now "" (the empty string),
+                        // then this is logically equivalent to what we used to have, which was:
+                        //
+						//      in = loaders[i].getResourceAsStream(fileName);
+                        //
+					in = loaders[i].getResourceAsStream(DefaultSearchPath.ROOT.value() + fileName);
 
 					// try resourceDirectory folder
 					if (in == null) {
@@ -1391,7 +1397,7 @@ public enum DefaultSearchPath {
 
     	RESOURCE_DIRECTORY("resourceDirectory/"),
     	SRC_MAIN_RESOURCES("src/main/resources/"),
-    	ROOT("/"),
+    	ROOT(""),
     	DOT_ESAPI(".esapi/"),
     	ESAPI("esapi/"),
     	RESOURCES("resources/");

src/main/java/org/owasp/esapi/reference/DefaultSecurityConfiguration.java

@@ -2,7 +2,7 @@
     <xs:element name="properties">
         <xs:complexType>
             <xs:sequence>
-                <xs:element name="property" maxOccurs="unbounded" minOccurs="0">
+                <xs:element name="property" maxOccurs="10000" minOccurs="0">
                     <xs:complexType>
                         <xs:simpleContent>
                             <xs:extension base="xs:string">
@@ -14,4 +14,4 @@
             </xs:sequence>
         </xs:complexType>
     </xs:element>
-</xs:schema>
+</xs:schema>