Wednesday, October 22, 2014

Java Extension Mechanism Loads All JARs

The Java Extension Mechanism is described in the Java Tutorial as a "standard, scalable way to make custom APIs available to all applications running on the Java platform." As described in Understanding Extension Class Loading, "the extension framework makes use of the class-loading delegation mechanism" with extension classes loaded after the bootstrap classes in rt.jar (and related JARs) but before the classes loaded from the typical classpath.

The extension directory works a bit like the classpath in that its part of the class loading mechanism and classes available within JARs in the extension directory are made available to Java applications. There are some key, differences, however, and some of these are highlighted next.

Characteristic Classpath Extension Mechanism (Optional Packages)
Scope Typically Application-Specific
Potentially All JREs on Host
All JVMs Running in Specific JRE All Host's JREs
  • Solaris: /usr/jdk/packages/lib/ext
  • Linux: /usr/java/packages/lib/ext
  • Windows: %SystemRoot%\Sun\Java\lib\ext
How Specified .jar Files
  • Explicitly specified by name (including .jar)
  • Wildcard (*) matching all all JAR files with .jar extensions
.class Files
  • Directory containing .class files specified
All JAR files (even if extension other than .jar or no extension at all) in designated directories are loaded.
Class Loading Order After bootstrap and extensions loading. After bootstrap but before classpath.

One of the most significant observations worth some more emphasis is that the extension mechanism will pick up all JAR format files in the extension directory even if the file does not have a .jar extension. The implication of this is that while one can change the name of a JAR located in a classpath directory to have an extension other than .jar so that the wildcard does not pick it up, this technique will not work with the extension directory.

I'm going to use some simple examples in this post to demonstrate some of these differences. The next two code listings are for a very simple HelloWorld class and a main application class called Main that uses the HelloWorld class.

HelloWorld.java
public class HelloWorld
{
   @Override
   public String toString()
   {
      return "Hello, World!";
   }
}
Main.java
import static java.lang.System.out;

public class Main
{
   public static void main(final String[] arguments)
   {
      out.println(new HelloWorld());
   }
}

To demonstrate a primary difference between classpath and the extension mechanism (optional packages), I will archive the compiled HelloWorld.class file into a JAR called HelloWorld.jar and place it in a different directory than the compiled Main.class file.

To demonstrate the use of the traditional classpath, I place the HelloWorld.jar file in a directory called C:\hello and will access that JAR via wildcard (*) for Main to use. This is demonstrated in the next two screen snapshots.

The two previous images demonstrate that the Java Main application can still load the HelloWorld.class file even though I had deleted it from the current directory because the Java launcher was explicitly told (via the -classpath option) to look for it in C:\hello. Using the extensions mechanism (optional packages), it is possible to have the class loaded without it being in the same directory and without explicit classpath specification. This is shown in the next screen snapshot.

The previous screen snapshot demonstrates that the Java launcher doesn't even need the HelloWorld.class in the same directory or specified on its classpath when that class is inside a JAR that is in the extensions (optional packages) directory. This is often cited as a benefit of using the extensions mechanism because all applications using that JRE (or potentially all applications on the host) can see the same classes without need to explicitly specify them on the classpath.

With the traditional classpath approach of instructing an application to load classes from JARs, the JAR file containing the .class file needs to end with the .jar extension. The next screen snapshot demonstrates what happens when the HelloWorld.jar is renamed HelloWorld.backup in the same classpath-referenced directory.

The last image demonstrates that a NoClassDefFoundError is encountered when the JAR file in the classpath-referenced directory does not have a .jar extension. Perhaps a bit surprisingly, the extensions (optional packages) mechanism does not work the same way. Instead, all JAR files in the extensions specified directory are loaded regardless of their extension and regardless of even if they have a file extension. This is demonstrated in the next screen image.

The previous image demonstrates that renaming the JAR file that resides within the extensions directory to not have any file extension whatsoever does not prevent the classloader from loading the classes of that JAR. In other words, the classloading mechanism loads all JAR files in the specified extensions directory based on file type rather than on file name or extension. As the Optional Packages Overview summarizes, "There is nothing special about any particular JAR file itself or the classes it contains that makes it an installed optional package. It is an installed optional package by virtue of its location in jre/lib/ext."

There are some risks and downsides associated with placing too many class definitions in JARs inside the extensions directory. It can be maddening to wonder why NoSuchMethodErrors, for example, are occurring when one can see that a class specified explicitly on the classpath has the method in question. I have previously written about one of the many potential causes of NoSuchMethodError, but forgotten outdated and obsolete class definitions residing inside of JAR files in the extensions directory are another potential cause. This is demonstrated next.

The next two code listings show revised versions of Main.java and HelloWorld.java. In particular, HelloWorld has an all-new method that the new version of Main invokes. In this case, I'm going to leave the newly compiled HelloWorld.class file in the same directory when I run the Main to demonstrate that the old, busted version of HelloWorld.class in the JAR in the extensions directory takes precedence over the new hotness HelloWorld.class in the current directory.

Revised Hello World.java (New Method)
public class HelloWorld
{
   @Override
   public String toString()
   {
      return "Hello, World!";
   }

   public String directedHello(final String name)
   {
      return "Hello, " + name;
   }
}
Revised Main.java
import static java.lang.System.out;

public class Main
{
   public static void main(final String[] arguments)
   {
      final HelloWorld helloWorld = new HelloWorld();
      out.println(helloWorld);
      out.println(helloWorld.directedHello("Dustin"));
   }
}

The last image demonstrates that the now obsolete class definition of HelloWorld in the extensions directory takes precedence over the new class definition of HelloWorld in the same directory. Even when I specify the current directory on the classpath, the old version in the extensions directory takes precedence. This is shown in the next screen snapshot, which also shows that the JAR in the extensions directory that is "hiding" the newer JAR and its class's newer method is still not even named with a .jar extension.

The example just demonstrated is not even the most difficult situation a forgotten JAR in the specified extensions directory (or directories) can cause. In that example, at least I had a NoSuchMethodError to alert me to a problem. A potentially even more difficult situation to debug can exist when the old class definition has the same method signature but has an outdated method implementation. In such cases, there may be no error, exception, or throwable of any kind, but the application logic will simply not work correctly or as expected. The old functionality could exist in the code base for some time before it's even recognized as an issue, especially if unit tests and other testing is lacking.

Use of the extensions directory can make things easier on developers because classes in JAR files residing in the extensions directory (or directories) are available to all applications in the JRE associated with the extensions directory (or with all JREs on the host if the operating system-based host-wide extensions directory is used). However, there are definite risks associated with too liberal use of the directory. It can be easy to forget that outdated class definitions residing in JARs in that directory are preventing classloaders from loading the newer and seemingly obvious versions of the class definitions. When this happens, the very extensions (optional packages) mechanism that made developers' lives easier now make it more difficult.

Elliotte Rusty Harold provides a warning about use of the extensions (optional packages) mechanism, "While this seems convenient, it is also a long-term mistake... Sooner or later (probably sooner), you'll load the wrong version of a class from a place you aren't even thinking about and waste hours debugging." The Java Tutorial also recommends caution (I added the emphasis), "Since this mechanism extends the platform's core API, its use should be judiciously applied. Most commonly it is used for well standardized interfaces such as those defined by the Java Community Process, although it may also be appropriate for site wide interfaces."

Although the extensions (optional packages) mechanism is similar to the classpath mechanism and both are used as part of class loading, the differences between the two are important to note. In particular, it is important to remember that all JAR files (even if they don't have .jar file extensions) that reside in the directory referenced as an extension directory will be loaded. Renaming these JARs and even changing their file extension will not be sufficient to have the classloading ignore them. With classpath, on the other hand, renaming the JAR is sufficient to prevent loading when the classpath specifies individual JAR files explicitly and changing the file extension is typically sufficient to prevent loading even when the classpath uses the wildcard (*) to specify all JARs in a directory.

There are situations when the extensions (optional packages) mechanism is the appropriate choice, but these seem fairly rare. It is also important to keep in mind the extensions (optional packages) mechanism in mind when dealing with unexplained NoSuchMethodErrors so that one can check it out to see if the offender lives in that directory or directories.

Wednesday, October 15, 2014

Java Minor Releases Scheme Tweaked Again

In 2013, Oracle announced the Java SE - Change in Version Numbering Scheme. The announcement stated that Limited Update releases (those "that include new functionality and non-security fixes") and Critical Patch Updates (CPUs) [those "that only include fixes for security vulnerabilities"] would be released with specific version number schemes. In particular, Limited Use Releases would have version numbers with multiples of 20 while Critical Patch Updates would have version numbers that are multiples of 5 and come after the latest Limited Use Release version number. The purpose of this scheme change was to allow room for versions with numbers between these, which allows Oracle "to insert releases – for example security alerts or support releases, should that become necessary - without having to renumber later releases."

Yesterday's announcement ("Java CPU and PSU Releases Explained") states, "Starting with the release of Java SE 7 Update 71 (Java SE 7u71) in October 2014, Oracle will release a Critical Patch Update (CPU) at the same time as a corresponding Patch Set Update (PSU) for Java SE 7." This announcement explains the difference between a CPU and a PSU:

Critical Patch Update CPU "Fixes to security vulnerabilities and critical bug fixes." Minimum recommended for everyone.
Patch Set Update PSU "All fixes in the corresponding CPU" and "additional non-critical fixes." Recommended only for those needing bugs fixed by PSU additional fixes.

Yesterday's announcement states that PSU releases (which are really CPU+ releases) will be released along with their corresponding CPU releases. Because the additional fixes that a PSU release contains beyond what's in the CPU release are expected to be part of the next CPU release, developers are encouraged to experiment with PSU releases to ensure that coming CPU features work well for them.

Tuesday, October 14, 2014

NetBeans 8.0's Five New Performance Hints

NetBeans 8.0 introduces several new Java hints. Although there are a large number of these new hints related to Java Persistence API, I focus on five new hints in the Performance category.

The five new "Performance Hints" introduced with NetBeans 8.0 are:

  • Boxing of already boxed value
  • Redundant String.toString()
  • Replace StringBuffer/StringBuilder by String
  • Unnecessary temporary during conversion from String
  • Unnecessary temporary during conversion to String

Each of these five performance-related Java hints are illustrated in this post with screen snapshots taken from NetBeans 8.0 with code that demonstrates these hints. There are two screen snapshots for each hint, one each showing the text displayed when the cursor hovers over the line of code marked with yellow underlining and one each showing the suggested course of action to be applied to address that hint (shown when clicking on the yellow light bulb to the left of the flagged line). Some of the captured screen snapshots include examples of code that avoid the hint.

Boxing of Already Boxed Value

Redundant String.toString()

Replace StringBuffer/StringBuilder by String

Unnecessary Temporary During Conversion from String

Unnecessary Temporary During Conversion to String

Unless I've done something incorrectly, there appears to be a minor bug with this hint in that it reports "Unnecessary temporary when converting from String" when, in this case, it should really be "Unnecessary temporary when converting to String". This is not a big deal as the condition is flagged and the action to fix it seems appropriate.

Conclusion

The five performance-related hints introduced by NetBeans 8.0 and illustrated here can help Java developers avoid unnecessary object instantiations and other unnecessary runtime cost. Although the benefit of this optimization as shown in my simple examples is almost negligible, it could lead to much greater savings when used in code with loops that perform these same unnecessary instantiations thousands of times. Even without consideration of the performance benefit, these hints help to remind Java developers and teach developers new to Java about the most appropriate mechanisms for acquiring instances and primitive values.

Thursday, October 9, 2014

JavaOne 2014 Observations by Proxy

I wasn't able to attend JavaOne this year, but have been happy to see some online resources covering what happened at JavaOne 2014. In this post, I summarize some of the observations made at JavaOne 2014 and provide links to references providing these observations or providing more background details on those observations. The listed observations are in no particular order and many of them come from the JavaOne 2014 Keynote Addresses.

Rapid Adoption of Java 8

The Oracle Press Release associated with JavaOne 2014 states, "Since its launch in March 2014, Java SE 8 has achieved record adoption rates. Overall, adoption is up more than 20 percent compared to the same post-launch time period for Java SE 7."1 George Saab highlighted this rapid adoption with the observation in the Strategy Keynote that there are already eight Java 8 publications available in eight different languages.4.

Intel and Java

Intel was an "Innovation Sponsor" of JavaOne 2014 and, because of that, had a portion of the JavaOne 2014 "Java Partner Community Keynote" address. In this address, it was stated that Java runs 32 times faster on Intel since 20072. It was also announced that Intel has joined the OpenJDK as a Contributing Member.2,5

New OpenJDK Partners

The Oracle Press Release for JavaOne 2014 mentions other recently added new partners to the OpenJDK team: FreeBSD Foundation, GE Digital Energy, and Microsoft Open Technologies, Inc.1

JDK Modularity / Project Jigsaw

It was confirmed in the "Java Partner Community Keynote" address that Oracle does intend to deliver modularity with JDK 9.2,6 Modularization was scheduled for previous versions of Java, but has been kicked down the road from JDK 8 and from JDK 7 before that. The Oracle Press Release announcing JavaOne 2014 states, "Oracle has begun work on the JDK 9 Project in the OpenJDK Community. New features will focus on modularity, performance, stability, and portability."1

Project Valhalla and Project Panama

In the Community Keynote2,5, Brian Goetz cited Project Valhalla (experimental JVM and language features and not to be confused with a much older Project Valhalla) and Project Panama ("Interconnecting JVM and native code"). The promise of value types was also discussed in this part of the keynote.2,5

Eclipse's Open IoT Stack

The Eclipse Foundation announced the Open IoT (Internet of Things) Stack at JavaOne 2014.3

JavaOne 2014 Talks on Parleys.com

It was announced that JavaOne 2014 talks will be on Parleys.com.5

Miscellaneous Java Usage Statistics

Oracle likes to announce splashy statistics related to "Java" (the language and the platform). This year's edition was no different1:

  • 9 million developers worldwide
  • More than 3 billion devices are powered by Java technology
  • More than 125 million Java-based media devices have been deployed
  • Over 10 billion Java Cards have been shipped since its introduction
Duke Has An Alias: Fang

One of the more important revelations from JavaOne 2014 for some of us is that Duke was formerly known as Fang.5.

JavaOne 2015

JavaOne 2015 will be October 25–29, 2015, in San Francisco, California.

Online References to JavaOne 2014

Tuesday, October 7, 2014

Book Review: AngularJS Essentials

The subtitle of Rodrigo Branas's AngularJS Essentials (Packt Publishing) is, "Design and construct reusable, maintainable, and modular web applications with AngularJS." This post is a review of the electronic (PDF) format of this book on AngularJS.

AngularJS Essentials consists of eight chapters and roughly 150 pages of substantive content. The Preface of AngularJS Essentials briefly describes the content of each of the book's eight chapters and recommends that readers have access to an IDE or text editor and a web browser to implement the examples included in the book.

Chapter 1: Getting Started with AngularJS

Chapter 1 of AngularJS Essentials devotes its first page to a history of HTML and web development that had led to frameworks such as AngularJS. Branas then describes AngularJS as "an open source, client-side JavaScript framework that promotes a high-productivity web development experience." I think Branas articulates well what AngularJS brings to web development when he writes that AngularJS is based on the idea that "declarative programming is the best choice to construct the user interface" and "imperative programming is much better and preferred to implement an application's business logic." AngularJS achieves this, Branas states, by extending HTML.

Chapter 1's introduction to AngularJS's architecture describes it as supporting applications implementing a model-view-whatever (MVW) pattern. The chapter briefly talks about the view, model, and controller/services/filters aspect of an AngularJS application. This chapter is also the first place I have seen POJO used as an identifying acronym for Plain Old JavaScript Object rather than for the more well-known use as Plain Old Java Object.

The first chapter of AngularJS Essentials introduces a parking lot application example that is used throughout the book. As part of this initial code listing, the author describes how to download regular or minified versions of AngularJS from the AngularJS site or from the Google Content Delivery Network (CDN).

AngularJS Essential's initial chapter concludes with a section briefly describing four potential styles of code organization ("most used and discussed styles in the JavaScript community") for an AngularJS application.

Chapter 2: Creating Reusable Components with Directives

The second chapter of AngularJS Essentials begins with a single page background discussion on the Document Object Model (DOM) and explains how AngularJS looks for "attributes known as directives" in the constructed DOM tree. The author then describes a directive as an "extension of the HTML vocabulary that allows us to create new behaviors."

After briefly describing which constructs in the DOM a directive can be applied to, Chapter 2 moves onto coverage of AngularJS built-in directives (ngApp, ngController, ngBind, ngBindHtml, ngRepeat, ngModel, ngClick, ngDisable, ngClass, ngOptions, ngStyle, ngShow/ngHide, ngIf, ngInclude) with one to three paragraphs and (in most cases) a code listing covering each built-in directive.

After a brief section on reorganizing the book's sample application, AngularJS Essentials covers custom directive creation. This coverage begins with a demonstration of creation of a Directive Definition Object and discussion of template, templateURL, replace, restrict, scope, transclude, link, require, and controller. The chapter references, but does not describe in any detail, the $compile service.

AngularJS Animation is the concluding topic of Chapter 2.

Chapter 3: Data Handling

The third chapter of AngularJS Essentials introduces "data handling using AngularJS" and specifically focuses on expressions, filters, and form validation. The author describes an AngularJS expression as "a simple piece of code that will be evaluated by the framework and can be written between double curly brackets."

An AngularJS filter is described as "the perfect solution to easily perform any data manipulation." The chapter briefly discusses several provided AngularJS filters (currency, date, number, filter, json, limitTo, lowercase, uppercase, and orderBy) used in conjunction with expressions and also briefly describes using filters generally with a controller. Chapter 3 also provides an overview of and simple examples of custom filter creation.

The section of Chapter 3 on form validation describes and illustrates use of several built-in directives commonly used with form validation and the input element (ng-required, ng-minlength, and ng-pattern). This portion of the third chapter also describes and illustrates used of objects $pristine, $dirty, and $error.

Chapter 4: Dependency Injection and Services

Chapter 4 of AngularJS Essentials begins with a discussion of AngularJS's support of dependency injection and provides an example and associated discussion to illustrate why array notation is often the preferred approach for AngularJS applications that are to be minified.

Chapter 4's coverage of "creating services" begins by defining a service as "a singleton object that has its life cycle controlled by the framework" that "can be used by any other component such as controllers, directives, filters, and even other services." This section examines creating of an AngularJS service from a factory and includes discussion on the Revealing Module Pattern and the Immediately-Invoked Function Expression pattern.

Chapter 4 concludes with coverage of AngularJS built-in services. The chapter includes this sentence that made me chuckle (I have added the emphasis in italics): "Introduced a long time ago by Roy Fielding, the REST method, or Representational State Transfer, has become one of the most adopted architecture styles in the last few years." Fielding's thesis on REST is dated 2000, which by some people's perspective certainly qualifies as "a long time ago." In addition to very brief coverage of REST, the chapter briefly describes HTTP (especially status codes), JSON, and Ajax. The chapter also illustrates applying an HTTP facade, HTTP headers, caching, and interceptors in an AngularJS application.

Chapter 4 devotes a section to single page application development with AngularJS's $route service and $location service. The chapter also introduces logging with the $log service and scheduling times for specific behaviors to occur with $timeout and $interval services. The "Asynchronous with a promise-deferred pattern" section of Chapter 4 describes use of AngularJS's deferred API and promise API.

Chapter 5: Scope

AngularJS Essentials's fifth chapter open by describing Scope as "an object that acts as a shared context between the view and the controller that allows these layers to exchange information related to the application model." Similarly, the AngularJS Developer Guide describes Scope as "the glue between application controller and the view." The author of AngularJS Essentials adds, "Both sides are kept synchronized along the way through a mechanism called two-way data binding."

The section of Chapter 5 on two-way data binding illustrates use of $apply and $watch. Another section of Chapter 5 covers best practices related to use of AngularJS scope. There is also very brief coverage of $rootScope.

The "Scope Broadcasting" section of Chapter 5 looks at use of the $broadcast and on functions.

Chapter 6: Modules

The focus of AngularJS Essentials's sixth chapter is on AngularJS modules. This relatively short chapter demonstrates how to create modules by creating three modules for the book's sample application. The chapter concludes with a references to some of the most popular publicly available AngularJS modules.

Chapter 7: Unit Testing

Chapter 7 of AngularJS Essentials focuses on unit testing AngularJS applications. As part of this coverage, it describes use of the ngMock, Jasmine framework, and Karma test runner. The chapter illustrates testing AngularJS Services, Controllers, Filters, and Directives. It also demonstrates mocking with AngularJS. In particular, use of $httpBackend to mock a backend for testing components that depend on the $http service is illustrated. I appreciated AngularJS Essentials dedicating a chapter to unit testing and mocking with AngularJS.

Chapter 8: Automating the Workflow

Grunt and Bower are the focus of the eighth and final chapter of AngularJS Essentials. This chapter provides step-by-step instructions with examples for creating a distribution package and executing the workflow with Grunt. It also demonstrates managing packages with Bower.

General Observations
  • As its title implies, AngularJS Essentials focuses on the more important aspects of AngularJS. Although some web development history and general web development techniques are very briefly mentioned, they are generally minimalist in nature and allow the focus to be directly on AngularJS. AngularJS Essentials is much shorter than other books on the subject (one is nearly 700 pages long), but also does not spend numerous pages covering CSS, HTML, JavaScript and other technologies that are well covered in other books and are often already familiar to developers learning AngularJS.
  • Code listings in AngularJS Essentials are black font on white background even in the electronic copy I reviewed and do not have line numbers. Several of the code listings use bold emphasis to highlight the specific concept being illustrated with the code listing.
  • AngularJS Essentials includes a couple graphics depicting described topics. These are attributed to the AngularJS documentation as the source of the graphics.
  • With some frameworks, libraries, or languages, a good book is important because of a lack of available resources on the subject. That is not the case with the wildly popular AngularJS that not only includes good standard documentation, but enjoys a large community of users generating useful resources. These numerous good resources covering AngularJS online include AngularJS documentation such as Official AngularJS Tutorial, AngularJS Developer Guide, and AngularJS API Reference, as well as third-party resources such as ng-newsletter articles, Thinkster/Egghead, StackOverflow, and Dan Wahlin's AngularJS posts.
  • Book reviews are by their nature opinionated and different books sometimes appeal to different people. Additional reviews of AngularJS Essentials can be found on Amazon.com and GoodReads.com. I think the composite impression one gets from reading the reviews as currently available on these sites provides a fair overall representation of AngularJS Essentials and its strengths and weaknesses.
Conclusion

AngularJS Essentials introduces the essentials of AngularJS and spends minimal time covering preliminary and historical details. As "essentials" implies, it is not intended to be a thorough reference. For those comfortable with learning from online tutorials and web sites, AngularJS Essentials is probably unnecessary in the well-documented world of AngularJS. However, I can recommend AngularJS Essentials for those looking for a good complement to the numerous useful online resources on learning AngularJS. AngularJS Essentials can provide a single source of quick structure for learning about AngularJS and how AngularJS can be used for web development. AngularJS Essentials is best suited for readers who already have basic familiarity with web development technologies such as HTML, JavaScript, and CSS, and want a book focused almost entirely on AngularJS and tools and techniques that can be used with AngularJS.

Tuesday, September 30, 2014

Coverity Code Spotter Beta

Earlier this summer, it was announced on the Coverity Blog and via Andy Chou's Tweet that Coverity Code Spotter Beta is available. That 8 July 2014 blog post describes Coverity Code Spotter as "a free and simple to use cloud-based service built upon Coverity source code analysis technology for finding often hard-to-detect bug-causing issues in Java source code." The blog post also states that "for the duration of the beta period, participants are welcome to upload as much code as they would like and submit builds for analysis as often as they would like.

Yesterday's (29 September 2014) press release (issued in conjunction with JavaOne 2014), "Coverity Launches Code Spotter™ in Free Beta Version to Speed Defect Detection in Java Code," restates some of these observations regarding Coverity Code Spotter. It states, "Built on Coverity’s static code analysis technology, Code Spotter is available for free to the software development community during the beta period." The press release, like the July blog post, describes the types of issues in Java code that Code Spotter detects: "the most common and critical issues in Java code bases, including resource leaks, race conditions, concurrency issues, control flow issues, null pointer dereferences, issues detected by the open source FindBugs tool, copy and paste errors, and many other software defects resulting in incorrect or unpredictable program behavior."

Dennis Chu, Senior Product Manager for Coverity, provided answers to some questions I had. Those questions and answers are shown next.

Q: Is this free for open source and proprietary code bases?
A: Yes, both open source and proprietary Java codes bases can utilize Code Spotter without any limitations during the beta period.

Q: Is the uploaded code made available in any way to others?
A: The uploaded code is kept completely private.

Q: Are the analysis results of the uploaded code available for others with traceability to the code that was analyzed?
A: It is currently possible for a user to download analysis results (which include issues detected as well as code snippets that help understand these issues) and share them with anyone they wish. We are working on a set of team-oriented features that would allow users publish their results to other users within the Code Spotter application.

Q: How long does the uploaded code remain on Coverity's cloud? Can it be completely removed if desired?
A: The code can be completely removed with a click of a button. By default, the code and the results are removed within 30 days of analysis completion. Further, the code is not actually stored on Coverity servers. Instead, the code (and the analysis results) are stored in Amazon's S3 under tight access control.

There is more description on Code Spotter Beta in Chu's blog post Code Spotter Beta: Now Available For Everyone!

Monday, September 22, 2014

ChoiceFormat: Numeric Range Formatting

The Javadoc for the ChoiceFormat class states that ChoiceFormat "allows you to attach a format to a range of numbers" and is "generally used in a MessageFormat for handling plurals." This post describes java.text.ChoiceFormat and provides some examples of applying it in Java code.

One of the most noticeable differences between ChoiceFormat and other "format" classes in the java.text package is that ChoiceFormat does not provide static methods for accessing instances of ChoiceFormat. Instead, ChoiceFormat provides two constructors that are used for instantiating ChoiceFormat objects. The Javadoc for ChoiceFormat highlights and explains this:

ChoiceFormat differs from the other Format classes in that you create a ChoiceFormat object with a constructor (not with a getInstance style factory method). The factory methods aren't necessary because ChoiceFormat doesn't require any complex setup for a given locale. In fact, ChoiceFormat doesn't implement any locale specific behavior.
Constructing ChoiceFormat with Two Arrays

The first of two constructors provided by ChoiceFormat accepts two arrays as its arguments. The first array is an array of primitive doubles that represent the smallest value (starting value) of each interval. The second array is an array of Strings that represent the names associated with each interval. The two arrays must have the same number of elements because there is an assumed one-to-one mapping between the numeric (double) intervals and the Strings describing those intervals. If the two arrays do not have the same number of elements, the following exception is encountered.

Exception in thread "main" java.lang.IllegalArgumentException: Array and limit arrays must be of the same length.

The Javadoc for the ChoiceFormat(double[], String[]) constructor states that the first array parameter is named "limits", is of type double[], and is described as "limits in ascending order." The second array parameter is named "formats", is of type String[], and is described as "corresponding format strings." According to the Javadoc, this constructor "constructs with the limits and the corresponding formats."

Use of the ChoiceFormat constructor accepting two array arguments is demonstrated in the next code listing (the writeGradeInformation(ChoiceFormat) method and fredsTestScores variable will be shown later).

/**
 * Demonstrate ChoiceFormat instantiated with ChoiceFormat
 * constructor that accepts an array of double and an array
 * of String.
 */
public void demonstrateChoiceFormatWithDoublesAndStringsArrays()
{
   final double[] minimumPercentages = {0, 60, 70, 80, 90};
   final String[] letterGrades = {"F", "D", "C", "B", "A"};
   final ChoiceFormat gradesFormat = new ChoiceFormat(minimumPercentages, letterGrades);
   writeGradeInformation(fredsTestScores, gradesFormat);
}

The example above satisfies the expectations of the illustrated ChoiceFormat constructor. The two arrays have the same number of elements, the first (double[]) array has its elements in ascending order, and the second (String[]) array has its "formats" in the same order as the corresponding interval-starting limits in the first array.

The writeGradeInformation(ChoiceFormat) method referenced in the code snippet above demonstrates use of a ChoiceFormat instance based on the two arrays to "format" provided numerical values as Strings. The method's implementation is shown next.

/**
 * Write grade information to standard output
 * using the provided ChoiceFormat instance.
 *
 * @param testScores Test Scores to be displayed with formatting.
 * @param gradesFormat ChoiceFormat instance to be used to format output.
 */
public void writeGradeInformation(
   final Collection<Double> testScores,
   final ChoiceFormat gradesFormat)
{
   double sum = 0;
   for (final Double testScore : testScores)
   {
      sum += testScore;
      out.println(testScore + " is a '" + gradesFormat.format(testScore) + "'.");
   }
   double average = sum / testScores.size();
   out.println(
        "The average score (" + average + ") is a '"
      + gradesFormat.format(average) + "'.");
}

The code above uses the ChoiceFormat instance provided to "format" test scores. Instead of printing a numeric value, the "format" prints the String associated with the interval that numeric value falls within. The next code listing shows the definition of fredsTestScores used in these examples.

private static List<Double> fredsTestScores;
static
{
   final ArrayList<Double> scores = new ArrayList<>();
   scores.add(75.6);
   scores.add(88.8);
   scores.add(97.3);
   scores.add(43.3);
   fredsTestScores = Collections.unmodifiableList(scores);
}

Running these test scores through the ChoiceFormat instance instantiated with two arrays generates the following output:

75.6 is a 'C'.
88.8 is a 'B'.
97.3 is a 'A'.
43.3 is a 'F'.
The average score (76.25) is a 'C'.
Constructing ChoiceFormat with a Pattern String

The ChoiceFormat(String) constructor that accepts a String-based pattern may be more appealing to developers who are comfortable using String-based pattern with similar formatting classes such as DateFormat and DecimalFormat. The next code listing demonstrates use of this constructor. The pattern provided to the constructor leads to an instance of ChoiceFormat that should format the same way as the ChoiceFormat instance created in the earlier example with the constructor that takes two arrays.

/**
 * Demonstrate ChoiceFormat instantiated with ChoiceFormat
 * constructor that accepts a String pattern.
 */
public void demonstrateChoiceFormatWithStringPattern()
{
   final String limitFormatPattern = "0#F | 60#D | 70#C | 80#B | 90#A";
   final ChoiceFormat gradesFormat = new ChoiceFormat(limitFormatPattern);
   writeGradeInformation(fredsTestScores, gradesFormat);
}

The writeGradeInformation method called here is the same as the one called earlier and the output is also the same (not shown here because it is the same).

ChoiceFormat Behavior on the Extremes and Boundaries

The examples so far have worked well with test scores in the expected ranges. Another set of test scores will now be used to demonstrate some other features of ChoiceFormat. This new set of test scores is set up in the next code listing and includes an "impossible" negative score and another "likely impossible" score above 100.

private static List<Double> boundaryTestScores;
static
{
   final ArrayList<Double> boundaryScores = new ArrayList<Double>();
   boundaryScores.add(-25.0);
   boundaryScores.add(0.0);
   boundaryScores.add(20.0);
   boundaryScores.add(60.0);
   boundaryScores.add(70.0);
   boundaryScores.add(80.0);
   boundaryScores.add(90.0);
   boundaryScores.add(100.0);
   boundaryScores.add(115.0);
   boundaryTestScores = boundaryScores;
}

When the set of test scores above is run through either of the ChoiceFormat instances created earlier, the output is as shown next.

-25.0 is a 'F '.
0.0 is a 'F '.
20.0 is a 'F '.
60.0 is a 'D '.
70.0 is a 'C '.
80.0 is a 'B '.
90.0 is a 'A'.
100.0 is a 'A'.
115.0 is a 'A'.
The average score (56.666666666666664) is a 'F '.

The output just shown demonstrates that the "limits" set in the ChoiceFormat constructors are "inclusive," meaning that those limits apply to the specified limit and above (until the next limit). In other words, the range of number is defined as greater than or equal to the specified limit. The Javadoc documentation for ChoiceFormat describes this with a mathematical description:

X matches j if and only if limit[j] ≤ X < limit[j+1]

The output from the boundaries test scores example also demonstrates another characteristic of ChoiceFormat described in its Javadoc documentation: "If there is no match, then either the first or last index is used, depending on whether the number (X) is too low or too high." Because there is no match for -25.0 in the provided ChoiceFormat instances, the lowest ('F' for limit of 0) range is applied to that number lower than the lowest range. In these test score examples, there is no higher limit specified than the "90" for an "A", so all scores higher than 90 (including those above 100) are for "A". Let's suppose that we wanted to enforce the ranges of scores to be between 0 and 100 or else have the formatted result indicate "Invalid" for scores less than 0 or greater than 100. This can be done as shown in the next code listing.

/**
 * Demonstrating enforcing of lower and upper boundaries
 * with ChoiceFormat instances.
 */
public void demonstrateChoiceFormatBoundariesEnforced()
{
   // Demonstrating boundary enforcement with ChoiceFormat(double[], String[])
   final double[] minimumPercentages = {Double.NEGATIVE_INFINITY, 0, 60, 70, 80, 90, 100.000001};
   final String[] letterGrades = {"Invalid - Too Low", "F", "D", "C", "B", "A", "Invalid - Too High"};
   final ChoiceFormat gradesFormat = new ChoiceFormat(minimumPercentages, letterGrades);
   writeGradeInformation(boundaryTestScores, gradesFormat);

   // Demonstrating boundary enforcement with ChoiceFormat(String)
   final String limitFormatPattern = "-\u221E#Invalid - Too Low | 0#F | 60#D | 70#C | 80#B | 90#A | 100.0<Invalid - Too High";
   final ChoiceFormat gradesFormat2 = new ChoiceFormat(limitFormatPattern);
   writeGradeInformation(boundaryTestScores, gradesFormat2);
}

When the above method is executed, its output shows that both approaches enforce boundary conditions better.

-25.0 is a 'Invalid - Too Low'.
0.0 is a 'F'.
20.0 is a 'F'.
60.0 is a 'D'.
70.0 is a 'C'.
80.0 is a 'B'.
90.0 is a 'A'.
100.0 is a 'A'.
115.0 is a 'Invalid - Too High'.
The average score (56.666666666666664) is a 'F'.
-25.0 is a 'Invalid - Too Low '.
0.0 is a 'F '.
20.0 is a 'F '.
60.0 is a 'D '.
70.0 is a 'C '.
80.0 is a 'B '.
90.0 is a 'A '.
100.0 is a 'A '.
115.0 is a 'Invalid - Too High'.
The average score (56.666666666666664) is a 'F '.

The last code listing demonstrates using Double.NEGATIVE_INFINITY and \u221E (Unicode INFINITY character) to establish a lowest possible limit boundary in each of the examples. For scores above 100.0 to be formatted as invalid, the arrays-based ChoiceFormat uses a number slightly bigger than 100 as the lower limit of that invalid range. The String/pattern-based ChoiceFormat instance provides greater flexibility and exactness in specifying the lower limit of the "Invalid - Too High" range as any number greater than 100.0 using the less-than symbol (<).

Handling None, Singular, and Plural with ChoiceFormat

I opened this post by quoting the Javadoc stating that ChoiceFormat is "generally used in a MessageFormat for handling plurals," but have not yet demonstrated this common use in this post. I will demonstrate a portion of this (plurals without MessageFormat) very briefly here for completeness, but a much more complete explanation (plurals with MessageFormat) of this common usage of ChoiceFormat is available in the Java Tutorials' Handling Plurals lesson (part of the Internationalization trail).

The next code listing demonstrates application of ChoiceFormat to handle singular and plural cases.

/**
 * Demonstrate ChoiceFormat used for differentiation of
 * singular from plural and none.
 */
public void demonstratePluralAndSingular()
{
   final double[] cactiLowerLimits = {0, 1, 2, 3, 4, 10};
   final String[] cactiRangeDescriptions =
      {"no cacti", "a cactus", "a couple cacti", "a few cacti", "many cacti", "a plethora of cacti"};
   final ChoiceFormat cactiFormat = new ChoiceFormat(cactiLowerLimits, cactiRangeDescriptions);
   for (int cactiCount = 0; cactiCount < 11; cactiCount++)
   {
      out.println(cactiCount + ": I own " + cactiFormat.format(cactiCount) + ".");
   }
}

Running the example in the last code listing leads to output that is shown next.

0: I own no cacti.
1: I own a cactus.
2: I own a couple cacti.
3: I own a few cacti.
4: I own many cacti.
5: I own many cacti.
6: I own many cacti.
7: I own many cacti.
8: I own many cacti.
9: I own many cacti.
10: I own a plethora of cacti.
One Final Symbol Supported by ChoiceFormat's Pattern

One other symbol that ChoiceFormat pattern parsing recognizes for formatting strings from a generated numeric value is the \u2264 (). This is demonstrated in the next code listing and the output for that code that follows the code listing. Note that in this example the \u2264 works effectively the same as using the simpler # sign shown earlier.

/**
 * Demonstrate using \u2264 in String pattern for ChoiceFormat
 * to represent >= sign. Treated differently than less-than
 * sign but similarly to #.
 */
public void demonstrateLessThanOrEquals()
{
   final String limitFormatPattern = "0\u2264F | 60\u2264D | 70\u2264C | 80\u2264B | 90\u2264A";
   final ChoiceFormat gradesFormat = new ChoiceFormat(limitFormatPattern);
   writeGradeInformation(fredsTestScores, gradesFormat);
}
75.6 is a 'C '.
88.8 is a 'B '.
97.3 is a 'A'.
43.3 is a 'F '.
The average score (76.25) is a 'C '.
Observations in Review

In this section, I summarize some of the observations regarding ChoiceFormat made during the course of this post and its examples.

  • When using the ChoiceFormat(double[], String[]) constructor, the two passed-in arrays must be of equal size or else an IllegalArgumentException ("Array and limit arrays must be of the same length.") will be thrown.
  • The "limits" double[] array provided to the ChoiceFormat(double[], String[]) constructor constructor should have the limits listed from left-to-right in ascending numerical order. When this is not the case, no exception is thrown, but the logic is almost certainly not going to be correct as Strings being formatted against the instance of ChoiceFormat will "match" incorrectly. This same expectation applies to the constructor accepting a pattern.
  • ChoiceFormat allows Double.POSITIVE_INFINITY and Double.NEGATIVE_INFINITY to be used for specifying lower range limits via its two-arrays constructor.
  • ChoiceFormat allows \u221E and -\u221E to be used for specifying lower range limits via its single String (pattern) constructor.
  • The ChoiceFormat constructor accepting a String pattern is a bit more flexible than the two-arrays constructor and allows one to specify lower limit boundaries as everything over a certain amount without including that certain amount exactly.
  • Symbols and characters with special meaning in the String patterns provided to the single String ChoiceFormat constructor include #, <, \u2264 (), \u221E (), and |.
Conclusion

ChoiceFormat allows formatting of numeric ranges to be customized so that specific ranges can have different and specific representations. This post has covered several different aspects of numeric range formatting with ChoiceFormat, but parsing numeric ranges from Strings using ChoiceFormat was not covered in this post.

Further Reading