Java Corpus Tools

Vision

1. Create a platform to enable language designers and the community to validate extensions to Java.
2. Encourage and facilitate empirical research of features in Java.

A Query Language for Language Designers

Java Source Code Query Languages

In this section, we give an overview of the seven query languages that we evaluate in this paper: Java Tools Language, Browse-By-Query, SOUL, JQuery, .QL, Jackpot and PMD. We selected these languages because they provide a variety of design choices and strictly provide a query language. For example, we didn't select Findbugs as it only lets programmers query source by creating new classes based on a Java framework. We also only selected source code query languages that included a guide or a working implementation.

Java Language Tools

The Java Tools Language (JTL) is a logic-paradigm query language to select Java elements in a code base. The current implementation is based on an analysis of Java bytecode classes. The JTL syntax is inspired by Query-by-Example ideas in order to increase productivity of users. JTL also relies on Datalog-like semantics. For example, one could find all methods taking three int parameters and returning a subclass of Java's Date class using the follow query:

public static D method(int, int, int), D extends* /java.util.Date;

In addition, JTL features variable binding and data flow queries.

Browse-By-Query

Browse-By-Query (BBQ) reads Java bytecode files and creates a database representing classes, method calls, fields, field references, string constants and string constant references. This database can then be interrogated through English-like queries. The syntax is motivated by the desire to be intuitive. For example, one could find all the methods that call a method whose name matches start by composing the following query:

methods containing calls to matching "start" methods in all classes

In addition, BBQ provides filtering mechanisms, set and and relational operators that can be combined to compose more complex queries.

SOUL

SOUL is a logic-paradigm query language. It contains an extensive predicate library called CAVA that matches queries against AST nodes of a Java program generated by the Eclipse JDT. SOUL facilitates the specification of queries by using example-driven matching of templates and structural unification to match a code excerpt with an AST node. In practice, this means a user can create a logic variable to match an AST node and reuse this variable within the query regardless of the execution path where the variable appears. For example, one could specify a query that finds instances of Scanner that is read after it was closed as follows: 

if jtMethodDeclaration(?m){
  public static void main(String[] args) {
    ?scanner := [new java.util.Scanner(?argList);]
    ?scanner.close();
    ?scanner.next();
  }
}

JQuery

JQuery is a logic-paradigm query language built on top of the logic programming language TyRuBa. The implementation of JQuery analyse the AST of a Java program by making calls to the Eclipse JDT. JQuery includes a library of predicates that allows querying Java elements and the relationships between them. For example, the following query finds all method declarations ?M that have at least one parameter of type Integer:

method(?M, paramType, ?PT), match(?PT, /Integer/).

.QL

.QL is an object-oriented query language. It enables programmers to query Java source code by composing queries that look like SQL. The motivation for this design choice is to reduce barrier to entry for developers that learn it. In addition, the authors argue that object-orientation provides the structure necessary for building reusable queries. An implementation is available, called SemmleCode, which includes an editor and various optimisations. As an example, the following query describes how to find all classes that declare a method equals, but which do not specify a method hashCode.

from Class c
    where c.declaresMethod("equals") and
        not (c.declaresMethod("hashCode")) and
        c.fromSource()
select c.getPackage(), c

Jackpot

Jackpot is a module for the NetBeans IDE for querying and transforming Java source files. Jackpot lets user query the AST of a Java program by composing rules under the form of a Java expression. In addition, one can specify variables to bind to a matching AST node. For example, the following query will match any code surrounded by a call to readLock() and readUnlock():

$document.readLock();
$statementsUnderLock$;
$document.readUnlock();

PMD

PMD is a ruleset based Java source code analyzer that identifies bugs or potential problems including dead code, duplicate code or overcomplicated expressions. PMD has an extensive archive of built-in rules that can be used to identify such problems. One can specify new rules by writing it in Java and making use of the PMD helper classes. Alternatively, one can also compose custom rules via an XPath expression that queries the AST of the program to analyze. For example, the following query finds all method declarations that have at least one parameter of type Integer:

//MethodDeclarator/FormalParameters
 [FormalParameter/Type/ReferenceType/ClassOrInterfaceType
  [@Image = 'Integer']]

Uses Cases

In this section, we describe the use cases examined for the evaluation. We selected use cases that are source of language design discussions and make use of a variety of Java features.

Final Array and Anonymous Inner Classes

Java lets programmers create inner classes, which is a nested class not declared static. There exists three different types of inner classes: non-static member, local and anonymous classes.

Inner classes have a restriction that any local variable, formal parameter, or exception parameter used but not declared in the inner class must be declared final.

However, programmers can circumvent this restriction by declaring a final array with only one element and mutate the element of the array. The following code illustrates this mechanism:

public class OutsideClass
{
    public void methodA()
    {
        final String[] s = new String[1];
        class InnerClass
        {
            public void methodB()
            {
                s[0] = "bypass"; // accepted by compiler
            }
        }
    }
}

Use Case 1: Find occurrence of an anonymous inner class whose code references a final array variable in the enclosing scope and which mutates array elements via that variable.

Generic Constructors

A constructor can have two sets of type arguments. A constructor can use the type parameters declared in a generic class. One can then specify the types after the class name: new Foo<Integer>(). In addition, a constructor can declare its own type parameters. The types are then specified between the new token and the class name: new <Integer> Foo<Number>(). The code below illustrates a constructor of class Foo which declares its own type parameter S that extends the class's own parameters.

class Foo<T extends Number> {
    <S extends T> Foo() {}
}

Use Case 2: Find generic constructors whose type parameters extend the enclosing class's own type parameters.

Capture Conversion Idiom

Java 5 introduced wildcards as a variance mechanism for generics. Safety is achieved by restricting accesses to fields and methods based on the position of the type parameter. The unbounded wildcard <?> represents any type and can be used to provide a simple form of bivariance. In practice, this means that List<?> is a supertype of List<T> for any T. 

The unbounded wildcard is frequently used as part of the capture conversion idiom. In the code below, the signature of reverse is prefered over rev as it doesn't expose implementation information to the caller. The argument List<?> is passed to rev, which takes a List<T>. However, allowing such a subtype relation would be unsafe. Java provides capture conversion as a solution: the unknown type <?> is captured in a type variable and T is infered to be that type variable within the method rev.

public static void reverse(List<?> list) { rev(list); }
private static <T> void rev(List<T> list) {
    List<T> tmp = new ArrayList<T>(list);
    for (int i = 0; i < list.size(); i++) {
        list.set(i, tmp.get(list.size() - i - 1));
    }
}

Use Case 3: Find occurrences of the capture conversion idiom.

Overloaded Methods

Overloading methods allows programmers to declare methods with the same name but with different signatures.

For example, one could write an add method that takes a different number of parameters:

public void add(T a) { ... }

public void add(T a, T b) { ... }

public void add(T a, T b, T c) { ... }

Often this pattern can be rewritten by using the varargs feature if the overloaded methods' parameters share a single type:

public void add(T a, T ... args) { ... }

Related to this use case, recent work has investigated overloading in Java and found that a quarter of overloaded methods are simulating default arguments.

Use Case 4: Find overloaded methods with multiple parameters that share a single type.

Covariant Arrays

In Java and C#, array subtyping is covariant, meaning that type B[] is considered a subtype of A[] whenever B is a subtype of A. However, this relation can cause runtime exceptions. Consider the following Java code where Banana and Apple are subtypes of Fruit:

Banana[] bananas = new Banana[5];
Fruit[] fruit = bananas;
fruits[0] = new Apple(); // ArrayStore Exception
peelBanana(bananas[0]); // Apple???

The assignment to the first element of the variable fruit on line 3 will cause an ArrayStore exception. Although statically, the variable fruit has type Fruit[], its runtime type is Banana[] and thus we cannot use it to store an Apple.

Use Case 5: Find occurences of covariant array uses in assignment, method calls, constructor instantiations and return statements.

Rethrown Exception

Java 7 introduced an improved checking for rethrown exceptions. Previously, a rethrown exception was treated as throwing the type of the catch parameter. Now, when a catch parameter is declared final, the type is known to be only the exception types that were thrown in the try block and are a subtype of the catch parameter type.

This new feature introduced two source incompatibilities with respect to Java 6. The code below illustrates the changes. Previously, the statement throw exception would throw a Foo exception. Now, it throws a DaughterOfFoo exception. As a consequence, the catch block catch(SonOfFoo anotherException) is not reachable anymore as the try only throws a DaughterOfFoo exception.

class Foo extends Exception {}
class SonOfFoo extends Foo {}
class DaughterOfFoo extends Foo {}

class Test {
    void test() {
        try {
            throw new DaughterOfFoo();
        } catch (final Foo exception) {
            try {
                throw exception; // first incompatibility
            } catch (SonOfFoo anotherException) {
                // second incompatibility: is this block reachable?
            }
        }
    }
}

Use Case 6: Find occurences of nested try/catch blocks that rethrow an exception.

Evaluation

[This section is work in progress]