Better Java Compiler Error Messages

This post represents almost a year of work from Andrew Arnold, Ataberk Cirikci, Noah Jamison, and Thalia La Pommeray¹.

Every part of this that you agree with, they deserve all the credit for. Every part that you do not can be blamed on me.

Also, this is about Java. We'll take a bit of a winding road to get there, but we will get there.

Background

A compiler's job is to take code - usually from text files - and produce some usable artifact from it. For Java that means taking *.java files and producing *.class files that can be fed into a JVM.²

The first and most important priority of a compiler is to be correct. If the class files produced by the Java compiler do not function in the way specified by the Java Language Specification then it would not be a Java compiler.

The second priority of a compiler has historically been to be fast and resource efficient. In the 90s, CPU and RAM were far more scarce resources. If a language couldn't be compiled efficiently then it would be impractical to use. ³

What has historically not been a focus, and has seen a renaissance in modern times, is error messages.

Elm

Elm is a very small and very focused language. It is built specifically for making frontend web apps and has a very restricted set of features.⁴

import Html

main = 
    Html.h1 "Hello, world"

It's pretty cool. Check it out if you have some time.

Due to its relatively small surface area⁵, the language designer was able to dedicate time to the user experience (UX) of its compiler errors.

And by most accounts⁶, that work has paid off.

When a programmer makes a mistake with their Elm code, the Elm compiler will

• always have a friendly tone
• do its best to show the relevant areas of the code
• provide a hint as to how to resolve the error

Say we took the example from above and tried to use a fictional h7 tag.

import Html

-- There is no h7 tag
main = 
    Html.h7 "Hello, world"

The error you would get is the following.

I cannot find a `Html.h7` variable:

5|     Html.h7 "Hello, world"
       ^^^^^^^
The `Html` module does not expose a `h7` variable. These names seem close
though:

    Html.h1
    Html.h2
    Html.h3
    Html.h4

Hint: Read <https://elm-lang.org/0.19.1/imports> to see how `import`
declarations work in Elm.

First to note is the personification of the compiler as an entity. When it says "I cannot find a variable" it subtly, but importantly, primes the user to think about the compiler as an entity unto itself. Its small stuff like that gives our monkey brains the hooks it needs to anthropomorphize.

And that is useful, because the only time a compiler talks to you is when something is wrong. Which would you prefer

• "You have cancer"
• "I have your test results, and unfortunately you have cancer."

The first one is a game over screen in a FromSoft game, the second is a human being with some bedside manner.⁷

Another aspect about this that is cool is that it points to the exact place in the code that is at issue. Not just the line, but specifically the Html.h7 expression.

5|     Html.h7 "Hello, world"
       ^^^^^^^

Before you can resolve an error, you need to find the code causing it. Seems pretty obvious.

With many compilers you get a location like program.x:43:22 that you have to decipher. Where is that file? Which one is the line? Which is the column? Okay, let me scan through my code. You also often get a pretty-printed version of the problematic code, but it looks nothing like the code you wrote. You again need to do a mental transformation to find it. So a lot of time is lost:

• converting row and column numbers into an actual file position
• converting pretty-printed code onto actual code to verify that position

And don't forget that hint! It's a pretty basic analysis⁸, but being able to suggest functions that the user might have meant is big. Even if we disregard the exact contents of the hint, that there is a dedicated place to give hints and for users to look for hints is great.

It is hard to show in this format⁹, but in addition to the layout of the message things like the ^^^^^^^ are colored red to draw our attention.

5|     Html.h7 "Hello, world"
       ^^^^^^^

Say we constructed a similar situation in Java.

class Html {
    static Object h1() {
       return null; 
    }
    static Object h2() {
        return null;
    }
    static Object h3() {
        return null;
    }
    static Object h4() {
        return null;
    }
    static Object h5() {
        return null;
    }
    static Object h6() {
        return null;
    }
}

public class Main {
    public static void main(String[] args) {
        System.out.println(Html.h7());
    }
}

The error message we get is streets behind.

/Main.java:24: error: cannot find symbol
        System.out.println(Html.h7());
                               ^
  symbol:   method h7()
  location: class Html
1 error

It is kind of shocking how much better things get when you focus on the user. I mean, on some level, it is not shocking at all though. Most terminal tools came into existence well before our industry really started focusing on making apps and websites feel great for their users. We all collectively realized that a hard to use app or website is bad for business, but the same lessons have not really percolated down to tools like compilers and build tools yet. Hopefully I have demonstrated that we can do better!

Rust

Rust is a systems programming language. That means that it targets the same use-cases as C and C++ where speed and predictable latency are hard requirements.

struct Position {
    x: u32,
    y: u32
}

fn main() {
    let position = Position {
        x: 0,
        y: 1
    };

    println!("x: {}, y: {}", position.x, position.y);
}

Rust's most famous feature is its borrow checker. This is what lets it compete in ergonomics with languages like Python and Java without automatic garbage collection at runtime.¹⁰

It does this by tracking the "lifetime" of individual variables and fields, putting some rules in place for when those lifetimes end, and what to do when they end and the variable is "dropped".¹¹

struct Position {
    x: u32,
    y: u32
}

fn main() {
    // Lifetime of the position starts here
    let position = Position {
        x: 0,
        y: 1
    };

    println!("x: {}, y: {}", position.x, position.y);
    
    // At this point the position variable is no longer "alive"
    // and all the memory allocated for it will be freed.
}

The tradeoff here is that the complexity of tracking lifetimes is pushed into the type system. This and other advanced features make Rust one of the most complicated languages out there. "Fighting the Borrow Checker" is a very common occurrence.

Despite this, it is overwhelmingly loved by those who have used it.

My hypothesis for why this is the case¹² is it is because very early on its development, there was dedicated focus given to the error messages its compiler produced.¹³ Even though people tend to produce malformed programs far more often¹⁴, the experience of the compiler being "helpful" offset that.¹⁵

With the importance of addressing Rust's learning curve a key theme in the Rust survey we're as motivated as ever to find any confusing or distracting part of the Rust experience and give it a healthy amount of polish. Errors are one area where we're applying that polish helps us improve the learning curve bit by bit, and we're looking forward to seeing how far we can go.

All error messages in Rust have a specific structure. There is a place for saying where an error occurred, why it occurred, what the error was, and potentially a hint as to how to resolve it.

For example, this code is malformed because enum variants need to be prefixed with the name of the enum.

enum Ex {
    A,
    B
}

pub fn main() {
    let ex = A;
}

The error message that the compiler produces reflects that.

error[E0425]: cannot find value `A` in this scope
 --> src/main.rs:7:14
  |
7 |     let ex = A;
  |              ^ not found in this scope
  |
help: consider importing this unit variant
  |
1 | use crate::Ex::A;
  |

For more information about this error, try `rustc --explain E0425`.

It says that the problem is that the value A could not be found in scope, shows exactly where in the code the problem is, and offers a hint as to how to resolve it.

Just like the Elm errors there is a dedicated section for giving hints, the exact place in the code where a problem happens is shown, and the message is written in a friendly tone.

error[E0425]: WHAT
 --> WHERE
  |
7 |     let ex = A;
  |              ^ WHY + (arrow gives implicit WHERE)
  |
help: HINT (can have many)
  |
1 | HINT
  |

Compare and contrast that to the error you get with similarly malformed Java code.

import java.util.List;

enum Ex {
    A,
    B
}

public class MyClass {
    public static void main(String args[]) {
        var ex = B;
    }
}

/MyClass.java:10: error: cannot find symbol
        var ex = B;
                 ^
  symbol:   variable B
  location: class MyClass

It still shows where the problem happened, but it offers no assistance for fixing it and uses a fittingly robotic tone.¹⁶

Scala

Scala is another language for the JVM like Java. I won't go that deep into an explanation because I am not qualified to do so, but as part of the work on Scala 3 they worked to improve their error messages in similar ways as Elm and Rust

We’ve looked at how other modern languages like Elm and Rust handle compiler warnings and error messages, and come to realize that Dotty is actually in great shape to provide comprehensive and easy to understand error messages in the same spirit.

That work is still ongoing, but the focus was there.

That doesn't mean anything by itself, but I choose to take it as social proof that I'm not crazy.

IDEs

It is tempting to say that it doesn't really matter what errors the compiler spits out because IDEs are in a better position to give feedback anyway.

To an extent, this makes sense. IDEs like IntelliJ are able to provide feedback in ways that a compiler cannot.

• If something is wrong, they can highlight it in red.
• If something is questionable, they can highlight it in yellow.
• If the IDE has a suggestion on how to fix something they can show that through other visual cues.¹⁷

That's all great, but unfortunately I don't think it is enough.

It's easy to forget when my M1 Mac is running Baldurs Gate 3 at 60 fps, but hardware powerful enough to run an IDE smoothly is a privilege. IntelliJ cannot run on a chromebook or whatever commodity hardware a chronically underfunded school system can afford.¹⁸

This is partly why many curriculums use online platforms like repl.it that are hosted remotely or rely on a student's workflow to be through the command line and a basic text editor.

In those cases especially, compiler errors are front and center in a student's learning.

This is likely going to become more true when the JEPs for Unnamed Classes, Instance Main Methods, and Multi-File Source-Code Programs are integrated and the ergonomics of teaching from the command line become more in line with that of other languages.

And while I can't demonstrate it in as strong a way, I maintain that the error messages matter when you are using an IDE as well. Not everyone sees the red squiggles, some students actively ignore them, and they will see the original compiler message when they try to run their code regardless.

If what they see is vague and unhelpful, that matters.¹⁹

Research

There was, to my knowledge, exactly one overview study done on compiler error messages. "Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research".

To those unfamiliar, an overview study is research that looks at existing research within a field and draws conclusions from the body of work in totality. You don't get to make claims like "studies consistently show X, Y, and Z" without looking at all the studies.

There are a few things from that study I think are worthy of note.

First is that there is very little actual research on error messages.

One of our most striking observations was that there was relatively little literature on the effect of programming error messages on students and their learning.

Which at the very least makes me feel better about "going with my gut." Everyone has to be.

Second is that the research that does exist does not produce any strong conclusions

While there have been many guidelines for programming error message design proposed and implemented, no coherent picture has emerged to answer even simple questions such as: What does a good programming error message look like?

But in the summation of the literature there are a few general guidelines that emerged.

• Increase Readability
• Reduce Cognitive Load
• Provide Context
• Use a Positive Tone
• Show Examples
• Show Solutions or Hints
• Allow Dynamic Interaction
• Provide Scaffolding
• Use logical argumentation
• Report errors at the right time

So while I would love to say "I'm right and science agrees with me"²¹, the best I can say is that all the properties of Elm and Rust compiler messages that I have noted are at least represented in the list of things that research suggests "might help."

That is

• Provide Context
• Use a Positive Tone
• Show Solutions or Hints

There are individual studies like this one that more directly support my claims, but I've heard of enough horrible Dr. Oz segments like "Chocolate - the new a superfood?!" to know that it's disingenuous to use single studies like that.

So yeah, best I can say is "I am not obviously wrong."

The Structure of javac

The reference compiler for Java is javac. It comes with every OpenJDK build, and it's what most people use to compile their java.²⁵

I will briefly explain some of its internal workings so that you have some context on what we changed and why.

compiler.properties

All the error message text for javac lives inside a set of compiler.properties files. There is one file for each language that has translations. German text is in compiler_de.properties, Japanese in compiler_ja.properties, and so on.

compiler.err.abstract.meth.cant.have.body=\
    abstract methods cannot have a body

Each message is keyed in a way that indicates its purpose. compiler.err.* are for error messages, compiler.warn.* for warnings, and compiler.misc.* for potpourri.

There are comments above messages with placeholders to indicate the type of data that needs to be filled in.

# 0: name
compiler.err.call.must.be.first.stmt.in.ctor=\
    call to {0} must be first statement in constructor

# 0: symbol kind, 1: name, 2: symbol kind, 3: type, 4: message segment
compiler.err.cant.apply.symbol.noargs=\
    {0} {1} in {2} {3} cannot be applied to given types;\n\
    reason: {4}

These files get processed into Java classes by this tooling.

The classes that get generated are subclasses of JCDiagnostic.DiagnosticInfo, where compiler.err.* properties are turned into instances of JCDiagnostic.Error, compiler.warn.* into JCDiagnostic.Warning, and so on.

The class CompilerProperties holds constants for each of these messages as well as static methods for the messages that had those special comments indicating that they need placeholders.

/**
 * compiler.err.anonymous.diamond.method.does.not.override.superclass=\
 *    method does not override or implement a method from a supertype\n\
 *    {0}
 */
public static Error AnonymousDiamondMethodDoesNotOverrideSuperclass(
        Fragment arg0
) {
    return new Error(
        "compiler", 
        "anonymous.diamond.method.does.not.override.superclass", 
        arg0
    );
}

/**
 * compiler.err.array.and.receiver =\
 *    legacy array notation not allowed on receiver parameter
 */
public static final Error ArrayAndReceiver 
        = new Error("compiler", "array.and.receiver");

// And so on

JCDiagnostic

I've been mostly talking about "error messages," but they are ontologically just one kind of "diagnostic".

JCDiagnostic - short for java compiler diagnostic I'm pretty sure - is the representation javac has for diagnostic messages.

It stores references to all the information needed to construct a message shown to the user. This includes the source which the diagnostic references, the position in that source being referenced, as well as other miscellaneous metadata.

A pointer to some text from the compiler.properties as well as the arguments needed for any placeholders in said text are stored in a sub-object under diagnosticInfo. These DiagnosticInfo objects come from what was generated in CompilerProperties.

The whole structure implements the Diagnostic interface, which is part of Java's public API.²⁶

Context

One of the more interesting concepts in javac is its Context mechanism.

In "regular" code that wants a singleton, you generally hide the constructor for your class and expose a single instance in some way.

final class Apple {
    private Apple() {}
    
    public static final Apple INSTANCE = new Apple();
}

The other option is to make your class "normal" but rely on some dependency injection framework to automatically create, manage, and provide singular instances of that class.

final class Apple {
    // ...
}

class UsageSite {
    private Apple apple;
    
    @Inject
    UsageSite(Apple apple) {
        this.apple = apple;
    } 
}

javac wants only single instances of many of its classes, but it also wants to allow for multiple instances of the compiler to run in parallel on the same JVM.

The solution they use is to have one class, Context, which holds a map of Context.Key to Objects.

Classes like JCDiagnostic.Factory have factory methods that do a get-or-create with their own constant Context.Keys.

public static class Factory {
    /** The context key for the diagnostic factory. */
    protected static final Context.Key<JCDiagnostic.Factory> diagnosticFactoryKey 
            = new Context.Key<>();

    /** Get the Factory instance for this context. */
    public static Factory instance(Context context) {
        Factory instance = context.get(diagnosticFactoryKey);
        if (instance == null)
            instance = new Factory(context);
        return instance;
    }
    // ...
}

And then this Context is threaded to every class in the compiler that wants to get instances of those "contextual singletons" or themselves participate in the mechanism.

protected Flow(Context context) {
    context.put(flowKey, this);
    names = Names.instance(context);
    log = Log.instance(context);
    syms = Symtab.instance(context);
    types = Types.instance(context);
    chk = Check.instance(context);
    lint = Lint.instance(context);
    rs = Resolve.instance(context);
    diags = JCDiagnostic.Factory.instance(context);
    Source source = Source.instance(context);
}

Log

During compilation, if a problem is encountered anywhere, the compiler constructs and "emits" a diagnostic.

It does this by getting an instance of the Log contextual singleton and using the generated constants from CompilerProperties.

public class Operators {
    // ...
    
    protected Operators(Context context) {
        context.put(operatorsKey, this);
        syms = Symtab.instance(context);
        names = Names.instance(context);
        log = Log.instance(context);
        types = Types.instance(context);
        noOpSymbol = new OperatorSymbol(
                names.empty, Type.noType, -1, syms.noSymbol
        );
        initOperatorNames();
        initUnaryOperators();
        initBinaryOperators();
    }
    
    // ...


    private OperatorSymbol reportErrorIfNeeded(
            DiagnosticPosition pos, 
            Tag tag, 
            Type... args
    ) {
        if (Stream.of(args).noneMatch(t -> 
                t.isErroneous() || t.hasTag(TypeTag.NONE)
        )) {
            Name opName = operatorName(tag);
            JCDiagnostic.Error opError = (args.length) == 1 ?
                    Errors.OperatorCantBeApplied(
                            opName, args[0]
                    ) :
                    Errors.OperatorCantBeApplied1(
                            opName, args[0], args[1]
                    );
            log.error(pos, opError);
        }
        return noOpSymbol;
    }
    
    // ...
}

Log internally holds onto the JCDiagnostic.Factory contextual singleton in order to construct JCDiagnostics from DiagnosticInfos like JCDiagnostic.Error.²⁷

public class Log extends AbstractLog {
    // ...
    
    private Log(
            Context context, 
            Map<WriterKind, PrintWriter> writers
    ) {
        super(JCDiagnostic.Factory.instance(context));
        context.put(logKey, this);
        this.writers = writers;

        @SuppressWarnings("unchecked") // FIXME
        DiagnosticListener<? super JavaFileObject> dl =
                context.get(DiagnosticListener.class);
        this.diagListener = dl;

        diagnosticHandler = new DefaultDiagnosticHandler();

        messages = JavacMessages.instance(context);
        messages.add(Main.javacBundleName);

        final Options options = Options.instance(context);
        initOptions(options);
        options.addListener(() -> initOptions(options));
    }
    
    // ...
}

All the diagnostics are then reported to a DiagnosticHandler.

public class Log extends AbstractLog {
    // ...

    @Override
    public void report(JCDiagnostic diagnostic) {
        diagnosticHandler.report(diagnostic);
    }

    // ...
}

DiagnosticFormatter

There are some steps I am skipping, but eventually diagnostics flow from DiagnosticHandlers to a DiagnosticFormatter which is responsible for formatting the diagnostic for display.

There are a few implementations of DiagnosticFormatter, but the most relevant is BasicDiagnosticFormatter.

BasicDiagnosticFormatter has three formats that it recognizes. Diagnostics with a position, diagnostics without a position, and diagnostics originating in a class file for which the source is not available. It uses custom format strings that describe how it should display each of those diagnostics.

private void initFormat() {
    initFormats("%f:%l:%_%p%L%m", "%p%L%m", "%f:%_%p%L%m");
}

For backwards compatibility reasons, javac also maintains an "old style" diagnostics format and a "normal" format. The old style diagnostics format can be enabled with compiler flags.

public static class BasicConfiguration 
        extends SimpleConfiguration {
    public BasicConfiguration(Options options) {
        // ...
        
        initIndentation();
        if (options.isSet("diags.legacy"))
            initOldFormat();
        String fmt = options.get("diags.layout");
        if (fmt != null) {
            if (fmt.equals("OLD"))
                initOldFormat();
            else
                initFormats(fmt);
        }
        
        // ...
    }
}

The format strings consist of "meta characters" that represent different components of the diagnostic.
The meta characters and other components are formatted independently and then concatenated.

For the normal format diagnostics with a position, each section of the string has a meaning as follows:

After these components, the source code at the position of the diagnostic is inserted.

The source code is inserted at the end of the diagnostic if the diagnostic message is a single line, or the source code is inserted after the first line of the diagnostic message if it is multiline.²⁸

Structural Problems

If we want errors closer to what Rust has, the most important structural deficiency to tackle is javac's message oriented-ness.

By that I am referring to the fact that every kind of diagnostic is "just a message." There is no clearly delineated place to put hints or other context.²⁹

This is something that has already had to be worked around. Take the following program.

public class Main {
    public static void main(String[] args) {
        Object o = 123;
        switch (o) {
            case Integer i -> System.out.println(i);
            default -> {}
        };
    }
}

This program uses pattern switches, which are a preview feature. This is the error you would get if you tried to compile it.

Main.java:5: error: patterns in switch statements are a preview feature and are disabled by default.
          case Integer i -> System.out.println(i);
               ^
  (use --enable-preview to enable patterns in switch statements)
1 error

While you might look at this and think that there is already a dedicated place for hints, you would be wrong.

This message come from this entry in the compiler.properties file.

# 0: message segment (feature)
compiler.err.preview.feature.disabled.plural=\
   {0} are a preview feature and are disabled by default.\n\
   (use --enable-preview to enable {0})

You will notice that the only thing separating the hint to use the --enable-preview flag from the initial message is a newline.

BasicDiagnosticFormatter just has a heuristic where it assumes that any newline in a message means that the lines following it should be displayed below the code that is the source of the issue.

public class BasicDiagnosticFormatter 
        extends AbstractDiagnosticFormatter {
    // ...
    public String formatMessage(JCDiagnostic d, Locale l) {
        // ...

        if (lines.length > 1
                && getConfiguration()
                .getVisible()
                .contains(DiagnosticPart.DETAILS)) {
            currentIndentation += getConfiguration()
                    .getIndentation(DiagnosticPart.DETAILS);
            for (int i = 1; i < lines.length; i++) {
                buf.append(
                        "\n" + indent(lines[i], currentIndentation)
                );
            }
        }
        
        // ...
        return buf.toString();
    }
    
    // ...
}

This causes a problem for diagnostics that have a newline that is not accounted for in that heuristic like the following.³⁰

# TODO 308: make a better error message
compiler.err.this.as.identifier=\
    as of release 8, ''this'' is allowed as the parameter name for the receiver type only\n\
    which has to be the first parameter, and cannot be a lambda parameter

If you construct a program that triggers this error like so:

public class Math {
    static int add(int a, int this) {
        return a + this;
    }
}

You will get the following.

Math.java:2: error: as of release 8, 'this' is allowed as the parameter name for the receiver type only
    static int add(int a, int this) {
                              ^
  which has to be the first parameter, and cannot be a lambda parameter
1 error

Which feels at the very least unintentional.

There are other places where the pressure to say more than one thing in a message leads to larger irreconcilable inconsistencies.³¹

Every year or so there is someone who complains about a specific error on the mailing list. This usually leads to a concrete improvement in whatever error they complain about, but the root problem here is structural.³²

Structural Solutions

Structural problems always require structural solutions, so that's what we aimed to do.

The way Rust deals with this is with structured diagnostics.³³ We translated that approach into the existing JCDiagnostic world by introducing two new concepts: Help and Info.

In our prototype all JCDiagnostics now carry an optional Help and an optional Info.

public class JCDiagnostic 
        implements Diagnostic<JavaFileObject> {
    // ...

    private final DiagnosticSource source;
    private final DiagnosticPosition position;
    private final DiagnosticInfo diagnosticInfo;
    private final Set<DiagnosticFlag> flags;
    private final LintCategory lintCategory;
  
    private final Info info;
    private final Help help;
    
    // ...
}

Help

"Help"s carry two pieces of information.

public record Help(
        HelpFragment message,
        List<SuggestedChange> suggestedChanges
) {
    // ...
}

First is a message. This is a fragment of text just like other DiagnosticInfos and it is where the actual text of "use --enable-preview to enable patterns in switch statements" would come from.

public static final class HelpFragment extends DiagnosticInfo {
    public HelpFragment(String prefix, String code, Object... args) {
        super(HELP, prefix, code, args);
    }
}

The second is a list of zero or more suggested changes.

public record SuggestedChange(
        DiagnosticSource source,
        RangeDiagnosticPosition position,
        String replacement,
        Applicability applicability
) {
    // ...
}

These SuggestedChanges all know where in the source they are referring to, what replacements to make in order to apply the suggestion, and to what degree the suggestion is mechanically applicable.

public enum Applicability {
    MACHINE_APPLICABLE,
    HAS_PLACEHOLDERS,
    UNKNOWN
}

Help messages would include:

• Suggesting importing a class or enum
• Suggesting fixing a misspelled identifier to a similarly named identifier that actually exists
• Explaining that an identifier doesn't exist, but it does exist in a class that is accessible
• Suggesting changing arguments to a function to satisfy the signature

Info

Infos are similar in spirit to Helps, but they only provide helpful context. They do not suggest a user change their code in any concrete way.

public record Info(
        InfoFragment message,
        List<InfoPosition> positions
) {
}

An Info holds a text fragment and a list of all the places in the code that are relevant to that message.

Info messages would include:

• Displaying related function signatures to show the programmer the expected signature
• Displaying the declaration of a class, field, or local variable to show what type was expected
• Providing more detailed information as to why something is not allowed, such as assert becoming a keyword in Java 1.4
• Displaying supplementary information related to valid values

compiler.properties

In order to get the text for help and info messages in a localization friendly way, we piggybacked on the existing conventions in the compiler.properties files.

We updated the tooling so that properties keyed by compiler.help.* and compiler.info.* are translated into fragments inside CompilerProperties the same as was done for compiler.error.* and company.

compiler.info.function.declared.here=\
    function declared here

# ...

# 0: kind name, 1: symbol
compiler.help.similar.symbol=\
    a similarly named {0} exists: {1}

public class CompilerProperties {
  public static class Infos {
    // ...

    /**
     * compiler.info.function.declared.here=\
     *    function declared here
     */
    public static final InfoFragment FunctionDeclaredHere
            = new InfoFragment("compiler", "function.declared.here");

    // ...
  }
  public static class Helps {
    // ...

    /**
     * compiler.help.similar.symbol=\
     *    a similarly named {0} exists: {1}
     */
    public static HelpFragment SimilarSymbol(
            KindName arg0, Symbol arg1
    ) {
      return new HelpFragment(
              "compiler", "similar.symbol", arg0, arg1
      );
    }

    // ...
  }
}

At the sites where diagnostics are emitted, there Helps and Infos can then be attached to a JCDiagnostic by referencing these generated classes and using the withHelp or withInfo methods.

class SymbolNotFoundError extends ResolveError {
    // ...
    @Override
    JCDiagnostic getDiagnostic(
            JCDiagnostic.DiagnosticType dkind,
            DiagnosticPosition pos,
            Env<AttrContext> env,
            Type site,
            Name name,
            List<Type> argtypes,
            List<Type> typeargtypes
    ) {
      // ...
      if (hasLocation) {
        // ...
        if (suggestMember != null) {
            diag = diag.withHelp(
                    new Help(
                            Helps.SimilarSymbol(
                                    Kinds.kindName(suggestMember),
                                    suggestMember
                            )
                    )
            );
        }

        return diag;
      }
            
      // ...
    }
}

The logic in the code above produces errors like the following.

Test.java:5: error: cannot find symbol
        TetsingMetho();
        ^
  symbol:   method TetsingMetho()
  location: class Test
help: a similarly named method exists: TestingMethod()

What is important here isn't the analysis being performed or how it is displayed exactly. Those are all things that can be disputed by reasonable people. It's that there is now a place to give that sort of advice.

With just that little bit of structure, it suddenly becomes tractable to build a feature like "suggest methods with similar names."

DiagnosticFormatter

In order to retrofit BasicDiagnosticFormatter to display help and info messages, we needed to add some more meta characters to its format strings.

public class BasicDiagnosticFormatter 
        extends AbstractDiagnosticFormatter {
    // ...
    public static class BasicConfiguration 
            extends SimpleConfiguration {
        // ...
        private void initFormatWithInfoAndHelp() {
            initFormats(
                    "%f:%l:%_%p%L%m%i%h",
                    "%p%L%m%i%h",
                    "%f:%_%p%L%m%i%h"
            );
        }
        // ...
    }
    // ...
}

This has the very convenient property that, at least for this design, we can put the new reporting format behind a flag.

public class BasicDiagnosticFormatter 
        extends AbstractDiagnosticFormatter {
    // ...
    public static class BasicConfiguration 
            extends SimpleConfiguration {
        // ...

        public BasicConfiguration(Options options) {
            // ...
          
            if (options.isSet(/* ... */)) {
                initFormatWithInfoAndHelp();
            } else {
                initFormat();
            }
            
            // ...
        }
        
        private void initFormat() {
            initFormats(
                    "%f:%l:%_%p%L%m",
                    "%p%L%m",
                    "%f:%_%p%L%m"
            );
        }
        
        private void initFormatWithInfoAndHelp() {
            initFormats(
                    "%f:%l:%_%p%L%m%i%h",
                    "%p%L%m%i%h",
                    "%f:%_%p%L%m%i%h"
            );
        }
        // ...
    }
    // ...
}

Remaining Work

There are many things left unfinished.

• Some of our modifications show suboptimal positions in particularly complex cases. Fixing this involves passing more context down through the compiler and keeping more positional information.
• The wording of our additional info can likely be improved to flow better with the writing style of the existing errors.
• The messages that make use of the newline heuristic, especially the --enable-preview ones, can be moved to having an attached Help. Doing so would mean that either the keys for those messages would need to be duplicated or the original messages would be degraded if the flag for this were turned off.
• Almost every error and warning could use tone audit.
• In general, the compiler tends to drop information between phases. We didn't find a system other than to go case by case. and just thread through the information we need as we need it.
• compiler.err.cant.apply.symbol is several different errors in a trench-coat.
• Ich spreche kein Deutsch.
• 私は日本語を話しません
• 我不会说中文
• And much, much more

But I am satisfied with the progress we made. You can find a bestiary of the specific errors that were tackled here.

Call to Action

Submitting a JEP, while technically an open process, in practice seems to be helped by having more free time than is reasonable³⁴, being paid to work on OpenJDK, or social capital.

If you want this work to continue, you should voice your support on compiler-dev@openjdk.org or in whatever forum you think would have the most impact. If you do want to use the mailing lists, take note that you need to sign up for the mailing list yourself in order for your emails to go through.

If you are interested in continuing this work yourself, the current state of this is on the hints branch here. There is quite a bit left to do and a lot of arcane knowledge we picked up along the way, so please reach out if you choose this path. I can at least help you get set up in IntelliJ to build the JDK.³⁵

If you are in a position of authority at of one of the large companies that has dedicated staff working on OpenJDK, hire some or all of us. Failing that, dedicate other person-power to the issue.³⁶³⁷

JEP

1: And well over a year of mental illness from me.

2: This definition isn't exactly accurate. JITs are compilers too but they do their work in memory. There is no artifact to speak of, but I think its close enough to what most people think of when they think of a compiler. The wikipedia entry for compilers has a more accurate definition.

3: Languages like C, C++, Rust, Zig, etc. also care quite a lot about optimizations. It might be a little disingenuous of me to gloss over that, but this post is about a language backed by a VM. The real optimizations happen at runtime and dwelling too long on that felt a bit much.

4: The Elm language is closest in spirit to Haskell. All code needs to be purely functional, the type system is strong, and the syntax is similar as well. What sets it apart is that it doesn't have Haskell classic features like type-classes, do-notation, or `IO` monads. Good case study in addition by subtraction.

5: I'm attributing causation here, but I don't actually have strong proof that the better error messages work benefited from Elm being small and focused. At least in my brain it tracks though. Evan Czaplicki is but one man. Doing what he did but with C++ would have been infeasible.

6: If twitter dies, just know that these links were to people saying nice things about Elm.

7: This isn't actually that good of an argument in and of itself. I know that. I just find it rhetorically compelling. I love Elden Ring, but I don't want to be playing it at every day at my job and I wouldn't want to force someone to deal with all the flavors of "lol, get rekt" it entails.

8: "Basic analysis" is doing a lot of legwork in that sentence. Its basic in large part because of the simplicity of the Elm language. Libraries, modules, and source files work in precisely one way. Doing the same kind of analysis with classes on the class-path, modules on the module-path, and code yet to be compiled might be quite a bit harder for Java.

9: I'm doing it for the example directly below, but it's hard to get the pandoc renderer which makes this site to like inline spans with color information. I'll only do that work when I'm talking about coloring and, because I'm writing these footnotes as I go, I won't rule out just falling back to images.

10: This is nowhere near a complete explanation of the borrow checker, how it functions, or what it accomplishes. For that you should read the Rust book or any of the many good references online. I'm trusting that most readers will have a passing familiarity.

11: This explanation conflates a little bit borrow checking and lifetime checking. There are also no "borrows" that happen in the following code, but I've seen "the borrow checker" used interchangably with what could be called "the lifetime checker" and at the level of zoom this post is at I think the difference isn't that crucial. Doesn't help that I'm not exactly sure where lines for these terms are regardless.

12: I think there is decent evidence for this, but it is really hard to prove.

13: The initial design for which seems to have been directly inspired by Elm.

14: During the aforementioned "fighting the borrow checker" phase.

15: You could probably make this same argument for any part of the Rust ecosystem like the polish in Cargo, Clippy, etc.

16: Compare how "cannot find symbol" sounds versus "cannot find value `A` in this scope". The difference is subtle, but it matters.

17: IntelliJ does this with a "light bulb" that appears when you hover over a bit of malformed code that contains a contextual menu with potential fixes.

18: I do not remember exactly what they are called, but my old High School had a bunch of machines that were basically just empty boxes that connected to a shared Windows server.

19: I hesitate to mention this, but the CS gender ratio is absolutely wack when famously it did not start out that way. There are lots of shitty reasons for this, and I'm not saying that its Java's fault, but it is worthy of note that Java has been an extremely popular first language in education ever since its release and as a consequence has presided over this depressing chart. I can't prove it²⁰ but I believe that small frictions like `public static void main` and obtuse error messages filter students down to the demographics that are willing to deal with that sort of hostility.

20: I haven't been able to find any meaningful research to back up or refute this claim. That must be because there is none, I'm bad at searching, or I've unconsciously ignored evidence that refutes my point.

21: "I'm right and science hasn't caught up to how right I am."

22: I will later be shown that I was at least somewhat wrong about this.

23: No relation.

24: These are the original PowerPoint and Design Document they produced for their class.

25: Other compilers like ECJ exist, but being the default means always mean javac is going to be what most people use. That's why the focus was there, it is where changes can do the most good.

26: Something that is merciful about wanting to change the internals of javac is that things like Diagnostic are all I need to worry about. If we're not changing the set of programs accepted by the Java compiler and aren't looking to change supported APIs, then we should be able to avoid sanction by Java's backwards compatibility policies.

27: The FIXME in this code is from 2011.

28: This explanation was taken almost verbatim from the students' paper.

29: Giving the information a dedicated and delineated position.

30: This FIXME is from 2013.

31: I'll take "irreconcilable inconsistencies" for $500.

32: Structural issues like this I don't think are because of incompetence. The goal of `javac` has always been to be a reference compiler for Java. Everyone wants "good" errors, but the status quo is a result of the natural trend when a diagnostic ~= a line of output.

33: Structure, structure, structure.

34: If you get enough commits into OpenJDK and are a committer you can submit a JEP. That requires fixing a lot of JDK issues and going through a nomination process. I have Hogans Heroes to binge.

35: That took awhile for us to figure out even with the tutorials that exist.

36: Even if you aren't convinced that the approach we've taken is the one to continue with, hopefully you recognize that there is a real problem here.

37: I, at least, am not particularly special. I do however have a lot of context on this area of the code by now.

<- Index