Java Build Scripts

I've written before about how I think that while they need some bolstering (here) using the CLI tools to build Java code is more practical than you might think (here and here).

What I didn't talk about, or tip-toed around, is that writing build scripts in bash, PowerShell, cmd.exe, etc. is not very cross-platform.

You can install bash on Windows and run in WSL, but that feels unideal. An extra setup step is one thing, but needing to ask students who just learned that the command line exists to also make sure they aren't accidentally running in PowerShell is painful.

You could also just ignore the problem. "Real" developers use Mac or Linux, right? Well those same developers sometimes pick an extra special shell for themselves like zsh, nushell, or fish. You have a similar, if less serious, problem.

Using any of those shells for experimenting or testing out commands is fine. Until you write $() or a file path they are all more or less the same. What we really need is some way of writing out commands that will work on Windows, Mac, and Linux and regardless of if someone is using bash, Powershell, cmd.exe, zsh, nushell, or fish.

If only we had a language that could be written once and then ran anywhere.

just

Since Java 22 we've been able to write java Main.java and execute a potentially multi-file program. Before that there was a significant bootstrap problem. If you write Java code to compile Java code, who compiles that Java code?

Now that that's there, we can start to consider what it would look like to run a CLI tool from Java code and compare that to the alternatives.

The alternative I've been using is just. just is a command runner similar to make but without any of the caching make does or the wild syntax and history make is burdened with.

You write the name of a "recipe", a : then an indented list of commands to run.

demo:
    javac --version
    jlink --version

For this, if you run just demo and then it will run each command in sequence.

$ just demo
javac --version
javac 22.0.1
jlink --version
22.0.1

If any command gives you a non-zero exit code it fails immediately.

javac --v
error: invalid flag: --v
Usage: javac <options> <source files>
use --help for a list of possible options

And, by default, it will echo the command its about to run before it runs it.

All of these properties are useful for different reasons.

• Printing the command before its run is useful when something fails. You can usually just copy-paste the command and tweak it until it works. Then you can just copy-paste the working command back in place.
• Failing immediately on a non-zero exit code is a good default for what I feel are obvious reasons.
• Being able to refer to a command or group of commands by an alias is almost required to do anything interesting. just compile is much more ergonomic to use than javac -d build --module-source-path "./*/src --module example.

Also, and it feels small but isn't, you can get a list of all the commands + a comment on how to use them with just --list.

$ just --list
Available recipes:
    demo

Run commands in Java

I want all of these properties so let's see how we can get them in Java.

To run a command we can use the ProcessBuilder.

import java.util.List;

public class Project {
    public static void main(String[] args) {
        var cmd = List.of("javac", "--version");

        var pb = new ProcessBuilder(cmd);
    }
}

Then all we need to do is start the command, wait for it to finish, and record the exit status. If its non-zero throw.

public class Project {
    public static void main(String[] args) {
        var cmd = List.of("javac", "--version");

        var pb = new ProcessBuilder(cmd);
        try {
            int exitStatus = pb.inheritIO().start().waitFor();
            if (exitStatus != 0) {
                throw new RuntimeException(
                        "Non-zero exit status: " + exitStatus
                );
            }
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    }
}

Yeesh. So that sucks. We haven't even gotten to printing out the command or labelling groups of commands yet.

🚧Construction Zone 🚧

I've been noodling on this for a time and this weekend² I think I finally came up with a half-decent API.

import dev.mccue.tools.ExitStatusException;
import dev.mccue.tools.Tool;

public class Project {
    public static void main(String[] args)
            throws ExitStatusException {
        Tool.ofSubprocess("javac")
                .run("--version");
    }
}

This will print out the command to System.err, run it, throw an exception if needed, and pipe output to System.out/System.err as needed.

Problem is that now we've introduced a dependency.

Hand-waving how you get the dependency¹, the command you need to run the code changes from java scripts/Main.java to java --module-path scripts/libs --add-modules ALL-MODULE-PATH scripts/Main.java.

That's simply too much to remember.

Argument Files

To deal with this we can use argument files.

If we make a file called project at the top level of our project with the following contents.

--module-path scripts/libs --add-modules ALL-MODULE-PATH

Now we can run the script with just java @project. This works as if all the arguments in the file were applied inline in the invocation. Most tools that come with Java, including the java launcher itself, support this.

picocli

As for identifying groups of commands, there is a solution there too. Now that we've opened the floodgates on our Java build script having dependencies, what's one more?

import dev.mccue.tools.ExitStatusException;
import dev.mccue.tools.Tool;
import picocli.CommandLine;

@CommandLine.Command(
        name = "project"
)
public final class Project {
    public static void main(String[] args) {
        new CommandLine(new Project()).execute(args);
    }

    @CommandLine.Command(name = "demo")
    public void demo() throws ExitStatusException {
        Tool.ofSubprocess("javac")
                .run("--version");
    }
}

If we use picocli, then it's trivial. Our build script is a CLI program like any other, why not use normal CLI libraries?

We get the ability to run commands by name.

$ java @project demo  
javac --version
javac 22.0.1

And we even get to list commands in a way.

$ java @project                                                               
Missing required subcommand
Usage: project [COMMAND]
Commands:
  demo

Yay!

Tool Tailored APIs

While Tool.ofSubprocess("javac").run("--version"); is complete in a sense, it's not very fun to use.

What we generally want from a Java API is method-level autocomplete. Having to separately reference a man page doesn't spark joy in me, and it shouldn't spark joy in you.

I started this particular adventure wanting to translate options from the CLI more or less 1-1. This is for two broad reasons.

1. I think learning how to use the CLI from Java should be transferable knowledge when writing commands the old fashioned way and vice-versa.
2. There are a lot of CLI tools and coming up with a creative name for every argument that can only be specified as -g is painful and a lot of work.

The transform I started with was for every argument that --looks-like-this I would add a method to an arguments object that looksLikeThis.

Javadoc.run(arguments -> {
    arguments
        .moduleSourcePath("./modules/*/src")
        .d(Path.of("build/javadoc"))
        .module("dev.mccue.tools")
});

This works pretty well, but look at these two options from javadoc.

javadoc --help
...
    --version     Print version information
...
    -version      Include @version paragraphs
...

It has both -version and --version and they do wildly different things. Great. Awesome.

This is a one-off example, but CLI tools are fundamentally textual apis. --some-thing to someThing isn't just a stylistic change, it's a lossy transformation.

So I gave up. My only strategy now is to take arguments that --look-like-this and turn them into ones that __look_like_this. It might be ugly, but at least I don't run into strange problems anymore. As a side benefit, it does now look a lot more 1-1 with the CLI api.

Javadoc.run(arguments -> {
    arguments
        .__module_source_path("./modules/*/src")
        ._d(Path.of("build/javadoc"))
        .__module("dev.mccue.tools")
});

Conclusion

I translated the spring demo repo I was using for some previous posts to use this approach. It includes running junit tests and managing multiple modules. You can find it here and the build script specifically here.

Note that the libraries I referenced, save for picocli, are very likely to change in backwards-incompatible ways as I iterate on them. Don't use them for anything serious yet, but you can find them here.

There are still some bootstrap and polish issues, but I think this approach is becoming more and more viable as its chipped away at.

Share thoughts, design feedback, etc. in the comments below.

1: jresolve --output-directory scripts/libs pkg:maven/dev.mccue/tools-jdk@2024.08.25.5.

2: This approach/outlook on tooling has a lot of similarities to bach and the work Christian Stein has been doing. Will likely elaborate more on the difference between this approach, bach's approach, bld's approach, etc. when I personally have more mental clarity on it.

<- Index