The title is a ? -> ! of this recent thread on Reddit and really a continuation of this series of posts (1, 2, 3, 4).
So first I am going to go over a few of the Maven/Gradle alternatives listed in that Reddit thread and explain why they don't really scratch the itch I think we need scratched, give a small status update on what I've been working on, and end with something that you the reader (yes, you!) could help me with.
I am also going to be terse, biased, and perhaps a bit too harsh. Otherwise, this would drag on and not be an entertaining read.
I want there to be a smooth on-ramp from java src/Main.java to running that program with dependencies, packaging it up to share with other people, and making use of the tools that come with Java (jlink, jpackage, etc.) and available in the Java ecosystem (junit, mapstruct, etc.) Ideally there should be a way to split dependencies over the --class-path, --module-path, and all the other paths as well.
Importantly I don't care, and I think the people who most need this on-ramp don't care, about maximal efficiency. Whoopty doo basil, you can incrementally compile a 100 million line codebase. Fart noises.
Easy one out-of-the-way first, Ant. Ant is a cross-platform scripting language with targets. It's just but you write XML.
By itself it doesn't handle dependencies at all and outsources that to Ivy. This is a second XML file and, once you've filled it out, it downloads jars to a folder.
Even if you are an Ant-head there is a problem in that to use Ivy you are expected to be using Ant. Ant makes sense if you start by compiling code with javac, but that's no longer a thing you need to do. So the path from java src/Main.java isn't great.
One thing Ant has going for it is that it is clear how to use built-in tools. It loses points in my book though because the arguments you'd give to its tasks don't match up that closely to the arguments you would pass to the command line tools. For example, compare the javac task to the man page for javac itself.
Ivy also dumps jars into a single folder. This means it isn't exactly easy to put a few dependencies on the --class-path and others on the --module-path.
java src/Main.javaMill is a Scala build tool. Scala is a programming language with a history of breaking changes and higher kinded types.
This is the example Mill program for a "simple Java program."
package build
import mill._, javalib._
object foo extends JavaModule {
def ivyDeps = Agg(
ivy"net.sourceforge.argparse4j:argparse4j:0.9.0",
ivy"org.thymeleaf:thymeleaf:3.1.1.RELEASE"
)
object test extends JavaTests with TestModule.Junit4 {
def ivyDeps = super.ivyDeps() ++ Agg(
ivy"com.google.guava:guava:33.3.0-jre"
)
}
}So very similarly to Ant, this jumps straight into building code. To understand what is happening here you need to understand Scala - an entirely separate programming language from Java. It uses a Task monad, which is like a burrito, to track what tasks depend on other tasks.
It is also very fast, a property I will reiterate I do not care about.
For built-in tools Mill bundles modules, like its JlinkModule, which implicitly give access to the functionality provided. This has a very similar problem to maven plugins in that the distance between "what command is actually run and when" and "what you actually specify" is relatively large.
Let's say you wanted to compile some .xsd files into Java classes with xjc. Perhaps a little niche nowadays, but how would you do it? Are you falling back to ProcessBuilder? Are you learning how the Task monad works?
As for splitting things over multiple paths, a quick look at that JlinkModule shows that its just reusing the --class-path for both the --class-path and --module-path.
val classPath = jars.map(_.toString).mkString(sys.props("path.separator"))
val args = {
val baseArgs = Seq(
Jvm.jdkTool("jmod", this.zincWorker().javaHome().map(_.path)),
"create",
"--class-path",
classPath.toString,
"--main-class",
mainClass,
"--module-path",
classPath.toString,
outputPath.toString
)
val versionArgs = jlinkModuleVersion().toSeq.flatMap { version =>
Seq("--module-version", version)
}
baseArgs ++ versionArgs
}java src/Main.javabld is the build tool from the Rife2 people. I've written promo material for it before and like to think appreciate it for what it is. That being said -
One problem is that it copies the maven directory layout and, to an extent, conceptual model. The education path gap issues that exist with Maven mandating src/Main.java -> src/main/java/Main.java are equally present. Same goes for the artifact focus, scopes, etc.
It very much is a build tool and, while that is in its tech-startup-de-voweled name, it's not the aspect we care about.
// User model is pretty similar to maven, just with Java instead of XML
public class MyAppBuild extends Project {
public MyAppBuild() {
pkg = "com.example";
name = "my-app";
mainClass = "com.example.MyApp";
version = version(0,1,0);
downloadSources = true;
repositories = List.of(MAVEN_CENTRAL, RIFE2_RELEASES);
scope(test)
.include(dependency("org.junit.jupiter", "junit-jupiter", version(5,11,4)))
.include(dependency("org.junit.platform", "junit-platform-console-standalone", version(1,11,4)));
}
public static void main(String[] args) {
new MyAppBuild().start(args);
}
}But unlike Mill there is no overarching Task abstraction. bld Operations are almost entirely ignorable if you want to run a tool separately. I'd count that as a clear path. Sure, you're using ProcessBuilder or AbstractProcessOperation, but at least you can "just" call the xjcs of the world and move on with your life.
Like Ivy, dependencies are dumped as just jars in folders. Unlike Ivy, there is explicit support for modules. You can opt to have a dependency be put on the module path and that just gets put in a different folder.
public class ProjectBuild extends Project {
public ProjectBuild() {
pkg = "project";
name = "project";
mainClass = "example.Main";
version = version(0,1,0);
downloadSources = true;
repositories = List.of(MAVEN_CENTRAL, RIFE2_RELEASES);
scope(compile)
// Will be on the --class-path
.include(dependency("commons-io", "commons-io", "2.18.0"))
// Will be on the --module-path
.include(module("com.fasterxml.jackson.core", "jackson-databind", "2.18.3"));
// Takes scopes from maven too
scope(test)
.include(dependency("org.junit.jupiter", "junit-jupiter", version(5,11,4)))
.include(dependency("org.junit.platform", "junit-platform-console-standalone", version(1,11,4)));
}
public static void main(String[] args) {
new ProjectBuild().start(args);
}
}But just because it does better than the other entries on this list doesn't mean I'll give it full points. It still doesn't have an obvious way of filling in any of the other paths you might want when executing a tool such as --processor-module-path, --module-path, -Dsystem.library.path, etc. You can at least get at the command line options (and the JaCoCo extension makes use of that) but it's not trivial to use fill them in via dependency resolution.
java src/Main.javaBach is a build tool built entirely around the assumption that you will be using Java modules. This means the path from java src/Main.java has to include a pit-stop where you add a module-info.java and put your code in a package like java src/somepackage/Main.java. That's at least a path, but it's not the best one.
Unlike the others on the list there is a focus explicitly on the JDK tools. So running different CLI tools is pretty directly supported.
It more or less abdicates getting dependencies to other tools, so it sorta just doesn't solve the core problem I care about. It doesn't have a solution that gets in the way, but it doesn't have a solution at all.
Also bach is lucky I'm not grading on documentation because "what documentation?" It's still a WIP from the author, and it's not like I do any better, but still.
java src/Main.javaSo pottery - it's another "maven, but." This time the "but" is that it has a set of defaults more suited to the author's preferences and it uses a yaml file for config.
parameters:
junit.version: "5.9.1"
snakeyaml.version: "1.33"
chalk.version: "1.0.2"
picocli.version: "4.7.0"
junit.platform.version: "1.9.0"
junit.engine.version: "5.9.1"
artifact:
group: "cat.pottery"
id: "pottery"
version: "0.3.2"
platform:
version: "17"
produces: "fatjar"
manifest:
main-class: "cat.pottery.ui.cli.Bootstrap"
dependencies:
- production: "org.yaml:snakeyaml:${snakeyaml.version}"
- production: "com.github.tomas-langer:chalk:${chalk.version}"
- production: "info.picocli:picocli:${picocli.version}"
- production: "org.junit.platform:junit-platform-launcher:${junit.platform.version}"
- production: "org.junit.jupiter:junit-jupiter-engine:${junit.engine.version}"
- test: "org.junit.jupiter:junit-jupiter-api:${junit.version}"You can make an uberjar, docker image, and native image out of the box. You can't put things on the --module-path though. Its also pretty tightly bundled so its unclear where you would put a hypothetical call to xjc. Its also taking over everything wholesale, so no easy path from java src/Main.java
java src/Main.javajbang got extremely lucky that nobody was already using the name for a porn site. That's worth celebrating.
The way jbang works is that you put a comment at the top of a file with the dependencies you want, then if you launch the program with jbang it will automatically download those dependencies and use them.
///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS ch.qos.reload4j:reload4j:1.2.19
import static java.lang.System.out;
import org.apache.log4j.Logger;
import org.apache.log4j.BasicConfigurator;
import java.util.Arrays;
class classpath_example {
static final Logger logger = Logger.getLogger(classpath_example.class);
public static void main(String[] args) {
BasicConfigurator.configure();
logger.info("Welcome to jbang");
Arrays.asList(args).forEach(arg -> logger.warn("arg: " + arg));
logger.info("Hello from Java!");
}
}So going from java src/Main.java to jbang src/Main.java is no trouble at all. Gold star.
Problems start when you want to put something on the --module-path. Unless I missed something in the documentation (which is possible) that is not possible. It has special carve-out support for JavaFX, but it's not doable in general.
If you want to use these dependencies in other tooling, like jlink, you are equally out of luck.
java src/Main.javajava-jpm is kinda what it says it is - it takes the same approach as npm and just downloads dependencies into a folder. Well it downloads dependencies then symlinks them into a folder but spiritually the same thing.
It just dumps into one folder, so no support for different paths. java src/Main.java -> java -cp deps/* src/Main.java is pretty straight forward as well.
Where I have to dock it points is in the platform-specificness. -cp deps/* isn't something that will work consistently across bash, powershell, fish, etc. Maybe that's unfair, but it's a real problem and none of the other options have it on account of making building CLI arguments outside a specific shell.
java src/Main.javaSo for a while now I've had the jresolve tool in my back pocket. It still has much the same deficiencies it had when I first shared it, sans a few bug fixes.
The tl;dr is that you could run
jresolve pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.3And it would print to standard output a path with all the dependencies on it, separated by the requisite platform specific path separator.
/Users/emccue/.jresolve/cache/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.18.3/jackson-databind-2.18.3.jar:/Users/emccue/.jresolve/cache/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.18.3/jackson-core-2.18.3.jar:/Users/emccue/.jresolve/cache/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.18.3/jackson-annotations-2.18.3.jarNesting commands is very platform specific
# only works in bash
java -cp $(jresolve pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.3) src/Main.javaSo I first added the --output-file argument, which would let you dump the path to a file without anything overly platform specific. You could then use the "argfile" syntax - @filename - to use the resolved path in future commands.
jresolve --output-file libs pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.3
java -cp @libs src/Main.javaBut what became clear very quickly is that I had no clue how to teach an IDE how to read dependencies from a file with a path in it.
So that's when I added --output-directory. You can at least tell an IDE to use all the jars in a folder.
jresolve --output-directory libs pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.3
java -cp libs/* src/Main.javaBut, as is the issue with java-jpm, the libs/* syntax is platform specific. It was then I started becoming biased towards using the --module-path. That at least you could just point to a folder.
jresolve --output-directory libs pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.3
java --module-path libs --add-modules ALL-MODULE-PATH src/Main.javaAnd when the list of dependencies got too big I could put them in an argfile of their own.
jresolve --output-directory libs @libs.txt
java --module-path libs --add-modules ALL-MODULE-PATH src/Main.javaAnd that's where jresolve sat for a bit over a year. I'd keep using it for one-off projects and bashing my head against the "--module-path or --class-path, pick one" problem every time I did.
Even in that state you can bootstrap build programs and I wrote libraries to support that goal. With a tasteful enough use of argfiles and picocli you could get it down to
jresolve @bootstrap
java @project compile
java @project testAnd that felt good, but problems remain. For one, the split path issue. For two, those libs.txt files kinda suck. It got me thinking about how Python started with requirements.txt and eventually that mutated into pyproject.toml.
So that's where I went with it. In the newest version of jresolve if you run jresolve install it will look for a jproject.toml
[project]
defaultUsage="--class-path"
[[project.dependencies]]
coordinate="pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.2"
[[project.dependencies]]
coordinate="pkg:maven/org.junit.jupiter/junit-jupiter-api@5.11.4"
dependencySets=["test"]
[project.dependencySets.test]
extends="default"And this will dump out argfiles in a predictable structure. So with the jproject.toml above you will end up with
dependencySets/
default
testWhere each argument file has not only the paths, but the --class-path preceding them. This finally gives a place to specify "this dependency goes on the --module-path."
[project]
defaultUsage="--class-path"
[[project.dependencies]]
coordinate="pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.2"
[[project.dependencies]]
coordinate="pkg:maven/commons-io/commons-io@2.18.0"
usage="--module-path"
[[project.dependencies]]
coordinate="pkg:maven/org.junit.jupiter/junit-jupiter-api@5.11.4"
dependencySets=["test"]
[project.dependencySets.test]
extends="default"It would even let you do exotic things like put libraries on the -Dsystem.library.path
[project]
defaultUsage="--class-path"
[[project.dependencies]]
coordinate="pkg:maven/com.fasterxml.jackson.core/jackson-databind@2.18.2"
[[project.dependencies]]
coordinate="pkg:maven/commons-io/commons-io@2.18.0"
usage="--module-path"
[[project.dependencies]]
coordinate="SDL/build"
usage="-Dsystem.library.path"
[[project.dependencies]]
coordinate="pkg:maven/org.junit.jupiter/junit-jupiter-api@5.11.4"
dependencySets=["test"]
[project.dependencySets.test]
extends="default"And running your program goes from java src/Main.java to java @dependencySets/default src/Main.java, perhaps with a --add-modules ALL-MODULE-PATH.
Now all I wish is that the java launcher itself accepted "nested" argfiles. If it did, we could put @dependencySets/default src/Main.java into a file named run and get java @run. Or have a dependency set for your build program and get java @project build. In that last situation you could slot in whatever build program you want or need. But that's where I am going to pause for the moment.
If you want to try any of that out you can run this command on Mac/Linux to get the tool.
bash < <(curl -s https://raw.githubusercontent.com/bowbahdoe/jresolve-cli/main/install)If you are on Windows then you can download the jar from the latest release and use java -jar jresolve.jar.
I still have no clue how to teach an IDE how to read dependencies from an argument file. I've had several false starts on making an IntelliJ plugin to do so. The closest I've gotten is with the Oracle extension for VSCode, but that has issues of its own
If you or someone you know has the knowledge required to pull that off please do.
Separately, if you can think of a convincing reason for the java launcher to support nested argfiles beyond it just making me happy file an issue in support of that. Also, there are tools like jshell which don't support argfiles for no reason in particular.
And one thing that seems to have happened in the Python world after the introduction of pyproject.toml is that a whole universe of tools started looking in that file for their own config. I don't know which is the chicken and which is the egg, but whatever goes in a jproject.toml it might be nice for a variety of things to use it for their config.
I also wrote this in a daze on a Sunday afternoon, there's likely things I missed or described wrong. I'll try to respond to comments below or wherever it ends up being shared.
EDIT: I've made some edits to this since I originally shared it. The older, somewhat meaner, text is on the wayback machine.