Maven

my life is short!

Peter Palaga

@ppalaga

Peter Palaga

↘     ↙

Camel Quarkus

Projects I build often

¹⁾ cloc --include-lang=Java .
²⁾ mvn clean && find . -type f -name 'pom.xml' | wc -l

Agenda

• Maven lifecycle
• Skipping mojos
• mvnd a.k.a. Maven Daemon
• Vertical scaling
• Incrememental builds with GIB

Maven lifecycle

-DskipTests

Why to skip mojos?

• After pulling from team git repository
• Source tree in a clean state
• Checked by the team CI
• Tests, and various source checks can be skipped

Find candidates for skipping

async-profiler or any other Java profiler

• export MAVEN_OPTS="-agentpath:/path/to/libasyncProfiler.so=start, event=cpu,file=mvn-profile.html"
• Outputs a flame graph
• Summed mojo execution times are easy to see

All possible skips for this project

$ mvn clean install \
    -DskipTests \
    -Dformatter.skip \
    -Dimpsort.skip \
    -Denforcer.skip \
    -Dcamel-quarkus.update-extension-doc-page.skip \
    -Dskip.installyarn \
    -Dskip.yarn \
    -Dlicense.skip \
    -Dquarkus.build.skip
    -Dgmaven.execute.skip # underway https://github.com/groovy/gmaven/pull/27
...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:34 min  # down from 23:27 min by mvn clean install -DskipTests
    

public class SomeMojo extends AbstractMojo {

    @Parameter(property = "some.skip", defaultValue = "false")
    boolean skip;

    @Override
    public void execute() {
        if (skip) {
            getLog().info("Skipping as requested by the user");
            return;
        }
        ...
    }
}

• Resolve and download plugin dependencies
• Setup a classloader, load the necessary classes
• Instantiate the Mojo
• Inject fields (potentially expensive objects)
• Call Mojo.execute()

Remove plugins
from Maven execution plan
altogether

<project>
    ...                      <!-- Proper skipping -->
    <build>
        <plugins>
            <!-- Only plugins essential for the fast build -->
            <plugin><artifactId>maven-compile-plugin</artifactId></plugin>
            <plugin><artifactId>maven-jar-plugin</artifactId></plugin>
            <plugin><artifactId>maven-install-plugin</artifactId></plugin>
            ...
        </plugins>
    </build>
    <profiles>
        <profile>
            <id>full</id>
            <activation>
                <property>
                    <name>!quickly</name><!-- Active by default unless -Dquickly is passed -->
                </property>
            </activation>
            <build>
                <plugins>
                    <!-- All other plugins -->
                    <plugin><artifactId>maven-enforcer-plugin</artifactId></plugin>
                    <plugin><artifactId>maven-surefire-plugin</artifactId></plugin>
                    <plugin><artifactId>maven-failsafe-plugin</artifactId></plugin>
                    <plugin><artifactId>groovy-maven-plugin</artifactId></plugin>
                    ...
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

$ mvn clean install -Dquickly
...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:01 min  # down from 04:34 min by
                               #
                               #   mvn clean install -DskipTests -Dformatter.skip ...
                               #

Java Warmup costs

• JVM boot
• JIT (Just in Time) compilation

Cost of starting Maven

Measurable through a simple experiment:

package org.apache.maven;
public class DefaultMaven implements Maven {
    ...
    private MavenExecutionResult doExecute( MavenExecutionRequest request ) {
        System.out.println("End of Maven init: " + System.currentTimeMillis());
        request.setStartTime( new Date() ); // Maven measures from here
    }
    ...
}

$ echo $(($(date +%s%N)/1000000)) && mvn clean install -Dquickly
1634117024986
...
End of Maven init: 1634117025765
$ echo "$((1634117025765-1634117024986))"
779 # Maven start time in milliseconds

Cost of JIT compilation

Eliminate JVM warm up

• Have a long living Java process, a.k.a. daemon
• Pass build requests through a socket
• Like Gradle daemon, but for Maven

- the Maven Daemon

https://github.com/apache/maven-mvnd

• Started by Guillaume Nodet in 2019
• 26 releases since then
• Donated to the ASF Maven Project in December 2021

mvnd - how to install

• ZIP file
• SDKMAN!
• Homebrew
• MacPorts
• Chocolatey
• ASDF

Maven Daemon overview

mvnd demo

$ git clone camel quarkus
$ cd camel-quarkus
$ mvnd install -Dquickly
...
# Try CTRL B, + and - keystrokes
# Check mvnd --status and mvnd --help in another terminal
$ mvnd install -Dquickly
...
$ mvnd install -Dquickly
...

mvnd speed gains

mvnd - Parallel builds

• Default number of threads:    Runtime.getRuntime().availableProcessors() - 1
• -T<n> or -T<n>C supported like with stock Maven

Maven builder

• Pluggable builder since Maven 3.2.1
• A strategy for scheduling and building modules
• singlethreaded (default)
• multithreaded (with -T)

mvnd - Smart builder

• Custom builder provided by mvnd
• Based on Takari Smart Builder

Hidden dependencies

•     A
     / \     Maven module dependencies
    B   C    (Lower depends on upper)
     \ /
      D
  

  No issues with serial builder
• Possible issues with a parallel builder:
  • C could be reading a file in B's target folder
  • C's test could dynamically read an artifact produced by B from the local Maven repository

Simple remedy: -1/--serial
to force the serial builder

Make hidden dependencies explicit

                                    <!-- Add this in C -->
    org.my-group
    B
    ${project.version}
    pom
    test
    
        
            *
            *
        
    

This won't add any real dependency to C but it will guarantee that B is fully built before C

Broken plugins

Plugins may do nasty things

• Mutable global state
• Race conditions

-1/--serial may help

Better: Report/fix the issue in the given plugin

mvnd drawbacks

• Blocks a few gigs of RAM (configurable)
• First official build (0.8.2) for Apple M1 these days
• Windows issues and testing catching up gradualy

Vertical scaling

Laptop vs. desktop

Costs laptop vs. desktop

*) Measured with a budget power meter from a hobby market
**) ENGIE single rate                                                                          

Wrap up

Techniques to speed up Maven builds

• Skip unessential mojos
• Maven daemon to keep the building JVM warm
• More CPU cores & RAM

Links

• https://github.com/mvndaemon/mvnd
• https://peter.palaga.org/blog.html
• https://github.com/gitflow-incremental-builder/gitflow-incremental-builder