Gr8

Gr8 is Gradle + R8.

Gr8 makes it easy to shadow, shrink, and minimize your jars.

Motivation

Gradle has a very powerful plugin system. Unfortunately, Gradle handling of classpath/Classloaders for plugins has some serious limitations. For an example:

• Gradle always forces its bundled version of the Kotlin stdlib in the classpath. This makes it impossible to use Kotlin 1.5 APIs with Gradle 7.1 for an example because Gradle 7.1 uses Kotlin 1.4 (See compatibility matrix for other versions).
• buildSrc dependencies leak in the classpath. This causes very weird bugs during execution because a conflicting dependency might be forced in the classpath. This happens especially with popular libraries such as okio or antlr that are likely to be used with conflicting versions by different plugins in your build.

By shadowing (embedding and relocating) the plugin dependencies, it is possible to ship a plugin and all its dependencies without having to worry about what other dependencies are on the classpath, including the Kotlin stdlib.

To learn more, read the "Use latest Kotlin in your Gradle plugins" blog post.

Gr8 is mostly focused at Gradle plugins but you can use it to relocate/shrink any library/binary. See Shrinking a Kotlin binary by 99.2% for a good illustration.

Usage

plugins {
  id("org.jetbrains.kotlin.jvm").version("$latestKotlinVersion")
  id("com.gradleup.gr8").version("$gr8Version")
}

dependencies {
  implementation("com.squareup.okhttp3:okhttp:4.9.0")
  // More dependencies here
}

/**
 * Create a separate configuration to resolve compileOnly dependencies.
 * You can skip this if you have no compileOnly dependencies. 
 */
val compileOnlyDependencies: Configuration = configurations.create("compileOnlyDependencies") 
compileOnlyDependencies.extendsFrom(configurations.getByName("compileOnly"))

gr8 {
  val shadowedJar = create("gr8") {
    // program jars are included in the final shadowed jar
    addProgramJarsFrom(configurations.getByName("runtimeClasspath"))
    addProgramJarsFrom(tasks.getByName("jar"))
    // classpath jars are only used by R8 for analysis but are not included in the
    // final shadowed jar.
    addClassPathJarsFrom(compileOnlyDependencies)
    proguardFile("rules.pro")

    // Use a version from https://storage.googleapis.com/r8-releases/raw
    // Requires a maven("https://storage.googleapis.com/r8-releases/raw") repository
    r8Version("8.8.19")
    // Or use a commit
    // The jar is downloaded on demand
    r8Version("887704078a06fc0090e7772c921a30602bf1a49f")
    // Or leave it to the default version 
  }
}

Then customize your proguard rules. The below is a non-exhaustive example. If you're using reflection, you might need more rules

# Keep your public API so that it's callable from scripts
-keep class com.example.** { *; }

# Repackage other classes
-repackageclasses com.example.relocated

# Allows more aggressive repackaging 
-allowaccessmodification

# We need to keep type arguments for Gradle to be able to instantiate abstract models like `Property`
-keepattributes Signature,Exceptions,*Annotation*,InnerClasses,PermittedSubclasses,EnclosingMethod,Deprecated,SourceFile,LineNumberTable

Using Gr8 for Gradle plugins

Using Gr8 to shadow dependencies in Gradle plugin is a typical use case but requires extra care because:

• The java-gradle-plugin automatically adds api(gradleApi()) to your dependencies but gradleApi() shouldn't be shadowed.
• gradleApi() is a multi-release jar file that R8 doesn't support.
• Since the plugins are published, the shadowed dependencies must not be exposed in the .pom/.module files.

To work around this, you can use, removeGradleApiFromApi(), registerTransform() and custom configurations:

val shadowedDependencies = configurations.create("shadowedDependencies")

val compileOnlyDependencies: Configuration = configurations.create("compileOnlyDependencies") {
  attributes {
    attribute(Usage.USAGE_ATTRIBUTE, project.objects.named<Usage>(Usage.JAVA_API))
  }
  attributes {
    attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, FilterTransform.artifactType)
  }
}
compileOnlyDependencies.extendsFrom(configurations.getByName("compileOnly"))

dependencies {
  add(shadowedDependencies.name, "com.squareup.okhttp3:okhttp:4.9.0")
  add(compileOnlyDependencies.name, gradleApi())
  // More dependencies here
}

if (shadow) {
  gr8 {
    create("default") {
      val shadowedJar = create("default") {
        addProgramJarsFrom(shadowedDependencies)
        addProgramJarsFrom(tasks.getByName("jar"))

        proguardFile("rules.pro")
        registerFilterTransform(listOf(".*/impldep/META-INF/versions/.*"))
      }

      removeGradleApiFromApi()
      
      // Optional: replace the regular jar with the shadowed one in the publication
      replaceOutgoingJar(shadowedJar)

      // Or if you prefer the shadowed jar to be a separate variant in the default publication
      // The variant will have `org.gradle.dependency.bundling = shadowed`
      addShadowedVariant(shadowedJar)

      // Allow to compile the module without exposing the shadowedDependencies downstream
      configurations.getByName("compileOnly").extendsFrom(shadowedDependencies)
      configurations.getByName("compileOnly").extendsFrom(compileOnlyDependencies)
      configurations.getByName("testImplementation").extendsFrom(shadowedDependencies)
    }
  }
} else {
  configurations.named("implementation").extendsFrom(shadowedDependencies)
}

Kotlin interop

By default, R8 removes kotlin.Metadata from the shadowed jar. This means the Kotlin compiler only sees plain Java classes and symbols and Kotlin-only features such as parameters default values, extension function, etc... are lost.

If you want to keep them, you need to keep kotlin.Metadata and kotlin.Unit:

# Keep kotlin metadata so that the Kotlin compiler knows about top level functions
-keep class kotlin.Metadata { *; }
# Keep Unit as it's in the signature of public methods:
-keep class kotlin.Unit { *; }

Note

Stripping kotlin.Metadata acts as a compile-time verification that your API is usable in Groovy as it is in Kotlin and might be beneficial.

Java runtime version

You can specify the version of the java runtime to use with systemClassesToolchain:

gr8 {
  val shadowedJar = create("gr8") {
    proguardFile("rules.pro")
    addProgramJarsFrom(configurations.getByName("runtimeClasspath"))
    systemClassesToolchain {
      languageVersion.set(JavaLanguageVersion.of(11))
    }
  }
}

FAQ

Could I use the Shadow plugin instead?

The Gradle Shadow Plugin has been helping plugin authors for years and is a very stable solution. Unfortunately, it doesn't allow very granular configuration and might relocate constant strings that shouldn't be. In practice, any plugin that tries to read the "kotlin" extension is subject to having its behaviour changed:

project.extensions.getByName("kotlin")

will be transformed to:

project.extensions.getByName("com.relocated.kotlin")

For plugins that generate source code and contain a lot of package names, this might be even more unpredictable and require weird workarounds.

By using R8 and proguard rules, Gr8 makes relocation more predictable and configurable.

Could I use the Gradle Worker API instead?

The Gradle Worker API has a classLoaderIsolation mode that can be used to achieve a similar result with some limitations:

• gradle-api and kotlin-stdlib are still in the worker classpath meaning you need to make sure your Kotlin version is compatible.
• classLoaderIsolation leaks memory
• Workers require serializing parameters and writing more boilerplate code.

Are there any drawbacks?

Yes. Because every plugin now relocates its own version of kotlin-stdlib, okio and other dependencies, it means more work for the Classloaders and more Metaspace being used. There's a risk that builds will use more memory, although it hasn't been a big issue so far.