bazelbuild/rules_jvm_external: Bazel rules to resolve, fetch and export Maven ar ...

原作者: [db:作者] 来自: 网络收藏邀请

开源软件名称（OpenSource Name）：

bazelbuild/rules_jvm_external

开源软件地址(OpenSource Url)：

https://github.com/bazelbuild/rules_jvm_external

开源编程语言(OpenSource Language)：

Starlark 91.7%

开源软件介绍(OpenSource Introduction)：

rules_jvm_external

Transitive Maven artifact resolution and publishing rules for Bazel.

rules_jvm_external

Features

WORKSPACE configuration
JAR, AAR, source JARs
Custom Maven repositories
Private Maven repositories with HTTP Basic Authentication
Artifact version resolution with Coursier
Integration with Bazel's downloader and caching mechanisms for sharing artifacts across Bazel workspaces
Pin resolved artifacts with their SHA-256 checksums into a version-controllable JSON file
Versionless target labels for simpler dependency management
Ability to declare multiple sets of versioned artifacts
Supported on Windows, macOS, Linux

Get the latest release here.

Prerequisites

Bazel 4.0.0 and above

Support for Bazel versions before 4.0.0 is only available on rules_jvm_external releases 4.2 or earlier.

Usage

List the top-level Maven artifacts and servers in the WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "4.2"
RULES_JVM_EXTERNAL_SHA = "cd1a77b7b02e8e008439ca76fd34f5b07aecb8c752961f9640dea15e9e5ba1ca"

http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

rules_jvm_external_deps()

load("@rules_jvm_external//:setup.bzl", "rules_jvm_external_setup")

rules_jvm_external_setup()

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "junit:junit:4.12",
        "androidx.test.espresso:espresso-core:3.1.1",
        "org.hamcrest:hamcrest-library:1.3",
    ],
    repositories = [
        # Private repositories are supported through HTTP Basic auth
        "http://username:password@localhost:8081/artifactory/my-repository",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],
)

Credentials for private repositories can also be specified using a property file or environment variables. See the Coursier documentation for more information.

rules_jvm_external_deps uses a default list of maven repositories to download rules_jvm_external's own dependencies from. Should you wish to change this, use the repositories parameter:

rules_jvm_external_deps(repositories = ["https://mycorp.com/artifacts"])
rules_jvm_external_setup()

Next, reference the artifacts in the BUILD file with their versionless label:

java_library(
    name = "java_test_deps",
    exports = [
        "@maven//:junit_junit",
        "@maven//:org_hamcrest_hamcrest_library",
    ],
)

android_library(
    name = "android_test_deps",
    exports = [
        "@maven//:junit_junit",
        "@maven//:androidx_test_espresso_espresso_core",
    ],
)

The default label syntax for an artifact foo.bar:baz-qux:1.2.3 is @maven//:foo_bar_baz_qux. That is,

All non-alphanumeric characters are substituted with underscores.
Only the group and artifact IDs are required.
The target is located in the @maven top level package (@maven//).

API Reference

You can find the complete API reference at docs/api.md.

Pinning artifacts and integration with Bazel's downloader

rules_jvm_external supports pinning artifacts and their SHA-256 checksums into a maven_install.json file that can be checked into your repository.

Without artifact pinning, in a clean checkout of your project, rules_jvm_external executes the full artifact resolution and fetching steps (which can take a bit of time) and does not verify the integrity of the artifacts against their checksums. The downloaded artifacts also cannot be shared across Bazel workspaces.

By pinning artifact versions, you can get improved artifact resolution and build times, since using maven_install.json enables rules_jvm_external to integrate with Bazel's downloader that caches files on their sha256 checksums. It also improves resiliency and integrity by tracking the sha256 checksums and original artifact urls in the JSON file.

Since all artifacts are persisted locally in Bazel's cache, it means that fully offline builds are possible after the initial bazel fetch @maven//.... The artifacts are downloaded with http_file which supports netrc for authentication. Your ~/.netrc will be included automatically. For additional credentials, add them in the repository URLs passed to maven_install (so they will be included in the generated JSON). Alternatively, pass an array of additional_netrc_lines to maven_install for authentication with credentials from outside the workspace.

To get started with pinning artifacts, run the following command to generate the initial maven_install.json at the root of your Bazel workspace:

$ bazel run @maven//:pin

Then, specify maven_install_json in maven_install and load pinned_maven_install from @maven//:defs.bzl:

maven_install(
    # artifacts, repositories, ...
    maven_install_json = "//:maven_install.json",
)

load("@maven//:defs.bzl", "pinned_maven_install")
pinned_maven_install()

Note: The //:maven_install.json label assumes you have a BUILD file in your project's root directory. If you do not have one, create an empty BUILD file to fix issues you may see. See #242

Note: If you're using an older version of rules_jvm_external and haven't repinned your dependencies, you may see a warning that you lock file "does not contain a signature of the required artifacts" then don't worry: either ignore the warning or repin the dependencies.

Updating `maven_install.json`

Whenever you make a change to the list of artifacts or repositories and want to update maven_install.json, run this command to re-pin the unpinned @maven repository:

$ bazel run @unpinned_maven//:pin

Without re-pinning, maven_install will not pick up the changes made to the WORKSPACE, as maven_install.json is now the source of truth.

Note that the repository is @unpinned_maven instead of @maven. When using artifact pinning, each maven_install repository (e.g. @maven) will be accompanied by an unpinned repository. This repository name has the @unpinned_ prefix (e.g.@unpinned_maven or @unpinned_<your_maven_install_name>). For example, if your maven_install is named @foo, @unpinned_foo will be created.

Requiring lock file repinning when the list of artifacts changes

It can be easy to forget to update the maven_install.json lock file when updating artifacts in a maven_install. Normally, rules_jvm_external will print a warning to the console and continue the build when this happens, but by setting the fail_if_repin_required attribute to True, this will be treated as a build error, causing the build to fail. When this attribute is set, it is possible to update the maven_install.json file using:

# To repin everything:
REPIN=1 bazel run @unpinned_maven//:pin

# To only repin rules_jvm_external:
RULES_JVM_EXTERNAL_REPIN=1 bazel run @unpinned_maven//:pin

Alternatively, it is also possible to modify the fail_if_repin_required attribute in your WORKSPACE file, run bazel run @unpinned_maven//:pin and then reset the fail_if_repin_required attribute.

Custom location for `maven_install.json`

You can specify a custom location for maven_install.json by changing the maven_install_json attribute value to point to the new file label. For example:

maven_install(
    name = "maven_install_in_custom_location",
    artifacts = ["com.google.guava:guava:27.0-jre"],
    repositories = ["https://repo1.maven.org/maven2"],
    maven_install_json = "@rules_jvm_external//tests/custom_maven_install:maven_install.json",
)

load("@maven_install_in_custom_location//:defs.bzl", "pinned_maven_install")
pinned_maven_install()

Future artifact pinning updates to maven_install.json will overwrite the file at the specified path instead of creating a new one at the default root directory location.

Multiple `maven_install.json` files

If you have multiple maven_install declarations, you have to alias pinned_maven_install to another name to prevent redefinitions:

maven_install(
    name = "foo",
    maven_install_json = "//:foo_maven_install.json",
    # ...
)

load("@foo//:defs.bzl", foo_pinned_maven_install = "pinned_maven_install")
foo_pinned_maven_install()

maven_install(
    name = "bar",
    maven_install_json = "//:bar_maven_install.json",
    # ...
)

load("@bar//:defs.bzl", bar_pinned_maven_install = "pinned_maven_install")
bar_pinned_maven_install()

Generated targets

For the junit:junit example, using bazel query @maven//:all --output=build, we can see that the rule generated these targets:

alias(
  name = "junit_junit_4_12",
  actual = "@maven//:junit_junit",
)

jvm_import(
  name = "junit_junit",
  jars = ["@maven//:https/repo1.maven.org/maven2/junit/junit/4.12/junit-4.12.jar"],
  srcjar = "@maven//:https/repo1.maven.org/maven2/junit/junit/4.12/junit-4.12-sources.jar",
  deps = ["@maven//:org_hamcrest_hamcrest_core"],
  tags = ["maven_coordinates=junit:junit:4.12"],
)

jvm_import(
  name = "org_hamcrest_hamcrest_core",
  jars = ["@maven//:https/repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar"],
  srcjar = "@maven//:https/repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3-sources.jar",
  deps = [],
  tags = ["maven_coordinates=org.hamcrest:hamcrest.library:1.3"],
)

These targets can be referenced by:

@maven//:junit_junit
@maven//:org_hamcrest_hamcrest_core

Transitive classes: To use a class from hamcrest-core in your test, it's not sufficient to just depend on @maven//:junit_junit even though JUnit depends on Hamcrest. The compile classes are not exported transitively, so your test should also depend on @maven//:org_hamcrest_hamcrest_core.

Original coordinates: The generated tags attribute value also contains the original coordinates of the artifact, which integrates with rules like bazel-common's pom_file for generating POM files. See the pom_file_generation example for more information.

Outdated artifacts

To check for updates of artifacts, run the following command at the root of your Bazel workspace:

$ bazel run @maven//:outdated

Advanced usage

Fetch source JARs

To download the source JAR alongside the main artifact JAR, set fetch_sources = True in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    fetch_sources = True,
)

Checksum verification

Artifact resolution will fail if a SHA-1 or MD5 checksum file for the artifact is missing in the repository. To disable this behavior, set fail_on_missing_checksum = False in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    fail_on_missing_checksum = False,
)

Using a persistent artifact cache

NOTE: Prefer using artifact pinning / maven_install.json instead. This is a caching mechanism that was implemented before artifact pinning, which uses Coursier's own persistent cache. With artifact pinning and maven_install.json, the persistent cache is integrated directly into Bazel's internal cache.

To download artifacts into a shared and persistent directory in your home directory, set use_unsafe_shared_cache = True in maven_install.

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    use_unsafe_shared_cache = True,
)

This is not safe as Bazel is currently not able to detect changes in the shared cache. For example, if an artifact is deleted from the shared cache, Bazel will not re-run the repository rule automatically.

To change the location of the cache from the home directory, set the COURSIER_CACHE environment variable. You can also use the --repo_env flag to set the variable on the command line and in .bazelrc files:

$ bazel build @maven_with_unsafe_shared_cache//... --repo_env=COURSIER_CACHE=/tmp/custom_cache

This feature also enables checking the downloaded artifacts into your source tree by declaring COURSIER_CACHE to be <project root>/some/directory. For example:

$ bazel build @maven_with_unsafe_shared_cache//... --repo_env=COURSIER_CACHE=$(pwd)/third_party

The default value of use_unsafe_shared_cache is False. This means that Bazel will create independent caches for each maven_install repository, located at $(bazel info output_base)/external/@<repository_name>/v1.

Using a custom Coursier download url

By default bazel bootstraps Coursier via the urls specificed in versions.bzl. However in case they are not directly accessible in your environment, you can also specify a custom url to download Coursier. For example:

$ bazel build @maven_with_unsafe_shared_cache//... --repo_env=COURSIER_URL='https://my_secret_host.com/vXYZ/coursier.jar'

Please note it still requires the SHA to match.

`artifact` helper macro

The artifact macro translates the artifact's group:artifact coordinates to the label of the versionless target. This target is an alias that points to the java_import/aar_import target in the @maven repository, which includes the transitive dependencies specified in the top level artifact's POM file.

For example, @maven//:junit_junit is equivalent to artifact("junit:junit").

To use it, add the load statement to the top of your BUILD file:

load("@rules_jvm_external//:defs.bzl", "artifact")

Full group:artifact:[packaging:[classifier:]]version maven coordinates are also supported and translate to corresponding versionless target.

Note that usage of this macro makes BUILD file refactoring with tools like buildozer more difficult, because the macro hides the actual target label at the syntax level.

Multiple `maven_install` declarations for isolated artifact version trees

If your WORKSPACE contains several projects that use different versions of the same artifact, you can specify multiple maven_install declarations in the WORKSPACE, with a unique repository name for each of them.

For example, if you want to use the JRE version of Guava for a server app, and the Android version for an Android app, you can specify two maven_install declarations:

maven_install(
    name = "server_app",
    artifacts = [
        "com.google.guava:guava:27.0-jre",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

maven_install(
    name = "android_app",
    artifacts = [
        "com.google.guava:guava:27.0-android",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

This way, rules_jvm_external will invoke coursier to resolve artifact versions for both repositories independent of each other. Coursier will fail if it encounters version conflicts that it cannot resolve. The two Guava targets can then be used in BUILD files like so:

java_binary(
    name = "my_server_app",
    srcs = ...
    deps = [
        # a versionless alias to @server_app//:com_google_guava_guava_27_0_jre
        "@server_app//:com_google_guava_guava",
    ]
)

android_binary(
    name = "my_android_app",
    srcs = ...
    deps = [
        # a versionless alias to @android_app//:com_google_guava_guava_27_0_android
        "@android_app//:com_google_guava_guava",
    ]
)

Detailed dependency information specifications

Although you can always give a dependency as a Maven coordinate string, occasionally special handling is required in the form of additional directives to properly situate the artifact in the dependency tree. For example, a given artifact may need to have one of its dependencies excluded to prevent a conflict.

This situation is provided for by allowing the artifact to be specified as a map containing all of the required information. This map can express more information than the coordinate strings can, so internally the coordinate strings are parsed into the artifact map with default values for the additional items. To assist in generating the maps, you can pull in the file specs.bzl alongside defs.bzl and import the maven struct, which provides several helper functions to assist in creating these maps. An example:

load("@rules_jvm_external//:defs.bzl", "artifact")
load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact(
            group = "com.google.guava",
            artifact = "guava",
            version = "27.0-android",
            exclusions = [
                ...
            ]
        ),
        "junit:junit:4.12",
        ...
    ],
    repositories = [
        maven.repository(
            "https://some.private.maven.re/po",
            user = "johndoe",
            password = "example-password"
        ),
        "https://repo1.maven.org/maven2",
        ...
    ],
)

Artifact exclusion

If you want to exclude an artifact from the transitive closure of a top level artifact, specify its group-id:artifact-id in the exclusions attribute of the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact(
            group = "com.google.guava",
            artifact = "guava",
            version = "27.0-jre",
            exclusions = [
                maven.exclusion(
                    group = "org.codehaus.mojo",
                    artifact = "animal-sniffer-annotations"
                ),
                "com.google.j2objc:j2objc-annotations",
            ]
        ),
        # ...
    ],
    repositories = [
        # ...
    ],
)

You can specify the exclusion using either the maven.exclusion helper or the group-id:artifact-id string directly.

You can also exclude artifacts globally using the excluded_artifacts attribute in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    excluded_artifacts = [
        "com.google.guava:guava",
    ],
)

Compile-only dependencies

If you want to mark certain artifacts as compile-only dependencies, use the neverlink attribute in the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact("com.squareup", "javapoet", "1.11.0", neverlink = True),
    ],
    # ...
)

This instructs rules_jvm_external to mark the generated target for com.squareup:javapoet with the neverlink = True attribute, making the artifact available only for compilation and not at runtime.

Test-only dependencies

If you want to mark certain artifacts as test-only dependencies, use the testonly attribute in the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact("junit", "junit", "4.13", testonly = True),
    ],
    # ...
)

This instructs rules_jvm_external to mark the generated target for junit:Junit with the testonly = True attribute, making the artifact available only for tests (e.g. java_test), or targets specifically marked as testonly = True.

Resolving user-specified and transitive dependency version conflicts

Use the version_conflict_policy attribute to decide how to resolve conflicts between artifact versions specified in your maven_install rule and those implicitly picked up as transitive dependencies.

The attribute value can be either default or pinned.

default: use Coursier's default algorithm for version handling.

pinned: pin the versions of the artifacts that are explicitly specified in maven_install.

For example, pulling in guava transitively via google-cloud-storage resolves to guava-26.0-android.

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ]
)