IssueTracker

We are currently using AGP internal task types to flag memory-intensive tasks to enforce a reduced parallelism at execution time. I've raised this separately (with a lot more detail) as a feature request (https://issuetracker.google.com/issues/224584845). However, I'm mentioning it here as our ability to implement this will be reduced or possibly eliminated by what is described int he roadmap.

Thanks #comment2 that's helpful context. I've marked that FR as blocking this and triaged it to the right component

Another use case that we have is to reactively respond to the creation of APKs and AABs. The new AGP APIs allow us to connect out tasks into the artifact pipeline via wiredWith but the best we can come up with to receive the completed artifact is to wire in toTransform. This A) does not guarantee that we will receive the final artifact as more transforms may be applied after our task is called, and B) requires us to copy the input property file/dir to our tasks output property file/dir in order to not break the build cache.

The reactive behavior of the above is the complicating factor.

A non-reactive approach could simply depend upon the task name and then look for a hardcoded path in the build directory (which is still sort of gross, since the build output paths are not documented as public API and change from time to time).

Another approach would be to wire a custom task to consume the output of the build via the built artifacts loader feeding an input property. However, this approach cannot be applied reactively. Either the custom task is included in the build and causes the creation of the binary artifact, or it is not included in the build and never gets invoked.

We didn't provide a task wiring helper for that case as there's only one thing to wire, but I can see how the inconsistency can be misleading

That last one (Comment #11) seems like a bug. Jerome?

Ran into another use case that the API does not yet seem to support: AndroidUnitTest configuration for offline Jacoco instrumentation. Given that we've had to tweak task outputs to get this to work reliably with the build cache it is probably best corrected on the AGP side. Probably easiest just to paste the relevant code here:

        /*
         * -Djacoco-agent.destfile arg is used to configure the offline mode behavior of jacoco.  The offline
         * behavior is what is used when dependency module code is executed as it has already been
         * instrumented by the jacoco-agent in previous executions.  We redirect this to record the offline
         * results under the build directory and give it a more explicit/identifiable name (it defaults
         * to the project dir as jacoco.exec).
         *
         * NOTE: Attempts at using JacocoTaskExtension.setDestinationFile were unsuccessful in capturing coverage
         */
        project.tasks.withType(AndroidUnitTest::class.java).configureEach {
            val execFile = project.layout.buildDirectory.file("jacoco/offlineDependencies.exec").get()
            jvmArgs("-Dfile.encoding=UTF-8", "-Djacoco-agent.destfile=${execFile}")

            // Register our file as a task output to ensure it is restored via the build cache when execution is avoided
            outputs.file(execFile)

            doFirst {
                // Make sure the coverage file is removed if it exists from a previous run
                execFile.asFile.deleteRecursively()
            }
        }

And one question:

We have some convention plugin code which is applied to many project module. It uses the components extension's onVariants callback to reactively trigger some data capture but needs to know whether or not minification has been enabled for the build type, configured by a separate convention plugin. We have code similar to the following:

        project.plugins.withId("com.android.application") { plugin ->
            val extension = project.extensions.getByType(ApplicationAndroidComponentsExtension::class.java)
            val android = project.extensions.getByType(ApplicationExtension::class.java)
            extension.onVariants { variant ->
                ...
                if (android.buildTypes.getByName(variant.buildType!!).isMinifyEnabled) {
                    ...
                }

ApplicationExtension feels like more of an input API for AGP and not something we should be programmatically querying within the onVariants callback. Given the exposure of other configuration values (e.g. variant.pseudoLocalesEnabled as Property<Boolean>), should isMinifyEnabled also be exposed?

Alex, can you look at #11 first, then at #13.

#14, yes, it should probably be offered in ApplicationVariantBuilder.

Alex and I looked a bit more carefully and we cannot make APK the result of the bundleToAPK task. The reason is that only one task can produce an artifact type at a time.

You cannot have an artifact type being produced by either the normal APK packaging task or the APKFromBundle task depending on what the user requested. In particular, if any script/plugin request the public APK artifact type, which task is supposed to run ?

The only way we can somehow satisfy #11 would be to have another public artifact type called APK_FROM_BUNDLE which would ensure that the bundle is created first, then the APK from the bundle. would that work ?

We should really file separate bugs for all these comments. We can't just use a single bug for all of this.

I filed the following specific bugs:

• comment #4 -> Issue 232323922
• comment #9 -> Issue 232324065
• comment #11 -> Issue 232325458
• comment #14 -> Issue 232325329

I have not yet filed anything related to Jacoco, as we probably need to discuss things a bit internally first.

We develop a convention plugin which adds some code quality tasks(detekt, checklist, lint) to the build, whenever assemble task is invoked. We achieved this on the old api by obtaining the assemble task provider with BaseVariant.getAssembleProvider() method then adding our tasks as a dependency to it. However, I couldn't find any equivalent method in the new Variant API. I thought of registering a custom Default task, which depends on SingleArtifact.APK and all of our other custom tasks, then host apps could use this new task on their local and CI machines to create apk. However, this method is not optimal since some developers still can run the assemble task directly and bypass our code quality tasks and also, this new task will not be executed when developers run their project on the Android Studio by default. So I would like a way to include some tasks to the artifact creation even if they do not produce or use this artifact. What I can suggest is a way to make artifacts depend on tasks similar to how tasks can depend on artifacts. For instance, something similar to the following would solve our use case:

variant.artifacts.get(SingleArtifact.APK).getTaskProvider().configure {
    it.dependsOn("detekt${variant.name.capitalize()}")
}

Currently, what we are doing to solve this problem is the following:

project.extensions.getByType(AndroidComponentsExtension::class.java).onVariants { variant ->
    project.afterEvaluate {
        project.tasks.named("assemble${variant.name.capitalize()}").configure {
            it.dependsOn("detekt${variant.name.capitalize()}")
        }
    }
}

which I know is not recommended. Therefore, could you consider adding a new api for this case?

Another thing is I couldn't find a way to obtain "lint" task for the current variant without tasks.named("lint${variant.name.capitalize()}"). Could you also create a method to obtain lint task for the current variant, add dependency tasks for it. Adding dependency for lint is important for us since we download our lint configuration with a custom task and we want this configuration to be ready when "lint" needs it. (This case could also be solved by using Provider API in the lint block of the finalizeDsl, however currently the "lintConfig" property is declared as File in this block)

And lastly, it would be nice if InternalArtifactType.JAVA_DOC_DIR was a public artifact. If our convention is applied to a library project, whenever we publish the aar, we also publish its javadoc(and kdoc) to a remote server. For this we need to add custom Javadoc task and configure it properly for the variant. However, I think, this process is a bit complex. I could not properly configure the custom Javadoc task using the new variant api. With the old API I was using the following configuration:

source = variant.getJavaCompileProvider().get().source
classpath += project.files(project.provider { androidExtension.bootClasspath })
classpath += project.files(variant.getJavaCompileProvider().map { it.classpath })

Therefore, making InternalArtifactType.JAVA_DOC_DIR public would greatly simplify our implementation and solve our problems. Currently, only solution I found was to add dependency to "javaDoc${variant.name.capitalize()}Generation" task and hardcode its output path as "intermediates${File.separator}java_doc_dir${File.separator}$componentName". But I know this is really fragile and would love to see an easier and more conventional way to accomplish this.

I'm developing a plugin for external sources compilation into *.so files.

How can I inject final *.so files into the final APK/AAR? I saw that AGP has AndroidArtifacts.ArtifactType.JNI{_SHARED} But I have no idea how to use it and can't find any samples. I also can't put SingleArtifact.APK because it is transformable but not appendable.

variant.artifacts.use(task)
    .wiredWith { it.outputSoFolder }
    .toAppendTo(...) // <-- what to put here?

EXTRA: how to add their debug symbols to LLDB during debugging from the plugin (same as Makefiles/CMake does).

AndroidArtifacts.ArtifactType is internal and not meant to be used by our API. The thing to pass to toAppendTo would have to be a MultipleArtifact but we don't expose many of them yet, and none that are useful for your use case.

At some point we may expose the intermediate artifact that is the final folder of all the .so files, but that may not be what you want either. If it's the final folder, then it's a single folder, so you cannot append to it, and you can only transform it (which means taking the content, processing it, and writing the output in a different folder). This is not efficient when you want to add new files to it.

So, your use case actually is better positioned to use sourcesets rather than inject in an intermediate. We recently introduced Component.sources which gives access to the different sources for a variant, and allows adding build-generated ones. This is the entry point you need.

I just ran into a need to modify android test manifests in my convention plugin. I intended to use the artifacts API to do this but it looks like there is no SingleArtifact.* making this available. As per the AGP 7.2 docs on MERGED_MANIFEST: "For each module, unit test and android test variants will not have a manifest file available.". Can we please get these added as well?

We are currently using the old AndroidSourceSet APIs to configure Checkstyle and Detekt for Android projects, as described in this issue: https://issuetracker.google.com/issues/170650362

As far as I can tell, this is not yet covered by the new APIs and this would likely apply to other static analysis tools that need to process source files as well.

My project currently uses the javaCompileProvider and preBuildProvider APIs of the BaseVariant class.

Fir the javaCompileProvider case, we use this mainly in Application projects. We have some custom code generation tasks that we run that require the classpath of the application be available (so we use the output of the task), and that we want to be done "at the same time" before tasks that depend on JavaCompile complete have the code we generate ready as well. Example usage is:

android.applicationVariants.configureEach { variant ->
  val customTaskOne = project.tasks.register("customTaskOne${variant.name.capitalized()") {
    dependsOn(variant.javaCompileProvider, kotlinCompileTask)
  }

  val javaCompileOutput = variant.javaCompileProvider.get().destinationDirectory.get().asFile
  val codeGenerationAction = CodeGenAction(javaCompileOutput, variant)

  variant.javaCompileProvider.configure {
    finalizedBy(customTaskOne)
    doLast(codeGenerationAction)
  }
}

I understand this is a bit janky, but I'm looking to update the code and make sure we're doing things "The Right Way(tm)" going forward. If there's a similar way to accomplish what we're looking for, I'd love to know about it, especially if it's a tool or technique that I'm not familiar with.

For the preBuildProvider usecase, we're essentially generating some code early on in the process that just needs to be ready. I can likely use some other method of having this task run (as it doesn't really depend on anything else other than being done before the APK is packaged). What would be the suggested way with the new gradle-api classes to perform this sort of work?

EDIT: I've spent the last few days looking through the Android Plugin Cookbook and found some stuff that looks like it's close to working, but doesn't quite let me solve the problem I'm trying to solve. To summarize the above, I have a set of tasks that I use to essentially scan the classpath and identify classes tagged with certain things (Annotations, Interfaces) and then generate either a new .class file for inclusion in the dex or generate a JSON file for inclusion in the application's assets.

For reference, all of this is with AGP 7.4.

I tried using the same task for both to see if AGP would know how to handle that:

                variant.artifacts
                    .forScope(ScopedArtifacts.Scope.ALL)
                    .use(scannerTask)
                    .toAppend(
                        ScopedArtifact.CLASSES,
                        ScannerTask::output
                    )

                variant.artifacts.forScope(ScopedArtifacts.Scope.ALL)
                    .use(scannerTask)
                    .toGet(ScopedArtifact.CLASSES,
                        ScannerTask::allJars,
                        ScannerTask::allDirectories)

but that led to things just not executing. I didn't see anything with the task name in the --debug output. So, I went ahead and tried using two tasks: one for toGet that would scan all the input and generate the .class file, and one that would then take that output of that scan task and then add it using toAppend. Attempting to do this led to a circular dependency, leading me to believe that toGet is ALWAYS executed last.

So, I went ahead and tried using toTransform:

variant.artifacts.forScope(ScopedArtifacts.Scope.ALL)
            .use(scannerTask)
            .toTransform(
                ScopedArtifact.CLASSES,
                ScannerTask::allJars,
                ScannerTask::allDirectories,
                ScannerTask::output
            )

And that worked! The class was generated, and included in the dex file. The problem was that the API was expecting me to essentially touch every input file and then add them to the output. That sounds like it's going to kill my build times.

Am I on the right track here and maybe just missing an API to use? Or is this use case not supported by the current APIs?

I'm sure I could get that to work, having a sort of incremental toTransform, though I don't think that would be the ideal solution for my particular use case. My attempt to use toTransform was based off it allowing me to essentially take a look at every class that's going to end up in the package. I'm not looking to actually modify any of the classes that I scan. With the API you suggested, we would essentially be marking everything we look at as "unchanged", and then attempting to add a new class/asset based on what we saw, or perhaps modifying one or two classes/assets by adding the results of our scans.

My end goal is to essentially "append" to the output using everything previously built as an input. So, maybe not "append", but almost "finalize". That's why we currently use the finalizedBy and doLast APIs for the JavaCompile task. I think my big concern would be using the "transform" API in a way that isn't strictly "transforming". If that's not one that you share, then this certainly would be worth trying out.

But in the meantime, maybe using a KSP or plain old annotation processor might be another solution, not exactly sure about your constraints.

That's certainly one of the avenues I'm investigating. I'm just trying to make sure I've taken a look at and understand all of the options that are available.

The problem of a finalize API is that only one thing can do it. If we expose this as a proper API, then we have to make sure only 1 plugin can do it and fail if 2 plugins try to do it. If we start having several published plugins using that APIs, they will not be compatible with each other.

Oh yeah, I absolutely understand the turmoil adding an API like that can cause, especially down the line. I don't blame you at all for not wanting to codify that potential nightmare in the public API. If the "transform" API is built and maintained in such a way that it accounts for folks not always actually wanting to transform classes, I think that would be sufficient.

you have to realize that the API cannot guarantee that you are last.

That's okay. No external dependencies should be using these, as they're strictly internal. We also don't need to worry about other internal plugins using/generating classes that we would be required to scan. We don't need to be "last" as much as we need to be "after compilation but before packaging", which this API would provide us.

I have been thinking about this a bit more and it's actually not easy to provide an API where you can identify some untouched inputs as outputs.

The main reason is that Gradle will complain if 2 tasks output the same file/directory, so one way of another, we must copy the inputs into outputs which is probably what you already do. At least we would save the merging step which is an improvement but there would still be a fair amount of I/O.