IssueTracker

It looks like we have more directories

›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/arm64-v8a/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/arm64-v8a/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/armeabi-v7a/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/armeabi-v7a/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/x86/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/x86/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/x86_64/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/debug/x86_64/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/arm64-v8a/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/arm64-v8a/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/armeabi-v7a/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/armeabi-v7a/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/x86/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/x86/build.ninja.txt📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/x86_64/build.ninja📋
›../../out/androidx/appsearch/appsearch-local-storage/build/../nativeBuildStaging/cmake/release/x86_64/build.ninja.txt📋

it also considers all of these input to configuration cache

Prefab feature also seems to cause the following to be added as configuration ache inputs

›../../out/androidx/external/libyuv/build/intermediates/prefab_package_header_only/prefab_publication.json/debug📋
›../../out/androidx/external/libyuv/build/intermediates/prefab_package_header_only/prefab_publication.json/release📋
›../../out/androidx/test/ext/junit-gtest/build/intermediates/prefab_package_header_only/prefab_publication.json/debug📋
›../../out/androidx/test/ext/junit-gtest/build/intermediates/prefab_package_header_only/prefab_publication.json/release📋

./gradlew zipTestConfigsWithApks

./gradlew zipTestConfigsWithApks in the androidx project

Do you mean to run from androidx-main/frameworks/support? If so, I did a run but it took too long and failed in the end with some 9 errors. Even if it finished successfully, it is too long for debug. Do we have to run that specific task or it can be reproduced with some lightweight task? If would be more helpful if we can even reproduce it in a sample project instead of using the androidx project.

Jomo, do you know where those cxx related files are being created? any pointers?

The easiest is to probably just look at https://android-build.corp.google.com/builds/pending/P58531861/androidx_incremental/latest/incremental/configuration-cache-reports/78r13m56v7yjbg14vhi2yg2nb/5o4wtl36u55s4qrlzmk4n19px/configuration-cache-report.html

in there see file system entry. all of these files are treated as input to the configuration cache. There should be no entries to ../../out/ once all the plugins are fixed.

Sadly, the report doesnt give you traces (I asked rodrigo from gradle to add it in the future version of gradle), so we only see it is coming from plugin 'com.android.internal.library'

As far as I know, the most effective way to get the traces is to debug a build running with gradle by adding a break point here https://github.com/gradle/gradle/blob/8129d2feae87affd001b82ada7fa57c233f66467/subprojects/core/src/main/java/org/gradle/internal/classpath/Instrumented.java#L367 which is why we need a sample project. Other than that, we have to guess based on path name from the report.

The folder path '../../out/androidx/appsearch/appsearch-local-storage/build/intermediates/cxx' originates from ProjectInfo::getIntermediatesDir() which is deprecated with a message that says "Use the version that returns a provider".

The files mentioned in comment#2 are created in ExternalNativeJsonGenerator. The presence of "build/../nativeBuildStaging" in the path indicates that the output location is customized in build.gradle by setting com.android.build.api.dsl.Cmake::buildStagingDirectory.

The files in comment#3 are related to a Gradle module with C++ depending on another Gradle module with C++. Though I don't see why they would have 'debug' or 'release' post-pended. You can exercise the general module-to-module logic by running ModuleToModuleDepsTest.

A question maybe for Ivan, I thought the tests under com.android.build.gradle.integration.ndk.* would fail if they violated configuration caching (I recall the original push for configuration caching required some of these tests to be temporarily exempted). Is there a new constraint arriving with Gradle 8.1? In other words, are these regressions or have they always existed? It might help narrow down the cause/fix.

Re #8:

if you upgrade androidx to 8.1 and run the following:

./gradlew appsearch:appsearch-local-storage:assembleDebug

you'll get that code path triggered.

There is also the whole gerrit topic you can apply if you want the 8.2-alpha** agp.

Re #9:

Gradle now instruments all file calls (exists(), isDirectory(), etc) and makes them part of configuration cache input if it happens during the configuration. This is a new requirement starting 8.1

Gradle now instruments all file calls (exists(), isDirectory(), etc) and makes them part of configuration cache input if it happens during the configuration. This is a new requirement starting 8.1

In that case, we should be able to find the issues directly by running tests under com.android.build.gradle.integration.ndk.* and failing when configuration caching misses now when it didn't before (I have no idea how to do that though).

Note, this affects the second run, so you need to make sure the tests call for a second build and validate that there is a cache hit on the second run.

Similar checks for lint added here

Thanks for your input Jomo! Do you have some cycles helping us making cxx compatible with the new constraint of config caching?

Running build twice in a row for cxx integration tests like com.android.build.gradle.integration.ndk.* and check if we have a config cache hit using link sounds like a good way to reproduce the problem.

For example, add the following code at here can reproduce part of the problem

        val result = executor.run(":app:build${appBuildSystem.build}Debug[arm64-v8a]")
        result.assertConfigurationCacheHit()

The way recommended by Gradle is to fix is to do the file calls inside ValueSource. AGP has a "helper" class ConfigPhaseFileCreator which is used for that purpose. An example of how we fix file creation issue during config phase is here

Add more details on debugging gradle on top of comment #8:

1. download the latest gradle source code from https://github.com/gradle/gradle/blob/master/CONTRIBUTING.md and open it in intellij
2. create a remote debugger with port 5005 in the gradle project
3. build a sample project from command line with -Dorg.gradle.debug=true --no-daemon
4. run the debugger from the gradle project