Using JaCoCo to Check Test Coverage

The team had no automated test coverage at the time, so the QA team wanted a sense of what manual testing actually covered. That is how this post came about. Plenty of articles online use Android’s instrumentation test framework and start the app from a command. The report is also generated after the launched activity finishes, which in complex scenarios cannot capture function calls on every screen.

The approach here overrides the Application code under a new build type, used only during manual testing, with no impact on existing code. Hope it helps. Sample code for this post is available here: trevorwang/jacoco-manual-coverage

In your project’s buildscripts directory, create a jacoco.gradle file and add the following code

apply plugin: 'jacoco'  // enable jacoco

jacoco {	
    toolVersion = "0.8.3"  // set the jacoco version
}

// If you use Robolectric, you must add the following
tasks.withType(Test) {
    jacoco.includeNoLocationClasses = true
}

In the build.gradle under the app directory, add code to apply the script
```
apply from: '../buildscripts/jacoco.gradle'
```
After running testDebugUnitTest, you will see testDebugUnitTest.exec under app/build/jococo/. Remember that file — we will use it later to generate the report.

Create a jacoco task

The Android gradle plugin generates different variants, so we need a different task per variant to generate reports.

project.afterEvaluate {
    android.applicationVariants.all { variant ->
        def variantName = variant.name
        def testTaskName = "test${variantName.capitalize()}UnitTest"
        tasks.create(name: "${testTaskName}Coverage", type: JacocoReport, dependsOn: "$testTaskName") {
          // TODO implement later
        }
    }
}

After clicking Sync Gradle, the gradle tasks gain two entries, testDebugUnitTestCoverage, testReleaseUnitTestCoverage. Next we add the implementation that generates the report.

Replace the TODO from the previous step with this code

 group = "Reporting"
   description = "Generate Jacoco coverage reports for the ${variantName.capitalize()} build."
 	// set report formats
   reports {
       html.enabled = true
       xml.enabled = true
   }

  // exclude classes we do not want counted
   def excludes = [
           '**/R.class',
           '**/R$*.class',
           '**/BuildConfig.*',
           '**/Manifest*.*',
           '**/*Test*.*',
           'android/**/*.*',
     	  'androidx/**/*.*'
   ]
   // Java class files
   def javaClasses = fileTree(dir: variant.javaCompiler.destinationDir, excludes: excludes)
   // Kotlin files
   def kotlinClasses = fileTree(dir: "${buildDir}/tmp/kotlin-classes/${variantName}", excludes: excludes)
   classDirectories = files([javaClasses, kotlinClasses])
   // source files
   sourceDirectories = files([
           "$project.projectDir/src/main/java",
           "$project.projectDir/src/${variantName}/java",
           "$project.projectDir/src/main/kotlin",
           "$project.projectDir/src/${variantName}/kotlin"
   ])
 	// the file we generated at the beginning
   executionData = files("${project.buildDir}/jacoco/${testTaskName}.exec")

Run the testDebugUnitTestCoverage task and the report shows up in the build directory
With that, we have finished generating a jacoco report.

Now the key step: how do you enable jacoco in the packaged app?

Create a new staging build type

    buildTypes {
        release {...}

        staging {
            initWith(debug)
            matchingFallbacks = ["debug"]
            testCoverageEnabled true    // packages the jacoco runtime into the app
        }
    }

Under src, at the same level as main, create a staging directory

Under staging, create a java directory, and add a StagingApp.kt file in the com.example.staging package, with this code:

package com.example.staging

import android.Manifest
import android.app.Activity
import android.app.Application
import android.os.Bundle
import android.os.Environment
import android.util.Log
import android.widget.Toast
import androidx.fragment.app.FragmentActivity
import com.tbruyelle.rxpermissions2.RxPermissions
import java.io.File
import java.io.FileOutputStream
import java.io.IOException


class StagingApp : Application() {
    override fun onCreate() {
        super.onCreate()
        Log.d(TAG, "StagingApp")
        registerActivityLifecycleCallbacks(object : ActivityLifecycleCallbacks {
            var activitySize = 0
            override fun onActivityPaused(activity: Activity?) {
            }

            override fun onActivityResumed(activity: Activity?) {
                // ask the first activity for SD card directory access
                if (activitySize == 1) {
                    (activity as? FragmentActivity)?.let {
                        val rxPerm = RxPermissions(it)
                        rxPerm.request(Manifest.permission.WRITE_EXTERNAL_STORAGE).subscribe({ result ->
                            if (!result) {
                                Toast.makeText(
                                    it,
                                    "You have to grant the permission to save coverage file",
                                    Toast.LENGTH_SHORT
                                ).show()
                            }
                        }, { e ->
                            e.printStackTrace()
                        })
                    }
                }
            }

            override fun onActivityStarted(activity: Activity?) {
            }

            override fun onActivityDestroyed(activity: Activity?) {
                activitySize -= 1

                if (activitySize <= 0) {
                  //once all activities are destroyed, write the report file
                    generateCoverageReport(createFile())
                }
            }

            override fun onActivitySaveInstanceState(activity: Activity?, outState: Bundle?) {
            }

            override fun onActivityStopped(activity: Activity?) {
            }

            override fun onActivityCreated(activity: Activity?, savedInstanceState: Bundle?) {
                activitySize += 1
            }

        })

    }


    private fun generateCoverageReport(file: File) {
        Log.d(TAG, "generateCoverageReport():${file.absolutePath}")
        FileOutputStream(file, false).use {
            val agent = Class.forName("org.jacoco.agent.rt.RT")
                .getMethod("getAgent")
                .invoke(null)


            Log.d(TAG, agent.toString())
            it.write(
                agent.javaClass.getMethod("getExecutionData", Boolean::class.javaPrimitiveType)
                    .invoke(agent, false) as ByteArray
            )
        }
    }

    fun createFile(): File {
        // under the SD card
        val file = File(Environment.getExternalStorageDirectory(), "jacoco/$DEFAULT_COVERAGE_FILE_PATH")
        if (!file.exists()) {
            try {
                file.parentFile?.mkdirs()
                file.createNewFile()
            } catch (e: IOException) {
                Log.d(TAG, "Exception : $e")
                e.printStackTrace()
            }
        }
        return file
    }

    companion object {
        const val DEFAULT_COVERAGE_FILE_PATH = "jacoco-coverage.ec"
        const val TAG = "StagingApp"
    }
}

Create an AndroidManifest.xml file in the staging directory with this content

<?xml version="1.0" encoding="utf-8"?>
<manifest
        xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.jacocomanual">
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
    <application android:name="com.example.staging.StagingApp"/>

</manifest>

The result looks like this:

jacoco-staging-build-type

In the IDE, select staging under Build Variants
Run the app and install it on a device or emulator, click around a bit, then press back to close every screen. This generates a jacoco/jacoco-coverage.ec file under the SD card directory.
Copy jacoco-coverage.ec into the jacoco folder at the project root

Now modify the jacoco task to generate the final report

            // the file we generated at the beginning
            executionData = files([
                    "${project.buildDir}/jacoco/${testTaskName}.exec",
                    "${rootDir}/jacoco/jacoco-coverage.ec"   // add a second data source
            ])

Run testStagingUnitTest and the report is there

Good stuff, honestly.~~