How to include custom ClassLoader coverage in EclEmma reports?

Issue

I have a number of classes that statically initialize constants based on system arch properties. To fully test them, I have used a custom class loader to reload the classes from files after changing properties. However EclEmma reports miss the coverage of classes loaded this way.

I am able to manually show coverage by adding Jacoco instrumentation in the class loader:

public class TestFlags {
    public static final int O_RDONLY = 0x0;
    public static final int O_WRONLY = 0x1;
    public static final int O_CREAT;
    public static final int O_TRUNC;
    
    public TestFlags() {}
    
    static {
        if (CoverageTest.isMac) {
            O_CREAT = 0x200;
            O_TRUNC = 0x400;
        } else {
            O_CREAT = 0x40;
            O_TRUNC = 0x200;
        }
    }
}

import java.io.IOException;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.jacoco.core.analysis.*;
import org.jacoco.core.data.*;
import org.jacoco.core.instr.Instrumenter;
import org.jacoco.core.runtime.*;
import org.junit.Test;

public final class CoverageTest {
    public static boolean isMac;

    @Test
    public void testFlagCoverage() throws Exception {
        try (Coverage cov = new Coverage(TestFlags.class)) {
            isMac = true;
            cov.instantiate(TestFlags.class);
            isMac = false;
            cov.instantiate(TestFlags.class);
            cov.showCoverage(true);
        }
    }

    /**
     * Captures coverage for specified classes only.
     */
    public static class Coverage implements AutoCloseable {
        private static final Map<Integer, String> lineStatusMap = Map.of(ICounter.NOT_COVERED, "-",
            ICounter.PARTLY_COVERED, ".", ICounter.FULLY_COVERED, "+");
        private final IRuntime runtime = new LoggerRuntime();
        private final Instrumenter instrumenter = new Instrumenter(runtime);
        private final RuntimeData data = new RuntimeData();
        private final Set<String> coverageClassNames;

        public Coverage(Class<?>... coverageClasses) {
            try {
                coverageClassNames = Stream.of(coverageClasses).map(Class::getName)
                    .collect(Collectors.toUnmodifiableSet());
                runtime.startup(data);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }

        @Override
        public void close() {
            runtime.shutdown();
        }

        /**
         * Use a new instrumenting class loader to instantiate the class.
         */
        @SuppressWarnings("unchecked")
        public <T> T instantiate(Class<T> cls) throws Exception {
            return (T) new InstrumentingClassLoader().loadClass(cls.getName()).getConstructor()
                .newInstance();
        }

        public void showCoverage(boolean lines) throws IOException {
            var exec = new ExecutionDataStore();
            data.collect(exec, new SessionInfoStore(), false);
            var cov = new CoverageBuilder();
            var analyzer = new Analyzer(exec, cov);
            for (var name : coverageClassNames)
                analyzer.analyzeClass(loadClassFile(name), name);
            for (var cc : cov.getClasses()) {
                var counter = cc.getInstructionCounter();
                System.out.printf("%s: %d/%d %d%%\n", cc.getName(), counter.getCoveredCount(),
                    counter.getTotalCount(), Math.round(counter.getCoveredRatio() * 100));
                if (lines) for (int i = cc.getFirstLine(); i <= cc.getLastLine(); i++)
                    System.out.printf("Line %2s: %s%n", i,
                        lineStatusMap.getOrDefault(cc.getLine(i).getStatus(), ""));
            }
        }

        private static byte[] loadClassFile(String className) throws IOException {
            var fileName = className.replace('.', '/') + ".class";
            try (var in = CoverageTest.class.getClassLoader().getResourceAsStream(fileName)) {
                return in.readAllBytes();
            }
        }

        private class InstrumentingClassLoader extends ClassLoader {
            @Override
            public Class<?> findClass(String name) throws ClassNotFoundException {
                try {
                    var bytes = instrumenter.instrument(loadClassFile(name), name);
                    return defineClass(name, bytes, 0, bytes.length);
                } catch (IOException e) {
                    throw new ClassNotFoundException(name, e);
                }
            }

            @Override
            public Class<?> loadClass(String name) throws ClassNotFoundException {
                if (!coverageClassNames.contains(name)) return super.loadClass(name);
                var cls = findLoadedClass(name);
                return cls != null ? cls : findClass(name);
            }
        }
    }
}

With EclEmma, launch VM args show -javaagent:.../jacocoagent.jar=...,output=tcpclient,port=NNNNN. I tried sending the data with a socket and RemoteControlWriter, but it didn't show on the report, perhaps because the session ids don't match.

Is there a way for EclEmma to pick up coverage from custom ClassLoaders?

Solution

After playing around in debug mode, it turns out the agent CoverageTransformer.transform() does get called, but returns null since inclNoLocationClasses is not set, and defineClass() doesn't pass in a ProtectionDomain.

If I pass in TestFlags.class.getProtectionDomain(), the coverage does show up.

Answered By - boot-and-bonnet