Accessing the Image Processing Pipeline in Android HAL3 / Camera2 from API

Issue

Is there a way to access the Image Processing pipeline in Android HAL3 / Camera 2 and inject your own algorithms ? For instance, in the HAL implementation as shown below and described here -- can developers inject their own algorithms for say color correction / noise reduction or maybe custom 3A algorithms ? Are there API level access endpoints to these either from Camera2 or NDK Camera API ?

Solution

No. The implementation of camera ISPs varies a great deal, including the internal representation of the data at the hardware level, so having a common interface to plug in custom algorithms would be challenging.

In addition, many of the stages are either completely or partially implemented in hardware fixed-function blocks directly chained together, with limited configurability, so replacing them is not trivial from that perspective either.

However, if the camera device supports RAW output, you can write your own complete processing pipeline if you want; it's unlikely to be able to operate fast enough (or power-efficiently enough) for running the viewfinder, but certainly that can be done for still captures.

In terms of 3A control, if the camera device supports the MANUAL_SENSOR capability, you can control the exposure parameters at the application level. The primary challenges there are not being able to use the ISP-generated statistics to speed up scene analysis for exposure, focus, and white-balance, and the slower feedback loop compared to the in-ISP algorithms.

Answered By - Eddy Talvala