When using on Android, why do you have to use allocatedirect to create ByteBuffer and ByteBuffer.Order(byteorder. Nativeorder()) this method

TensorFlow General Discussion
, , ,
1

I am now using an Android demo of tensorflow Lite. I want to try to use my model on it, but the input data required by my model is float type, but the floatbuffer does not have allocatedirect, and its order cannot be passed into any method, which leads to the empty output of model recognition. What do I need to do?

This is how I convert mat to floatbuffer

private fun convertBitmapToByteBuffer(mat: Mat): FloatBuffer {

    val floatBuffer = FloatBuffer.allocate((BATCH_SIZE * inputSize * inputSize * PIXEL_SIZE))
    floatBuffer.order()
    val intValues = IntArray(inputSize * inputSize)

    val channels: Int = mat.channels()
    val width: Int = mat.cols()
    val height: Int = mat.rows()
    val data = ByteArray(channels)

    for (i in 0 until width) {
        for (j in 0 until height) {

            mat.get(i, j, data)

            val b: Byte = (data[0] and 0xff.toByte())
            val g: Byte = (data[1] and 0xff.toByte())
            val r: Byte = (data[2] and 0xff.toByte())
            floatBuffer.put(b.toFloat())
            floatBuffer.put(g.toFloat())
            floatBuffer.put(r.toFloat())

        }
    }

    return floatBuffer
}
1 Like
3

Hi, @jun_yin

I apologize for the delayed response, as far I know due to the compatibility issues between Kotlin (Java) Float and TensorFlow Float(float32 or float64 ) datatypes, we need a helper function to convert the Kotlin Float(s) to float buffers before sending them to the TFLite model. We will use the method below

  fun floatArrayToBuffer(floatArray: FloatArray): FloatBuffer? {
        val byteBuffer: ByteBuffer = ByteBuffer
            .allocateDirect(floatArray.size * 4)

        byteBuffer.order(ByteOrder.nativeOrder())

        val floatBuffer: FloatBuffer = byteBuffer.asFloatBuffer()

        floatBuffer.put(floatArray) 
        floatBuffer.position(0)
        return floatBuffer
    }

Please refer this blog, you’ll often need to use a ByteBuffer.allocateDirect() and byteBuffer.order() because ByteBuffer.allocateDirect() will creates a ByteBuffer that is stored in the devices native memory which is faster than using the JVM heap which is useful for large amounts of data or I/O operations.

For byte order compatibility we use byteBuffer.order() because devices may have different byte orders (big-endian or little-endian) so ByteOrder.nativeOrder() ensures that the ByteBuffers byte order matches the devices native byte order.

If the byte order of the ByteBuffer does not match the devices native byte order data corruption can occur when reading or writing binary data so to prevent that we use byteBuffer.order()

If I have missed something here please let me know.

Thank you for your cooperation and patience.

1 Like