Edge tpu(google coral), quantizitation

The 8 bit quantizitation as requirement for google coral is an issue – that makes sense for image’s and text characters (keyboard=8bit), but in my opinion an intelligent structure is in the minimal case a tuple, like two 4 bit (6,8) or (0,5) or better a 8 bit tuple (127,65) or (17,120). That is how i learned about artificial intelligence… a tupel is a minimal link of information to another information (neural network)…
Maybe some problems do not need this “minimal link”, i don’t know exactly.
(maybe image layer to layer acceleration is without link between each pixel-byte (8bit))?

My question is how to make such a tuple like in python or C:

int a = 14
int b = 17
int result = (b<<4) | a
printf(result)

, because the accuracy of 32 bit is very important for my application. I cannot lose it, i would loose to much…like artificial intelligence makes little sense then on this high precision data. Because the error is very small.
I need to bitwise split the 32 bit to 4 * 8 bit int. And then bitwise merge. But it is not possible (as i know) without tuples like (1,2)(2,3)(3,4) to create the structure. Without that structure the 4*8bit int are independent and useless for my application. They must be connected… How on coral api???

There is a method
pycoral.utils.dataset. read_label_file (file_path )

taking entries like 0:cat, 1:dog, 2:snake, 3:imageclass

…maybe it is possible to classify the int8 values as 0:first 1:sesond 2:third 3:fourth?

The edge api is very disappointing…
on a to high “level”…proud of quantizitaion … but in practice… there is no such framework/method to optimize and use the quantizitation for optimization… Anyone feels the same, what i feel? :smiley:

The final question is: Did google failed with it’s 8bit quantizitation, as a nonsense feature?
Did nvidea simply better? :wink:

I think like “quantizitation sounds good, but must be fully implemented”, and it isn’t…

When this limitation is true…google can call it “circumcised api for bloody beginners(don’t use it for professional tasks, you will fail!!!)”

If there is an option, it must be here… and not at the python api (high level)…

there is something like “scale” “quantize” and “dequantize”… templates…

but i will make the structure on my own.
I can simply define a quantized tensor with 3 dimensions, so i will get 3 * 8 bit = 24 bit.

Thanks a lot. But google should offer a function like this…
2d_tensor_quantum = 16_bit_convert(value1 int8, value2 int8)
3d_tensor_quantum = 24_bit_convert(value1int 8,value2 int8,value3 int8)
and with uint8…
2d_tensor_quantum = 16_bit_convert(value1 uint8, value2 uint8)
3d_tensor_quantum = 24_bit_convert(value1uint 8,value2 uint8,value3 uint8)

…but what is better? Better on one dimension?
1d_tensor_16bit = 16_bit_convert(value1 int8, value2 int8)
1d_tensor_24bit = 24_bit_convert(value1int 8,value2 int8,value3 int8)

simply direct…

3d_tensor = 24_bit_from_float(float32 value)
(will loose 32-24=8bit precision, but it is the default for high resolution audio)
1d_tensor = 24_bit_from_int(int32 int24)
(from int == equal weights, = one dimension)

…first…

tf.Tensor([a b c], shape=(3,), dtype=int8)

And after all --multiply the dimensions with weights…pseudocode… vectors…
float32 merge0 = shape0(int8)#vectors
float32 merge1 = shape1(int8) * (-(2^8))
float32 merge2 = shape2(int8) *(-(2^16))

or float24

I know this is not right! Floating point conversion follows other rules… it is pseudo conversion.
Suitable if the boundary is -127 … 128 !!!

for i in dimensions:
    result = merge0[i] + merge1[i] + merge2[i]

…google should fix this common sense of quantization …

I can give a solution for the boundary -127 … 128 (data must be normalized to this range!)

    def samples_to_tensor(file_path):
    sr, wav_file = scipy.io.wavfile.read(file_path)#capable of float32
    one_array = numpy.full((len(wav_file)), 0)
    two_array = numpy.full((len(wav_file)), 0)
    three_array = numpy.full((len(wav_file)), 0)

    for i in range(len(wav_file)):
        float_32 = wav_file[i]
        #the trick
        one = 0
        two = 0
        three = 0
    if float32>=0:
        while (float_32 > 1) and (one < 128):
            one += 1 # weight 1
            float_32 -= 1
        while (float_32 > (1/128) and (two < 128)):
            two += 1 #weight 1/128
            float_32 -= (1/128)
        while (float_32 > (1/16384) and (three < 128)):
            three += 1 # weight 1/16384
            float_32 -= 1/16384
    if float_32<0:   
        while (float_32 < (-1) and (one > (-127))):
            one -= 1#weight 1
            float_32 += 1
        while (float_32 < (-(1/128)) and (two > (-127))):
            two -= 1 #weight 1/128
            float_32 += (1/128)
        while (float_32 < (-(1/16384)) and (three > (-127))):
            three -= 1#weight  1/16384
            float_32 += 1/16384
        one_array[i] = one
        two_array[i] = two
        three_array[i] = three
        #the quantum is useless, as you see..but let us define it:
        #tensor_quantum = tf.Tensor([[one] [two] [three]], shape=(3,1,1), dtype=tf.int8)#from 0 to 256
        # maybe syntax error...
    tensor_summary = tf.Tensor((one_array, two_array, three_array),shape=(3,len(wav_file)), dtype=tf.int8)  

… the crucial point is – a “quantizment must be in a context; we looking at the minimal “template” for an intelligent structure”…
And in my opinion – it is simply an array like np.array([12,23]);
The minimal structure is a sequence of two entries. A tuple.
And the minimal (quantize) datatype is a bit, 0 or 1.
So there is a poor understanding if google introduce such a concept of quantize… of AI…

it cannot be an int8 datatype, but they did. And they simply convert to the nearest value… like if we get a int16 with value 16171 … google drops all precision… and we got int8 value of 126. disappointing concept.

and a tip if you want to buy a google coral:
Anything is deprecated… And it seems like nobody cares, even google.

Maybe an approach for an AI datatype …an ordered pair (set theory)
Hausdorfs definition of an ordered pair is most promising:
(a,b) := {{a,1}{b,2}}

where a and b are distinct objects, different from 1 and 2.
I believe this is the best definition, because it showes, that the order itself is also an mathematical object. distinct from the values a and b.

You know that the order of a sequence is like (pseudo…) (0,1)(1,2)(2,3)(3,4) or…
(0,1)(1,3)(3,2)(2,4), or (0,1)(1,3) | (4,2)(2,4)
the last is a loop…

But which value belongs to (4) ?

… Look at the definition, and we see that this information is necessary, before set up a loop or…another structure…

Look at intel manufacture…the 2 bit pair is maybe useful for the concept of high density AI - chips.
That is how – i think about “quantization”, …

And this is realy smart:
using four different threshold voltages, two bits could be stored per cell.
This could help to make a better AI- circuit cell ,
thus cells are like “quantized” objects…

Another topic: The AI cloud market…

… intels lunar lake chip is as strong as the best nvidea chips… with up to 120 tops.
And i believe on my own experience that intel chips are quiet excellent.
intels public relation is an understanding, i guess. They are technological better than people think…
But maybe not as good in business as others…

…nvideas. blackwell costs 30.000 dollar, if compared to intels solutions, … it is a question of cash…
and nobody knows exactly the benefit of thus ultra high power devices…
precision and excellence is not communicable, it is like elon musks rocket science…
And if we look at the potential speed of quantum computers, it is maybe just a short runner…but never economical scalable.

Quantum computers just need a good/suitable interface – and they will break record for record in the high performance computing field. It will start up like a rocket. And people forget all this high power devices from nvidea…that’s the trick … and you know it is true…

If quantum computers can deliver exaflops for just a penny…0.01USD… in the cloud/rent market…