The past years have seen more and more companies focus on AI (Artificial Intelligence) as a means to distinguish their product from the competition. Uses of AI range from understanding voice commands to recognizing scenarios to executing direct orders, thus rendering it necessary to reduce the friction between a customer and a service. Because of its exploding popularity, AI is now widely misused as a buzzword, and it is about time some system of measuring this functionality is set up.

AnTuTu, known for its popular benchmarking app, has taken it upon itself to provide a quantifiable standard for everyone to judge the difference in AI performance of different platforms. To set the foundation stones for this purpose, AnTuTu has worked with chip manufacturers to release a benchmarking app called “AI Review” that focuses on measuring the AI performance of smartphones.

AnTuTu’s blog post for AI Review begins by pointing out the difficulties in measuring something as vast as Artificial Intelligence. Currently in the smartphone segment, there does not exist any unified standard for AI, which in turn has led to a situation wherein every chip manufacturer has their own understanding and implementation of AI. Qualcomm handles some AI operations through the Hexagon DSP; Huawei’s HiSilicon handles it through an independant NPU; Samsung and MediaTek also handle AI operations through dedicated chips referred to as NPU and APU respectively. This situation is further complicated by the synergy between hardware and software, which is crucial for effective AI performance. Each vendor provides their own SDK for AI — Qualcomm has SNPE, MediaTek has NeuroPilot, HiSilicon has HiAI, and so on.

AnTuTu’s AI Review benchmark is divided into two sub-categories: Image Classification and Object Recognition. The Image Classification test reviews test data comprising of 200 images, and is based on the Inception v3 neural network, while the Object Recognition test reviews a 600-frame video and is based on the MobileNet SSD neural network. These neural networks are then translated into the neural network that is supported by the manufacturer through the SDK provided by the vendor. If the chip does not support AI-related algorithms, the benchmark app uses TFLite for benchmarking, the results of which AnTuTu themselves warn as being unsatisfactory and unreliable.

The benchmark scoring is directly related to both speed and accuracy. If accuracy is traded off for speed, AnTuTu assigns penalties to the score. This would discourage AI benchmark cheating that would have relied on simply providing quick but wrong results.

Download link:-

Read full article here :-