Cheating in Android Benchmark Is Widespread

Latest Samsung is caught in optimizing their products under benchmark tests, but they are certainly not alone.

Benchmark tests cannot be used to much if manufacturers do not fight on a level playing field. An illustration created by anandtech.com, shows that it is relatively common to try to cheat the benchmark programs, so one’s own devices will get a better result than his competitors.

A form of cheating is to detect that the device will be tested by a benchmark program and turn up the performance during this period as the unit perform significantly better than the consumer will experience under normal use.

The table shows a subset of the devices that have been tested and the benchmark programs recognized by the phones which then screws up for performance.

It is certainly quite a few which builds their telefonkøb on these benchmark tests (and rightly so), but nevertheless it is a pity that it is not possible to form a reasonable basis for comparison.

AnandTech concludes in the article (can be read in its entirety here), that you can not do much other than to avoid using benchmark programs which are compromised, talking with producers about to stop this trend, as well as a few benchmark developers to circumvent optimizations as they are discovered.

Some more keen than others

As you can see from the graphic (larger image here) is some manufacturers more keen on getting their phones at the top of the tables than others. Development of Samsung from Galaxy S4 Note 3 clearly shows an effort in this area-to catch multiple types of benchmark programs.

Conversely, similar to what a faint-hearted effort from LG who clearly could use some more “optimization-engineers”.

So far, it looks as if Google with their Nexus devices, Motorola and Apple do not participate in this show.

According to Anandtech fought there on the 0-5 percent additional CPU performance and maximum 10% GPU performance when the various benchmark tests are finished.

 

Leave a Reply

February 25, 2016

Posted In: Cellular Phone

Tags: , , , , , ,

Leave a Comment