Benchmarking the Qualcomm Snapdragon 8 Gen 1: Setting performance expectations for 2022’s flagships

Benchmarking the Qualcomm Snapdragon 8 Gen 1: Setting performance expectations for 2022’s flagships

Just last week saw the advent of the new Qualcomm Snapdragon 8 Gen 1 at the company’s tech summit in Hawaii. Qualcomm’s latest chipset packs a whole host of major improvements across the board, making this one of the most exciting Qualcomm devices in a long time. While the company was hesitant to provide in-depth technical details in some aspects (including neglecting to mention an Adreno or Kryo version name), we were still able to run a gamut of popular benchmarks on the Snapdragon 8 Gen 1 reference device. These benchmarks help set the baseline of performance expectations for upcoming flagships in 2022, giving us one more thing to look forward to next year.

On the Snapdragon 8 Gen 1 reference device, we ran one holistic benchmark (AnTuTu), a CPU-centric benchmark (Geekbench), a GPU-centric benchmark (GFXBench), and MLPerf benchmarks. Each benchmark was run three times, and we took the average of the three results. Qualcomm had enabled a “UI Perf Mode” option by default that we disabled, as it effectively tries to force the benchmarking apps to run on Prime cores in order to eke out a slightly higher score in certain benchmarks. It is also worth noting that once we get our hands on a commercial device with the Qualcomm Snapdragon 8 Gen 1, we will be rerunning these benchmarks.

If you’re interested in reading up on all the specifications and features of the Qualcomm Snapdragon 8 Gen 1 mobile platform that the company has made available so far, then I recommend reading our explainer on the Qualcomm Snapdragon 8 Gen 1. For quick reference, I put together a chart comparing the key specifications of the Qualcomm Snapdragon 8 Gen 1 reference device compared to the other two references devices used in this benchmark comparison. You can find that chart below ahead of the benchmark results.

Qualcomm provided us with a set of expected benchmark scores based on their own testing. We used this for reference only, and a table is available at the bottom of this article containing the benchmark scores that Qualcomm expected the reference device to achieve.

About this article: Qualcomm sponsored my colleague, Rich Woods, to attend the Snapdragon Tech Summit in Kona, Hawaii. The company paid for his flight and hotel. However, Qualcomm did not have any input regarding the content of this article.

Qualcomm Snapdragon 8 Gen 1 Benchmark Results

Test Device Specifications

Qualcomm Snapdragon 8 Gen 1

(Qualcomm Reference Device)

Qualcomm Snapdragon 888

(Qualcomm Reference Device)

Qualcomm Snapdragon 865

(Qualcomm Reference Device)

CPU
  • 1x Kryo (ARM Cortex-X2-based) Prime core @ 2.995GHz, 1MB L2 cache
  • 3x Kryo (ARM Cortex A710-based) Performance cores @ 2.5GHz
  • 4x Kryo (ARM Cortex A510-based) Efficiency cores @ 1.79GHz
  • ARM Cortex v9
  • 6MB L3 cache
  • 1x Kryo 680 (ARM Cortex X1-based) Prime core @ 2.84GHz, 1x 1MB L2 cache
  • 3x Kryo 680 (ARM Cortex A78-based) Performance cores @ 2.4GHz, 3x 512KB L2 cache
  • 4x Kryo 680 (ARM Cortex A55-based) Efficiency cores @ 1.8GHz, 4x 128KB L2 cache
  • 4MB L3 cache
  • 1x Kryo 585 (ARM Cortex A77-based) Prime core @ 2.84GHz, 1x 512KB L2 cache
  • 3x Kryo 585 (ARM Cortex A77-based) Performance cores @ 2.4GHz, 3x 256KB L2 cache
  • 4x Kryo 385 (ARM Cortex A55-based) Efficiency cores @ 1.8GHz, 4x 128KB L2 cache
  • 4MB L3 cache
GPUNew Adreno GPU (unspecified)Adreno 660Adreno 650
Display
  • FHD+
  • 144Hz
  • 2340 x 1080 resolution
  • 120Hz refresh rate
  • 2880 x 1440 resolution
  • 60Hz refresh rate
AI
  • Hexagon DSP with Hexagon Vector eXtensions, Hexagon Tensor Accelerator, and Hexagon Scalar Accelerator
  • 7th generation AI Engine
  • 3rd generation Qualcomm Sensing Hub
    • Always on
    • Always secure
  • Hugging Face Natural Language Processing
  • Leica’s Leitz Look mode
  • Hexagon 780 with Fused AI Accelerator architecture
  • 6th generation AI Engine
  • Qualcomm Sensing Hub (2nd generation)
    • New dedicated AI processor
    • 80% task reduction offload from Hexagon DSP
    • 5X more processing power YoY
  • 16X larger shared memory
  • 50% faster scalar accelerator, 2x faster tensor accelerator YoY
  • 26 TOPS
  • Hexagon 698 with Hexagon Vector eXtensions and new Hexagon Tensor Accelerator
  • 5th generation AI Engine
  • Qualcomm Sensing Hub
  • 15 TOPS
Memory8GB LPDDR5 @ 3200MHz, 16GB
  • 12GB LPDDR5
  • 3MB system level cache
  • 12GB LPDDR5
  • 3MB system level cache
Storage512GB UFS 3.1512GB UFS 3.0128GB UFS 3.0
ISP
  • Triple 18-bit Spectra 680 ISP
  • 3.2 Gigapixels per second throughput
  • Triple 14-bit Spectra 580 ISP
  • 2.7 Gigapixels per second throughput
  • Dual 14-bit Spectra 480 ISP
  • 2.0 Gigapixels per second throughput
Manufacturing Process4nm (likely Samsung)5nm (Samsung’s 5LPE)7nm (TSMC’s N7P)
Software versionAndroid 12Android 11Android 10

Overview of the benchmarks. Click to expand.

Benchmarks Overview

  • AnTuTu: This is a holistic benchmark. AnTuTu tests the CPU, GPU, and memory performance, while including both abstract tests and, as of late, relatable user experience simulations (for example, the subtest which involves scrolling through a ListView). The final score is weighted according to the designer’s considerations.
  • GeekBench: A CPU-centric test that uses several computational workloads including encryption, compression (text and images), rendering, physics simulations, computer vision, ray tracing, speech recognition, and convolutional neural network inference on images. The score breakdown gives specific metrics. The final score is weighted according to the designer’s considerations, placing a large emphasis on integer performance (65%), then float performance (30%), and finally cryptography (5%).
  • GFXBench: Aims to simulate video game graphics rendering using the latest APIs. Lots of onscreen effects and high-quality textures. Newer tests use Vulkan while legacy tests use OpenGL ES 3.1. The outputs are frames during test and frames per second (the other number divided by the test length, essentially), instead of a weighted score.
    • Aztec Ruins: These tests are the most computationally heavy ones offered by GFXBench. Currently, top mobile chipsets cannot sustain 30 frames per second. Specifically, the test offers really high polygon count geometry, hardware tessellation, high-resolution textures, global illumination and plenty of shadow mapping, copious particle effects, as well as bloom and depth of field effects. Most of these techniques will stress the shader compute capabilities of the processor.
    • Manhattan ES 3.0/3.1: This test remains relevant given that modern games have already arrived at its proposed graphical fidelity and implement the same kinds of techniques. It features complex geometry employing multiple render targets, reflections (cubic maps), mesh rendering, many deferred lighting sources, as well as bloom and depth of field in a post-processing pass.
  • MLPerf Mobile: MLPerf Mobile is an open-source benchmark for testing mobile AI performance. It was created by MLCommons, a non-profit, open engineering consortium, to “deliver transparency and a level playing field for comparing ML systems, software, and solutions.” MLPerf Mobile’s first iteration provides an inference-performance benchmark for a handful of computer vision and natural language processing tasks. For more information, refer to the paper “MLPerf Mobile Inference Benchmark: Why Mobile AI Benchmarking Is Hard and What to Do About It.
    • Image classification: This test involves inferring a label to apply to an input image. Typical use cases include photo searches or text extraction. The reference model used is MobileNetEdgeTPU with 4M parameters, the dataset is ImageNet 2012 (224×224), and the quality target is 98% of FP32 (76.19% Top-1).
    • Image segmentation: This test involves partitioning an input image into labeled objects. Typical use cases include self-driving or remote sensing. The reference model used is DeepLab v3+ with 2M parameters, the dataset is ADE20K (512×512), and the quality target is 93% of FP32 (0.244 mAP).
    • Object detection: This test involves drawing bounding boxes around objects as well as providing a label for those objects. Typical use cases involve camera-input such as for hazard detection or traffic analysis while driving. The reference model is SSD-MobileNet v2 with 17M parameters, the dataset is COCO 2017 (300×300), and the quality target is 97% of FP32 (54.8% mIoU).
    • Language processing: This test involves responding to questions colloquially. Typical use cases include online search engines. The reference model is MobileBERT with 25M parameters, the dataset is mini Squad (Stanford Question Answering Dataset) v1.1 dev, and the quality target is 93% of FP32 (93.98% F1).


Benchmark Results

AnTuTu

AnTuTu Overall Results for the Snapdragon 8 Gen 1

Starting off with AnTuTu, we can see that the Qualcomm Snapdragon 8 Gen 1 reference device scored substantially higher than previous years, with a major year-on-year increase of roughly 240,000 points. This is a pretty major improvement of roughly 33%, well above Qualcomm’s promises of a 20% speed improvement. This is only one test, but AnTuTu is a great tool for comparing raw computational ability between devices, even if not all of that translates into real-world usage.

Geekbench

In Geekbench 5.0, however, it’s a different story entirely. The Snapdragon 8 Gen 1 demonstrates little to no gain over the Snapdragon 888 reference device from last, and in fact, even regresses in performance when it comes to multi-core. It’s not as if we got a dud either — Qualcomm gave us an informational page with benchmark scores to expect from the Snapdragon 8 Gen 1, and these fall in line with what should be expected. In fact, the single-core performance was higher than the upper-bounds that Qualcomm gave us, whereas the multi-core performance was 20 points below the lower-bounds of what to expect.

Having said that, I think it’s clear that whatever the test is, it’s not testing where Qualcomm has made improvements. Our other tests show major improvements over previous years’ chipsets.

GFXBench

Qualcomm hasn’t disclosed much about the new Adreno GPU in the Snapdragon 8 Gen 1, so we have little to say about the GPU other than its gains in performance. We don’t know the core count, we don’t know the frequency, and we don’t even have a version number. In GFXBench’s Manhattan test, which uses the OpenGL ES 3.0 API and renders a 1080p scene offscreen, the Snapdragon 8 Gen 1 had an average framerate of 221fps, about 31% and 75% higher than the framerates achieved by the Snapdragon 888 and 865 respectively. In GFXBench’s Aztec Ruins test, which uses the Vulkan graphics API and renders a 1080p scene offscreen, the Snapdragon 8 Gen 1 had an average framerate of 41fps. There’s a caveat to this though; the previous results on reference devices were tested in 1080p, whereas the only Aztec Ruins test we had access to was in 1440p. Increasing to 1080p requires 43.75% more pixels to be rendered at a time, which is why the performance decreased in this test.

Only some of the very best Android games require a lot of GPU horsepower, but improved GPU performance is useful for more than just gaming. Having said that, gaming is definitely the biggest reason why people will care about these benchmark results, and the Snapdragon 8 Gen 1 seems to deliver with its 35% faster graphics rendering and 20% better power efficiency year on year. These results only demonstrate the peak GPU performance, however, so we’ll have to revisit GFXBench — once we get our hands on commercial hardware — in order to run the benchmark’s long-term performance tests.

MLPerf

Qualcomm has been particularly cagey about specifics when it comes to artificial intelligence improvements. We don’t have any figures for TOPS (Trillion Operations Per Second), though we do have some information about other improvements. There’s the third generation of the company’s sensing hub for example, and it also demonstrated a number of other SoC-specific features in Hawaii.

It is difficult for us to demonstrate just how significant this leap in performance actually is, however. We’ve talked in-depth about the difficulties of AI benchmarking during our interviews with Qualcomm’s Travis LanierGary Brotman, and Ziad Asghar. The good news is that, since our discussions with Qualcomm execs, there have been significant advances in the field of AI benchmarks.

The most promising benchmarks currently available come in the form of MLPerf Mobile, which is an open-source mobile AI benchmark backed by multiple SoC vendors, ML framework providers, and model producers. Its initial batch of mobile inferencing results is public, so we used those results to compare the Snapdragon 8 Gen 1 to the Snapdragon 888 in the Xiaomi Mix 4, the Dimensity 1100 in the Vivo S9 5G, and the Exynos 2100 in the Samsung Galaxy S21 Plus. We didn’t get latency results — only throughput figures — so we did not plot the full results as submitted by the vendors for verification by MLCommons.

In these select computer vision and natural language processing inferencing benchmarks, we can see that the Qualcomm Snapdragon 8 Gen 1 reference device achieved the highest scores in all four tests by far. The Demensity 1100 fared pretty poorly overall. Qualcomm’s Snapdragon 888 still comfortably beat the others in this test, but the Snapdragon 8 Gen 1 is on another level in all of these tests.

It will be interesting to see what applications and features developers and OEMs can create using the AI prowess of the Snapdragon 8 Gen 1. Computer vision will play an especially important role in the many AI-enhanced videography features we’ll likely see in 2022, while improved NLP performance can likewise affect video adjacent aspects like audio recording. Companies like Google working on Google Tensor will also push other SoC vendors in this department.

Conclusion

The table that Qualcomm provided us with expected benchmark scores is below, which you can see falls mostly in line with the results that we achieved above.

Expected benchmark scores for Snapdragon 8 Gen 1 reference design (from Qualcomm)

BenchmarkVersionMethodExpected Score Range
SystemGeekbench STv5.4.2Average of 3 iterations~1220 – 1233
SystemGeekbench MTv5.4.2Average of 3 iterations~3770 – 3810
SystemAnTuTuv9.2.1Average of 3 iterations1st run: ~1m

Avg of 3 iterations: ~980K

SystemPCMarkv3.0.4061Average of 3 iterations~17k
Browser (Chrome v95.0.4638.74 64-bit)JetStreamv2.0Average of 3 iterations~135 – 140
BrowserSpeedoMeterv2.0Average of 3 iterations~123 – 126
BrowserWebXPRTv3.0Average of 3 iterations~194 – 197
AIAITuTuv2.0Average of 3 iterations~2,550,000 – 2,600,000
AIAIMarkv3.0Average of 3 iterations~97K
AIMLPerf (on a separate QRD since it takes 30 mins to run)v1.1Average of 3 iterations
  • Image Class: ~2435 – 2450
  • Object Detect: ~1180 – 1250
  • Image Segment: ~520 – 540
  • Language Understanding: ~38 – 40
  • Image Class (Offline): ~3580 – 3650
AIETH AIBv4.0.4Average of 3 iterations~530 – 550k
GraphicsGFXBench Manhattan 3.0 Offscreen (1080p) (FPS)v5.0Average of 3 iterations~267 – 268 FPS
GraphicsGFXBench T-Rex – Offscreen (1080p) (FPS)v5.0Average of 3 iterations~450 – 452 FPS
GraphicsGFXBench Manhattan 3.1 Offscreen (1080p) (FPS)v5.0Average of 3 iterations~176 FPS
GraphicsGFXBench Car Chase Offscreen (1080p) ES3.1 (FPS)v5.0Average of 3 iterations~97 – 98 FPS
GraphicsGFXBench Aztec Ruins Vulkan (High Tier) Offscreen (1440p) (FPS)v5.0Average of 3 iterations~49 FPS
GraphicsGFXBench Aztec Ruins OpenGL (High Tier) Offscreen (1440p) (FPS)v5.0Average of 3 iterations~43 FPS

The Snapdragon 8 Gen 1 brings a ton of improvements over last year’s chipset, especially when it comes to AI. While there is definitely some weirdness in the CPU-bound results from Geekbench, it’s clear that there are across-the-board improvements. If you’re upgrading from a device that’s two years older (or older) then the improvements will likely be noticeable, though the massive gains in AI performance will likely go unnoticed by most. Companies seldom leverage the full potential of AI when it comes to Qualcomm’s chipsets, and it’s likely that will be the same again here.

It’s also worth noting that with competition heating up, it might be worth waiting to see what Samsung and MediaTek do next. The Dimensity 9000 chipset has the potential to take the Snapdragon 8 Gen 1 head-on when it comes to performance, and there’s not a lot that we actually know about the upcoming Exynos 2200 just yet. I’m personally looking forward to revisiting these benchmarks on a commercial device in the future, particularly in a more controlled setting.

About author

Adam Conway
Adam Conway

I'm a senior editor at XDA-Developers. I have a BSc in Computer Science from University College Dublin, and I'm a lover of smartphones, cybersecurity, and Counter-Strike. You can contact me at [email protected] My Twitter is @AdamConwayIE and my Instagram is adamc.99.