By Gabriel, 30 Jun 2018 , updated 07 Jul 2018
Reviewing an ARM-based, ie not a traditional x86 cloud based hosting provider: Scaleway.
As a web developer I have the opportunity to work with a lot of different hosting providers. The good, the bad and the ugly. In this post I’m sharing my latest experiment with an ARM-based (as opposed of traditional x86) cloud based hosting provider: Scaleway.
ARM and x86 are two different architectures for Computer Processor Unit (CPU) ARM CPU are used in most mobile devices today. While x86 continues to dominate the compute-intensive usages: workstation and server processor. Using ARM processor for server is like asking the question: Can we use a lot of those cheap and power-efficient processors to replace a big and complex CPU AND deliver the same service?
There are essentially 2 manufacturers for x86: Intel and AMD. But Intel have around 80% market share in total (laptop, desktop and server) and even more when we look only at the server market. Very concentrated. On the other hand there are currently a lot of designers, manufacturers for the ARM CPU: and among them some big names: Apple, AppliedMicro, Broadcom, Cavium, Nvidia, Qualcomm, Samsung and others… Quiet a competitive environment.
ARM has been very successful in mobile devices because it is cheap and and power-savy. Those criteria are more important than raw computing power in this area. But with the thriving market of smartphones and other portable devices those last 10 years (Iphone 1 was released in 2007) the ARM architecture has evolved a lot: clock speed increased, multi-cores… Now with the 8th generation (ARMv8) the CPU went from 32 bits to 64-bits, bringing it closer to the current architecture and power of the x86. Not here yet but it’s coming. It’s coming fast.
On the software side, most of the Linux distribution and other critical projects (I mean critical for the web: webserver, browser, database, hypervisor…) are already available and supported on ARM. That’s mean that from a enduser point of vue: it doesn’t matter if the underlying CPU is ARM or x86, all the important softwares of the web ecosystem can run on it.
So now, can we or can we not safely and efficiently replace x86 server by an ARM servers? If we can, we shall expect to reduce hosting cost because ARM chip are both cheaper to build and to run (per MIPS).
Scaleway is the first cloud hosting provider, to my knowledge, to offer ARM-based server. And it is cheap. Let’s try to create a Virtual Private Server (VPS) and configure it to run a website.
The admin interface is nice and sleak. No further comment needed on that. Creating a VPS is straight forward. Most of the VPS and other products of the hosting company are based on x86. ARM is just a recent addons to their offer. There are currently 7 ARM-based VPS: from 4 cores to 64 cores ! (and from €2.99/month to €279.99/mo ). In this test I will review the 4 cores and the 8 cores. All are based on ARMv8 64bits SoC by Cavium ThunderX.
Admin page: choice of the VPS model
After the account is created and credit card entered, You choose your server and it will be created in less than 2 minutes. You choose the OS among 9 differents Linux distro officially supported (Ubuntu, Debian, Fedora…) + the community provided OS: some more specific image: wordpress, node.js ; or even Docker!
Admin page: VPS is ready to be used
You will need to register your ssh public key in the admin interface before the server creation (use to grant you ssh access to the VPS). A (optional and paid) public IP can be affected to your VPS.
I chose to install Ubuntu 16.04 + the ordinary stack to powered a Drupal website:
et voilà!
Drupal Status report page at the end of the configuration
No issues, no difference between the configuration of this LAMP stack and the configuration of a LAMP stack on x86. And performance, admin pages are working fine. But this is just 1 test with 1 user (me!), let’s try to do a load test and compare it to a traditional server.
I chose to compare the Scaleway servers with a very popular server EC2 of AWS. The comparison included 4 real life servers: 2 of each types:
Those servers are in the low to middle-end for each company. Bear in mind that the price differ a lot, but the goal is just to give an idea of what value you can get for your money.
Note: cost at the time of publication, Scaleway Euro price converted to USD. For AWS I used Paris datacenter pricing, to have a better comparison with Scaleway (based in Paris too).
sysbench is a lightweight benchmarking tool for linux. It supports testing CPU (multithreading), memory, file I/O, and other stuff. It is available as a package in all popular linux distribution. (But in that case I’ve downloaded the binary package directly from the vendor website to use the latest version: (1.0.14 on arm, 1.0.15 on x86).
| test | 4 ARMv8 | 8 ARMv8 | t2.small | t2.medium |
|---|---|---|---|---|
| CPU: Results: events per second more is better |
2,530 evt/s | 5,059 evt/s | 875 evt/s | 1,752 evt/s |
| File I/O random read Results: Read per second Throughput |
66,616 reads/s 1,040 MiB/s |
182,974 reads/s 2,858 MiB/s |
7,535 reads/s 117 MiB/s |
431,607 reads/s 6,743 MiB/s |
| Memory: Results: events per second |
4,064,073 evt/s 3,968 MiB/sec |
7,292,855 evt/s 7,121 MiB/sec |
793,445 evt/s 774 MiB/sec |
1,302,535 evt/s 1,271 MiB/sec |
Commands used:
sysbench cpu --threads=8 runsysbench fileio --threads=16 --file-test-mode=rndrd preparesysbench fileio --threads=16 --file-test-mode=rndrd runsysbench fileio --threads=16 --file-test-mode=rndrd cleanupsysbench memory --threads=16 runSummary
Those 2 Multi-core ARM servers seem to perform clearly better CPU-wise and memory-wise for those indiviudal tasks. It is not a surprising results: 8 cores is doing more operation than 2. It is interesting to notice that I/O are still much better for the AWS servers. Although I think it is more related to the fact that SSD disk powering those servers are more efficient.
95% of the servers I have installed are used to host a website. They combine a HTTP web server (eg: Apache), a database (MySQL), a PHP framework (or CMS) and some custom PHP code: this is a LAMP stack. In this scenario the global performance of the entire stack is important. It is the global performance that will defined the response time of the server and the capacity to handle heavy traffic.
So in this section I’m using ApacheBench (ab): with this tool we can send to the server a lot of HTTP requests in a short period of time, simulating big traffic on the website, and measure the average server response time. This tool is not perfect because it won’t represent exactly “real life” traffic but it is ok to compare different servers like we are doing here:
We are requesting with ab the “List of content” admin page in Drupal. In this test website I have created 50 test pieces of content to populate the page.
Command sent
ab -n100 -c4 -v1 -C 'SESSYYYYYyyyy=ZZZZZzzzz' http://<SERVER_IP>/d8/admin/content
In this commands:
The results:
| test | 4 ARMv8 | 8 ARMv8 | t2.small | t2.medium |
|---|---|---|---|---|
| number=1 concurrency=1 |
total=0.569s request=569ms |
total=0.611s request=611ms |
total=0.12s request=120ms |
total=0.127s request=127ms |
| number=100 concurrency=4 |
total=15s request=147ms |
total=15s request=153ms |
total=11s request=107ms |
total=6s request=60ms |
| number=200 concurrency=8 |
total=28s request=142ms |
total=17s request=86ms |
total=23s request=116ms |
total=12s request=61ms |
| number=400 concurrency=16 |
total=60s request=149ms |
total=34s request=86ms |
total=48s request=120ms |
total=25s request=62ms |
Summary
The resultats are more comparable: the 2 best servers are the t2.medium and 8-ARM with the t2.small performing slightly better than the 4-ARM and the t2.medium performing slighly better than the 8-ARM. In this more complex test, those 2 arm servers stand slighltly behind the x86 EC2 instances chosen, but still show decent capacity.
It seems to me that those ARM-based server by Scaleway are working very well to power websites. In fact it doesn’t make any difference for me to create and configure those servers as opposed as a traditional x86. And as a developer/devops this is important: this encourages technology adoption. Of course, this was just a short test and we will need more time to make sure it doesn’t come with drawbacks. And I’d like to see other cloud provider entering the market. But, and this is the main point, if it turnout that those processors are significantly cheaper to produce and to run (because less electricity consumption) for the same MIPS (million instructions per second) then it will mean that hosting cost will be reduced. And this is good for all!