Welcome to Inkbunny...
Allowed ratings
To view member-only content, create an account. ( Hide )
GreenReaper

Delivering Inkbunny using the chips in your phone

Mobiles have gotten powerful in recent years. Many browse Inkbunny with one. Of course, being a geek, I'm more interested in serving it with one!

The idea of hosting websites on the same low-power ARM chips used for phones has been building for a while, albeit with many false starts. And when a new technology arises, it's natural to test it on animals…

Test box

To this end, I obtained an ARM-based virtual server from RunAbove - a "test lab" division of European and Canadian hosting provider OVH. It was, for a day, handling ~1/20th of Inkbunny's image traffic.

The system in question has a ThunderX 2.0Ghz ARMv8-A core, working out to about 3/8 the processing power of our Virginia cache, at roughly 3/8 the price. It also has 2GB RAM and a 10GB HDD. The RAM is important. Too little memory on a 64-bit node doesn't work out so well. Pair it with too much disk, and you end up barely able to keep the filesystem cached, let alone any of the files, leading to thrashing.

The system runs Ubuntu, which is great, as we use the similar Debian. RHEL for ARM is in development.

The good, the bad, and the ugly

So after a day, what's the verdict? CPU-wise, the system might well be capable of pushing Inkbunny's entire content traffic load (~50Mbit). Our caching software, nginx, is very efficient - and the CPUs have integrated SSL-acceleration hardware, so content encryption isn't a burden.

Would we transfer everything to an ARM-based server? Well, no - we already have dedicated servers in Europe, with CPUs more suited to databases, web servers, or PHP. Scale doesn't help so much here - if it takes a second to generate a page, that's still a problem even if you can do many pages at once.

There's also a reason these servers have four real drives each - a single virtual disk wouldn't keep up. It's fine to be doing hundreds of operations per second if you're the only person using one disk. Having 48 people do the same thing doesn't work so well. (Hosting companies rely on you not all using it at once.)

Still, for use in a private CDN? It'd work, if it were more reliable. We encountered significant networking issues which also appeared to impact disk access (possibly because they're virtual disks). These must be resolved before such systems can be used in production for any length of time. Edit: It was!

Conclusion

Overall, ARM-based servers seem capable of taking on at least low-level tasks. But will this be enough to take market share from Intel? The Xeon D is very exciting; it's something we'd consider for dedicated servers a few years down the line, once they're available for second-hand lease.

ARM may become the norm for the low-end VPS; or it may never be more than a niche product. Either way, we're interested, because we want to get the best value possible for your donations. At a minimum, it forces Intel to compete in product areas previously neglected. Availability of an alternative can be used to extract price concessions by hosting companies, which trickle down to us.

Try it yourself! (no longer available, sorry!)

We have the test box available as the Hamburg Cache (we think it's around there).

RunAbove's ARM-based virtual servers are currently free. That probably won't last (and they appear to have a 100-user limit), but the advertised beta price is $1.13/month, so you'd not be on the hook for much. If you fancy giving one or two a go yourself, here's instructions. Sign up here (SMS required) and you'll get $10 to try out the other services they're offering, such as a VPS with a dedicated disk.

Note that RunAbove is a test lab; support is minimal, and few niceties are available (due to the new architecture, even getting a non-SSH console requires an extensive tutorial). VMs will need to be retired, or migrated to OVH if the system is successful - as happened recently with other cloud services.

In short, if you want a long-term VPS, this isn't it. If you want a ARM server to play with, it's worth a go.

While I've been messing around with ARM and putting HTTP/2 on our cache nodes, the rest of the team has been working on several very cool features for the next site revision. Stay tuned by watching
Inkbunny
Inkbunny
!
Viewed: 194 times
Added: 1 year, 8 months ago
 
DizziMorhlis
1 year, 8 months ago
Can't wait for the next phase of Inkbunny! Keep up the amazing work guys ^^
Tycloud
1 year, 8 months ago
For once in my life I was able to read tech talk and not find it dry. You sir are really good at writing this stuff. You should consider teaching as a secondary career.
xTwoTails
1 year, 8 months ago
That's because a true teacher knows how to keep a post entertaining, interesting and always keeps you reading until the end.
He indeed, may be a teacher ... or at least I'd like to think he is @w@
GreenReaper
1 year, 8 months ago
I'm not a teacher, but I do have a decade of writing for WikiFur and Flayrah under my belt! Non-fiction writing and editing isn't quite the same as teaching, but both improve with practice.
Zapthechu
1 year, 8 months ago
I have a doubt about how Opera Mini works when i use my phone to browse inkbunny. Well everything loads well, i have no issues at all, it's just that i can't do some things like favorite a post unless i go to the browser options and uncheck the mobile view so that it loads the page like it was from a desktop pc, but it is really laggy. ANother thing i can't do is to open someone's journal from the notices, i must directly go to their profile page and click the journals from there (the same go with the comments).

I know this mostly limitations on the cellphone i use, an old brand Nokia 2690. So i was wondering what are the chances of seeing an app that gathers the basics features (like facebook for everyphone) and works to open inkbunny without needing a mobile browser?
GreenReaper
1 year, 8 months ago
We don't plan to make a mobie app ourselves - if one were made, it'd probably be for Android, not JavaME, which is the only option for a 2010 feature phone. However, if in the future we change our UI to be more responsive, it may work better with your phone's browser. This is something we want to do, but I cannot give you a date for it, in part because we are currently working on other things.

If you can manage to get a device that works with Android 2.3 Gingerbread or higher, you are likely to have a better experience, but I understand such devices haven't yet filtered down everywhere.
Zapthechu
1 year, 8 months ago
I see. That's ok then. I'm not unhappy with browser, since it works decent enough to keep enjoying onkbunny  when can't use my pc :P
imer
1 year, 8 months ago
Generally, the inkbunny website is fine for mobile stuff
The only issue is stories/text submissions really since the font size is so tiny (with phones that cant handle all the formats to download)
Eiko
1 year, 8 months ago
My old web server ran on Arch Linux for ARM. It was pretty limited in what it could handle, and the community developing for it seemed to lose interest as updates got buggier and buggier. I still have it in a box somewhere now that web services are handled by a VM. bunnycloud.ca isn't anywhere near as fancy as inkbunny though, and my needs are pretty minimal.
GreenReaper
1 year, 8 months ago
ARM suffered for a long time from platform fragmentation. With the support of large Linux distributions for AArch64, I think things are finally turning the corner. We have a script for setting up cache nodes and I experienced zero difference with this one. It was just another server from my perspective.

Performance has improved a lot with the 64-bit architecture, too, in certain areas. For a start, AES is eight times faster on 64-bit. (And of course newer chips with better compiler optimizations work faster.) Still, it's no i5, for sure - and there will still be a demand for them.
ZekromLover
1 year, 8 months ago
I'm afraid to say I didn't understand what is going to change.
GreenReaper
1 year, 8 months ago
Probably nothing, for now. We're just testing alternative server technologies. In the future, it might mean we can confidently use a cheaper technology to distribute files in some areas. From a user perspective the site should work as normal.
imer
1 year, 8 months ago
You might want to give https://www.scaleway.com/ a try
The traffic you get is so dirt cheap.. its unbelieavable
GreenReaper
1 year, 8 months ago
I had a look at those yesterday, in fact! They're using ARMv7 cores, which are less powerful and I believe don't have AES acceleration. But you do get four of them, and 50GB SSD, which is nice. Probably a better machine for a cache node, and worth considering if we weren't based in the Netherlands with 100TB over 1Gbps available (we used ~13TB before launching our cache network).

Traffic is cheap because it's France. Europe has cheep peering - it will be throttled (from the other side) if your users are in North America. That's one reason we added cache nodes in the USA.
Tholmeir
1 year, 8 months ago
I would like to see a mobile version, if only for the easier format than the normal version when using a phone. Perhaps an app for some kind for easy uploading, favoriting, etc.? Just an idea that's been kicking around in my head for a while. Not sure how much it would cost though, likely a fair bit of cash.
My apologies if this is off-topic.
GreenReaper
1 year, 8 months ago
It's basically time for us. Nobody gets paid to write code for Inkbunny. Of course, time is money. :-)

We're not that interested in writing apps - in part because we'd rather make the site better for everyone, but also because we lack the relevant technical knowledge (I could probably hack one together for Windows, but getting it onto the Store might be tricky). However, we have an API for submissions, and if anyone else wants to give it a go we'd be glad to list them on our apps page.

In contrast, we are interested in writing a website that works better on mobile. It's just one of the things we're interested in, and not yet at the front of our queue.
fluffdance
1 year, 8 months ago
Sounds like a promising little path to watch develop!  In my head, I'm picturing a closet full of second-hand smartphones and a cheap SOHO-type Wireless bridge in the back of a datacenter somewhere, with a medusa of charging cables to keep everything running.  I'm sure the reality is a lot different, though.

When you initially mentioned serving InkBunny off a mobile device, I was thinking you were devising some sort of P2P-style distributed processing with users' smartphones; think SETI with a fuzzy twist.  ;D

I am mildly concerned, though, as you say that you were running part of IB off the technology for a day.  I wish I could remember what day it was, but there was ONE day where things went a little...weird.  Some images (thumbnails, submissions) would load while other wouldn't; sometimes on the same page (submission view).
GreenReaper
1 year, 8 months ago
It's a bit more like this and this - extended motherboards in racks, rather than mobile phones, though it's fun to imagine just slotting another phone in for more capacity (you can do that with drives).

The day in question was yesterday, and the test was restricted to members in southern Europe. We have had other days when specific caches have had outages, and you probably encountered one of those - it sounds like what happens when a cache gets full, and can't add all the new files.
fluffdance
1 year, 8 months ago
Ooh, awesome!!  Makes me kind of curious, though.  If it isn't already happening, I can imagine that within the next few years, a lot of the "throw-away" mobile phone hardware that's out there may serve a huge benefit to impoverished, remote communities in undeveloped/underdeveloped countries and regions.  Imagine being able to set up an entire, functional network of clients and servers with nothing but old smartphones, and the impact that could have on things like education and healthcare.  :)

As for the testing you did and the little blip I had, I'm definitely not in Europe, and the blipping was definitely longer ago than yesterday, so I guess I can rule that out!  *laughs*  I've noticed a few little hiccups with the cache system here and there, but really nothing major, and nothing a quick switch in the settings can't cure.  Is there potentially some way to integrate a "sensing" system that can tell when a cache is being troublesome, and automatically re-route a user to the next-closest cache?
GreenReaper
1 year, 8 months ago
Well, where do you think Zapthechu got his? Most are still useful as phones! :-)

But yes, it's a fun idea, and potentially feasible with the newer smartphones, though I'm not sure the economics would work out.

We have it set up in the database to do be able to do just what you describe, and we have a good idea of how the script to test the caches and set the flag would work, we just haven't done it yet. Staff can manually disable caches as of the latest revision; the DB query to select the cache checks both flags and picks the highest-priority active cache serving you.
Flygon
1 year, 8 months ago
Kinda has me curious how desktops will go.

Like... if dual-arch computers will become a widespread thing due to how ingrained x86 and derivatives are now.

Dual-Hybrid ARM/x86 machines anyone?
GreenReaper
1 year, 8 months ago
In that sense most machines are dual-arch. Consider your video card, network controller, RAID, etc.

In terms of "main chip" I don't think there's a need for two-in-one. Instead, you network two machines together, as we did, and communicate via protocols which both can produce and consume, whether or not they run the same OS. High-CPU tasks can use x64, content streaming can use ARM.
Flygon
1 year, 8 months ago
Heheh, I forgot about GPUs and other such things.

I was mainly referring to CPUs though. Kinda rambling in my own head a bit, but wouldn't we eventually hit the point where it's easier to squeeze more performance out of a separate or new CPU arch. than to try and keep squeezing more out of the (in hindsight) rather frustratingly designed x86?

Wanna keep compatibility with x86? Then keep the x86 as secondary CPU! All in one machine. :D
GreenReaper
1 year, 8 months ago
Well, this approach has been done, most recently with the PS3. But this is rarely perfect, and adding the necessary hardware can be quite expensive, for a niche use. It might be more cost-effective to emulate the CPU instead, as has been done for x86 on ARM devices several times. Of course, emulation is far from perfect, either. Really, it might be best just to keep the old box around until you have developed an alternative.
Flygon
1 year, 8 months ago
Mm, the entire reason for the concept was to avoid emulating x86. Precisely because it's such a nightmare to emulate.

With regards to the PS2-in-PS3. Considering the two pieces of hardware were more completely divided from eachother, apart from a few obvious things, I wouldn't consider it a true example of two separate CPU archs working together.

If we were to bring up historical examples, I'd actually drag in the Sega Mega Drive! In fact, even working for much the reasons I'd suggest dragging the concept into the PC area.

The console is, from an extremely simplified point of view, a Sega Master System mashed together with a derivative of Sega's late 80s arcade hardware. This includes having both the PSG (Master System sound system) and FM synthesis chips working in tandum, the 68k and Z80 CPUs being able to work cooperatively in tasks (though, for most usages, the Z80 was simply used as a slave for controlling both sound systems. Probably due to how difficult it would be to try and make the two things work together in a multi-CPU environment, heheh).

The real main exemption with the two pieces of hardware mashing together well would be the VDP (the graphics chip). The Master System video modes worked completely independently of the Mega Drive's, and wasted a lot of silicon space in the process. I suspect they simply found it easier to design a new VDP and miniaturize the Master System's for use in SMS video mode, rather than try and upgrade the SMS's. I'm not too sure on the intricacies of this one, nor does it particularly pertain to this discussion very well.


Anyway, the point I'm trying to make is, is that I think it could be possible to introduce a dual-CPU multi-arch environment to desktop computing, in some sort of an industry-wide effort to try and retire the x86 arch (as much as Intel would probably try and succeed to make such an effort impossible), for the sake of further performance improvements in a modern architecture not bogged down by 40-50 years of 'winging' it, and creating a tremendous kludge of a CPU arch that's become a nightmare to squeeze performance improvements out of


I do wish to stress, please, do not take me for any sort of an expert. I probably have gotten a lot of things wrong. I'm not an engineer. I'm not even very smart. I'm just a guy sitting in an armchair trying to make sense of how these things work. And from my casual observations, the x86 arch is pretty much fucked. :D
GreenReaper
1 year, 8 months ago
Interesting read! I only ever had a Master System. :-(
Literally, that was the last console I ever had. PC all the way! :-D

I suspect the x86-64 architecture will be around for a long time to come. We'll see!
Flygon
1 year, 8 months ago
Of course. Raw inertia helps with that one!
darkdoomer
1 year, 7 months ago
haha, i just imagine Inkbunny running on just a tablet.
"nah, really, that's where all of the data are stored."

ah, won't criticize, i'm actually keeping a blackberry connected all the time on wifi as an home server for the stereo (FTP and sometimes IRC)
GreenReaper
1 year, 7 months ago
Not quite, but we do have one VPS serving the U.S. East/Central on just 10¢/day. This server could probably do that, too, although it'd be nice to have a little more disk space. (Plus, it's in Europe.)

You do need "real" servers to process PHP/SQL and resize images at a reasonable speed. But the space of what is possible with less becomes larger every day.
New Comment:
Move reply box to top
Log in or create an account to comment.