Welcome to Inkbunny...
Allowed ratings
To view member-only content, create an account. ( Hide )
GreenReaper

Two caches, both alike in symmetry…

Inkbunny has historically served content, processed data, and produced web pages from just one server. This kept administration low, but divides one machine's resources between these tasks. It also means you had to fetch files from the Netherlands – a bit of a slog if you live on the U.S./Canadian west coast, let alone Australia, Brazil or Japan.

This week we launched two content caches, both to bring the files closer to you, and to relieve the main server from the burden of serving an average of 35MBit/sec (peaking at 50MBit in the evenings).

Both caches check their RAM and disks for a copy of the requested file, and give it to you directly if they have it; if they don't, they fetch it from another server and store it; but in other respects they differ.

The USA cache, a VPS provided by toy maker Bad Dragon, is a pure web cache - starting from nothing, and storing the most recently-used files up to its capacity. (Edit: Our new cache in AU does the same.) Our secondary server in the Netherlands contains a backup of Inkbunny's image library, updated nightly, and as files are accessed; it uses this library as its cache, rather than having a separate set of files.

The difference between the two is like the difference between a large local library and a national library, such as the Library of Congress - it might take a little longer to get it from the latter, but it's almost certain to have it; the local library will be a bit faster if it does have it, but slower if it has to order it.

Donors were already paying for an on-site backup, so why not use it to the full? As a bonus, we now have a fresher backup with redundant storage, updated over the course of the day. (Private and hidden files are not accessible via the cache because they require authentication; they continue to be backed up daily.)

Why not store a full backup in the USA? For a start, Inkbunny is 900GB and growing; the cache has 50GB of SSD and 50GB of hard disk to play with. This is enough to store around 35,000 original files and 1.35 million screen- and thumb-sized derivatives - all non-private files accessed within three days of regular use. We're using SSD for everything but original files, where random access is of less concern.

We also prefer to keep your private files and personal data under staff control. As much as we appreciate Bad Dragon's generosity, they're a separate organization in a separate jurisdiction. (That's also why JS and CSS comes from our main server.)

This cache has a positive impact for users basically everywhere to the west of Europe - most of North and South America should be seeing significant improvements, and access times from Japan and Australia have been cut by half. The U.S. East Coast, however is directed to the Netherlands because it isn't any faster to go to Arizona. Perhaps a spot for a future cache? In any case, traffic is fairly evenly divided, although it peaks at different times - midnight server-time in Europe, and six hours later for the USA.

This feature was a team effort – Salmy handled the heavy lifting code-wise, including integration of the GeoLite2 location database, while I specified the cache architecture, determined the traffic division, and tweaked the nginx configurations. (And thanks to Dari and varka on the Bad Dragon side!)

I'm very pleased with the performance of the system so far, and it should reduce or eliminate several potential bottlenecks in the future - most notably, bandwidth.
Viewed: 170 times
Added: 2 years, 3 months ago
 
Charliemon
2 years, 3 months ago
Never knew it was so close to me in Ireland x
GreenReaper
2 years, 3 months ago
A lot of sites are hosted in the Netherlands, at least those targeted towards European users. Others have had to take a submarine cable over the Atlantic to get to us - but this should help some of them.
Tycloud
2 years, 3 months ago
I can have butts closer to me?! Best news eveh!  Also
Salmy
Salmy
is amazing! :3
GreenReaper
2 years, 3 months ago
Just 20ms difference for you, but every little helps!
RollerCoasterViper59
2 years, 3 months ago
Wow, didn't know USA was brought to you by Bad Dragon... Cool :3
GreenReaper
2 years, 3 months ago
Well, we haven't done the full announcement yet. :-)
A few eagle-eyed members noticed the change in the footer, though.
RollerCoasterViper59
2 years, 3 months ago
What footer change?
GreenReaper
2 years, 3 months ago
The bit where it says what content server you're on.
RollerCoasterViper59
2 years, 3 months ago
ah.., :3
IrritatedCharizard
2 years, 3 months ago
I have a skill for spotting changes~ =p
Swampwulf
2 years, 3 months ago
How, exactly, is InkBunny associated with Bad Dragon?
GreenReaper
2 years, 3 months ago
They have a VPS which they offered for use to us as a cache, in return for attribution as its sponsor. There's no other business relationship similar to that which certain other furry sites have with them.

We have root access to the cache, but of course it is ultimately their machine, which is why we are not storing any private data there or causing you to send cookies to it - just requests for the same public files on inkbunny.net. It is possible to switch to another server, although it may impact performance.
Swampwulf
2 years, 3 months ago
Thank you for the succinct reply and pointing out alternatives at too routing.
I'm still a bit leery of an outside org deciding if they want to clear their cache or retain it should I delete content here.
Gonna have to think on that for a bit.
GreenReaper
2 years, 3 months ago
We don't currently have an eviction system set up for the U.S. cache. However, that cache is barely 1/10th the size of Inkbunny, so if a work is not accessed - for example, because it is deleted - it will be replaced within a week by other files.

I'm sure a few people have indexed Inkbunny's public files already; that was possible ever since we launched. The U.S. cache server has no special access.
Swampwulf
2 years, 3 months ago
No doubt. There's no official association between those caches and a specific org like BD however.
Zapthechu
2 years, 3 months ago
Thanks for keeping inkbunny working smoothly.
Flygon
2 years, 3 months ago
I thought the technical term for this sort of scenario was Dragonfruit? :P
tanuki
2 years, 3 months ago
Congratulations to the staff team (and donors) for this effort! I don't speak for the community but I'm 99% sure that everyone is thankful for the hard work you guys put on your free time to keep IB running!
graymuzzle
2 years, 3 months ago
Comforting to know that I'm closer to the Netherlands!
Now, about those Cannabis cafe's and legal brothels.....
GreenReaper
2 years, 3 months ago
The first are only for residents, I'm afraid!
Salmy
2 years, 3 months ago
Are they? I went to one :o (didn't smoke, though!)
GreenReaper
2 years, 3 months ago
Well, I heard the city council discourages foreigners coming there for that. You were probably un-American enough, though. ;-)
Salmy
2 years, 3 months ago
Or maybe they just didn't give a shit in that cafe ;)
graymuzzle
2 years, 3 months ago
Not to worry! Here in the People's Republic of Connecticut, they'll legalize as soon as they figure out how to tax it. They're already licensing farms and dispensaries.
Veratai
2 years, 3 months ago
coolio to know you folks are upgrading :)
IrritatedCharizard
2 years, 3 months ago
Keep up the great work, everything is looking superb and even more snapier now. ^^
xTwoTails
2 years, 3 months ago
Its like kissing bad dragon~
GreenReaper
2 years, 3 months ago
I swear, we only went to first base! >_>
xTwoTails
2 years, 3 months ago
You should of went for a Home Run! X3
Pierce
2 years, 3 months ago
Sounds bitchin'.

Does this mean you'll have access to all their coca-cola and cum lube?
GreenReaper
2 years, 3 months ago
Alas, no; but it does make for better use of furry resources - your yiff-dollars at work!

For you, it'd be a 70ms vs. 105ms difference - not huge, but perhaps noticeable. :-)
Pierce
2 years, 3 months ago
When things start dropping below 100ms, it's a noticeable change.
Good work.
I still need to get used to the fact you make progress on the site and the staff is competent. I'm not used to that with furry art sites.
New Comment:
Move reply box to top
Log in or create an account to comment.