Welcome to Inkbunny...
Allowed ratings
To view member-only content, create an account. ( Hide )
Inkbunny

Site Updates - Speed Boost

Hi everyone

We have just released build 55 of Inkbunny (well okay it was 12+ hours ago but who's counting?). :P

This update was mainly to take advantage of new features in the database software we use. Many users may notice speed improvements with general browsing and searching.

The technical mumbo-jumbo

This may be of interest to anyone using Postgresql or databases in general.

Search results (which for IB includes gallery view, new submissions view, pools, etc.. anywhere you see galleries of thumbnails... they're all just searches on the database) are stored per user session in new tables as you click the Gallery, Search, etc links. These results tables are then used to show you the data page by page as you click 'next page'. This saves a lot of time as we don't have to run the search all over again just to show you page 2, page 3, etc.

Previously this temporary search results data was saved to disk by the database as it was created but now we tell the database to keep those particular search results tables in memory only (not quite true but it's the best way to explain it quickly). This is a neat trick called "Unlogged Tables" that wasn't possible before the newest stable version of Postgres.

http://www.postgresql.org/docs/9.1/static/sql-createta...

http://wiki.postgresql.org/wiki/What's_new_in_Pos...

http://michael.otacoo.com/postgresql-2/unlogged-table-...

You may have heard of "temporary" tables which have been around a long time in the database world but unlogged tables in Postgresql are not the same. Temporary tables provide data that should only exist per DB session or per transaction but then is deleted as soon as the request ends, which is useless when we want to access it again potentially for several hours like we do with search results on IB.

Since upgrading to use unlogged tables for this semi-permanent search data, the performance graphs show a huge reduction in Postgres buffer usage and disk access time, which means faster queries and less strain on the server.

Unlogged tables are also much faster to create, access and destroy than normal tables.

The downside is the data in them (just the rows actually) is lost in the event of a server crash, but considering the data in our search results tables is transient and destroyed within minutes or hours of use anyway, it is not a problem for us.

TL;DR - Inkbunny is faster now!


Other updates

* API - Added: There is now a complete current list of error codes at http://wiki.inkbunny.net/wiki/API#Errors Previously this listed just a few of the codes.

* API - Changed: The default time that search results exist after the last time they were accessed has been shortened to 30 minutes (was 2 hours). See "rid_ttl" under Search http://wiki.inkbunny.net/wiki/API#Search

* Fixed: A few other minor bugs were zapped.


That's all for this version!

Thanks

IB
Viewed: 389 times
Added: 5 years, 7 months ago
Site News Item: yes
 
Flygon
5 years, 7 months ago
By this point, the speed boosts will only be noticeable should I ever get NBN coverage. :p
starling
5 years, 7 months ago
It still means you're waiting a second or so less! :D But yes NBN would be nice. ;.;
Flygon
5 years, 7 months ago
I'll have to move to Bendigo. Entire city and area around it should be getting NBN FiOS by 2014. And they have Trams!
Skash
5 years, 7 months ago
srsly? Bendigo is still waiting for NBN? I still have 4 years to wait for mine
Flygon
5 years, 7 months ago
It's getting it sooner than where I live.

Despite being inside Zone 2...
Xanthe
5 years, 7 months ago
Madprops on the speed boost! Definately noticing an improvement. =D (My connection speed peaks at 84Mbps, I live right next to the exchange, heehee)

BT is trialing Gigibit broadband in Suffolk later this year I believe. I can't help but wonder how much it'll cost to upgrade when it's available though...
Jewel
5 years, 7 months ago
As a developer, I love hearing about this stuff, BTW, so please keep the technical details coming :)
KNIFE
5 years, 7 months ago
Able to get porn faster? What's NOT to like!. :D
Shokuji
5 years, 7 months ago
Neat! Glad to hear it. =3 Thank you for your efforts, Starling. ^^
EtherSaga
5 years, 7 months ago
I'm really curious and annoyed at the same time. Recently my notices have always been at a -16 or so. if I nuke my notices it would take about 16 notices until they actually showed up. Is there anyway I could fix this? I even tried deleting my cookies.
trashbunny
5 years, 7 months ago
That seems to be a software problem on Inkbunny's side, which means that you probably can't fix it.
starling
5 years, 7 months ago
Yeah something is messing it up. Will try to track it down asap! Nuke really should fix it though, at least until the problem comes back again.
EtherSaga
5 years, 7 months ago
Thanks! I'm not sure if you found it, but it seems to be working now. Haha!
starling
5 years, 7 months ago
I ran a quick fix script to make all the counts right but it doesn't fix the cause sadly! Can you PM me when it happens again and don't nuke/clear your notices? I'll then take a look to see what count is messing up!
Zoichi
5 years, 7 months ago
I love to read all these details about how you make the site better =D I haven't seen it done in other sites and it really gives more insight as to how you improve it. Thanks for that!
octamous
5 years, 7 months ago
Is there a suggested limit to how many search API requests we can make per hour? I'm working on an app that finds related submissions but even with caching it can produce a lot of requests.
Inkbunny
5 years, 6 months ago
Sorry for the late reply. No limit for now but just try to make as few as possible!
ralesk
5 years, 6 months ago
Ahhh PostgreSQL <3
New Comment:
Move reply box to top
Log in or create an account to comment.