Welcome to Inkbunny...
Allowed ratings
To view member-only content, create an account. ( Hide )
Inkbunny

Site Updates - Speed Boost

Hi everyone

We have just released build 55 of Inkbunny (well okay it was 12+ hours ago but who's counting?). :P

This update was mainly to take advantage of new features in the database software we use. Many users may notice speed improvements with general browsing and searching.

The technical mumbo-jumbo

This may be of interest to anyone using Postgresql or databases in general.

Search results (which for IB includes gallery view, new submissions view, pools, etc.. anywhere you see galleries of thumbnails... they're all just searches on the database) are stored per user session in new tables as you click the Gallery, Search, etc links. These results tables are then used to show you the data page by page as you click 'next page'. This saves a lot of time as we don't have to run the search all over again just to show you page 2, page 3, etc.

Previously this temporary search results data was saved to disk by the database as it was created but now we tell the database to keep those particular search results tables in memory only (not quite true but it's the best way to explain it quickly). This is a neat trick called "Unlogged Tables" that wasn't possible before the newest stable version of Postgres.

http://www.postgresql.org/docs/9.1/static/sql-createta...

http://wiki.postgresql.org/wiki/What's_new_in_Pos...

http://michael.otacoo.com/postgresql-2/unlogged-table-...

You may have heard of "temporary" tables which have been around a long time in the database world but unlogged tables in Postgresql are not the same. Temporary tables provide data that should only exist per DB session or per transaction but then is deleted as soon as the request ends, which is useless when we want to access it again potentially for several hours like we do with search results on IB.

Since upgrading to use unlogged tables for this semi-permanent search data, the performance graphs show a huge reduction in Postgres buffer usage and disk access time, which means faster queries and less strain on the server.

Unlogged tables are also much faster to create, access and destroy than normal tables.

The downside is the data in them (just the rows actually) is lost in the event of a server crash, but considering the data in our search results tables is transient and destroyed within minutes or hours of use anyway, it is not a problem for us.

TL;DR - Inkbunny is faster now!


Other updates

* API - Added: There is now a complete current list of error codes at http://wiki.inkbunny.net/wiki/API#Errors Previously this listed just a few of the codes.

* API - Changed: The default time that search results exist after the last time they were accessed has been shortened to 30 minutes (was 2 hours). See "rid_ttl" under Search http://wiki.inkbunny.net/wiki/API#Search

* Fixed: A few other minor bugs were zapped.


That's all for this version!

Thanks

IB
Viewed: 402 times
Added: 11 years, 7 months ago
Site News Item: yes
Commenting Locked
 
Flygon
11 years, 7 months ago
By this point, the speed boosts will only be noticeable should I ever get NBN coverage. :p
Jewel
11 years, 7 months ago
As a developer, I love hearing about this stuff, BTW, so please keep the technical details coming :)
KNIFE
11 years, 7 months ago
Able to get porn faster? What's NOT to like!. :D
Shokuji
11 years, 7 months ago
Neat! Glad to hear it. =3 Thank you for your efforts, Starling. ^^
EtherSaga
11 years, 7 months ago
I'm really curious and annoyed at the same time. Recently my notices have always been at a -16 or so. if I nuke my notices it would take about 16 notices until they actually showed up. Is there anyway I could fix this? I even tried deleting my cookies.
trashbunny
11 years, 7 months ago
That seems to be a software problem on Inkbunny's side, which means that you probably can't fix it.
Zoichi
11 years, 7 months ago
I love to read all these details about how you make the site better =D I haven't seen it done in other sites and it really gives more insight as to how you improve it. Thanks for that!
octamous
11 years, 7 months ago
Is there a suggested limit to how many search API requests we can make per hour? I'm working on an app that finds related submissions but even with caching it can produce a lot of requests.
Inkbunny
11 years, 6 months ago
Sorry for the late reply. No limit for now but just try to make as few as possible!
b4818529c406
11 years, 6 months ago
Ahhh PostgreSQL <3
New Comment:
Move reply box to top
Log in or create an account to comment.