Forums faster and new Google search

[Stuart Wright]

Couple of changes to the forums.
Firstly a bug introduced into a recent version of PHP has meant that all vBulletin (the software running these forums) installations suffered significant performance hits.
This bug has been fixed and we upgraded to the new version of PHP a few minutes earlier.
This should have a significant effect on the speed of the forums.

Secondly I have added a Google site search. We have well over 250,000 pages in Google, and where our own search facility falls short (in searching 2 letter words) the Google search should do better. It can be found in the Search drop-down menu of the navigation bar.

[GrahamC]

Wooot, passes the dreadded 'Z2' test...

[OldSkoO1]

The google search is very useful as you've outlined

Is gzip compression enabled and have you setup some table indexes? As this forum is based on a box solution you can't really optimize the code (of course you could always opt to install Turck MMCache) but setting up indexes and compressing the text output really does help imho especially when you have that many pages spidered and quite a lot of inbounds.

p.s. thanks very much for running this forum i've found it extremely useful and very informative in my short time here

[Stuart Wright]

Originally Posted by OldSkoO1

The google search is very useful as you've outlined

Is gzip compression enabled and have you setup some table indexes? As this forum is based on a box solution you can't really optimize the code (of course you could always opt to install Turck MMCache) but setting up indexes and compressing the text output really does help imho especially when you have that many pages spidered and quite a lot of inbounds.

p.s. thanks very much for running this forum i've found it extremely useful and very informative in my short time here

How would zipping up files help?
I'm not sure how adding indexes to the database beyond what the vB teak have already done would help.
We're already running Turck MMCache.
I have a whitespace removing hack which reduced the size of the pages sent to the browser significantly.
Aside from that I'm just waiting for the new vBulletin database driven search to released.

[OldSkoO1]

gzip is a pretty common compression handler for php which will compress the text content before it is sent to the browser/parsed. You should have the option to turn this on in the admin for VB.

I see that this forum is very well optimized in terms of not that many images and more css which is good but you can increase the speed of text using compression by up to 80%.

Creating table indexes basically puts the primary and foreign keys in db cache for each table. When you do a sql lookup it looks through the indexes before querying the table. E.g. if you set postID, accountID, forumID etc etc as the indexes for each table (where applicable) it knows what to use straight away (by looking in the cache) instead of enetering into the depths of massive db tables.

Although i'd be suprised if such a popular forum such as VB didnt already have table indexes. You can see in phpMyAdmin by clicking on a table and checking the indexes info report.

A search mod would be nice, the one that only allows 1 search every 60 seconds for each user. Searches are by far the biggest drain, the query string has to match masses of data which ties up a lot of resources per search. Hence forum search are always poop and as you mentioned if it bought back less than 2/4 characters (and,then,the,for,this etc) it would match far to many things and everythings is sooo slow. Easy solution let the google API do the donkey work

[eviljohn2]

Originally Posted by Stuart Wright

Couple of changes to the forums.
Firstly a bug introduced into a recent version of PHP has meant that all vBulletin (the software running these forums) installations suffered significant performance hits.
This bug has been fixed and we upgraded to the new version of PHP a few minutes earlier.
This should have a significant effect on the speed of the forums.

Secondly I have added a Google site search. We have well over 250,000 pages in Google, and where our own search facility falls short (in searching 2 letter words) the Google search should do better. It can be found in the Search drop-down menu of the navigation bar.

No chance of a member search in the same menu?

[Stuart Wright]

Originally Posted by eviljohn2

No chance of a member search in the same menu?

What enter a member's name and it finds their posts?

[eviljohn2]

So that it finds matching members so you can check their profile. The same feature as there is in the actual Member List but available on the standard menu.

[Stuart Wright]

Originally Posted by OldSkoO1

gzip is a pretty common compression handler for php which will compress the text content before it is sent to the browser/parsed. You should have the option to turn this on in the admin for VB.

Checked with our hosts that the required module is available - which it is - and gzip is now enabled. Thanks for the tip.

Originally Posted by OldSkoO1

Creating table indexes basically puts the primary and foreign keys in db cache for each table. When you do a sql lookup it looks through the indexes before querying the table. E.g. if you set postID, accountID, forumID etc etc as the indexes for each table (where applicable) it knows what to use straight away (by looking in the cache) instead of enetering into the depths of massive db tables.

Although i'd be suprised if such a popular forum such as VB didnt already have table indexes. You can see in phpMyAdmin by clicking on a table and checking the indexes info report.

Oh blimey, yes - it's one of the advantages and essential elements of an SQL database. It's so obvious that I thought you were talking about something else.
Take the new way banners are selected, for example. Each time an advert is served, we record the ad id, the zone id (location) and the IP address in an SQL table. This table is then archived daily to a stats file since there are over 820,000 rows per day. Every time an advert is served, it checks the table to find the advert which has been seen least by the particular person (IP address).
For the SQL bods out there it's

SELECT ad_adverts.adid, ad_adverts.advertiserid, ad_adverts.filename, ad_adverts.html, ad_adverts.alttext, ad_adverts.text_below, ad_adverts.url, ad_adverts.width, ad_adverts.height, ad_adverts.border,
COUNT(ad_views.ad_id) as views
FROM ad_campaigns, ad_adverts LEFT JOIN ad_views ON ad_adverts.adid = ad_views.ad_id
AND ad_views.ad_zone_id=$zoneid
AND ad_views.ad_ip='$ip'
WHERE FIND_IN_SET( ad_adverts.adid, ad_campaigns.adids )
AND FIND_IN_SET( '$zoneid', ad_campaigns.zones )
AND ad_adverts.active =1
GROUP BY ad_adverts.adid
ORDER BY views,RAND()
LIMIT 1

If it wasn't for the fact that there is a 3-column index in ad_views on the ad_id, ad_zone_id and the ad_ip, the search through 800,000 records (dozens every second) would slow the forums down so much that the server would grind to a halt.
So indexes - yep we have them covered.

[OldSkoO1]

Thought as much

The site seems quicker to me now although we are on a 10Mbit connection at work. Was on last night and it seemed faster too with apache serving up the pages being the achilles heel. The actual content seem rather quick. Since the upgrades mentioned i can definatly say that the forums are now faster

[Ray]

Stuart,

Great work - certainly speeded up since last week ! On thing though - when it was going really slow (3-4 days last week ? Can't rember now!) it would have been great to have a small box on the front page stating what the current serviice was like e.g red,Amber,green - like the ISPs use. That would help users realise it's a site/server problem not their ISP or machine. I was tempted to e-mail you to ask why it was so slow but then I thought you'd be getting loads of e-mails and wouldn't appreciate another and you'd be well aware of the problem anyway. A small 'Current Site Status' box on the front page would have let us know that the problem was known and in hand.

Can I second the Member Search request as well. i.e. put in a partial name, get a list back, click on the name to send PM - or go to profile.

Cheers

Ray

[Stuart Wright]

Originally Posted by eviljohn2

So that it finds matching members so you can check their profile. The same feature as there is in the actual Member List but available on the standard menu.

Done.

[eviljohn2]

Stunning.

Thanks Stu.