[darcs-users] GSoC: network optimisation vs cache vs library?

Max Battcher me at worldmaker.net
Sun Apr 18 21:02:18 UTC 2010


Much of your comments have been discussed at length in other posts.

Zooko Wilcox-O'Hearn wrote:
> Currently Trac-Darcs issues these queries only when the user has 
> indicated that this is what they want to see, for example by browsing 
> the Trac-Darcs page about a patch and then clicking the link to see the 
> file as it existed (in this repository -- the canonical one) before 
> and/or after the patch was applied.

Browsing/link-following (HTTP GET) is not a clear indicator that a user 
wishes to see a page. It is generally a bad idea in web development to 
perform actions that may have side-effects, or may require significant 
computation effort (as is this case) only on an HTTP GET request 
(browse/link-follow). This is because on the web, this can just as 
easily be a web crawler (search engine) or web cache tool making the 
HTTP GET request. Even when only expecting "legitimate" user 
interactions, a website should always keep in mind that multiple, and 
possibly many, simultaneous requests can and will occur...

I've made the suggestion to Lele that Trac+Darcs, in the case of "hard 
operations" such as this one, should make use of a processing queue 
(which is the standard, simple model most scalable websites use) and 
provide a better user experience in the case where the cache is not 
populated ("Please try again in a few minutes" and/or with an AJAX 
spinner and auto-refresh when completed).

I also made the suggestion that Trac+Darcs could provide a command to 
pre-populate the cache (easily built on top of a processing queue), 
which could be invoked outside of normal "business hours" (via cron) or 
even as a post-hook to darcs operations. (My own "darcsforge" system 
attempts to do as much of its caching as it can in an apply post-hook.)

(Also, if I may throw in a further recommendation, Zooko: sqlite is not 
well-optimized for web serving needs and you may want to consider 
installing a more traditionally optimized database server for your Trac 
needs, such as PostgreSQL.)

--Max Battcher--

More information about the darcs-users mailing list