[darcs-users] GSoC: network optimisation vs cache vs library?
me at worldmaker.net
Sun Apr 18 21:02:18 UTC 2010
Much of your comments have been discussed at length in other posts.
Zooko Wilcox-O'Hearn wrote:
> Currently Trac-Darcs issues these queries only when the user has
> indicated that this is what they want to see, for example by browsing
> the Trac-Darcs page about a patch and then clicking the link to see the
> file as it existed (in this repository -- the canonical one) before
> and/or after the patch was applied.
Browsing/link-following (HTTP GET) is not a clear indicator that a user
wishes to see a page. It is generally a bad idea in web development to
perform actions that may have side-effects, or may require significant
computation effort (as is this case) only on an HTTP GET request
(browse/link-follow). This is because on the web, this can just as
easily be a web crawler (search engine) or web cache tool making the
HTTP GET request. Even when only expecting "legitimate" user
interactions, a website should always keep in mind that multiple, and
possibly many, simultaneous requests can and will occur...
I've made the suggestion to Lele that Trac+Darcs, in the case of "hard
operations" such as this one, should make use of a processing queue
(which is the standard, simple model most scalable websites use) and
provide a better user experience in the case where the cache is not
populated ("Please try again in a few minutes" and/or with an AJAX
spinner and auto-refresh when completed).
I also made the suggestion that Trac+Darcs could provide a command to
pre-populate the cache (easily built on top of a processing queue),
which could be invoked outside of normal "business hours" (via cron) or
even as a post-hook to darcs operations. (My own "darcsforge" system
attempts to do as much of its caching as it can in an apply post-hook.)
(Also, if I may throw in a further recommendation, Zooko: sqlite is not
well-optimized for web serving needs and you may want to consider
installing a more traditionally optimized database server for your Trac
needs, such as PostgreSQL.)
More information about the darcs-users