[darcs-users] a practical application of whitespace within a token

[Albert Reiner]

[Bill Trost <trost at cloud.rain.com>, Sat, 03 Sep 2005 21:05:42 -0700]:
> The darcs' manual's description of the theory of tokens solicits
> practical applications of whitespace within a token.

Another example: in literate programming, you often want to name
chunks by some text that includes whitespace.  E.g., in noweb you
would write

<<do this>>=
code to do this
@

, and logically the chunk name "do this" is like a token.  Currently
(using "1.0.3 (release)") you can do the replace by specifying
--token-chars '[^<>]' (which does, of course, break as soon as you
have a chunk name like the perfectly legal "do that if $a>b$"), but
its effect is recorded as a normal line changes.

Regards,

Albert.