[darcs-users] machine-readable formats (Was: the state of the adventure)

Jason Dagit dagitj at gmail.com
Sat Sep 4 06:32:57 UTC 2010

Eric Kow <kowey <at> darcs.net> writes:

> Perhaps it'd be good to try to think a bit globally about this.

Please allow me to weigh in on this change to annotate (perhaps with
gmail's new priority inbox I'll actually get back on the list).  I've
tried to sectionize my rather long post for easy skimming.

----- Context --------------------------
Things that I see as important (ie., the context for my opinion):
  * Satisfying stakeholders
  * Backwards compatibility
  * Keeping orthogonal things orthogonal
  * I love open standard (when they make sense)
  * freedom is important

Let me also state up front that, I'm not demanding anything, just
giving my $0.02.  I'm just voting.  If you disagree with my points
then, well you disagree with my points.  Thanks for taking the time to
read them.  Hopefully you are able to approach my points with an open

------ Main Discussion -------------------

  * I think Zooko is absolutely right that we need to ask the right
    people how they feel about a format change.

  * I really dislike making user visible changes without involving
    users.  I just feel like it's less thoughtful than we can be, and
    thus potentially rude.

  * I think our current annotate is totally lame, ridiculous,
    unreadable, and just plain silly.

  * Our current annotate might be the way it is for a reason?

So...based on all my bullet points, I would say, that we should
definitely have a discussion about the annotate format.  Do we have
any volunteers who enjoy institutional knowledge that would like to
report back about how the current annotate format came to be?  It's
possible, although admittedly unlikely, that it could help inform the

In the mean time, I think we should keep with the existing format so
that we:
  * keep backwards compatibility
  * separate the format change from the speed improvements
  * give users a chance to speak up

There are two reasons why I think we should separate the format change
and the speed improvements. The first is that I think that keeps
orthogonal things orthogonal.  Second, for anyone who will be affected
by us changing the format in a non-backwards compatible way, I believe
we should provide a release of darcs "2.Recent" that features fast
annotate.  That's just a way to say, "hey, we're sorry to change the
format on you, please enjoy this last release of the old format," to
our users.

When we switch to a new format, I think we should try to be compatible
with other vcs's annotate output. I think this helps take care of my
desire to use open standards, even de facto ones.

Is it possible for us to match git-annotate?  I think the hurdle here
is that they give hashes in their output.  I'm not sure if they are
change hashes or tree hashes.  The feasibility of matching git depends
on which one it is, I think.

We could give hashes of patches, which might be close enough.  If they
are tree hashes we could propose to make tree hashes meaningful for
darcs now that we use hashed-storage, and then wait to change the
annotate format after that change is in effect.  I've been thinking
about this lately in terms of user visible changes and I'm not sure
it's a good idea but I should really write up my thoughts and send it
in for discussion in separate thread.

About: XML vs. Yaml vs. regular language vs. ??

I think having an extensible, attribute rich, machine parsable format,
like XML or Yaml, is a worthy and valuable feature and an honorable
goal.  I think using a (mostly) stable regular language format for
reading by humans is also very valuable.  I don't think having both
formats has high maintenance costs in a language like Haskell.  It's
just a matter of choosing a pretty printer in some sense.  In other
words, in my professional opinion the cost of having both is low
relative to the value it will bring to users.

In conclusion, my stance is:
  * Keep the current format for now
  * In the future, redesign annotate output:
     * Provide one attribute rich machine format (XML, Yaml, etc)
     * Provide one regular language for humans and shell scripts/grep
     * Use existing formats, if possible

-------Meta Discussion-----------------------

Now, a meta-note for the process geeks in the crowd. :-)

I think we have really long debates about compatibility-breaking
changes.  Healthy debates are good!  However, I wonder if darcs, as a
project, would benefit from having guidelines about deprecation of
features and compatibility breakage.  That is to suggest, a formal
process by which we agree to change things.  To my knowledge, we don't
have a "standard" approach to these debates.  Please correct me if I'm
wrong about that.

My rough idea (and I'm not good at these process things; I'm sure Eric
will have much better intuition about it than me):

  * Jane Hacker or Joe User submits an idea for a
    compatibility-breaking change (feature redesign, feature removal,

  * We have a healthy debate about how to best achieve it, isolated
    from the discussion of the work itself (ie, the patch review), in
    part to avoid miscommunications/hurt feelings.

  * Once the proposal is accepted, the implementation work can begin

  * The feature(s) affected by the proposal is marked as DEPRECATED in
    the manual, UI, maybe we issue warnings at run-time, etc.

  * After N stable releases *featuring the deprecation notice*, we are
    allowed to completely gut the feature from darcs.

Perhaps the default value of N would be 2?  Giving users one year to
get used to the idea.  Longer or shorter on a case-by-case basis.

I think what I've described is actually standard practice for some
large projects (python and GHC come to mind for me).  I think having
this process helps users to know what to expect while giving them time
to adapt and voice objections.  It helps proposers by giving them a
way to vet their idea.  Also it gives everyone an agreed upon standard
for evolving user visible aspects of darcs.

I could see this process applying to annotate starting NOW and
changepref in the near future :)

Thanks for keeping an open mind,

More information about the darcs-users mailing list