[darcs-users] The future of darcs - glowing or gloomy

Thorkil Naur naur at post11.tele.dk
Sat Mar 7 19:40:04 UTC 2009


Hello,

So, with his permission, here is Eric's initial response to 
http://lists.osuosl.org/pipermail/darcs-users/2009-March/018181.html 
(Message-Id: <200903072035.11602.naur at post11.tele.dk>)

Best regards
Thorkil

On Friday 06 March 2009 18:29, Eric Kow wrote:
> Hi Thorkil,
> 
> Here's a first batch of thoughts
> 
> On Tue, Mar 03, 2009 at 11:06:47 +0100, Thorkil Naur wrote:
> > I am writing to you in private, not because any of what I would like to 
say 
> > couldn't be posted publicly, but because I am going to voice some 
> > disagreements with you that could lead to some discussion that one or both 
of 
> > us would perhaps prefer to be conducted privately.
> 
> Feel free to voice disagreements in public.
> 
> For darcs, my feeling is that I really have no business being its
> maintainer (my re-entry into the project was all about rallying the
> troops, not actually leading them, argh!), and so being stuck in this
> position I really do not want to be in, I muddle along, do my best, and
> most of all, try to learn from people who are kind enough to offer up
> their criticisms.
> 
> In an ideal world, David would still be maintaining darcs (without
> creating all that tension), and I would be scurrying around doing the
> boring admin stuff (treasurer type stuff, getting sprints going, etc).
> But here we are and so be it.
> 
> > For a while, I have had some concerns about the future of darcs, the
> > community and all those things.  The thing that eventually made me
> > write the present letter is your response "a newbies confusion with
> > repositories - darcs or git"
> > (http://lists.osuosl.org/pipermail/darcs-users/2009-March/018024.html)
> > with its statement "darcs will always preserve user intent" - sorry,
> > but this is not true in any significant sense.
> 
> This was really an innocent remark.  I was just saying that darcs does
> not use heuristics to determine where hunks should be applied; it does
> so with exact knowledge and hence, preserves user intent.
> 
> On the other hand, it is true that conflicts are problematic.  I think
> I can issue some sort of statement clarifying my remarks.
> 
> > The famous patch theory: What is it that makes darcs tick?
> > ----------------------------------------------------------
> > 
> > I dip into your reponse quoted above and find the statement "So darcs has 
a 
> > clean theory which makes it possible to use a revision control system in a 
> > new seamless way."
> > 
> > Honestly, this is not my impression. It could be that there really is such 
a 
> > clean theory in somebody's head, but I, for one, have not been able to 
find 
> > any trace of a comprehensive description of this theory.
> 
> I was thinking only in terms of the relationship between patch
> commutation and merging, which I felt was quite clean.
> 
> Conflicts may not be very well understood yet, and I can see if you
> think I should have taken greater pains to stress that.
> 
> > I will hasten to add that this fact does not, in itself, detract from the 
> > value or usefulness of darcs, but I do believe that darcs must be 
evaluated 
> > in a much more down-to-earth manner.
> 
> Fine.
> 
> > Now, rather than leaning against some fancy theory here, I believe that we 
can 
> > firmly attribute the usefulness of the original darcs to the brilliance of 
> > David Roundy.
> 
> Now there's two things here.  Yes, a lot of the friendliness of the user
> interface is probably a result of David's foresight.  That's not what I
> was talking about.  What I was alluding to is the fact that interactive
> hunk selection, patch merging and cherry picking and selective undo
> all fall out of the single operation of patch commutation.  These aren't
> separate things; they're all just things you get for free once you know
> how to commute patches.  That's what I meant in my enthusiasm for the
> patch theory, an enthusiasm for which I was absolutely sincere.
> 
> Maybe it was wrong for me to this enthusiastic.  I do find it unsettling
> that nobody really deeply understands what happens when there are
> conflicts (I think Ian does), and I am sad that there doesn't yet exist
> a single reference document you can go to to learn what darcs is doing
> on the inside.
> 
> > In any case, what worries me profoundly here is the apparent lack of 
> > familiarity in the community with central parts of the darcs 
implementation, 
> > probably what in some places is called the darcs core. As indicated, I do 
not 
> > believe that we can lean against any patch theory, I would even go so far 
as 
> > to claim that this theory doesn't really exist.  All we have is this 
> > wonderful program, initially written by David Roundy.
> 
> So I'm not a mathematician, or much a computer scientist for that
> matter.  So maybe my standards for calling things beautiful or elegant
> are too lax.  I wasn't speaking formally about the "theory" of patches
> (I'm just not used to thinking about whether or not things can be
> considered a proper theory or not, and wasn't trying to sell it from
> that standpoint).  All I was saying was that you get exact patch
> application once you have a history, and that you have a lot of
> seemingly disparate features that emerge from a single operation.  That
> sounded "clean" to me, and that was all I meant.
> 
> > On various occasions, I have had the opportunity to ask questions in
> > the community about the darcs handling of patches and other technical
> > issues. And I have followed discussions of these matters in the
> > community.  My impression may certainly be wrong, nevertheless I
> > believe that absolutely nobody in the present community would be able
> > to handle some fundamental difficulty with the central patch handling
> > code in darcs that may come up.
> 
> You're right to point out the fact that not enough people understand
> darcs.  I think Ian gets this patch theory work very well.
> 
>   http://wiki.darcs.net/index.html/PatchTheoryPeople
> 
> But you also have to look at the bigger picture.  This stuff will come.
> If anything, I may just have to wade in myself and learn the darcs core
> better.  We have to focus on building the community and hopefully
> bringing David back into the project....
> 
> > This conclusion is, admittedly, based on rather skimpy evidence. I try
> > not to be too forward in the way I communicate. But I clearly sense a
> > lack of enthusiasm in the community when the subject closes in on
> > these central issues.
> >
> > A few observations that may support this view are: (1) The lack of 
response to 
> > Trent's very relevant observation of duplicated code in darcs "Duplication 
> > between Prim and 
> > 
Commute" (http://lists.osuosl.org/pipermail/darcs-users/2008-December/016633.html);  
> > (2) The lack of relevant response to "Issue 1363 mark-conflicts says there 
> > are none, but push says there are" (http://bugs.darcs.net/issue1363) which 
> > could, as I see it, indicate some fundamental (although not necessarily 
> > significant) problem with darcs 2.
> 
> I don't really think that's very fair.  Sure, for #1 we just don't have
> enough people looking into core code to answer (I'm still tied up doing
> admin).  #2 looks like triage failure on my part.
> 
> Attention is scarce.  I have a workflow which is based on conserving
> attention by getting rid of things once I feel I have done what I can
> (for example, if I'm waiting on a response).  I'm not always very
> diligent about marking when I should follow up on something should I not
> get a response... which is what happened there.
> 
> > Lack of familiarity in the community with the darcs core is, of course, 
> > extremely serious in the long run.  I could be wrong, but if familiarity 
with 
> > the darcs core is as thinly spread as I believe it to be, we should be 
much 
> > more expressive about it and work towards increasing familiarity.  Exactly 
> > how to do that is not clear, of course, but the problem needs first of all 
to 
> > be acknowledged and perhaps even mapped in some detail.
> 
> Fair enough.
> 
> > The matter of the "borked CRC" (Issues 842, 844, and 1239)
> > ----------------------------------------------------------
> > Let me tell you first how I would react if I found that some routine error 
> > checking had been left out in a program that I was responsible for and 
that, 
> > furthermore, could be handling the crown jewels of my customers: I would 
> > introduce the required error checking, unconditionally, in the working 
> > version of my program. And if there were several versions (such as in this 
> > case where several ways to link to zlib exist), I would introduce error 
> > checking for all the versions.
> 
> Sounds like a project for you to work on during the sprint.  Are you
> interested?
> 
> > Furthermore, I would institute, at a very high priority, a review of
> > all the code, to find any additional cases of such lack of care. And
> > correct them.  And I would make sure that no further versions of my
> > program were released without these routine checks.
> > 
> > Only then would I take stock and consider my further moves. Most
> > likely, my program would now be in some more or less crippled state,
> > not working properly.  And the task ahead would be to fix that
> > problem, including, of course, taking care of the mess (in this case,
> > repositories with CRC errors in them) that has been created. But I
> > would not accept the continued existence of a circumstance (lack of
> > routine error checking) that increased the risk of my customers being
> > hit by additional, independent, problems, undetected.
> 
> I appreciate the criticism.  Things like these are good to be vocal
> about from the beginning.  But realistically, given the resources
> we have right now, who is going to do these things?  Are you?  I
> have 8 hours I try to spend on darcs (since getting back into things
> last month).  For now, I spend all of it writing emails.  Hopefully
> the writing emails phase of things will draw to a close, and I can
> start thinking about doing some hacking.  Sometimes we go with a
> suboptimal solution.  It's not out of irresponsibility, we simply do
> what we can.
> 
> > Obviously, such a reaction could not be played out without my customers 
> > noticing. And this could, of course, become somewhat unpleasant, having to 
> > explain that you have made this glaring blunder and so on. But that's what 
I 
> > would do, face the music, explain carefully and seriously the exact nature 
of 
> > the problem, how it could be detected, its immediate resolution (stopping 
the 
> > leak) and, as soon as possible, publish a repair.
>  
> > 1. duncan.coutts observed that gzclose error returns were ignored, but the 
> > matthias.andree investigations showed that gzwrite were called with a zero 
> > length buffer (or something like that) and got a response from Vladimir 
> > Nadvornik <nadvornik at novell.com> on the Novell BTS that indicates that 
darcs 
> > is also ignoring error returns from its gzwrite calls. Has this been 
adressed 
> > by anyone?
> 
> Please chase up on this.
> 
> > 2. The question has been raised, but no answer given, as to why darcs is 
> > writing zero-length buffers at all. I am not at all saying that writing 
zero 
> > length buffers is necessarily a bad thing: To maintain regularity of one's 
> > code, writing zero-length buffers could be a perfectly sensible thing to 
do. 
> > But the droundy patch to resolve this makes me worry again: If some part 
of 
> > darcs is writing zero-length buffers, another part could be expecting to 
read 
> > zero-length buffers. And if they are no longer being written, how will the 
> > reader be told?  There could be several simple answers to this, but I have 
> > not seen any.
> 
> I would welcome a code audit effort.  Again, I think the sprint would be
> a good time to do this because we actually have the time to concentrate
> on something for a longish time.
> 
> > But my main frustration has been this rather fundamental disagreement with 
the 
> > approach chosen to resolve the problem. And the lack of answers to certain 
> > urgent questions, as noted earlier. I have voiced my opinion and asked 
> > questions on #darcs a few times, but the reaction I got from there left 
> > little doubt in my mind as to what people thought about this. And this is 
> > where I believe we move into project policies rather than mere issue 
> > management. And hence, calls for a kowey rather than a thorkilnaur.
> 
> I think we should talk about this more at the sprint.
> 
> > The role of tests and documentation when working with darcs
> > -----------------------------------------------------------
> > would like to add "missing documentation" as a further sore spot to 
"missing 
> > tests".  I am not sure about internal (primarily Haddock?) documentation, 
but 
> > I certainly have the feeling that the external documentation, I mean the 
> > darcs user's manual (the one I have printed in the 1.0.8 edition) and the 
> > darcs --help texts are not being cared for sufficiently.
> 
> Again, this you should be taking up with us in general and offering
> concrete ways to move forward on.  I think we're doing the best we can
> here.
> 
> > I know that 
> > certainly Trent has been doing a lot of work here, but still, I have the 
> > impression that changes are allowed into darcs that modifies its 
behaviour, 
> > as seen by the user, but without necessarily also changing the 
description, 
> > also seen by the user, of that behaviour.
> 
> Specifics would be good
> 
> > And here is another point where I have disagreed with what has happened: 
At 
> > some point, some mechanism which was used to help keep (some of the) 
> > documentation and code synchronized was removed, the code and 
documentation 
> > separated and thereby allowed to diverge in peace. The argument for doing 
> > this was that this complex mechanism would tend to scare off people who 
were 
> > considering contributing to darcs. My priorities in such a matter would be 
> > very different here.
> 
> Here too
> 
> > I don't believe that we are doing this at present, not very well at least. 
> > What I fear might happen as a result of this is a steady decline in the 
> > quality of our documentation: Not only are developers allowed to 
contribute 
> > code without documentation, but people who may want to contribute 
> > documentation are discouraged because whatever work they have done today 
may 
> > become out-dated without notice by tomorrow's code change.
> 
> This is where you need to chime in when you see patches on the mailing
> list.
> 
> > So, I am all in favor of strict requirements to darcs contributions: Both 
> > tests and documentation should be included with code changes. Easier said 
> > than done, I know. But if not said, it will not be done. And darcs will 
> > wither.
> 
> I don't really mind strict policies and can see how they will pay off in
> the long run.  But it's the same with the testing.  I have to see them
> in practice, make sure they are doable before I will try and implement
> them.  I'm not being very firm here (although I also get complaints
> about policy on the other end, that it should be more lax), and I am
> emphasising a lot of decentralisation of the decision making here.  All
> of this is a strategy for conserving time and attention.  It may not
> lead to the best results.  If we continue making progress building up to
> something more robust
> 
> > I am not sure what the conclusion here is. From my point of view, it has 
been 
> > useful to express these things that I have had difficulties expressing 
> > elsewhere, so thank you for reading as far as this. I don't have any 
> > particular expectations as to what your response to this will be, but I 
hope 
> > that you will find the insight that you get into how I look at things 
useful.
> 
> You should never hesitate to be critical.
> 
> I'm sure my responses here are not very satisfactory, and I think the
> best thing is just to talk about it more over that beer that I owe you.
> 
> The thing I would really emphasise, like I did with Petr is that if you
> want to change a policy, the best thing to do is take action yourself.
> I'm still trying to work out what my role in this team in.  I'm still
> trying to put out fires, so to speak, (although things are starting to
> settle down a bit), so the amount of higher level thought I'm going to
> put into these things is limited.
> 
> -- 
> Eric Kow <http://www.nltg.brighton.ac.uk/home/Eric.Kow>
> PGP Key ID: 08AC04F9
> 


More information about the darcs-users mailing list