[darcs-users] Re: need to jump in with both feet

Eric S. Johansson esj at harvee.org
Tue May 25 13:11:54 UTC 2004


David Roundy wrote:

> On Mon, May 24, 2004 at 07:08:33PM -0700, Samuel A. Falvo II wrote:
> 
>>On Monday 24 May 2004 06:51 pm, Martin Pool wrote:
>>
>>>I don't think that darcs would need the high-volume or anti-censorship
>>>properties of something like freenet.  The volume of patches
>>>developers can produce in a week is pretty small.
>>
>>My suggestion for P2P networking was intended to reduce the load of 
>>individual contributors in the SourceForge-replacement service.  I must 
>>have missed something where P2P made the jump to an intrinsic feature of 
>>Darcs itself.
> 
> 
> The idea would be to better support many users darcs getting a largish
> repository.  For example, if I wanted to publish the linux kernel
> repository on the abridgegame.org website, all it would take would be 15
> people running darcs get on it in order to use up my 30G limit for the
> month.  A P2P system might allow more people to use the repository without
> swamping the server.  That would be the theory, anyways.
> 
> Anyhow, the idea is that the high-volume properties could come in handy,
> *especially* if you don't have a nice high-bandwidth server as I do.  In
> that case P2P may be your only hope of allowing many people access to a
> large project you host.

this raises an interesting question.  In order for a peer-to-peer 
network to work, all copies must be identical.  Now if memory serves, 
all repositories would have all the patches up to the point of last 
synchronization.  So it would be possible for someone to get the 
baseline patches for a project off of any number of repositories even if 
they have created their own variant branches.  The challenge becomes 
distributing knowledge of where to get which patches from.

It would probably be a good idea to take a look at mute 
(http://mute-net.sourceforge.net/) for a potential architecture.  The 
anonymity feature, while not strictly necessary, comes for free.

a bit torrent model only works if you have a) an authoritative center, 
b) reasonably large files.  I seem to remember some discussion saying 
that mp3's may be on the borderline of not large enough for effective 
bit torrent used but that's a fuzzy memory.

the discussion started with the idea of a distributed sourceforge 
replacement.  I'm not sure this is necessarily a good model because it 
does describe software development with a rigid center.  I would argue 
for a somewhat different model.  One that is more dynamic and movable. 
For example, when people describe a project in a sourceforge context, 
they are typically describing services of:

Textual information presentation (i.e. documentation, news etc.)
binary information presentation (i.e. distributions etc.)
source code control
bug tracking
common communications channel (i.e. mailing list, forum)

each one of these services comes with a different distributed model. 
for example, textual information is typically represented by http 
services and can be handled in a variety of ways ranging from 
distributed caches to browsers as servers model.  distributing binary 
information is also a well-known problem with lots of solutions

but what I suspect is most interesting to this group is the distribution 
problem for source code control.  It seems to me that the basic "here 
are some changes, now I will integrate" part of the problem has been 
solved nicely and with a great deal more rigor than most other systems. 
  What I think is necessary for a truly distributed workgroup is a way 
of becoming entering or exiting a group, push notifications (not 
necessarily patches), and automated mechanisms for making patches 
available in a world where end-to-end connectivity is broken.

One way to solve both the communications distribution problems and the 
push notification mechanism is to develop a distributed mailing list 
mechanism.  there could be multiple mailing lists carrying both human 
and machine to machine traffic.

entering and exiting the group should be automated but it still only 
Tuesday and my brain is not fully in gear.

Making patches available in the "consume only" Internet world may have 
an interesting solution that was proposed to me earlier which is to keep 
your repository behind the firewall and push a disposable copy out to a 
publicly visible web site.  Now this model doesn't solve the distributed 
load problem but it's a point from where you can create a solution.

I really need to finish this anti-spam projects so I can go do something 
interesting like this.  Solo development sucks after awhile.

---eric





More information about the darcs-users mailing list