[Pydra] GSoC Web Crawler

Yin QIU qiuyin at gmail.com
Wed Apr 7 05:09:31 UTC 2010


I was a SoC student for Pydra last year. I just read your proposal and
it looks good!

I have one piece of advice though. Can you give more specific details
on how you would leverage Pydra's power to implement the crawler? I
mean, your proposal did show a good map of a distributed web crawler,
but maybe it's better to include your thoughts on how to realize your
idea with the existing Pydra task interface (e.g., ParallelTask and
MapReduceTask). We are talking about a *Pydra* application, right? ;-)

On Sun, Apr 4, 2010 at 6:45 AM, Jesus Mager <fongog at gmail.com> wrote:
> Hi all!
> My name is Jesús Mager, student at the National Autonomous University
> of Mexico, and I'm very interested to articipate in this summer of
> code for the Web Crawler Idea. I already submitted my proposal, and
> discussed some issues with peter. I hope you can give me some
> feedback. If sombody that isn't a mentor want read my poposal the link
> is: http://code.kiutz.com/gsoc/pyder.html
> --
> Jesus Mager
> [www.h1n1-al.blogspot.com]
> _______________________________________________
> Pydra mailing list
> Pydra at osuosl.org
> http://lists.osuosl.org/mailman/listinfo/pydra

Nanjing University, China

More information about the Pydra mailing list