My latest open project is a “SETI-at-home-like infrastructure for massively distributed CPU-intensive jobs based on HTML5 WebWorkers and node.js for distributing tasks” (quote from Pedro Teixeira’s blog).

It’s called Map Crowd Reduce and it’s available on github.

This project made into the final SAPO Codebits 2010 programming contest, but unfortunately didn’t win :)

The project consists of a node.js server that starts a HTTP server on port 3000. To start a new computation, you must supply some javascript functions: a split/segmenter function that generates jobs, a map function, and a reduce function.

After that, the system gives you an URL that you can share with anyone wanting to contribute with CPU to the computation. The client side functions are run inside a WebWorker so it never blocks the main UI thread.

With the code there are 2 prewritten examples:

• find the number of primes between 0 and 60000 (brute force)

this uses a brute-force method of testing if each number is a prime, from 0 to 60000. it runs very fast and can be used to debug.

• find a MD5 collision with the MD5 on all words with 5 or less alfa words

this lasts 15 minutes on a single machine. when I tested it on the main stage, 50 people connected their browsers at it lasted like 20 seconds (#win).

We are currently trying to write a parallel Mandelbrot fractal using map reduce :) it will be awesome.

Finally, I couldn’t have built this project without the help of some friends.

Today I’m announcing version 2 of twittersslinks.

TwitterRSSLinks is a Twitter (OAuth) service that filter all your timeline tweets that contain HTTP links, so you can aggregate and watch them later and so on. The website outputs the tweets in a variety of formats including JSON, RSS and ATOM.

Version 1 was shutdown because I had no plan for monetizing the service, and at the time, there was a good alternative. Now the alternative is dead and Twitter UserStreams is in Beta.

There’s a good excuse to learn node.js. :)

Without further ado, here’s version 2 architecture (sorry for the quality, I am definitly not a drawing guy…):

The site is completely free, and now beneficts from the real-time user-stream feeds. Also, I’m using pusherapp for the first time, just in case you are on the browser and a new tweet arrives :)

The live website runs on a EC2 Micro instance, and I will definitly open the source in the near future. Happy tweets :)

Sunday monday fun: gowalla gem, google maps, some Javascript stuff, and a Rails app.

Gowalla spots Google Maps Mashup [http://gowallamaps.heroku.com]

It tries to use HTML5 geo location to get a position on the map (defaults to somewhere on USA). Probably best used on the iphone (tried on a friend’s Android and it failed :S). Screenshot:

Simple hack, and a reminder for Gowalla to open up their API :) Thnks @pengwynn for the awesome gem :)

ActivityStreams is the next big thing ^TM. It defines a protocol to syndicate activities taken from web applications (mainly social ones) and services. Just imagine a common spec for describing the activities a user do on Facebook, FriendFeed, Delicious, etc.

The protocol originally targeted the Atom XML Standard, via an extension. Later, some work has been done on standardizing a JSON format too.

With that in mind, I’ve tried to implement a Ruby parser that defines a common API to access both the Atom and the JSON specification. Is is now available on github and also as a Ruby gem.

Here’s a simple demonstration on how it works. We take the excellent Cliqset feed proxy and get an activity streams annotated Twitter Atom feed:

The JSON API is very similar. Some documentation is up on rdoc.info and it is worth checking the spec too. Have fun :)

Quick post to let you know about my latest hack: basic support for OpenCL on MacRuby and Snow Leopard.

• it is my first Ruby C extension ever
• it is my first OpenCL approach ever
• it is my first MacRuby hack (ever)

With this in mind, don’t expect real world usage from this code, nor that it will get merged into upstream: it was just written for fun!

The MacRuby branch with the changes is located here

Example

The hello world on OpenCL is probably “calculate each element’s square on a given array”. The code below shows how this can be done in MacRuby:

This code runs the same computation on all available OpenCL devices on your hardware. The results on my hardware (2 GPUs, 1CPU) are:

It was fun, and maybe it can inspire someone :) The OpenCL API is kinda complex and I’m not sure if there’s a good way of making it Rubyish enough. Suggestions are welcome! Have fun :)