Jon Aquino's Mental Garden

Engineering beautiful software jon aquino labs | personal blog

Tuesday, June 14, 2005

Processing a mountain of email -- attack it from both ends

When you have a mountain of email that you want to process, there are two schools of thought: start with the oldest, and start with the newest. They both have pros and cons. If you start with the newest, you might not get to the oldest for months. And if you start with the oldest, you might not get to the newest for months.

I'm going to try alternating between the two ends. I'll process the oldest email, then the newest, then the oldest, then the newest, etc. Ideally I would randomize my inbox, but Gmail doesn't support this. Anyway this is probably close enough.

Part of the reason I'm getting so much email these days is because of YubNub, that popular web application I wrote last weekend. Here's a great picture of a blog post from a very distant part of the world. (Anyone know what they're saying?).



  • Yes, I've had similar tasks, and I've discovered that thinking about it as garbage collection is useful. As in garbage collection, it turns out that working from both ends is often the right thing.

    This approach is called Generational Older-First. You keep the youngest heap cells (excuse me, messages) in a nursery, where you can get rid of most of them right away. The messages that survive then get "tenured", as the GC folks say. Tenured messages probably follow a radioactive-decay model, that is, they become junk randomly and independently, so the longer a message has been around, the more likely it is to be junk. So for cleaning up old messages, the right thing to do is to start from the oldest, since then you get to delete more messages per minute.

    Anyhow, this algorithm is called Generational Older-First, and it's due to Will Clinger and Lars Hansen. You should check out their papers on this subject.

    By Blogger lcmaven, at 6/15/2005 7:36 AM  

  • Looks like you're famous Jon. Yubnub's featured on today!

    By Anonymous CHaN, at 6/16/2005 9:52 AM  

  • lcmaven - I'm delighted! I didn't know this technique had been researched in the informational sciences. Cool!

    CHaN - Really! I'll check it out! Yippee!!!

    By Blogger Jonathan, at 6/16/2005 1:00 PM  

  • hey very cool :) you've redone the 24 hour dotcom site in the rails competition! We actually still have up and running since end of last summer...i'm totally addicted to it but I haven't bothered with updating it since I thought no one would use I realise maybe they will..?! anyway: you (or we) have a competitor: ;) ..see u in cyberspace..

    By Blogger Eric Wahlforss, at 6/20/2005 6:06 AM  

  • congrats Jon!! you've now been slashdotted for your great 24hrRoR project.

    By Anonymous nessa, at 6/20/2005 11:08 AM  

  • Eric - Hm! I had read about your 24-hour dotcom project. Great minds think alike! Let a thousand flowers bloom, as they say.

    nessa - Yippee!

    By Blogger Jonathan, at 6/20/2005 12:51 PM  

  • Spreading the word... I´m writing a whole cover page article on YubNub in the newspaper I work for, and that is even after suggesting it as a great tool for the Web.

    Excellent work, Mr. Aquino.

    Cheers from México.

    By Blogger Talya, at 6/20/2005 9:06 PM  

  • Wow, thanks Talya! I'm thrilled!

    By Blogger Jonathan, at 6/20/2005 9:08 PM  

  • Wow, I didn't realize YubNub was such a new phenomenon. You basically became famous overnight with it. Congrats.
    (found your blog while googling YubNub at random)

    By Blogger xybre, at 6/22/2005 12:16 AM  

  • Hi xybre - Yeah, guess this is my 15 minutes of fame, heh. Everyone is entitled to 15 minutes!

    By Blogger Jonathan, at 6/22/2005 12:47 AM  

Post a Comment

<< Home