The April Fool’s joke that caused data migration headaches

On April 1, 2005, del.icio.us started getting a few strange bookmarks that looked like this:

2019790245_ff88ac7b3f_o

Color bookmarks! If you posted something like color:FF0000,FFFFFF,0000FF in the URL field (instead of a normal URL), you’d get a little set of color swatches.

Joshua had quietly implemented this and saved a few colors, and friends noticed and started saving colors too. We watched the “recent” page as people discovered it and played with it. The popular page for the tag “color” started to get wonderful. Another example:

432118726_04d1f1862b_o

Color bookmarks were an April Fool’s joke, from a time when April Fool’s jokes on the web were possibly-believable fun things. I see it also as a gentle experiment in expanding what del.icio.us could “bookmark” and seeing how people responded. This was before the Yahoo! acquisition (October 2005) and a year before Twitter started. It wasn’t clear whether del.icio.us should only be for links, or whether the scope and ambition should expand. What else would people want to save and share? For example, should people be able to post short notes without an associated link? What kind of community management would that need, and what could go wrong? Twitter answered those questions later.

On April 2, 2005, Julia West wrote a blog post pointing out that color bookmarking was oddly off-brand, and also that “the color syntax in place of the URL is useless outside of the del.icio.us interface, botching RSS feeds and the importing of bookmarks into a user’s browser”. del.icio.us had a mailing list for users and team members, and John Resig expanded on her post with a detailed complaint and suggestion about the new color bookmarks:

I was reading a delicious RSS feed of one of my friends and saw a link to one of these new color: items, I clicked it (not thinking) and was immediately treated to an error message. Up until this point I had thought that this addition was amusing, and mildly interesting at best. Now I was just annoyed and very upset. Essentially, this URI will only work on delicious and will only ever have meaning to those who view this information on the delicious web site. This is a serious problem.

John’s post goes on to recommend a different URI for color bookmarks, with rigorous reasoning about the purposes and uses of URIs.

I love these posts because they’re a wonderful example of early del.icio.us: people who loved the web, loved tools for the web that used structured data (such as in RSS feeds and APIs) to enable new things, built new things, and debated them with each other. It was a time of geotagging, microformats, the semantic web, and machine-readable ontologies for friendship.

What happened after April’s Fools Day? del.icio.us had a principle of not deleting user data, and some people were fond of color bookmarks, so the tiny undocumented feature stuck around.

Later at Yahoo!, del.icio.us spent a couple years preparing to become Delicious 2.0: a complete rewrite of the aging code, a major redesign, a massive data migration. The color bookmarks were still there, and we had to keep them. There was no way we could jettison the color bookmarks after years. So the backend and frontend engineers rebuilt what had been a tiny bit of Perl in the new enterprise-class PHP framework. The designer had to accommodate color bookmarks. But oh, the data migration — the color bookmarks caused many complications for data migration, and the team complained and fixed bugs in their Erlang. It was amazing that such a small thing had such an effect.

I believe color bookmarks finally died when AVOS rewrote the site in 2011.

Reddit’s origin in a conversation about Delicious

Reddit’s visual style hasn’t changed much since its early versions in 2005, and if you squint, there’s something similar to how del.icio.us looked at the time:

October 2005 reddit

September 2004 delicious

It’s not a coincidence. The co-founders of Reddit (started in 2005) were inspired by the del.icio.us popular page, via a conversation with Paul Graham of Y Combinator. Reddit co-founder Alexis Ohanian has told this story in a few places, and each gives a perspective on how this happened.

An interview with him in December 2005: “We were actually inspired by del.icio.us/popular, it’s a great indicator of what people are busy bookmarking, but we were interested in the more ephemeral links that are popular because they are interesting and new — not just good reference material.”

A comment by him in 2013 that explains what happened after Y Combinator rejected the team’s original application and wanted them to try a different startup idea:

“PG asked if we’d heard of del.icio.us (neither of us had) and pulled it open on a browser to show it was getting at (tho not directly) a solution for finding out what was new and interesting online…’That’s it! You two need to build the frontpage of the internet!'”

“At this point we had no idea what the functionality would look like, other than something like del.icio.us with submitting links and headlines, but we knew we needed an emphasis not on reference material, but on ephemeral ‘news’ and some kind of voting mechanism, which we’d figure out when we graduated and moved to Boston in a couple months.”

This excerpt from Ohanian’s book, published in 2016, goes into more detail about that story.

Mashable wrote a history of Reddit (2014) that includes interviews with founders of Reddit, del.icio.us, and Digg, and it explains how Reddit’s adoption of a subreddit model propelled it past del.icio.us and Digg (which didn’t):

“Then in early 2008 they decided to let users create their own custom reddits, or subreddits. The move ignited the Reddit community and truly differentiated it from Digg. Alexis had been in favor of a more traditional tagging system, but Steve pushed for subreddits.”

“Joshua [Schachter], founder of Del.icio.us, says he proposed creating a similar feature called Islands, but Yahoo, which acquired the startup in late 2005, was against the idea. The same was true at Digg. ‘I actually advocated for something like subreddits,’ says Owen from Digg. He had planned to call it ‘Personal Digg,’ but founder Kevin Rose, who he refers to snidely as the ‘arbiter of coolness,’ balked at the feature for fear of “potential chaos that would bring.””

“Rose was right: It would bring chaos. Reddit users later created subreddits showing pictures of minors and creepshots of women. It is what makes Reddit vibrant, but also what makes it volatile.”

Chaos is a light term for problems Reddit has had and caused. Here’s the del.icio.us popular page today:

Screen Shot 2017-06-11 at 1.42.46 PM

The elaborate scam of automated bookmark spam

One of the things I worked on at Delicious was anti-spam. We rarely deleted anything, we just marked a lot of spam accounts NIPSA (Not In Public Site Areas), a technique we had in common with our sister Flickr for dealing with accounts that broke certain rules. The spammer would keep posting spam, it just wouldn’t be seen by anyone unless they directly visited the spammer’s page.

This meant most spammers didn’t notice they had been blacklisted. This was mostly good, because it meant they didn’t complain or develop more sophisticated techniques, but it had a weird side-effect of enabling a third-party scam business in spamming tools.

We received floods of automated spam links submitted by make-money-from-home wannabes who hoped bookmarks were the key to mega traffic and top Google rankings. Spamming Delicious had no effect – nobody clicks random spam bookmarks, and the bookmarks were “nofollow” links ignored by search engine rankings – but that didn’t stop them.

The key was that more-sophisticated entrepreneurs were making money by selling spamming tools to these poor souls, with lovely promises of an “Unlimited Supply Of High PR Backlinks And Laser Targeted Traffic From Major Bookmarking Sites… All Done In Minutes On Autopilot!”

The most popular scammy spamming tool was BookmarkingDemon, with its first release in 2006 and regular updates until 2014. Sadly, even the Delicious spam market has fallen apart — the site says “BookmarkingDemon is not available for sales and will cease operation after 31st Oct 2017”.

You can still enjoy the intense marketing copy and bad graphics for now! For some reason, these spam tools always had an imaginary software box in their graphics. I think they all got sold a box-rendering tool by another entrepreneur.

image

The spam ecosystem went even deeper. Many people bought tools like Bookmarking Demon and sold spam-bookmarking-as-a-service, which was probably the most effective way to make back your $147 investment, as seen in this screenshot from another site:

image

Other miniature kingpins built bookmark spamming tools and set up affiliate programs so that aspiring kingpins could sell the tools and take a cut. That made the affiliates fight each other to get the best SEO for their spam tool sales websites, even selling each other bookmarking-spam-tool-related domains with grand promises that their domain would get you the best results.

I saw it. There are many levels of hell in shady webmaster forums, and I clicked on all of them.

I was on a mission to persuade our engineering team to implement my ideas for improving the algorithms in our small semi-automated anti-spam tool, since my hands were at risk of getting repetitive stress strain from hiding spam with manual clicks. I wanted us to detect and blacklist the automated spam. A kind engineer built a few tool improvements over time, but it was very hard for the team to prioritize engineering time for internal anti-spam tools.

Here are a few more screenshots from the Bookmarking Demon site, including a guest appearance from the Delicious 2.0 interface.

image
image

image

image

Those who went before us: Web 1.0 bookmarking sites

Since Pinboard bought Delicious and already has its own blog, I’m a former Delicious community manager back here on my old blog, commandeering it just for fun.

After Delicious and most of its clones fell apart, I got into researching its forgotten Web 1.0 predecessors: who were the people who tried to build bookmarking sites before we did? What happened to them? I put my research into Wikipedia’s article about social bookmarking. Here are some screenshots and weird quotes in case you’re ready for some very bright color schemes and the burst dreams of 1999.

itList launched in April 1996. Here it was in 1998, with a nice image map:

itlist

According to the Washington Post in 1997, it had “the ‘itList Puppy Dog,’ a small window, floating over your main browser window, into which you paste a Web address and click the ‘Submit your URL’ button to bookmark it.” Puppy dog! This review from 2000 details itList’s features, including: “You can make comments about any URL you save; you can give it a ‘Cool’ rating and also a ‘Useful’ rating.” I wish I could find that old tiny sunglasses icon that indicated a cool site. I hope this site had one. Looking around the Internet Archive snapshots, it looks like the site fell apart into parked-domain spam by 2001.

Around 1999, it was time for the dot-com bubble, and people got tens of millions of dollars in venture capital for this kind of site: Backflip, Blink, Clip2, ClickMarks, HotLinks, others.

Backflip had some purple-and-orange style. I like this snapshot from 2005 since it has a nice “win a pink Razr” ad gif:

backflip

The 2001 about page talked about their fancy investors and board, which included Ron Conway: “Backflip was founded by Tim Hickman and Chris Misner, who left Netscape to create a company that would solve a real problem and dramatically improve people’s everyday Web experiences. Backflip is a privately held company backed by some of the most successful new economy venture investors. Our Board of Directors includes 21st Century Venture Partners, Angel Investors LP and Rosewood Venture Group, who are responsible for early investments in name-brand start-ups like AskJeeves, AdForce, Avantgo and PeoplePC.”

A weird article from 1999 (not actually 2002 as dated there) mysteriously includes: “The technology is the product of 15 years of research performed at Stanford University at the behest of the CIA, Misner said. The service provides users with a list of the top 10 pages they visit and lets them upload bookmarks into their categories.”

A rather optimistic article from 2000 said “Backflip insists it isn’t a search engine. Maybe so, but it may well turn out that Backflip will be the web navigation story of 2000, just as Google blew us all away in 1999. Or, if they have their way, it will be the next big story of community – perhaps as interesting as About.com.“

This 2000 article about Backflip says: “I would chalk up the relatively low rankings for Yahoo! in many categories to the fact that most people don’t need a bookmark to remember Yahoo!!”

In 2001, “Backflip’s parent company realigned its business objectives” and sold it to “a small team of ex-Backflip employees, who, even though we no longer work there, still love the Backflip service and would like to see it survive (and thrive!)”. The status page from 2001 (at the bottom) to 2010 tells an increasingly sad story of outages, data loss, and eventual shutdown.

Blink, “The web’s favorite bookmarking community” (“We think of ourselves as Hotmail for bookmarks”) started in 1999. Here it was in 2000:

blink

In 2005, when del.icio.us got acquired, the founder of Blink wrote about why he thought Blink failed, despite $13 million in investment and 1.5 million users. He wrote about product decisions that made the site less useful than del.icio.us, but his post doesn’t say anything about money or business model.

Hotlinks was another site in this generation, started in 1999. “Where to find and keep what’s cool on the web”. Snapshot from 2000:

hotlinks

According to this 2000 article, the idea with HotLinks was to build a Yahoo-style-but-better internet directory out of people’s bookmarks, but since that wasn’t working out: “Currently, Chairman and CTO Jonathan Abrams is seeking partners and investors to help HotLinks port its functionality into the enterprise space.” (Edit: as noted by Andy Baio, Jonathan Abrams went on to start Friendster, and now Nuzzel.)

And now the shocking conclusion to what happened to all of them: lack of viable business models. Oh well.

(Edit: The HotLinks founder says “ultimately I don’t think the issue was business model, it was product market fit.” Noting that here for completeness!)

Site Problems

Dear Delicious Users,

We are aware of the many difficulties that the site is currently experiencing, and we’re doing our best to resolve them. Thank you for your patience while we work through these problems. Please be assured that we will not quit — we will keep working hard to get the site working smoothly and well.

Sorry for the trouble.

Site Revisions

Dear Delicious Users,

We have deployed a revised design for our site using the old python version of Delicious that has been living at previous.delicious.com. Thank you for your patience while we iron out the bugs during this transition.

Sponsored Links in RSS Feeds

Dear Delicious Users,

Today we introduced sponsored items in our RSS feeds in an effort to help pay the significant costs we are currently incurring to support them. We understand that some of you will not care for this, but the bottom line is that it is quite expensive for us to support the tremendous volume of simultaneous requests that we currently receive to our RSS and JSON feeds.

Thank you for your continued support of Delicious.

Discontinuing Delicious Premium

Dear Premium Delicious Users,

After careful consideration, we have decided to discontinue the Premium service at Delicious. While we appreciate the passionate users who paid to use our service, the bottom line is that there have not been enough to support the resources required to maintain separate functionality.

We have canceled all premium subscriptions at paypal, and have also refunded all monthly payments since Dec. 13, 2015 and all yearly payments since Dec 1, 2015.

If you were a yearly subscriber who made payment before Dec, 1, 2015, and you would like a pro-rated refund, please contact support@delicious.com.

Thank you again for your support of Delicious, and your patience with us while we strive to improve functionality for all users.

Delicious Changes

Dear Delicious Users,

My name is Tony Aly, and I’m the CEO of Delicious Media, a new company formed in alliance between my company, Domainersuite, and Science, the company that has managed Delicious since 2013. Science has transitioned control of Delicious to our new entity so that my team and I can dedicate ourselves to the long-term success and stability of this wonderful, useful, trailblazing site.

As part of this transition, over the next few weeks, we will be a making a few fairly prominent changes to the site.

The first big change you’ll notice is our transition from the javascript front-end framework that has been powering the content at https://www.delicious.com. The engineers who crafted this version of the site are incredibly talented, and their code is amazing. It’s beautiful and powerful, but it has posed several significant challenges for us. For example, the search engines have a real problem reading our content, hindering users’ efforts to use Google or Bing to find what they’re looking for on Delicious.

Fortunately for us, the version that the javascript site replaced has been kept alive at previous.delicious.com. This was built using a much more traditional framework, and it’s great! In fact, many of our longtime users have continued to prefer it over the main site, and frankly, so do we. Therefore, we are switching to this platform for our main site, and this transition will position us to quickly iterate in our ongoing efforts to keep Delicious thriving.

The next change will arguably be a bit bigger. One of the terms of the transition is that we must revert to the original URL: http://del.icio.us. This will pose a few challenges, as there a lot of moving parts involved. But it’s something we have to do.

Another exciting change that many of our users have been asking for is to bring back stacks. We agree — stacks are awesome! Please keep an eye out for this very cool and exciting feature to make a comeback.

In a nutshell, we’re going back to our roots. Delicious is the original social bookmarking site, and we’re honored to be its caretakers.

Please be patient with us as we make these changes, and know that while we fully expect to break things…our team is fully dedicated to making things right.

Please email support@delicious.com to let us know how we’re doing and how we can make the site better for you.

Thanks!
Tony Aly