Earlier this year, BBC technology hacks announced a crowdsourcing experiment: an app to measure the UK’s mobile broadband speeds.

Although, broadly, this was a crowdsourcing success story the fact that one of the UK’s best resourced organisations and certainly the one with the largest online presence and most straightforward social media interactions faltered at all says a lot about these types of promotions.

Here are three things the BBC’s effort can teach us about crowdsourcing in general.

Make sure it works

The BBC didn’t take any chances: they commissioned a firm called Epitiro to create their app and manage the data collected. The company had an ideal track record: they’d just completed Ofcom’s recent report on mobile broadband speeds. Not all firms will able to hire the leaders in their fields. But a strong track record is a must.

Have a clear aim… and deliver

The BBC’s project had a clear aim: record 3G signal and plot the findings on a map which is searchable by postcode. It would also fill an obvious need. Earlier that year, an Ofcom report found that, “the availability of 2G, 3G or HSPA networks, and the performance delivered, vary significantly” and concluded that there was a lack of adequate information on coverage for consumers.

So far, so good. Unfortunately, though the BBC chose to ignore criticisms of Ofcom’s own speed reports which pointed out the research didn’t measure actual network speed.

Bandwidth is limited and operators increasingly throttle services: it’s all very well having a strong 3G signal but what good is it when an operator chooses to divert your data use to 2G / EDGE levels? As a result, while the crowdsourcing results were interesting they were of limited use and tech writers mourned a lost opportunity.

“Any study of mobile broadband availability needs to look at measuring capacity as well as coverage,” said Tom Brewster in ITPRO, for example. Oh, and BBC journo John Hunt said that the project would, “help to name and shame those who are reporting coverage incorrectly.” No word on any shaming as yet.

Keep an eye firmly on data

Finally, if you’re collecting data on a mobile app for goodness sake keep an eye on mobile data and make sure your users do the the same. The BBC’s 3G survey app recorded phone signal data all the time while the phone was switched on with no way to exit the background task. Although data use was low and if there were any instances of bill shock I never heard of them, the Beeb did get some considerable stick for this.