Searching Big Data for ‘Digital Smoke Signals’

But the office in Manhattan is not dedicated to the latest app. It is the base camp of the United Nations Global Pulse team — a tiny unit inside an institution known for its sprawling bureaucracy, not its entrepreneurial hustle. Still, the focus is on harnessing technology in new ways — using data from social networks, blogs, cellphones and online commerce to transform economic development and humanitarian aid in poorer nations.

“We work hard, play hard and tend to stay well-caffeinated,” said Robert Kirkpatrick, who leads the group. “This is an exercise in entrepreneurship.”

The efforts by Global Pulse and a growing collection of scientists at universities, companies and nonprofit groups have been given the label “Big Data for development.” It is a field of great opportunity and challenge. The goal, the scientists involved agree, is to bring real-time monitoring and prediction to development and aid programs. Projects and policies, they say, can move faster, adapt to changing circumstances and be more effective, helping to lift more communities out of poverty and even save lives.

Research by Global Pulse and other groups, for example, has found that analyzing Twitter messages can give an early warning of a spike in unemployment, price rises and disease. Such “digital smoke signals of distress,” Mr. Kirkpatrick said, usually come months before official statistics — and in many developing countries today, there are no reliable statistics.

Finding the signals requires data, though, and much of the most valuable data is held by private companies, especially mobile phone operators, whose networks carry text messages, digital-cash transactions and location data. So persuading telecommunications operators, and the governments that regulate and sometimes own them, to release some of the data is a top task for the group. To analyze the data, the groups apply tools now most widely used for pinpointing customers with online advertising.

“We’re trying to track unemployment and disease as if it were a brand,” Mr. Kirkpatrick said.

Global Pulse is small, employing 11 people in New York. Seven more people work at a laboratory in Jakarta, Indonesia, that opened last fall. And Global Pulse is hiring for another lab in Kampala, Uganda, to open this fall.

The research labs are initially working on demonstration projects to show the potential of the technology. “But the larger role of Global Pulse is as a catalyst to foster a data ecosystem for development, bringing together the private sector, universities and governments,” said William Hoffman, an associate director who leads the data-driven development program at the World Economic Forum, which has worked with Global Pulse.

Its United Nations pedigree helps Global Pulse serve as an impresario for data-driven development efforts. “Global Pulse has been central in raising awareness,” said Alex Pentland, a data scientist and director of the Human Dynamics Lab at the Massachusetts Institute of Technology. “And it is a trusted party in an area that is sensitive for many governments and companies.”

The group traces its origins to the 2008 financial crisis and concerns about how the economic pain would sweep through the developing world. But as Secretary General Ban Ki-moon of the United Nations said in a speech, “Our traditional 20th-century tools for tracking international development cannot keep up.”

Global Pulse is intended as a 21st century answer to that problem. It was set up in 2009, as an innovation arm in the office of the secretary general. Mr. Kirkpatrick joined in early 2010, began assembling a team and emphasized tightly focused projects and rapid experimentation, while traveling the world to spread the data-for-development gospel at conferences and in private meetings.

Gadgetwise Blog: Three Tips for Better Google Searching

Want to improve your Google search skills?
Here are the three tips — basic, intermediate and advanced — from Dan Russell at Google. He studies how people use the search engine and teaches classes on how to do it better, including a free online course this month, for which registration started Tuesday. He promises these tips will make you happy, and he cares a lot about that — his official title at Google is über tech lead for search quality and user happiness.
Basic tip:
Say you know you were mentioned in an article online, so you search to read what was said about you. But when you click on the link, you get the entire article and no idea where to find your name. The solution is simple: Control- F. Typing those two keys allows you to enter a term to find it anywhere it appears on the Web page. Ninety percent of United States Internet users do not know how to find the word they are looking for on a Web page, according to Mr. Russell’s studies. “Control F changes the way you read anything.”
Intermediate tip:
You tried to snowboard and ended up back at your computer with a possibly broken arm. Instead of typing “What do I do with a busted arm” into Google, try to use words that you imagine someone else would have written about broken arms, Mr. Russell said. “Put yourself in the mind of the author of a perfect Web page,” he said. “Emulate the language of the person you want to read the answer from.” But do not take that too far, by making up words that you think are expert, but are not real words. “Don’t overdo it using words you think are medical, like ‘fracturated,’ because you will find results, at which point you’ll be in even bigger trouble,” Mr. Russell said. Which leads us to his final tip.
Advanced tip:
Before you ask Google what to do with a “fracturated” arm, look up the word, or any word, by typing “define:” followed by the word. Mr. Russell advises promiscuous use of the word “define” followed by colon. “Before you search, make sure you’re searching the right word. The worst thing you can do is to search for something you think means one thing but in fact means something else, but you believe the answer when you asked the wrong question.”