‘Bots’ That Siphon Off Tickets Frustrate Concert Promoters

That enemy is the bot.

“Bots,” computer programs used by scalpers, are a hidden part of a miserable ritual that plays out online nearly every week in which tickets to hot shows seem to vanish instantly.

Long a mere nuisance to the live music industry, these cheap and widely available programs are now perhaps its most reviled foe, frustrating fans and feeding a multibillion-dollar secondary market for tickets.

According to Ticketmaster, bots have been used to buy more than 60 percent of the most desirable tickets for some shows; in a recent lawsuit, the company accused one group of scalpers of using bots to request up to 200,000 tickets a day.

Ticketmaster and its parent company, Live Nation Entertainment, have stepped up efforts to combat bots, in part to improve the ticket-buying experience for concertgoers, but also to burnish the company’s reputation with consumers. The result has been a game of cat and mouse between the company and the bots.

“As with hackers, you can solve it today, and they’re rewriting code tomorrow,” said Michael Rapino, Live Nation’s chief executive. “Thus the arms race.”

In late 2011, Ticketmaster hired John Carnahan, an expert on machine learning who fought online advertising frauds at Yahoo, to lead its anti-bot effort.

By monitoring the behavior of each visitor to Ticketmaster’s site, the company can determine the likelihood of a customer being human or a machine. For example, a human may click a series of buttons at a range of speeds and in different spots on a screen, but bots can give themselves away by rapidly clicking on precisely the same spot each time.

A screen on Mr. Carnahan’s desk in Los Angeles shows Ticketmaster’s incoming traffic, with a rainbow of colors at the bottom and splotches of red on top representing suspicious activity. On a recent Thursday afternoon, the screen showed that the red visitors were making 600 times more ticket requests than those the system identified as being most likely human.

Bots are not kicked off the system, but rather “speedbumped” — slowed down, sent to the end of the line or given some other means of interference, to allow a regular customer through.

“We’re not trying to stop anybody from buying tickets,” Mr. Carnahan said. “We’re just trying to make sure that a fan can buy the tickets.”

Ticketing bots are often inexpensive and programmed in countries beyond easy reach of American law enforcement. Rob Rachwald of the computer security company FireEye, which is not working with Ticketmaster, points out that one site — available in English and Russian — charges just $13.90 for the keys to 10,000 Captchas, those squiggly lines that test whether a potential customer is human.

In January, Ticketmaster replaced most of its old Captchas with newer and more sophisticated versions. The company is also introducing a system for mobile devices that aims to eliminate Captcha-style tests altogether.

Live Nation will not say how many of the 148 million tickets it sells each year are bought using bots, and in many cases it may not know. Few ever admit to using the programs; official groups like the National Association of Ticket Brokers, which represents many of the biggest resellers, condemn them and say they support anti-bot measures. But people at nearly every level of the concert business blame bots for wreaking all kinds of economic havoc.

“There are sold-out shows in reserved-seat houses in New York City where we will have 20 percent no-show, and that 20 percent will be down in the front of the house,” said Jim Glancy of The Bowery Presents, an independent concert promoter in New York. “It’s speculators who bought a bunch of seats and didn’t get the price they wanted.”

Concert promoters, artist managers and ticketing services say that bots are now an ever-present force, not only during the high-traffic moments when a big show officially goes on sale, but also at the odd moments when a promoter releases a few dozen extra seats with no announcement.

Darlene Schild, of Lincroft, N.J., may well have experienced the reach of bots firsthand recently when she tried to buy Justin Bieber tickets as an 11th birthday present for her daughter. Like any well-trained concertgoer, she fired up Ticketmaster’s iPhone app just as the tickets went on sale, but after 15 fruitless minutes she gave up.

“The first thing that crossed my mind was that some ticket-buying service bought them all,” Ms. Schild said. “Or someone could dial quicker than me. Some technology — something.”

Last month, Ticketmaster sued 21 people in federal court, accusing them of fraud, copyright infringement and other offenses in using bots to search for millions of tickets over the last two years.

The legal status of bots is unclear. They are banned in a handful of states, but those laws have proved largely ineffectual, and enforcement at the federal level has also been a disappointment to the concert business.

Three years ago, four men connected with a company called Wiseguy Tickets were indicted on conspiracy, wire fraud and other charges, for apparently using bots to get tickets to Bruce Springsteen, Hannah Montana and other concerts.

The case hinged on whether the men had committed actual crimes or had merely violated the terms of service on Ticketmaster’s site; in the end three of the men were sentenced only to probation and community service (one remained at large).

“They got a slap on the wrist,” Mr. Rapino said. “It wasn’t much of an actual deterrent.”

Not everyone is convinced that bots are the primary villain of the everyday concertgoer. The Fan Freedom Project, a nonprofit group financed by StubHub, has pushed for anti-bot laws around the country, and Jon Potter, its president, praised Ticketmaster for filing its lawsuit last month.

But he also criticized the industry practice of “holds,” in which sometimes large blocks of tickets are reserved for sponsors, fan club members and industry contacts, and never go on sale to the general public.

When it comes to the secondary ticket market, Live Nation has a complicated position. As much as it is trying to block bots, it also profits from the ticket resale market through TicketsNow — its own version of StubHub — as well as through deals with major sports groups, like the National Basketball Association. Mr. Rapino sees no contradiction in Live Nation’s stance.

“I have no problem if you bought a Justin Timberlake ticket and you decide to go sell that ticket to somebody,” he said. “We would first and foremost want to make sure that the first ticket sold, that the fan has a shot to buy that ticket.”

Cyberattacks on 6 American Banks Frustrate Customers

Frustrated customers of Bank of America, JPMorgan Chase, Citigroup, U.S. Bank, Wells Fargo and PNC, who could not get access to their accounts or pay bills online, were upset because the banks had not explained clearly what was going on.

“It was probably the least impressive corporate presentation of bad news I’ve ever seen,” said Paul Downs, a small-business owner in Bridgeport, Pa., and a contributor to The Times’s small-business blog, You’re the Boss. “This is extremely disconcerting.”

The banks suffered denial of service attacks, in which hackers barrage a Web site with traffic until it is overwhelmed and shuts down. Such attacks, while a nuisance, are not technically sophisticated and do not affect a company’s computer network — or, in this case, funds or customer bank accounts. But they are enough to upset customers.

A hacker group calling itself Izz ad-Din al-Qassam Cyber Fighters — a reference to Izz ad-Din al-Qassam, a Muslim holy man who fought against European forces and Jewish settlers in the Middle East in the 1920s and 1930s — took credit for the attacks in online posts.

The group said it had attacked the banks in retaliation for an anti-Islam video that mocks the Prophet Muhammad. It also pledged to continue to attack American credit and financial institutions daily, and possibly institutions in France, Israel and Britain, until the video is taken offline. The New York Stock Exchange and Nasdaq were also targeted.

On Friday, PNC became the latest bank to experience delays and fall offline. Customers said they had been unable to get access to PNC’s online banking site, and those that visited the bank’s physical locations were told it was because PNC, and many others, had been hacked.

Fred Solomon, a PNC spokesman, said Friday afternoon that the bank’s Web site was back online, but that it was still working to restore online bill payment. Asked why the bank was not better able to withstand such an attack, he said that while PNC had systems in place to prevent delays and disruption from hacker attacks, in this case “the volume of traffic was unprecedented.”

Representatives for other banks also confirmed that they had experienced slow Internet performance and intermittent downtime because of an unusually high volume of traffic.

Security researchers said the attack methods were too basic to have taken so many American bank sites offline. The hackers appeared to be enlisting volunteers for the attacks with messages on various sites. On one blog, they called on people to visit two Web addresses that would cause their computers to flood banks with hundreds of data requests a second. They asked volunteers to attack banks according to a timetable: Wells Fargo on Tuesday, U.S. Bancorp on Wednesday and PNC on Thursday.

But experts said it seemed implausible that this method would create an attack of this scale. “The number of users you need to break those targets is very high,” said Jaime Blasco, a security researcher at AlienVault who has been investigating the attacks. “They must have had help from other sources.”

Those sources, Mr. Blasco said, would have to be a group with money, like a nation, or botnets — networks of infected computers that do the bidding of criminals. Botnets can be rented through black market schemes that are common in the Internet underground, or lent out by criminals or governments.

Last week, Senator Joseph I. Lieberman of Connecticut, chairman of the Senate Homeland Security Committee, said in an interview on C-Span that he believed Iran’s government had sponsored the attacks in retaliation for Western economic sanctions. The hacker group rejected that claim. In an online post, it said the attacks had not been sponsored by a country and that its members “strongly reject the American officials’ insidious attempts to deceive public opinion.”

The hackers maintained that they were retaliating for the online video. “Insult to the prophet is not acceptable, especially when it is the last Prophet Muhammad,” they wrote.

It is very difficult to trace such attacks back to a particular country, security experts say, because they can be routed through different Internet addresses to mask their true origin.

But experts said they had seen an increase in such activity from Iran and in the number of so-called hacktivists, hackers who attack for political purposes rather than for profit, based in Iran.

“We absolutely have seen more activity from the Middle East, and in particular Iran has been increasingly active as they build up their cyber capabilities,” said George Kurtz, the president of CrowdStrike, a computer security company, and former chief technology officer at McAfee. “There is also a strong activist movement underfoot, which should be concerning to many large companies. The threat is real, and what we are seeing now is only the tip of the iceberg.”

James A. Lewis, a computer security expert at the Center for Strategic and International Studies, said that in this case, the attack methods used were “pretty basic” to have been state-sponsored. But he added that even if the attacks were not the work of Iran’s government, the state would be aware of them because Iran monitors its networks extensively.

For Mr. Downs, the small-business owner in Pennsylvania, such half explanations were of little consolation.

“A major bank has a problem and gives no indication of what’s happening, when it started or when it will stop,” he said. “That’s pretty freaky if it’s your own business’s money and you need to do things with it.”