Rates are low. Regulation is high. But business is booming at innovative banks.

A number of fast-growing banks with niche business models are making big gains, the research team of Keefe, Bruyette & Woods recently observed. Their report — "America's Challenger Banks" — underscores recent data showing how regional banks are outperforming many larger competitors.

Banks with $5 billion to $50 billion in assets have steadily increased their market share, in terms of overall assets, since the financial crisis, while larger banks like Citigroup and Bank of America have scaled down, KBW's analysts noted. Regionals have nearly 8% of total industry assets, compared to 5.4% in 2010, but those modest gains highlight how new approaches to banking are slowly reshaping — and redefining — the market.

"The idea of being a financial institution where you just bank everybody within a 25-mile radius is just a flawed model," Stephen Gordon, chief executive at the $5.6 billion-asset Opus Bank in Irvine, Calif., said in an interview.

"It's not as though there's a lack of financial institutions in the United States," Gordon added, noting that more than 6,000 banks exist. "You need to differentiate yourself."

KBW's report discussed how a number of banks are adopting new practices in order to succeed.

Many have developed expertise in fast-growing industries, including Opus, which focuses on high-growth sectors such as technology and health care. Doing so helped increase the bank's loan book by 37% in the first quarter compared to a year earlier, to $4.3 billion.

Other banks, such as the $5.5 billion-asset BofI Holding in San Diego, have grown quickly by focusing exclusively on their digital platforms.

"At the core of what it means to be a challenger bank is that you're disrupting the existing community banking model," Catherine Mealor, a KBW analyst who co-authored the report, said in an interview. "They're forcing the other large banks in their market to rethink how they should grow."

The report, at times, wades into murky political waters. It suggests that post-crisis regulations have given smaller lenders a leg up in the industry.

Banks with $5 billion to $50 billion in assets are "optimally positioned for higher growth" and profitability because they don't have to deal with the costs of being a systemically important financial institution, KBW's analysts said. Smaller institutions are also big enough to handle a range of compliance costs.

"The large banks are too big," Mealor said. "They've got too much regulatory scrutiny."

Some industry experts cautioned against using asset thresholds to evaluate a bank's performance.

"I wouldn't bet against a great company that was regulated, and I wouldn't bet against a great company that wasn't regulated," said Marshall Lux, a professor at Harvard University.

Still, Lux, whose research on community banks was cited in KBW's report, said it is important to keep tabs on midsize companies with innovative business models.

"That's how Capital One started," Lux said. American Express, he added, developed a "core that was different."

Still, banks highlighted as "challenger banks" in KBW's report didn't pass up the opportunity to take a few swipes at their bigger competitors.

"The largest banks can't get out of their own way," Gordon said, describing them as acting "like countries in and of themselves" with big bureaucracies. "We pick up a lot of clients inside of our niches," he added.

Read more: