The IRS Is Hunting Millionaire Tax Cheats: Here’s Who Is Most at Risk

When the nation’s top tax official said last week that the Internal Revenue Service was taking “swift and aggressive” action against millionaire scofflaws, he praised the agency’s recent haul of $38 million in unpaid taxes.

“This is just the start,” IRS Commissioner Danny Werfel declared in a statement July 14 about the money collected in recent months from roughly 175 deadbeat millionaires. He added that “we will continue to go after delinquent millionaires as we ramp up enforcement capabilities.” One millionaire non-filer who was snagged had used money owed to the Treasury Department to buy a Maserati and a Bentley.

But the response of some tax lawyers working for wealthy and ultra-high-net-worth Americans was: Um, good luck with that?

“That’s a rounding error,” laughed Michael Kosnitzky, a co-leader of the private client and family office practice at white-shoe law firm Pillsbury Winthrop Shaw Pittman. “Bar change. If they only got $38 million out of some of the wealthiest people in the United States, they’re not doing their job.”

Armed with an extra $80 billion in funding for the next decade, the tax collector is using the bulk of its war chest to put taxpayers who make at least $400,000 a year under a microscope. The funding, part of last year’s Inflation Reduction Act to lower drug prices and offer potentially $1.2 trillion in clean energy incentives, according to Goldman Sachs, is intended to focus on the wealthy, partnerships and large corporations. The Treasury Department estimates the “tax gap,” or the difference between taxes owed and those actually paid, to be $600 billion annually, equivalent to $7 trillion in lost government revenue over a decade.

Plunging Audits

The pledge to ramp up investigations of high earners across the board comes as audits of Americans of all income levels have declined for more than a decade, the victim of agent retirements, computer systems that date back to the Kennedy administration and the proliferation of sophisticated tax strategies involving trusts and overseas entities. Still, the chances of falling in the crosshairs can vary wildly from year to year.

In 2012 the IRS scrutinized 136 of every 1,000 taxpayers making at least $10 million, or 13.6% according to the agency's most recent data book. By 2017 the rate had plunged by more than half to 6.3%. 

But the next year, it ticked up to 9.2%. Then the pandemic hit in 2020, and the IRS was overwhelmed by the processing of hundreds of millions of federal stimulus checks and incentives for small business. That year, the audit rate for those earning $10 million or more plunged again, to 2.4%. The IRS hasn’t yet released data for the past two years.

Still, Werfel put two niche groups of affluent Americans on notice. The July 14 IRS statement said the agency “recently identified about 100 high-income individuals claiming benefits in Puerto Rico without meeting the residence and source rules involving U.S. possessions … we expect many of these cases to proceed to criminal investigation.”

U.S. citizens who live in the U.S. territory of Puerto Rico for at least 183 days each calendar year don’t owe U.S. federal tax on income “sourced” in Puerto Rico, including capital-gains profits. They can also be eligible for a 4% Puerto Rican corporate rate, with distributions from earnings and profits tax-free. Over roughly the past decade, hedge fund managers, property investors and cryptocurrency traders have flocked to the island, which has its own special tax system, to erase tax bills on their profits.

Amid Werfel's call-out, law firm Caplin & Drysdale issued an alert on Monday saying that “this is a good time for anyone claiming Puerto Rico residency to take stock of the strength of their tax return position.“ Kosnitzky said taxpayers who get audited over their benefits should be prepared to fork over cell phone records, travel logs and credit card bills proving they were on the island for half the year.

Werfel also flagged Americans who use pension plans in Malta, the tiny Mediterranean archipelago, to avoid capital-gains taxes. The IRS most recently flagged Maltese retirement schemes on its “Dirty Dozen” tax scams list in April. The agency’s extra funding “will enable us to forcefully find tax avoiders who leverage these offshore schemes,” the IRS said.

Trophy Billionaires.

The IRS’s trophy of tax kills is Houston-based billionaire Robert T. Brockman, the former chief executive officer of an Ohio-based software company who was indicted last August for concealing $2 billion in income from the IRS and defrauding investors. Brockman, whom federal prosecutors accused of tax evasion, wire fraud and money laundering, was due to stand trial this year but died last August. Also in 2020 the IRS sealed a deal for Robert Smith, the billionaire founder of private equity firm Vista Equity Partners, to pay more than $139 million in taxes and penalties. 

“Those are the types of cases the IRS wants to bring,” said Nathan Hochman, a former assistant attorney general at the Justice Department Tax Division, which oversees both civil and criminal tax cases. But whether a broad crackdown on wealthy tax cheats gains steam will depend in large part on whether the IRS can hire and train sophisticated agents with top-notch knowledge of accounting, business law, trusts, cryptocurrencies and overseas banking. 

“It’s going to take years for them to ramp up,” Hochman said. “I don’t expect to see anything monumental happening in the next year or two. We’ll know in years three to five.” Asked to comment on Werfel’s remarks, an IRS spokesperson emailed the July 14 notice.

Mark Everson, who served as IRS commissioner from 2003-2007 and is now the vice chairman of Alliant Group, a tax and business consulting firm in Houston, said Werfel’s comments “signal that they’re getting started.” The agency can throw out those numbers anytime, he added, so it’s “a shot across the bow.”

Charles Ruchelman, a partner at Caplin & Drsydale in Washington, D.C., who focuses on tax litigation and controversies with the IRS, said that though $38 million is not a lot of money, the dollars collected from delinquent millionaires and those who failed to file returns could have “an impact as a deterrence. “It has a huge ripple effect, and it incentivizes millionaires and billionaires to be a not take risks with their tax reporting, do things appropriately and don't fall for the next great thing.”

But Werfel’s warning volley may not really high-net-worth taxpayers any time soon. 

“It is going to be no easy task to build up the enforcement staffing and to hire and train competent personnel who are going to be able to do this work,” Everson said. Meanwhile, what Everson called “production” — audits of taxpayers — will “deteriorate in the near term.”