The Internal Revenue Service will largely diminish the amount of unannounced visits it makes to homes and businesses, citing safety concerns for its officers and the risk of scammers posing as agency employees, it announced Monday.

Typically, IRS officers had done these door visits to collect unpaid taxes and unfiled tax returns. But effective immediately, they will only do these visits in rare circumstances, such as seizing assets or carrying out summonses and subpoenas. Of the tens of thousands of unannounced visits conducted annually, only a few hundred fall under those circumstances, the agency said.

"These visits created extra anxiety for taxpayers already wary of potential scam artists," IRS Commissioner Danny Werfel said. "At the same time, the uncertainty around what IRS employees faced when visiting these homes created stress for them as well. This is the right thing to do and the right time to end it.

Instead, certain taxpayers will receive letters in the mail giving them the option to schedule a face-to-face meeting with an officer.

The IRS typically sends several letters before doing door visits, and typically carry two forms of official identification, including their IRS-issued credentials and a HSPD-12 card, which is given to all federal government employees. Both IDs have serial numbers and photos of the person, which you may ask to see.

"We are taking a fresh look at how the IRS operates to better serve taxpayers and the nation, and making this change is a common-sense step," Werfel said.