What happens if you just don’t file your taxes?

My 2020 bank statements tell a story, and it’s kind of a bummer. The first four months of the year were flush with semi-regular cash infusions from my former full-time employer. In mid-April, after the pandemic just about demolished what was left of the local newsroom where I was working, those payments abruptly stopped, replaced by direct deposits from the New York State Department of Labor. As the year went on, those unemployment benefits were joined by a trickle of cash from freelance writing.

Like millions of Americans who lost their jobs in 2020, I not only have a depressing accounting project ahead of me this tax season, but a more complicated tax situation than I’m used to dealing with. I’ve got 1099s to assemble and deductions to calculate before the extended May 17 deadline. I’ve also got to wait to see how my state will handle the new federal guidance that makes the first $10,200 in unemployment benefits tax-free.

Given all that, is it any wonder that I’m pondering what it would look like if I just…ignored Tax Day this year?

Of course, I don’t plan to (if you’re reading this, IRS agent, thank you for your service.) But how dire could the consequences really be? And what are the odds that the IRS would simply “overlook” a missing return from a modest earner like me?

Not high, according to Susan Allen, senior manager for tax practice & ethics for the American Institute of CPAs. That’s because tax forms documenting the income taxpayers have received are not only sent to the taxpayer, but also to the IRS. From there, it’s a simple matching game to confirm that all the income reported to the agency has been claimed on a corresponding return.

“The ‘mismatch’ process is fairly straightforward and automatic. If you received a W-2 or 1099 form that wasn’t reported on your tax return, you have a very good chance of receiving a notice where the IRS is asking about the income,” says Allen. “And this process is true no matter if you’re a high-income or low-income taxpayer.”

Auditing of suspicious information on a tax return — like enormous deductions or questionable dependents — is also an equal opportunity enterprise for the IRS (depending on your point of view), with the poorest Americans about as likely to face an audit as the top 1% of earners.

Even if the IRS didn’t receive any records of payments to you in 2020, breaking a years-long pattern of filing income taxes can itself set off alarm bells at the agency. Once the missing return is flagged, the IRS will begin to send notices. If you ignore these, the agency may prepare a substitute return on your behalf based on information compiled from other sources.

Having them create a tax return sounds easier than doing it yourself, right? Unfortunately, the return prepared by the IRS probably won’t include any of the expenses or deductions you’re entitled to claim, Allen says, and may overstate your actual tax liability.

Dawn Delia, tax attorney and founder of Delia Law P.C., says one of the biggest mistakes clients make is not filing their taxes because they’re worried about not having the records they need, or not being able to pay the taxes they anticipate owing. “They may think, ‘I can’t pay it, they’re going to come after me. If I don’t file then I have more time,’” says Delia. In reality, not filing a return doesn’t extend the due-date for taxes you may owe, doesn’t stop interest from accruing on those taxes, and is punishable by hefty fines. “The goal is to file anyway, to get it on record. Because there’s a huge failure-to-file penalty from the IRS.”

That penalty, also known as the late filing penalty, can be as high as 25% of the tax owed. If the return is more than 60 days late, the minimum late filing penalty is $435, or 100% of the tax owed, whichever is smaller. This is separate from the IRS’s late payment penalty, which is .5% of the balance of unpaid taxes each month they’re late.

If a taxpayer ignores a tax bill from the IRS, they can eventually have their wages garnished, their bank accounts levied, and/or lose out on refunds in the future. “If you owe any money from past tax debt, they will funnel that refund straight back to the debt and you’ll never see it,” says Delia. “So to lose out on that refund, that’s a big mistake.”

Taxpayers can request a filing extension to Oct. 15 to avoid the late-filing penalty. Asking for an extension to file doesn’t extend the due-date to pay any taxes owed, however. The IRS will work out a payment plan if you can’t afford to pay the full balance upfront, which will help minimize penalties for late payment.

There also exists the small but frightening possibility of being criminally prosecuted for not filing taxes. “Willful failure-to-file” a return is a misdemeanor that carries a potential sentence of up to one year in prison and a $25,000 maximum fine for individuals. The “willfulness” element comes into play when the government can prove beyond a reasonable doubt that the defendant knew about their obligation to file taxes and chose not to do so. Particularly egregious cases may be elevated to tax evasion, a felony charge that carries a maximum penalty of $100,000 and up to five years in prison.

In 2019, only 114 investigations from the IRS’s Non-Filer Program led to indictments. Following repeated budget cuts from Congress, the IRS has reduced its staff by one third since 2011, and its enforcement of failure-to-file cases and of tax fraud generally have been on the decline ever since.

Still, it’s best not to gamble on escaping the IRS’s notice. Even those who earn their money through illegal means, such as embezzlers or drug dealers, are required by law to pay income taxes. Sorry, business expenses for many of these activities, like duffel bags for drug trafficking, or bribes paid to corrupt officials, cannot be deducted from your tax bill.

Alas, the only taxpayers who aren’t subject to hefty penalties for not filing their taxes are those who don’t have a filing requirement, or those who would have received a refund. In that case, if you don’t file the return within three years, you’ll likely lose the refund for good. The bottom line is: Not doing your taxes might actually be more stressful than sitting down and knocking them out with coping mechanisms (hello, vodka) in place for after you’re through.