How We Plan to Spend $80,000 a Year & Pay Nearly $0 Taxes

In my Q3 Income Report, I mentioned one strategy of ours that I want to expand on: paying next to no taxes in retirement. There are a bunch of strategies for reducing taxes – Backdoor Roth IRA conversions, Roth IRA Ladders, tax gain/loss harvesting – just to name a few.

The more I read about different strategies for reducing taxes when withdrawing funds the more I realize there are soooo many ways to do it. What strategy you choose will rely mostly on two key facts:

• Which accounts you’re using (401k, Roth IRA, Brokerage)
• How much you’re withdrawing each year.

I’m going to go through these two questions for my own accounts and show just how we plan to be able to spend $80,000 a year and pay absolutely taxes. Not “low taxes” but $0 a year in federal tax.

Note: The debate on civil responsibility is a separate issue. In this case, assume that this is all income that is from existing funds earned from working that have already been taxed if that’ll make you feel better about the idea.

Which Accounts Am I Using?

Due to a number of windfalls I’ve received in my life, coupled with working at some startups that didn’t offer 401ks, I am heavily skewed towards brokerage accounts for my investments. The quick breakdown of our household investment accounts looks something like this:

• 8% – 401k
• 6% – Roth IRA
• 86% – Brokerage

Note: This is only looking at my investments, not as a percentage of my total net worth.

I don’t read too many investment blogs about how to handle this situation. The most common route to retirement involves saving a LOT more in a 401k. I’ve been maxing mine out every year I had one since my 20s, but sadly many years it wasn’t offered.

At first, I thought minimizing taxes from a brokerage account would be nearly impossible – hardly anyone talks about it. Having this large a position in a brokerage account actually is incredibly useful for tax purposes – it’s almost crazy to me how useful it is. Let’s recap how taxes work for these three account types real quick before moving forward.

401k Taxes

There are (at least) 3 different ways to get money from and pay taxes on a 401k:

• After age 59.5, anything withdrawn will be taxed at your ordinary income rate.
• Before 59.5, you can convert a 401k to a Roth IRA at your ordinary income rate.
• Before 59.5, you can withdraw funds and pay taxes at your ordinary income rate + a 10% penalty.

There are a number of situations where you won’t be hit with that 10% penalty (leaving a job at age 55 using the Rule of 55, disability), but for the most part, you’ll be hit with the early distribution penalty tax. The money you withdraw from a 401k is considered ordinary income and has the potential to raise your tax bracket.

The ideal way to use a 401k for tax purposes is when your ordinary income is low – which would allow you pay very little taxes on a withdraw (after age 59.5) or a conversion. A Roth IRA Conversion Ladder is one technique used to access money without that 10% penalty, but you’ll still pay the ordinary income rate (which could be low enough to pay nothing).

Since we only have about 8% of our portfolio in 401ks, we don’t plan to touch them until we’re at least 60. In that time they’ll hopefully grow a bunch (all tax free) and then we can withdraw funds when we’re in a very low tax bracket.

One we are able to draw from our 401k, we plan to strategically withdraw only enough so that the 401k amount + Roth IRA amount + Capital gains + Dividends amount still puts us in a 0% tax bracket. As of today in 2018, that would mean less than $24,000 in income – the standard deduction for a child-free couple filing jointly. More on how we’ll have this low income while spending $80k/year ahead!

Roth IRA Taxes

Roth IRAs are easily the best retirement account when you need money but don’t want it to impact taxes. Similar to 401ks, there are a few different tax rates depending on how you’re accessing the money from a Roth:

• You can withdraw contributions at any age tax-free.
• Before age 59.5, if you withdraw gains on an investment in a Roth, those gains will be taxed at your ordinary income rate + a 10% penalty.
• After Age 59.5, you’ll pay $0 taxes taking anything out from a Roth.

There’s one other reason why Roth IRAs are so amazing is that the withdrawals from them are not considered ordinary income. This is extremely helpful if you need to artificially get your income below a certain point.

Take healthcare. To qualify for a 50% reduction in healthcare through the ACA, your income needs to be below about $64,000. If your income was actually $70,000, you’d be over the limit! If instead, you withdrew $64,000 from a 401k and $6,000 from a Roth IRA, then your income would be $64,000 and you’d be in great shape. (note: technically these numbers would be $24k higher due to the standard deduction).

We don’t plan to access our Roth IRA account before age 60, and hopefully not until much later. Ideally, these will be used for emergencies when we want to stay under a certain tax bracket and need a little more tax-free income. They have another 24+ years to grow, which means they could potentially double 3 more times — up to $800,000 by the time we start withdrawing form them.

Brokerage Taxes

Taxes on brokerage accounts are one of the most simple to explain – mostly because they don’t rely on age! Instead, they rely completely on two things: how long you held the investment and what your tax bracket is.

There are two types of gains when selling investments:

• Short Term Capital Gains – Taxes on gains from investments held for less than a year.
• Long-Term Capital Gains – Taxes on gains from investments held for more than a year.

The difference in taxes on these can be HUGE. Take a look at this chart of rates by income.

2018 Income Tax Brackets

What should stand out is that with short-term capital gains you’ll pay taxes no matter what, but with long-term capital gains you could pay $0 in taxes.

That’s our plan. By keeping our income under $77,201 ($101,201 with the standard deduction) , we’ll pay $0 in long-term gains. Keep in mind that long-term capital gains count towards this income.

For example, let’s say you bought $50,000 in AAPL stock a few years ago. That stock has more than doubled and is worth $150,000. How much tax would pay if you file taxes jointly and you sold it all at once (and did nothing else that year?)

The answer is $0. This is because your total income would be $100,000 ($150,000-$50,000) for the year. You would have $100,000 in capital gains, get a personal deduction of $24k, then have income of $76,000. That would land you in the 0% tax bracket for capital gains – leaving you with $150,000 in total money, $100,000 in income and $0 in taxes. Craziness.

This is one of the main reasons why $77,000 is a super-sweet spot for spending. Having your income below that point ensures the most beneficial tax treatment.

Brokerage Dividend Taxes

There is one other side to these though – dividends. Within a brokerage account, you’ll end up paying taxes on dividends every year. I’ve written about how I avoid dividends like the plague, which should say a lot about what I think about them.

The main reason for this is because dividends count towards income. I’m trying to reduce my income, so if I have dividends that means my income is higher. Take the Apple example from above. If you had $3,000 in dividends, all the sudden your income would’ve been $78,000 – placing you in the 15% tax bracket. What happens then?!

Well, luckily capital gains is a progressive tax. Going over the limit doesn’t shift all of your funds into that tax-bracket, only the part that’s over it. This means for the $799 you’re over ($78,000 – $77,201) you would pay a 15% capital gains tax. Not awful but something to think about.

Alternative Minimum Tax

There’s one other tax-rate to think about – the Alternative Minimum Tax or AMT. This is a tax I won’t even go into because it only applies if you have income above $191,500 (jointly) or $95,750 (filing single) in 2018. I haven’t fully looked into this tax except to say I’ve never been bitten by it, and it appears to be way above any spending most people reading this would have during retirement. If you’re aiming to retire with more than $5 million, you might want to read up on it though.

How Much Are You Withdrawing Each Year?

This has a huge impact on your tax rate and insurance costs. The sweet spot that I’ve seen as of today is somewhere around $64,000 in income (filing jointly). You can still spend more than that, but if you manage to lower your income to that level you’re in a good spot.

By keeping your income this low, you’ll have a bunch of advantages:

• Pay 0% long-term capital gains
• 50% reduction in ACA Healthcare costs
• 12% tax bracket for dividends and other income

That 50% reduction in ACA costs could save you $500 a month, or $6,000 a year right there! Would you rather make $64,000 and pay $6,000 have healthcare ($58k left), or make $70,000 and pay $12,000 to have the exact same healthcare (also $58k left)? Save yourself the trouble and find out.

Our spending has grown lately. As our incomes have grown, we’ve traveled much more, bought more luxury items, clothes and upgraded our apartment. While those additions to our lifestyle have been awesome, it’s still possible to have them and spend a reasonable amount.

We plan to spend somewhere around $70/yr, drawing almost entirely from brokerage accounts. That includes:

• $29k/year a year for an apartment, utilities, insurance and minor improvements.

• $15k/year in travel & luxuries. This is a very large, flexible category we could reduce in a down-market year.

• $10k/year in food – split mostly even between restaurants and groceries.

• $3k/yr in transportation

That’s roughly $60k right there. The other $10k is personal care, clothes, pets, entertainment, software, phone, and a few other things. There is plenty of room to bring this budget down if we want/need. It also doesn’t include large-one off purchases like a new car.

On the investment side, the cost basis on our brokerage accounts is about 32% gains and 68% our initial investments. That means that if we were to withdraw $70k a year, it would break down to something like this:

• $70k sold from a brokerage account
• $22,400 of that would be taxed at capital gains.
• $47,600 would be tax-free because that was the amount put in initially.

If that $22,400 was my income for the year, then my long-term capital gains tax rate would be well in the 0% range. Actually, I could almost quadruple that amount and still pay $0 taxes.

Let’s say that instead I wanted my income to equal $77,200 a year – how much stock would I need to sell? Turns out it’s:

income x 0.32 = $77,200 + $24,000

income = $316,250

So if I were to sell $316,250 in stock, my taxable income would be $77,200. If we continue our current spending, then we’d have an extra $246,250 — what should we do with it?

The answer is we’d just invest it immediately back into the stock market! This is called tax-gain harvesting. We’re realizing those capital gains, paying the taxes on them (which happen to be $0) and putting that money back to work for us. By taking it out and investing it right back in we’re effectively lowing our tax rate down the line.

Each year we do this, we lower our cost basis even more. It might be 32% the first year, 22% the second, 15% the third. and 7% the fourth. By the 5th year we’d struggle to harvest long-term gains and likely need to hold longer.

If we repeat this process for a decade, all of the sudden that 32% cost basis will very quickly drop to $0. At that time we could take out any amount from our portfolio and it would generate nearly no income whatsoever from long-term capital gains.

Having money in brokerage accounts makes for some very tax-advantaged situations!

The Standard Deduction

There’s one area I haven’t touched on and that’s the standard deduction in taxes. For 2018, this is a $24,000 deduction or $12,000 for individuals. This means that whatever your income is, you can effectively subtract $24,000 from it if you’re filing jointly.

Honestly seeing these numbers I’m kind of in shock. If you think the tax-code isn’t skewed in favor of high income earners, I hope this helps to highlight just how skewed it is.

The one thing to think about in all this – if you make above a certain amount you’ll be required to pay the alternative minimum tax. Before writing this article I hadn’t fully investigated the AMT, but now I’m starting to realize just how incredibly important it is to close loopholes that would allow people to withdraw 10x, 100x, 1000x or more what I’m doing every year. Even in my situation, it seems amazing that I’m not hit by it. If I was I would fully understand and pay for it.

Our Plan Recap

Ok, that was a lot. Let me recap that real quick.

• We plan to live mostly off long-term capital gains from brokerage accounts.
• We’ll pay $0 in taxes as we withdraw from them.
• We’ll keep our income low enough to be in a 0% capital gains bracket and a low bracket for health care.
• As we start tapping into our 401ks, we’ll lower our income bracket even more, down to under $24,000 a year. (by then the cost basis of our brokerage account will be nearly $0).
• We’ll withdraw more than we need every year and tax-gain harvest the difference.
• After age 60, we’ll use our Roth IRA to get tax-free money if needed.

Given all of this, there seem to be a number of routes that we can go that all lead to paying $0 in taxes while still spending $80k or less. While I wouldn’t recommend using a brokerage account over a 401k if you are in a position where you have a brokerage account you can use it to the fullest!