How We Plan to Spend $80,000 a Year & Pay Nearly $0 Taxes

Written by .
9 min read. Investing, Personal, Taxes.

In my Q3 Income Report, I mentioned one strategy of ours that I want to expand on: paying next to no taxes in retirement. There are a bunch of strategies for reducing taxes – Backdoor Roth IRA conversions, Roth IRA Ladders, tax gain/loss harvesting – just to name a few.

The more I read about different strategies for reducing taxes when withdrawing funds the more I realize there are soooo many ways to do it. What strategy you choose will rely mostly on two key facts:

  • Which accounts you’re using (401k, Roth IRA, Brokerage)
  • How much you’re withdrawing each year.

I’m going to go through these two questions for my own accounts and show just how we plan to be able to spend $80,000 a year and pay absolutely taxes. Not “low taxes” but $0 a year in federal tax.

Note: The debate on civil responsibility is a separate issue. In this case, assume that this is all income that is from existing funds earned from working that have already been taxed if that’ll make you feel better about the idea.

Which Accounts Am I Using?

Due to a number of windfalls I’ve received in my life, coupled with working at some startups that didn’t offer 401ks, I am heavily skewed towards brokerage accounts for my investments. The quick breakdown of our household investment accounts looks something like this:

  • 8% – 401k
  • 6% – Roth IRA
  • 86% – Brokerage

Note: This is only looking at my investments, not as a percentage of my total net worth.

I don’t read too many investment blogs about how to handle this situation. The most common route to retirement involves saving a LOT more in a 401k. I’ve been maxing mine out every year I had one since my 20s, but sadly many years it wasn’t offered.

At first, I thought minimizing taxes from a brokerage account would be nearly impossible – hardly anyone talks about it. Having this large a position in a brokerage account actually is incredibly useful for tax purposes – it’s almost crazy to me how useful it is. Let’s recap how taxes work for these three account types real quick before moving forward.

401k Taxes

There are (at least) 3 different ways to get money from and pay taxes on a 401k:

  • After age 59.5, anything withdrawn will be taxed at your ordinary income rate.
  • Before 59.5, you can convert a 401k to a Roth IRA at your ordinary income rate.
  • Before 59.5, you can withdraw funds and pay taxes at your ordinary income rate + a 10% penalty.

There are a number of situations where you won’t be hit with that 10% penalty (leaving a job at age 55 using the Rule of 55, disability), but for the most part, you’ll be hit with the early distribution penalty tax. The money you withdraw from a 401k is considered ordinary income and has the potential to raise your tax bracket.

The ideal way to use a 401k for tax purposes is when your ordinary income is low – which would allow you pay very little taxes on a withdraw (after age 59.5) or a conversion. A Roth IRA Conversion Ladder is one technique used to access money without that 10% penalty, but you’ll still pay the ordinary income rate (which could be low enough to pay nothing).

Since we only have about 8% of our portfolio in 401ks, we don’t plan to touch them until we’re at least 60. In that time they’ll hopefully grow a bunch (all tax free) and then we can withdraw funds when we’re in a very low tax bracket.

One we are able to draw from our 401k, we plan to strategically withdraw only enough so that the 401k amount + Roth IRA amount + Capital gains + Dividends amount still puts us in a 0% tax bracket. As of today in 2018, that would mean less than $24,000 in income – the standard deduction for a child-free couple filing jointly. More on how we’ll have this low income while spending $80k/year ahead!

Roth IRA Taxes

Roth IRAs are easily the best retirement account when you need money but don’t want it to impact taxes. Similar to 401ks, there are a few different tax rates depending on how you’re accessing the money from a Roth:

  • You can withdraw contributions at any age tax-free.
  • Before age 59.5, if you withdraw gains on an investment in a Roth, those gains will be taxed at your ordinary income rate + a 10% penalty.
  • After Age 59.5, you’ll pay $0 taxes taking anything out from a Roth.

There’s one other reason why Roth IRAs are so amazing is that the withdrawals from them are not considered ordinary income. This is extremely helpful if you need to artificially get your income below a certain point.

Take healthcare. To qualify for a 50% reduction in healthcare through the ACA, your income needs to be below about $64,000. If your income was actually $70,000, you’d be over the limit! If instead, you withdrew $64,000 from a 401k and $6,000 from a Roth IRA, then your income would be $64,000 and you’d be in great shape. (note: technically these numbers would be $24k higher due to the standard deduction).

We don’t plan to access our Roth IRA account before age 60, and hopefully not until much later. Ideally, these will be used for emergencies when we want to stay under a certain tax bracket and need a little more tax-free income. They have another 24+ years to grow, which means they could potentially double 3 more times — up to $800,000 by the time we start withdrawing form them.

Brokerage Taxes

Taxes on brokerage accounts are one of the most simple to explain – mostly because they don’t rely on age! Instead, they rely completely on two things: how long you held the investment and what your tax bracket is.

There are two types of gains when selling investments:

  • Short Term Capital Gains – Taxes on gains from investments held for less than a year.
  • Long-Term Capital Gains – Taxes on gains from investments held for more than a year.

The difference in taxes on these can be HUGE. Take a look at this chart of rates by income.

2018 Income Tax Brackets

Tax RateIndividualsMarried, Filing JointlyLong-Term Capital Gains Rate
10%Up to $9,525Up to $19,0500%
12%$9,520 to $38,700$19,050 to $77,4000%
22%$38,701 to $82,500$77,401 to $165,00015%
24%$82,501 to $157,500$165,001 to $315,00015%
32%$157,501 to $200,000$315,001 to $400,00015%
35%$200,000 to $425,800$400,001 to $479,00015%
35%$425,800 to $500,000$479,001 to $600,00020%
37%Over $500,000Over $600,00020%

What should stand out is that with short-term capital gains you’ll pay taxes no matter what, but with long-term capital gains you could pay $0 in taxes.

That’s our plan. By keeping our income under $77,201 ($101,201 with the standard deduction) , we’ll pay $0 in long-term gains. Keep in mind that long-term capital gains count towards this income.

For example, let’s say you bought $50,000 in AAPL stock a few years ago. That stock has more than doubled and is worth $150,000. How much tax would pay if you file taxes jointly and you sold it all at once (and did nothing else that year?)

The answer is $0. This is because your total income would be $100,000 ($150,000-$50,000) for the year. You would have $100,000 in capital gains, get a personal deduction of $24k, then have income of $76,000. That would land you in the 0% tax bracket for capital gains – leaving you with $150,000 in total money, $100,000 in income and $0 in taxes. Craziness.

This is one of the main reasons why $77,000 is a super-sweet spot for spending. Having your income below that point ensures the most beneficial tax treatment.

Brokerage Dividend Taxes

There is one other side to these though – dividends. Within a brokerage account, you’ll end up paying taxes on dividends every year. I’ve written about how I avoid dividends like the plague, which should say a lot about what I think about them.

The main reason for this is because dividends count towards income. I’m trying to reduce my income, so if I have dividends that means my income is higher. Take the Apple example from above. If you had $3,000 in dividends, all the sudden your income would’ve been $78,000 – placing you in the 15% tax bracket. What happens then?!

Well, luckily capital gains is a progressive tax. Going over the limit doesn’t shift all of your funds into that tax-bracket, only the part that’s over it. This means for the $799 you’re over ($78,000 – $77,201) you would pay a 15% capital gains tax. Not awful but something to think about.

Alternative Minimum Tax

There’s one other tax-rate to think about – the Alternative Minimum Tax or AMT. This is a tax I won’t even go into because it only applies if you have income above $191,500 (jointly) or $95,750 (filing single) in 2018. I haven’t fully looked into this tax except to say I’ve never been bitten by it, and it appears to be way above any spending most people reading this would have during retirement. If you’re aiming to retire with more than $5 million, you might want to read up on it though.

How Much Are You Withdrawing Each Year?

This has a huge impact on your tax rate and insurance costs. The sweet spot that I’ve seen as of today is somewhere around $64,000 in income (filing jointly). You can still spend more than that, but if you manage to lower your income to that level you’re in a good spot.

By keeping your income this low, you’ll have a bunch of advantages:

  • Pay 0% long-term capital gains
  • 50% reduction in ACA Healthcare costs
  • 12% tax bracket for dividends and other income

That 50% reduction in ACA costs could save you $500 a month, or $6,000 a year right there! Would you rather make $64,000 and pay $6,000 have healthcare ($58k left), or make $70,000 and pay $12,000 to have the exact same healthcare (also $58k left)? Save yourself the trouble and find out.

Our spending has grown lately. As our incomes have grown, we’ve traveled much more, bought more luxury items, clothes and upgraded our apartment. While those additions to our lifestyle have been awesome, it’s still possible to have them and spend a reasonable amount.

We plan to spend somewhere around $70/yr, drawing almost entirely from brokerage accounts. That includes:

  • $29k/year a year for an apartment, utilities, insurance and minor improvements.
  • $15k/year in travel & luxuries. This is a very large, flexible category we could reduce in a down-market year.
  • $10k/year in food – split mostly even between restaurants and groceries.
  • $3k/yr in transportation

That’s roughly $60k right there. The other $10k is personal care, clothes, pets, entertainment, software, phone, and a few other things. There is plenty of room to bring this budget down if we want/need. It also doesn’t include large-one off purchases like a new car.

On the investment side, the cost basis on our brokerage accounts is about 32% gains and 68% our initial investments. That means that if we were to withdraw $70k a year, it would break down to something like this:

  • $70k sold from a brokerage account
  • $22,400 of that would be taxed at capital gains.
  • $47,600 would be tax-free because that was the amount put in initially.

If that $22,400 was my income for the year, then my long-term capital gains tax rate would be well in the 0% range. Actually, I could almost quadruple that amount and still pay $0 taxes.

Let’s say that instead I wanted my income to equal $77,200 a year – how much stock would I need to sell? Turns out it’s:

income x 0.32 = $77,200 + $24,000

income = $316,250

So if I were to sell $316,250 in stock, my taxable income would be $77,200. If we continue our current spending, then we’d have an extra $246,250 — what should we do with it?

The answer is we’d just invest it immediately back into the stock market! This is called tax-gain harvesting. We’re realizing those capital gains, paying the taxes on them (which happen to be $0) and putting that money back to work for us. By taking it out and investing it right back in we’re effectively lowing our tax rate down the line.

Each year we do this, we lower our cost basis even more. It might be 32% the first year, 22% the second, 15% the third. and 7% the fourth. By the 5th year we’d struggle to harvest long-term gains and likely need to hold longer.

If we repeat this process for a decade, all of the sudden that 32% cost basis will very quickly drop to $0. At that time we could take out any amount from our portfolio and it would generate nearly no income whatsoever from long-term capital gains.

Having money in brokerage accounts makes for some very tax-advantaged situations!

The Standard Deduction

There’s one area I haven’t touched on and that’s the standard deduction in taxes. For 2018, this is a $24,000 deduction or $12,000 for individuals. This means that whatever your income is, you can effectively subtract $24,000 from it if you’re filing jointly.

Honestly seeing these numbers I’m kind of in shock. If you think the tax-code isn’t skewed in favor of high income earners, I hope this helps to highlight just how skewed it is.

The one thing to think about in all this – if you make above a certain amount you’ll be required to pay the alternative minimum tax. Before writing this article I hadn’t fully investigated the AMT, but now I’m starting to realize just how incredibly important it is to close loopholes that would allow people to withdraw 10x, 100x, 1000x or more what I’m doing every year. Even in my situation, it seems amazing that I’m not hit by it. If I was I would fully understand and pay for it.

Our Plan Recap

Ok, that was a lot. Let me recap that real quick.

  • We plan to live mostly off long-term capital gains from brokerage accounts.
  • We’ll pay $0 in taxes as we withdraw from them.
  • We’ll keep our income low enough to be in a 0% capital gains bracket and a low bracket for health care.
  • As we start tapping into our 401ks, we’ll lower our income bracket even more, down to under $24,000 a year. (by then the cost basis of our brokerage account will be nearly $0).
  • We’ll withdraw more than we need every year and tax-gain harvest the difference.
  • After age 60, we’ll use our Roth IRA to get tax-free money if needed.

Given all of this, there seem to be a number of routes that we can go that all lead to paying $0 in taxes while still spending $80k or less. While I wouldn’t recommend using a brokerage account over a 401k if you are in a position where you have a brokerage account you can use it to the fullest!

  1. Adam – Great article about a very important topic. One thing – I think when you are tax-gain harvesting, you are attempting to increase your cost basis in your brokerage account. Your cost basis is the portion you have paid for the investments, and the remainder is gains that are subject to taxes.

  2. Take healthcare. To qualify for a 50% reduction in healthcare through the ACA, your income needs to be below about $64,000. If your income was actually $70,000, you’d be over the limit! If instead, you withdrew $64,000 from a 401k and $6,000 from a Roth IRA, then your income would be $64,000 and you’d be in great shape. (note: technically these numbers would be $24k higher due to the standard deduction).

    I don’t believe the 24K standard deduction has anything to do with MAGI which is used to determine eligibility for ACA premium and cost subsidies.

    1. Aiee, thanks for the catching that. I was so deep in tax research I think I assumed that ACA eligibility took that into account as well. That’s incredibly useful to know it doesn’t. I’ll read a bit more on that and update the article to match — thanks Dan!

  3. It seems wiser (and safer) to use part or all of the first $24k in standard deductions to convert your 401k to a Roth, and bring your 401k balance to $0 over the first 5-10 years of retirement. Doing so permanently de-risks your 401k from any taxes in the future. If you let your 401k grow for another 24 years, there’s a small chance it could balloon to be too big (unless they’re filled with only bonds?), exposing you to expensive RMDs at high taxes. The tax rates could also change so dramatically by the time you’re 60 that your tax rate against your 401k ordinary income (at age 59.5) isn’t so close to 0% anymore. Why take a chance, when you can guarantee a 0% rate today?

    The only catch with this strategy is that newly Roth converted money has to stay put for 5 years, but looks like you have plenty of reserves in your taxable account to stay afloat during that time. The other bigger catch is that if you Roth convert say $10k per year, you have to show $10k in income that year, but not actually be able to use that $10k for spending. This could affect ACA thresholds. You would have to make up that $10k difference by selling shares from your taxable account at a price equal (or close enough) to its cost basis to counter this. This will give you $10k to spend without showing $10k as income on your tax return. Thoughts?

    1. > It seems wiser (and safer) to use part or all of the first $24k in standard deductions to convert your 401k to a Roth, and bring your 401k balance to $0 over the first 5-10 years of retirement.

      That is a good point. You’re going with risk later if you do choose to go that route. I think for me, more of my taxable assets (if I were to withdraw now) are in capital gains, making me more worried about a change to the long-term capital gains tax code. Maybe whichever way your portfolio is most heavily skewed should be the area to try to take tax-advantage of first. For me, my 401k is 100% bonds, so there’ll be less growth there as well.

      On the conversion side, it does make it tricky with ACA involved – since you’d get taxed on $10k 401k->roth and $10k withdrawn from capital gains. I think for someone who is going that route, they’d only be able to make it work if their income was under $54k (in this example — $64k minus 401k conversion). If that works for budget though, that would be a great way to take advantage of that low tax rate!

  4. Fantastic read! I’ve been interested in this topic since I started reading a bit about it over on Go Curry Cracker and this is the best summary I’ve seen that matches my own concerns. Keep the great content coming!

    1. Thanks! I think when I first read GCC’s post and didn’t fully understand all of this, I assumed it would only be possible due to child deductions (I was extremely naive at the time on how the tax code worked). Great to see there are options for those without kids to do the same.

  5. Nice work, Adam! I am bookmarking this one for future reference. We’ll have a mix of real estate and 401K when I hit 60. The houses will eventually be sold or passed on to the kids. The 401K is what we plan to live off – and as you show, with a low income, the tax should be close to zip!

    1. Nice, seems like a solid plan! I haven’t dug too deep into real estate myself, but that sounds like it’ll also really help with the lower income. If my house was paid off and my income was that much lower then taxes would be even easier.

  6. Excellent article Adam. The threshold for 0% long term capital gains is higher than I thought!

    What type of holdings do you have in your brokerage accounts?

Leave a Reply

Your email address will not be published. Required fields are marked *

Share2
Pin2
Tweet
4 Shares