I have a big-ass tax problem.

When I was in my late twenties and early thirties I did some stock-picking—not day-trading, mind you, but long term buy-and-hold stuff—and I beat the ever-blazing behoobas out of the market.

Which leaves me with a big-ass tax problem.

I carry three legacy post-tax accounts at my full-service broker. Two are actively-managed funds; the third is money I manage myself. (Yeah, yeah, I know…I’m not writing to defend having those fund managers sucked onto my back, just to deal with same.)

Above is a screenshot of my self-managed account. In the left-hand box you see the names of the stocks I picked: Amazon, Apple, Berkshire Hathaway Class B, and Altria/Phillip Morris. I bought the AAPL, BRK.B, and MO between Y2K and 2004, and the AMZN between 2012 and 2014. The other two line items are essentially cash.

In the right-hand box you see the unrealized capital gains for each stock. The oval contains the sum total. In my first post-tax account I’m up $408,406 on an initial investment of $85,050.90.

Now: I believe in carrying six to eighteen months’ expenses in cash. We have a budget of roughly $50,000 a year, so I ought to have from $25,000 to $75,000 on the books. But I’m down to about $12,000.

So here’s the problem. How the hell am I supposed to raise cash to meet our expenses without taking a ginormous capital gains tax hit? How the hell do I keep the IRS from [REDACTED REDACTED REDACTED REDACTED] until my esophagus blisters?

You’re familiar with the concept of tax loss harvesting, yes? If so, skip down to the next horizontal line. And if not, here’s a summary.

In the absence of other passive income sources, early retirement requires you to live on the proceeds from your savings—capital gains, dividends, interest, etc.—with capital gains usually being the major component. Since you need cash to pay the bills, at some point you’ll have to raise it by selling off your investments.

Selling off winning investments incurs a capital gains liability. Depending on your income, it can be from 15% to 19.6% if you’ve owned them for more than a year, and as much as 39.6% if you’ve owned them for less.

Obviously a significant hit.

So loss harvesting is the practice of selling off losing or break-even investments in order to manage down your income and therefore your tax liability. It’s not that you want to lose money, of course, but rather that your situation is such that taking a loss is preferable to taking a tax hit.

Otherwise known as a “perverse incentive.”

Unfortunately, in my situation tax loss harvesting won’t work.

Like I said, besides the self-managed account I just showed you, I have two other post-tax accounts that are actively managed. They cost me approximately $6,000 a year in advisory fees on a fund balance of $350,000. That’s 1.7%, way higher than it ought to be.

I’d very much like to cash out of that $350K, use some of it for our annual expenses, and transfer the balance to Vanguard low-fee funds. But the investments in those accounts are also all winners, meaning to sell I’d have to take a tax hit of, at the bare minimum, 15%.

Again: ouch.

So how do I get out of this, quote, problem and raise some cash in the process? I see four options:

Option one: do nothing. Continue paying $6,000 a year in advisory fees. My foregone market return at 7% across 40 years would compound to -$1,371,504.17.

Option two: sell $58,823 from the managed funds, take the 15% hit to raise $50,000, and meet our spending budget for a while. Foregone annual market return at 7% of $8,823 across 40 years compounds to $132,119.64.

But: to that amount you have to add an estimated $12,000 in increased current-year healthcare costs (for reasons I won’t get into here)—so the total current-year hit would be $20,823 for an opportunity cost of -$311,813.14 in that same forty years.

Note, however, that this assumes that taking the total $20,823 hit is a one-time event to meet current-year expenses only. If we continue having to realize gains on $58,820 to meet our budget each year (probably impossible given our balance), our foregone annual market return of the tax hit alone would be -$2,016,796.88, plus whatever the foregone market return on increased healthcare costs would be—which is impossible to estimate in the current regulatory environment.

But it’s a snowdrift of currency, for sure.

Option three: cash out of the entire $350,000 managed fund balance, keep $50,000 for expenses, and transfer the rest to Vanguard’s low-fee funds. This action would require eating the tax hit on our gains of roughly $105,000, which’d be $15,750—and leaving an estimated $12,000 of increased healthcare costs on the table. The foregone market return on that $27,750 works out to -$415,541.21 across 40 years at 7%. The least-cost option so far, but still: ouch.

But…at Vanguard I could at least start collecting 7% on the annual managed funds fee of $6,000 I’d no longer have to pay, which compounds to +$1,371,504.17 across 40 years. So if you net the -$415,541.21 out of the +$1,371,504.17, this option is roughly +$955,962.96 in the green. Clearly the best option so far.

There’s a scary problem here, though. The thought of dumping $273,000 ($50K expenses plus $27.5K tax plus healthcare hit) into the current market all at once makes me soil my loincloth.

Option four: borrow $50,000 against a 4.5% line of credit we carry at our brokerage. No foregone market-rate compounding, but the 40-year cost of principal plus compounded interest would be -$290,818.23 if we let it ride. No tax hit, no healthcare cost hit.

Which looks kind of meh on its own, but note again that this assumes borrowing $50,000 would be a one-time event to meet current-year expenses only. Worst case: if we have to borrow $50K every year and pay the same interest rate on it—which is a terrible assumption, because the rate floats—by year 40 we’re out -$5.6 million.

Option five: come up with a blend of foregone market return and gradually dripping into Vanguard that stays within my comfort zone.

To sum up, here are the foregone returns of these options:

Option one: -$1,371,504.17

Option two: -$311,813.14 to -$2,016,796.88

Option three: +$955,962.96

Option four: somewhere between -$290,818.23 and -$5,600,000

Option five: ???

And so: at the moment option three—sell everything in the managed funds and transfer to Vanguard—seems like the most financially efficient option. However, option one seems like the most defensive option given my aversion to dumping $272,000 into the market all at once (or even dripping it in.)

But option one’s impossible…I have to raise cash from SOMEWHERE.

And hence my big-ass tax problem. Whaddaya think?

Footnote: somebody’s bound to ask, “Well, if you’re paying all these advisory fees, why don’t you ask your advisors about this?” Believe you me…I’m drubbing, nay, clubbing them with it. When I get all this sorted out and decide what action to take, I’ll post an update. Also: pictures of them in Little Bo Peep costumes.

Author: ER Dude

Sick of your job? After a thirteen-year career, Early Retirement Dude fled corporate America for good. You can do it too! Visit http://EarlyRetirementDude.com or email EarlyRetirementDude@gmail.com.

11 thoughts

  1. Regarding your managed account, can you just have them detach from it and transfer the mutual funds into your brokerage account, if not there already? This is the way it works when you give LPOA over something like a Schwab subaccount to a portfolio manager. Or are these some kind of special funds that are non transferrable?

    Getting locked into funds with large cap gains is a bummer, and sometimes the purveyors of such funds know this and realize that people are loath to cash out and this lets them continue with high ERs, kind of like what Blackrock did with Wealthfront, https://blog.wealthfront.com/avoid-blackrock-etfs/

    1. >can you just have them detach from it and transfer the mutual funds into your brokerage account

      Slightly different situation: these managed funds are made up of of individual equities, which means I can cherry-pick. From a capital gains perspective it doesn’t matter which account I hold them in.

  2. I have the exact same problem with almost the exact same stocks…
    Depending on your state and assuming they are long term gains based on your writing you can have capital gains that make up your income up to $90+k before realizing capital gains. This is all dependent on your other income streams and number of dependents.

    IRS tax topic 409:
    “Some or all net capital gain may be taxed at 0% if you’re in the 10% or 15% ordinary income tax brackets”

    You can play with it on the free calculator at H&R Block to figure out your best balance:

    1. >you can have capital gains that make up your income up to $90+k before realizing capital gains

      You mean before paying tax on CG, right? We’ve always managed our liability down to zero using loss harvesting; that’s what’s brought us to the situation we’re in now.

  3. I think you need to get over the fear of investing that $273K all at once and eliminate that from your evaluation.

    I’m about the same age and net worth and had to take a similar deep breath to invest almost $250K at once last year so I understand your hesitation. Wasn’t nearly as hard the second time.

      1. You know…I have the same aversion to market timing. I’m sitting here realizing that it’s not so bad as I make it out to be, since I’m not going from cash to equities, but EQUITIES to equities. If a crash came, I’d crash in much the same amount either way. Maybe not so bad as I thought except for the tax and healthcare hits.

  4. Isn’t there a GoCurryCracker article on this? Basically you want to get your AGI under the 15% tax bracket and you will not pay any capital gains tax. You used the term “we” when referring to the 50K yearly budget, so I’m guessing you file jointly, therefore the 15% bracket cutoff is around 75K. Meaning you can “recognize” the 50K you need to live off of, plus another 25K to get money shifted to non-fee accounts.

    1. >Isn’t there a GoCurryCracker article on this?

      No idea, but it’s a common problem with the common solution that you describe. Since the time I wrote that article I’ve identified $67,000 I can take out of those two managed funds while realizing only about $677 in gains–basically selling a bunch of recently-purchased break-evens–which gives me room to get some of those home runs off the table if I want. And since I’m 48 I’m going to start rolling IRA money into Vanguard’s 10-year target fund, which will mitigate the risk even further.

Leave a Reply

Your email address will not be published. Required fields are marked *

Important Disclaimers