How do I quit my job when I know my boss is going to lay on guilt and make me feel like the bad guy?

LONG post sorry I need advice and help!

I have been at my job for 7 plus years and I looking for a new job because I can’t stay here any longer.

Coming to work literally gives me anxiety and some days being around my boss literally makes my skin crawl and I just want to leave and never come back. And when I do finally find a job and decide to leave I know it will be a huge thing and my boss will act like I am betraying him and not being loyal.

The problem is its a small family office and I am literally doing everything except for 1 assistant we just hired. We also have labor/maintenance staff off site from the office.

I started as an office admin, but that role morphed into more complex roles, now I also do all the bookkeeping and I trade stock and options for my older boss as well.

This place is truly toxic, its a father and son who buy buildings, rehab them then rent them. The older one (my main boss) is 93 years old and he is not tech savvy, extremely ornery and expects everything immediately, he constantly interrupts any work you are doing because he “needs to know” what’s happening, its beyond annoying, if you go out to get lunch or go to the bank, he will call you to see where you are and what’s taking so long. His sense of time is warped. Once he kept calling me when I was at the dentist for a dental emergency and I couldn’t answer the phone because I had cotton in my mouth.

He doesn’t think people are actually working. He also acts like he is doing everyone a service and he is teaching you and you are lucky to be working here as you are getting an education from him.

He doesn’t understand that the world has changed and the compensation is a major struggle. HR is non existent. We have no benefits here, no healthcare, they do not contribute to 401K, no paid time off. Taking a vacation day is like pulling teeth, if you call in sick he will call you serval times during the day to see how you are and what you are doing and I think its because he doesn’t really think you are sick. He has zero boundaries, if you don’t answer he will just keep calling. He complains out loud about people that call in sick or have a problem, that he doesn’t think its going to work out and they aren’t motivated to work here.

The younger boss (his son) mostly just ignores all the behaviors and just lets his dad run the show, he cuts side deals to pay the off site guys (I see the checks etc) but he won’t discuss compensation with me and refers me to his father.

When I recently asked for a raise (its been almost 2 years) My boss threw it in my face that I get quarterly bonuses and he thinks I am making plenty of money. I even reminded him that we were hiring someone new, I would have to train them and I took on the bookkeeping back on last year after he let the part time bookkeeper go because he thought she was working too many hours and wasting time.

I don’t even clear 100K with that and I live on the East Coast in an area where taxes and cost of living are outrageous. My healthcare was costing me $600 a month with a huge deductible, my only saving grace was I got remarried and my husband thankfully has healthcare. He also claimed I only put in 35 hours a week because I leave at 4:30 most days, I come in early some days and leaving at 4:30 is 37.5 hours a week. I usually report I work 38-39 hours a week. I do run banking and shipping and return errands on my time sometimes as well.

My other assistant recently left because she needed more money and more flexibility for her kids and she found a great hybrid job making almost as much as I am now. So we needed to hire someone which is impossible because the pay is so low and the quality of candidates is not great for his expectations, We have a new assistant, he is fine but I don’t think is the sharpest or will cut it here in the long run. My boss also said he would hire another bookkeeper instead of giving me a raise and I said no because frankly I don’t want to have to train a bookkeeper on all the accounts and intricacies of the company LLCs. More work for me.

Also the guy that was doing all the stock trading that I took over for 2 years ago made more than me and I still haven’t even reached that number and I am doing way more things then he ever did. Part of me thinks that is sexist because I am woman.

Once when I asked for a raise my boss said don’t you make enough money with your alimony (From my previous marriage *which I no longer get) and I was taken aback and I said my alimony has nothing to do with you or the value of my work here and my worth.

Its gotten to he point where he just expects everything from you and there are no boundaries there are in even personal assisting things he asks for and he thinks its okay. Like swapping out his cable boxes for Apple Tv then having to revert is all back because he couldn’t figure out the Apple tv. He will yell out for you get someone on the phone for him or call the pharmacy for him to get a refill. I have too many other things I am doing and I feel like the other assistant should be handling all of that – sometimes I just say can so and so do that I’m busy. Last week he somehow blocked 2 people on his phone then he insisted his phone was broken and it did it in the middle of the night by itself. Then asked me if I could I go through all his contacts and check them to make sure they weren’t blocked. I flat out refused.

He also pushes all the time for me to “find money” in the options contracts when it flat out isn’t there unless you are willing to risk losing positions for $100 commissions on the option. the market is volatile these days.

He falls asleep at his desk (everyone just ignores it), he gets hooked on dumb things, like a mortgage banker he spoke to didn’t return his call so now he wants to call the president of the bank (a LARGE bank) to complain about her. Its unnerving I want to yell out just STOP! Last week he made everyone waste a half a day looking for an old space heater and insisted it was going to be problem if “it walked off” (ie someone stole it). They are very well off so who really cares about a $25 dollar space heater. I think it was old and it probably got thrown out – no one could even recall what space heater he was even talking about.

I am at the point where I hate this job and I am doing my job but the bare minimum I have zero motivation.

I am interviewing and I have a potential job that is bookkeeping alone and the pay is a little less then here with some benefits. I am now having anxiety about how to leave this job and tell them I am quitting. I am horrible at confrontation and I usually just get railroaded and bullied into putting up with things the way they are.

I know I can’t stay here, its killing my soul, I am losing sleep at night thinking about having to tell them I am quitting but I am just done.

Any advice would be helpful, I am also dreading leaving here because I know they will freak out because I am the only one who knows how to do everything here and about everything here. HELP!!