I’m a little confused and, just in case you don’t know, I’m from CA.
How does a barn figure into the equation, would that be the structure that’s connected to the original house?
Is the lot 2+ acres? If so, then you have plenty of room to build a 2nd larger structure, depending on your zoning laws.
Can you subdivide the 2+ acre lot? If so, it might make more sense financially to build the 2nd structure on the newly created lot.
I remember a old house in a city here in Silicon Valley, this area of the city had an “architectural control committee.” Typically, the people on these committees aren’t builders, designers or architects, so well, they tend to be very, er, uh, myopic. Yes, that’s a good word for it. This committee made the builder save a portion of this “architecturally important” old house. In the view of most, it wasn’t. Anyway, the house ultimately turned out great, though it could have been better, if started from scratch, but the amount of extra work and the compromises that had to be made on the design showed, at least to “Architectural Digest” crowd.
Thankfully, no HOA in your case. IMO, they’re terrible. I’m not familiar with septic, that’s rare here unless you live out in the “boondocks.” :))
Just remember, it’s about time and money. You could make almost anything work with time and money.
Ultimately, I think you can run the risk of a “Frankenstein” house or “Winchester Mystery House” syndrome. And that’ll hurt resale value.