The federal IRS is not gonna care. Some states like CA and NY certainly care.
Since CA is ‘state A’, income earned elsewhere is taxable in CA. But you would receive an offset against any taxes paid to State B. But if State B is one of those with no state income tax, then no need to file in State B.