Have you figured out the financial aspect? Have you discussed with whoever pays your tuition? Do those things first.
If you are scared to transfer, ask yourself if you’re more scared of what will happen to your state of mind if you stay. That will help you decide.