I would choose Nantucket, but I have a vacation home there, so I’m not objective. It is an amazingly beautiful island that is 30 miles out to sea. There are many places to stay and sights to see on island, depending on what activities you enjoy.
One thing to know—Memorial Day is the weekend of the Figawi Regatta (a sail boat race from Hyannis to Nantucket). Google Figawi and read the story of the name. It’s funny.
Anyway, the point is that this weekend will be really crowded. There will be many, many young people partying on the beaches and in town. Since you are arriving on Thursday, you will avoid most of the crowds and you will have options if you act quickly.
IMO your best bet is to fly from Boston to Nantucket on Cape Air. If you are just going overnight, you will probably spend as much money flying as you would to take your car over or renting a car on island. If you drive to the ferry, you will also pay for parking in Hyannis. Plus, you will have the cost of ferry tickets. That means, in addition to the cost of taking the car (which is probably $250) you also purchase passenger tickets. You can take a fast ferry (1 hour) or a slow ferry (2 hours). The slow ferry is cheaper and it will be easier to get a reservation.
It is 70 miles from Boston to Hyannis, add to that the mileage from your NH destination to Boston plus ferry time and you’ll realize that just getting to the island will eat up a big chunk of the time you could have on island. The flight from Boston to Nantucket is about an hour (it’s a small plane). If you had more time and if you were coming on another weekend, I would take the ferry.
Stay at one of the hotels in town—the White Elephant or the Nantucket Hotel. If you want to stay on the beach, go to the Wauwinet. It’s far out of town (same company that owns the White Elephant) and you can sit by the beach or pool and stay away from the crowds. I think the hotel arranges a shuttle/car to town and you can probably get a ride from the airport to the hotel.
Another reason to stay at one of the hotels is that the concierge can arrange for anything you need—have bikes delivered to the hotel, arrange tours or reservations for dinner, etc. There is also a shuttle bus service on island that goes to most beaches.
PM me—I am happy to make recommendations about the beaches to visit, places to stay, what to see, where to eat, etc. Consider staying another day—there is lots to see and do. My daughter got married on island and we created a family guide to the island that we have updated every year. I would be happy to send it to you.