Yes that’s right. Its about 30-40 mins from Lee Vining where you go up the canyon to Tuolumne Meadows (~60 mins from Mammoth to the Yosemite entrance, note there may be entry passes you need to book in advance, but they don’t sell out too quickly and last for multiple days). Definitely doable for a day, they usually try to open the road for Memorial Day weekend unless its a very heavy snow year. But the trip to Yosemite Valley is much longer (at least 2.5 hours each way). You can see down to Half Dome etc from Olmstead Point (at the west end of Tenaya Lake), which is about 90 mins from Mammoth (and there’s a long but stunning day hike to Clouds Rest from there).
We hiked from Mammoth to Yosemite Valley in August: