I know multiple people who have been quite successful. As far as I can tell, none of them took the shortest path to get from wherever they started to wherever they are now. They all tried a bit of this and a bit of that. Some partial failures or dead ends are just part of the road to success.
I doubt that you wasted three years working. I think that you learned more about the world and the high tech industry and also learned more about what you want to do. Your current plan to get a master’s degree in CS sounds like a very reasonable plan based on what you have done up to now.
For machine learning I hope that you got a strong background in mathematics. With a combined physics / CS degree this does sound likely.
I got my master’s degree at Stanford. The other students in the same program had come from a very, very wide range of undergraduate universities. It was rare to find two people who had come from the same undergraduate school (the only exception that I found was probably a coincidence, and was NOT a “top 50 in the US” university). “Barely top 200 worldwide” is still quite a good university, and graduating Summa Cum Laude should help also.
You will need recommendation letters. When I applied to graduate schools my recommendation letters were from more senior employees were I worked. One daughter is currently applying to graduate schools and one reference is from her thesis advisor from her bachelor’s degree, but the other two are from more senior employees where she works. Yes, this does mean that you need to tell your boss that you are considering returning to graduate school. Hopefully they will be supportive.
If you are three years out of university, you might want to contact your faculty advisor and/or a few professors that you happened to know when you were in university and see whether they will feel qualified to write a recommendation letter. They might want to see a current resume or a list of “what you have done since graduating”.
You might be running up against application deadlines (for starting in September 2024). I am not sure which schools are already past the deadlines and which are not. However, it would not be a bad idea to take some time to figure out where you want to apply, and you are way ahead for starting in September 2025.
And you might want to apply to a range of universities. Given what you said, it might be worth including a few top schools on the list, possibly including Stanford.
And a lot of students are 22 when the graduate university with no work experience or with only an internship over the summer. Life is not a race, but if it were you would still be ahead of most of the pack. To me it sounds like you are doing well.