You can visit during office hours. It is definitely okay. They are happy to talk to you about their area of expertise, or give you advice about careers etc.
What you don’t want to do is:
Take time away from students who are in that class…like if someone is there for questions, let them go first.
Take up too much of their time…by that I mean talking about non-relevant topics for the entire time and coming back multiple days in a row.