So How Does it Work?

So if someone comments to my post and I then reply to their comment does it go to their blog?  Or do they have to read my response on my blog page?  The later is what I am thinking, so if that's the case we go visit other blogs and comment and then check back on their page to see what they respond back????...Still trying to figure it all out.  Hope that makes sense.