I think he does owe an apology. Even if you are not 100% convinced you did something wrong, if you bother someone, the honest thing to do is say your sorry. It is just the right thing to do in a case like this.