Notes security

Let assume,

we have two department A and B

Department A user  has taken notes for some contact Like Rahul and now Department B user comes and take note for the same contact Rahul. Now my requirement is A department notes should not be seen by department B user for same account.They should able to see notes taken by there department user only not other department user

Thanks,

Rahul Tiwari