Hmmm, no .... custom properties do not depend on permissions.
To filter by permissions you need information to be in a different workspace. So the only alternative seems to be to create a note in another workspace with the confidential info, and have it linked to the contact.