The easiest way I can think of to implement this is to create two user groups: one for family and one for friends. Make the family group have the family album as a managed album and the opposite for friends. Then, unpublish both albums. It should work out that only the members of the family group can see the family album and the opposite for friends.
This should work because unpublished albums are hidden from users who don't have access. As long as the user is a member of the respective group, he can see the album because of the group's access. I believe this method will also not yield access to editing the album, though.
The only drawback of this method is that you'll have to manually assign each user to the respective group. But I think that would be necessary no matter what method.