`»` is the html entity encoding of » Likewise the second image shows `&``amp;raquo;` which is basically a double encoded `»`.
Presumably your theme has passed these encoded strings to the Zenphoto breadcrumb and page nav functions. As explained in the 1.4.3 release notes, Zenphoto "print" functions should be passed un-encode strings as they will encode them as necessary to meet WEB validation requirements.