Is there an option to list all existing albums under Edit Album > Images > image > 'Set album thumbnail for:"

I'm looking to be able to use an image within 'Album A' as a thumbnail for 'Album B'.

By default that selector uses only images of the album. You can enable subalbum images of that album on Option > Gallery > Gallery behaviour. But note it may cause overhead if you have lots of subalbums and lots of images. You cannot enable images from totally different albums. You would have to upload the image to that album as well.