It would be nice to have a preview-/thumbnail-image of the video shown. Is this an already included feature that just has to be activated somewhere?

I tried to throw a .jpg into the video folder and naming the picture like the video file only with changed extensions. This didn't work.

Thanks in advance

This is the valid way to do this:
http://www.zenphoto.org/2009/03/troubleshooting-zenphoto/#19

And it works as you can see here on this demo (last thumb):
http://zenpage-demo.maltem.de/album1/

I have to admit that somwhoe the movie does not play right now, but that is an encoding issue as flvplayer does play it. Also try the nightly as there has been some work on that plugin recently.

It is as easy as it could be:

I just didn't rename the file extensions correctly as it seems to be case sensitive.
file.JPG -> won't work
file.jpg -> works

Maybe this is something to be changed in future releases?

Please file a ticket about that. Thanks.

Done: http://www.zenphoto.org/trac/ticket/1291

Unfortunately, this is a file system issue. Zenphoto just does a filesystem call to see if a file exists. It would be really unreasonable (and quite a bit of overhead) to check for each possible file type with each pemutation of character case. So, the rules of the game are that zenphoto will look for lower cased suffixes. If your filesystem insists on case sensitive file names, you will have to name your files accordingly.