On the admin pages, on the Albums tab, when you have many and many albums in an album, the list is very long to load.

So, is it possible to implement a "albums pagination" feature for it ?

No, this would need a separate page because then drag and drop album sorting would not be possible.