It looks like the problem is that some directories are still bind mounted in the container, and aren't being unmounted for some reason. If you reboot into recovery, you can delete the libertine container's root filesystem directory by hand, and nothing will be mounted at that point.
You can also check mount -l | grep libertine to see what is still mounted, and try to forcefully unmount, and then re-run the libertine-container-manager command to destroy the container once no mounts are left for it.