If you need to delegate Azure Bastion to a user without assigning too many rights, here are some tips for you:

• User needs to have Virtual Machine login or Virtual Machine Administrator Login on VM or Resource Group where the VM is located
• Read rights are needed on Bastion Resource Group
• Make sure to wait 5-10 mins after applying settings