Hi Kevin,
Open the respective Application/Application Configuration in SE80 and go to
Web Dypro configuration-> Test-> Execute in Administrator mode. Right click on the two buttons and select Settings for Current Configuration.
Now change the Visibility property to Invisible and save it to a TR. That's it.
Please let me know, are you able to resolve or you need some alternate way.
BR,
RAM