Of course you can! Once you have created your own cashback offer for your business' shop see how here, you are able to deactivate it whenever you want, without any restrictions.
How? Open the payzy pro app and select the "cashback" section in the home screen → There you will find the cashback that you have created → You need only click on the Icon "On-Off" to deactivate a currently enabled cashback.