To clear all cache,
Brave Setting -> Privacy & Security -> Clear browsing data
Then select how much you want to clear.