I would say afterward, but really it won't matter in the long run. Changes to the appearance and content of your site won't be reflected in the cached files until the cache is updated.
If you modify your theme or install new plugins that affect the appearance of your site I recommend clearing the outdated cache files. Not doing so will cause discrepancies in the appearance of your site from one page to another; if one cached page has been updated recently it will appear different from a page that hasn't been updated since the changes were made. This issue would resolve itself over time, but I think it is best to avoid it all together.
