I know there are programs out there that does the job, but I can't find any...

Any suggestions?

double click on a zipped file and it will show the files in that zip.
Just delete the file in question.

Is that what you wanted ?

No that was not what i wanted, thanks though. I have about 500 .zip files and I did a mistake and added the wrong file to many of them. What I wanted was to delete that file from all the archives at once.

Figured it out myself. just did a search for the file in that directory and windows XP also searched inside all the archives. Then it was simple to delete them.