I think I might be a little confused as well. Are you saying you found such a resource or that you want to create one?
I'm assuming the later. A few ideas: Have the tests be reviewed both before and after they are run. It can be easy to set up a bad test (for example, having more than one variable), but a peer review before should be able to point something like that out. I would guess data would also be absolutely vital. A browser plugin or some kind of API might be nice for entering data, as would integration with Google Analytics (I'm fairly certain you can download your data from GA).
A common set of tests, with specific criteria, would also be helpful for gathering the most reliable results as one test is hardly ever conclusive. Suppose you have Test 1A: The Title Tag, which tests whether or not a title tag is useful in SEO. If 1000 people run the same test, you can probably make a fairly good hypothesis with the data. Think Project Euler, but for a purpose and with data collection. Another advantage to crowd sourcing it would be that it is more cost effective to spread the tests around rather than having one person or company do it.
Last edited by VirtuosiMedia; 11-11-2008 at 11:28 PM..
|