Trial and Error

I'm pretty convinced that trial and error is the most consistently powerful method for attaining knowledge. There's just way to many ways a person to fail and make mistakes in reason. So, it's interesting that Ben Goldacre over at the Guardian should write an article at the Guardian titled "How can you tell if a policy is working? Run a trial." In other words, if you're considering making a law, run a trial to see what the best way to write the law.

A couple of caveats that Goldacre doesn't mention, of course. For one, trials are not always feasible; sometimes they're too expensive or simply unethical. Second of all, trials can only tell you how well a certain policy will achieve a specified goal; it can't tell unconditionally what the best policy is. For example, if you have a policy that increases safety, but compromises privacy, it's an open question whether the increased safety is worth the compromised privacy, and it certainly isn't self-evident that it is lawmakers that should always be making these decisions on behalf of the populace. Third of all, always remember to include a control. In other words, if you're trying to evaluate Law A vs Law B, you want to make sure you include (if possible) a control that evaluates what happens without either Law A or B; sometimes not creating a law to begin with is better. Lastly, it's a bit unrealistic to expect that governments, after a long history of ignoring scientific evidence and expert opinion, to suddenly start building policy on evidence. Alas, there's been too much research in public choice theory to make one think that politicians will consistently enact the laws that best reflect the evidence.

I'm pretty convinced that trial and error is the most consistently powerful method for attaining knowledge. There's just way to many ways a person to fail and make mistakes in reason. So, it's interesting that Ben Goldacre over at the Guardian should write an article at the Guardian titled "How can you tell if a policy is working? Run a trial." In other words, if you're considering making a law, run a trial to see what the best way to write the law.

A couple of caveats that Goldacre doesn't mention, of course. For one, trials are not always feasible; sometimes they're too expensive or simply unethical. Second of all, trials can only tell you how well a certain policy will achieve a specified goal; it can't tell unconditionally what the best policy is. For example, if you have a policy that increases safety, but compromises privacy, it's an open question whether the increased safety is worth the compromised privacy, and it certainly isn't self-evident that it is lawmakers that should always be making these decisions on behalf of the populace. Third of all, always remember to include a control. In other words, if you're trying to evaluate Law A vs Law B, you want to make sure you include (if possible) a control that evaluates what happens without either Law A or B; sometimes not creating a law to begin with is better. Lastly, it's a bit unrealistic to expect that governments, after a long history of ignoring scientific evidence and expert opinion, to suddenly start building policy on evidence. Alas, there's been too much research in public choice theory to make one think that politicians will consistently enact the laws that best reflect the evidence.

Trial and Error