I was meeting with colleagues today about a research project. We’ve just recently started getting our first experimental results.
The meeting was largely do decide what we should do next. We could go in many directions, but we had to focus and choose one.
There is always a temptation to immediately go large based on that first early data, to gather lots more data, to instrument up in grand style. But we realized that this would be a mistake.
If you try to figure out too many things at once, it not only takes a lot of time and effort, but you also end up creating more opportunities for confusion and error.
So we decided we would run some modest benchmark tests to figure out how much we could trust those first measurements. The results of those benchmarks will tell us how much of what we have so far is actual signal, and how much is just noise.
At the meeting, here is how I ended up describing the next thing we need to figure out: “What the hell are we looking at?”
Just from a practical perspective, I think those are seven very useful words in research.