In recent analysis sessions at a local university, the worrying influence of human judgement in the reproducibility of results was made very clear. Students completing an exercise had shown great consistency in their analysis results using Progenesis SameSpots, right up to the point where they had to make a judgement call. And what was the decision that caused such divergence?
“Will this spot yield a reliable protein identification, if picked?”
The fact that SameSpots supported such reproducible, objective analysis up to this point was great to see. In fact, a large percentage of those taking part had managed to analyse their images 100% in accordance with our own expert’s analysis. This article, however, aims to support you better in making the final step of selecting your picking spots from the list of interesting spots found in your analysis.
We’ll start by looking at the advice sheet given to the students who took part in our exercise. The examples it gives are exaggerated to illustrate problems clearly, but the types of problem are not unusual:
The following table gives a few examples of why a given spot might not produce a reliable protein identification:
As mentioned above, the problems shown are deliberately exaggerated; the case for rejecting the spot is clear. Often, however, it’s not so clear and even 2 experts may disagree over the reliability of a given spot. Where does one draw the line? What is an acceptable tolerance for alignment? What about for intensity? And how can you be sure that’s really 2 spots and not just an uneven running of the second dimension?
So what can you do to reduce the subjectivity here? First and foremost is to make the most of the tools available to you in the software. Secondly, take a quantitative rather than qualitative approach. In Progenesis SameSpots, this means:
- Use the controls in the Filtering step to remove very small spots
- This is a measure that can be made completely objective within your lab. Choose an area value and use it in your standard analysis protocol. The value itself may be based on the corresponding size of your robot’s picking head, the smallest spot size you can reliably cut by hand, or either of these plus a margin area (to account for the protein’s density profile across the spot area).
- Use the contrast control
- Sometimes, your spots may not stand out clearly against background, when judged by eye. This often illustrates the strength of robust statistical measures (being able to identify meaningful differences that can’t be done by eye alone), but there’s still a limit to how little protein material can be used to get a reliable identification. For a quantitative measure, define the contrast level at which spots must be readily apparent, and also the number of gels in which it must be evident.
- Use the 3D view to confirm merged proteins
- Where a spot outline suggests it may actually consist of two partially-separated proteins, the 3D view can be an invaluable tool to confirm this. In the View Results screen, select the 3D Montage tab. The 3D spot image can then be moved around using the mouse. If you can clearly see a valley inside the spot boundary, this may well be a pair of merged proteins. The Peak Scale option can help clarify things further.
- Before investigating in the 3D view:
- After rotating and increasing the peak scaling view option:
- As with the contrast level, you may wish to define the number of gels in which the valley is present before rejection as part of your standard analysis protocol.
Of course, these are just some of the things you can do to address the issue of subjectivity-based variation. Here at Nonlinear, we’ll continue to look for further ways that we can eliminate it in the software itself. That may, ultimately, be an unreachable goal — there’s always another new situation to cater for — but we remain committed to proving that proteomics can work, reliably and repeatably.
The questions raised when considering a spot’s picking potential are not always simple to answer, but it’s very clear that you need to be aware of the issues. It could mean the difference between identifying a key protein and missing it entirely. For our part, here at Nonlinear, with our analysis sessions’ confirmation of the serious implications of using human judgement in your analysis, we’ll be refocusing our development efforts on increasing objectivity and eliminating subjectivity.
If the results from your current software or analysis service still rely heavily on subjective decisions — and the provider’s not addressing the situation — you may be left wondering why.