HUPO drives the creation of protein test standards

Over the last ten years the Proteomics field has been a technologically dynamic area. New methods and techniques help drive the field to achieve more sophisticated measurements that yield increasingly larger volumes of data and information. This creates several problems.

The first is to question how scientists should test and validate new methods and technologies. Currently, proteins or protein mixtures used to document performance are purchased from vendors or prepared in the laboratory (e.g. cell culture). The samples are not necessarily designed to serve as test standards, but have done so in the absence of other alternatives. The second issue is how to test a method you are trying to replicate in your laboratory to ensure you are able to achieve the results reported by others. In other words, how can you calibrate your use of the methods of others to ensure you obtain the same results? Replication requires exact duplication of the methods including the use of the same test sample. A common problem has been the use of different types of protein mixtures to document the performance of a technique or to demonstrate the enhancements of a method which makes it difficult to judge the magnitude of improvement. For example, if a paper demonstrates a technique with a cell lysate from yeast and another paper uses a Hela cell lysate, comparing the techniques is made difficult because the two samples have different levels of complexity. Ideally, a technique should be compared with the same protein mixture prepared in the same manner in order to minimise sample related variability. To this end HUPO is coordinating the identification and production of suitable standards for a variety of different types of proteomic analyses.

Standardising proteomic analyses

A driving force behind the development of protein test standards is comparability and calibration between laboratories and techniques. To achieve this goal good quality test standards need to be readily available and often this means commercially available (Figure1). A commercial standard has the resources and integrity of the business behind it and is likely to lead to a widely available test standard, preferably at a reasonable cost. To achieve this goal the proteomics field needs to decide which test standards should be developed, how they should be made and provide guidelines on analysis and the expected results. Through close collaboration with industry, informed decisions can be made on the suitability of standards for commercialisation.

A recent HUPO workshop in Montreal last July was designed to initiate discussions on the development of protein test standards. As a first set of test standards several candidates were proposed. It is intended that a test standard will be applicable to a wide variety of analytical techniques, e.g. gel electrophoresis, LC/MS/MS and LC/LC/MS/MS. Two pre-commercial products were reviewed with details on the purification strategies and results from preliminary analyses. These included a 20 protein mixture from Invitrogen and a 49 protein mixture from Sigma-Aldrich. Both products are designed to be highly pure, equimolar simple protein mixtures. Beyond uses for direct analysis they could be spiked into more complex mixtures to create more challenging mixtures to evaluate method performance. Initially, the protein test standards are expected to be equimolar, but different amounts of each protein could be mixed together to test other variables, such as dynamic range. A naturally occurring protein mixture proposed as a test standard was the T4 E. coli phage. T4 phage is easy to grow and enrich but, more importantly, it has a natural and known protein abundance difference thus making this protein mixture suitable to test 1-D and 2-D gel electrophoresis conditions. A concern about the T4 phage as a test standard is the ability of laboratories to duplicate purification protocols, ease of purification and the impact of E. coli contamination on the results. A commercial source of the material as a test standard is being considered. Results from initial analyses of the test mixtures were further discussed at the HUPO 2006 International Congress (Oct 28th-November 2nd).

An area of increasing effort in proteomics is the development and application of methods to analyse post translational modifications. These efforts encompass the development of methods to both enrich and analyse modified peptides and thus represent an important and rapidly evolving area. As methods are developed they are usually tested and validated using a particular type of sample. In some cases these test standards may be difficult to duplicate precisely, or may require specialised expertise to create (e.g. cell line cultures) and consequently other laboratories attempting to replicate the method may use a different test sample. In this situation it is difficult to establish whether or not replication of the method achieved the same or similar results. For example, one of the first applications of metal ion affinity chromatography to large scale phosphoproteome profiling used a yeast cell lysate to test the method.1 An alternate strategy was published a few years later using a HeLa cell nuclear extract by Beausoleil et al.2 Due to the fact that the methods were demonstrated on two different types of samples, it was difficult to establish the magnitude of the improvement from the method alone. Ideally the methods should have been demonstrated on a well established test standard and then applied to a specific biological system. The proteomics community would be able to gauge the improvement if methods were compared on the same type of standard protein mixture. Test standards to evaluate methods to enrich and analyse post translational modifications are a good next step and discussions at the Test standard workshop at HUPO will propose and evaluate the merits of particular types of standards. It is important that test standards be readily accessible and of reasonable cost, so these issues will be discussed.

Improving the quality of analyses

Proteomics is having a large impact on the analysis of organelles, cells, tissues and body fluids. Large scale studies are now possible, but the analyses are less than comprehensive and thus the development of technology to improve analyses is on-going. These studies are complicated by the increased complexity of the protein mixtures in addition to the range of protein abundances present. Abundance differences can range from 103 to 1012. Furthermore, organelles, cells and tissues contain lipid bilayers or membranes into which proteins are inserted. Extraction, digestion and analysis are complicated by the presence of these hydrophobic proteins and thus specific strategies are often needed to address this issue. Body fluids have a particular set of challenges associated with them, not the least of which is the dynamic range of protein abundances. Ninety-five per cent of the mass of serum consists of approximately 20 proteins creating barriers to the analysis of the lower abundance proteins. Creating test standards to assist method development in these areas is a difficult challenge. There are few examples of cell systems where an analysis has been substantiated as complete or at least nearly complete. One example can be found in S. cerevisiae. Yeast cells have been used as a test bed for proteomic technology development for years, but the exact (or at least a good estimate) number of proteins expressed in log phase growing cells and their abundances was unknown until Ghaemmaghami et al. measured the expression levels of all yeast proteins.3 We will never be sure that the exact same number of proteins is present at the measured levels, but we will at least have an accurate estimate of what should be there. Additional issues such as the strain to be used, growth conditions, cell lysis and digestion conditions need to be addressed as these factors can lead to analytical variability. Thus, yeast cells may make a good test standard for eukaryotic cells. It is not likely that issues surrounding test standards for these more complicated types of analyses will be resolved quickly, but hopefully experiences and lessons learned from the development of less complicated test standards will help inform the process.

HUPO has led the drive to create standards for data formats, presentation and analysis and is now studying the creation of test standards to allow evaluation and calibration of methods and developments in this rapidly evolving field. These efforts will help minimise duplication of efforts and allow the field to rapidly assess improvements that merit replication.

References

1. Ficarro, S. B.; McCleland, M. L.; Stukenberg, P. T.; Burke, D. J.; Ross, M. M.; Shabanowitz, J.; Hunt, D. F.; White, F. M. Nat Biotechnol 2002, 20, 301-305.
2. Beausoleil, S. A.; Jedrychowski, M.; Schwartz, D.; Elias, J. E.; Villen, J.; Li, J.; Cohn, M. A.; Cantley, L. C.; Gygi, S. P. Proc Natl Acad Sci U S A 2004, 101, 12130-12135.
3. Ghaemmaghami, S.; Huh, W. K.; Bower, K.; Howson, R. W.; Belle, A.; Dephoure, N.; O’Shea, E. K.; Weissman, J. S. Nature 2003, 425, 737-741.