Friday, April 19, 2013

Common ptifalls of FDR estimation part one

Today’s most widely used method for FDR estimation is the target-decoy strategy. This is a well-established method in statistics and started to be used in proteomics around 2007.

In this approach, a decoy database that contains the same number of proteins as the target database are searched together by the database search engine to identify peptides. The blue colors indicate the target hits and the orange colors indicate the decoy hits, the squares are the false hits, and circles are true hits. 
The decoy proteins are randomly generated so that any decoy hit is supposedly a false hit. Since the search engine doesn’t know which sequences are from target and which are from decoy, when it makes a mistake, the mistake falls in the target and decoy databases with equal probability. Thus, the total number of false target hits can be approximated by the number of decoy hits in the final result. And the FDR can be estimated by the ratio between the numbers of decoy hits and the number of target hits. 

The target-decoy strategy is a powerful method for FDR estimation. However, as we will discover in the next little while, such a powerful method must be used with caution to avoid FDR underestimation. 

The first pitfall in the use of target-decoy approach for FDR estimation is due to the so-called multiple round search strategy in today’s database search software. 

This multi-round search was popularized by the X!Tandem program published in 2004, in order to speed up the computation. The first round uses a fast but less sensitive search method to quickly identify a shortlist of proteins from the large database. Then, the second round uses a more sensitive but slower search method to identify peptides, but only from the short list of proteins. This effectively speeds up the search without sacrificing too much sensitivity. Indeed, X!Tandem is one of the fastest search algorithm used today.

However, as pointed out by a paper published in JPR in 2010, this multiple-round search strategy screws up the target-decoy estimation of the FDR. The reason is that after the first round, there will be more target proteins than the decoy in the short list. Thus, if the second round search makes a mistake, the mistake will be more likely in the target proteins. So, we will end up with fewer decoy hits than the actual false target hits. This causes the FDR underestimation.
The JPR paper in 2010 provided a fix to this problem. But a year later, in another JPR paper, Bern and
Kil pointed out that the fix was wrong, and proposed a different fix that required the change of the search engine’s algorithm. This shows that the FDR estimation is very tricky, even the experts can sometimes get it wrong. 

In PEAKS, we used a new approach, called decoy fusion to solve this problem. 

Instead of mixing the target and decoy databases, we append a decoy sequence to each target protein.

So, after the fast search round, the protein shortlist will still contain the same length of target and decoy sequences. And the false hits of the second round will have the equal chance to be from the target and decoy sequences. This recreates the balance and can accurately estimate the FDR in the multiple-round search setting.

*The content of this post is extracted from "Practical Guide to Significantly Improve Peptide Identification Sensitivity and Accuracy" by Dr. Bin Ma, CTO of Bioinformatics Solutions Inc. You can find the link to the guide on this page.


  1. عزيزي العميل إذا كنت تبحث عن أفضل شركات التنظيف في الدمام ، فيجب عليك الاتصال بشركة التنظيف فيلات في الدمام على الإطلاق ، فالشركة تنتظرك الآن للقيام بكل ذلك
    شركة النجوم لخدمات التنظيف
    شركة نقل اثاث بجدة
    شركة كشف تسربات المياه بجدة
    نصائح لرائحة بيت منعشة

  2. such a important information, i like your writing skills and everything about this blog just caught my attention.
    - Heley robert

  3. Put more information on your website like this one, I really like these types of blogs. I am also wanted to write blog kindly guide me if my topic is facial tissue suppliers then what should I do first and how will I create new and unique content on this topic

  4. Amazing blog and about beauty and personal care because everyday in fashion industry have new changes. Find out more about These: leather handbag manufacturer

  5. yes this is right that this strategy is useful in many ways even proof of funds providers are also talking about it

  6. having a rough estimation for any thing helps a lot even 100 cotton t shirt suppliers have estimate about the cotton usage

  7. The moderator and admin teams at 808 teens are always on hand to assist with the enforcement of chat rules and safety precautions. Users have the ability to report messages and bullying, as well as ignore irritating users. Our list of features is constantly developing in response to Teen Chat customer demand, which will make the site even more enjoyable!

  8. At Inleit Ingredients, we ensure that all of our products are in complete compliance with the highest food safety requirements by continuously improving the safety and inspection of our working process. food safety quality management system We are committed to adopting, adhering to, and maintaining a Food Quality and Safety Policy, with the satisfaction of our customers' requirements and expectations taking precedence above all other considerations.