Friday, April 19, 2013

Common ptifalls of FDR estimation part one

Today’s most widely used method for FDR estimation is the target-decoy strategy. This is a well-established method in statistics and started to be used in proteomics around 2007.

In this approach, a decoy database that contains the same number of proteins as the target database are searched together by the database search engine to identify peptides. The blue colors indicate the target hits and the orange colors indicate the decoy hits, the squares are the false hits, and circles are true hits. 
The decoy proteins are randomly generated so that any decoy hit is supposedly a false hit. Since the search engine doesn’t know which sequences are from target and which are from decoy, when it makes a mistake, the mistake falls in the target and decoy databases with equal probability. Thus, the total number of false target hits can be approximated by the number of decoy hits in the final result. And the FDR can be estimated by the ratio between the numbers of decoy hits and the number of target hits. 

The target-decoy strategy is a powerful method for FDR estimation. However, as we will discover in the next little while, such a powerful method must be used with caution to avoid FDR underestimation. 

The first pitfall in the use of target-decoy approach for FDR estimation is due to the so-called multiple round search strategy in today’s database search software. 

This multi-round search was popularized by the X!Tandem program published in 2004, in order to speed up the computation. The first round uses a fast but less sensitive search method to quickly identify a shortlist of proteins from the large database. Then, the second round uses a more sensitive but slower search method to identify peptides, but only from the short list of proteins. This effectively speeds up the search without sacrificing too much sensitivity. Indeed, X!Tandem is one of the fastest search algorithm used today.

However, as pointed out by a paper published in JPR in 2010, this multiple-round search strategy screws up the target-decoy estimation of the FDR. The reason is that after the first round, there will be more target proteins than the decoy in the short list. Thus, if the second round search makes a mistake, the mistake will be more likely in the target proteins. So, we will end up with fewer decoy hits than the actual false target hits. This causes the FDR underestimation.
The JPR paper in 2010 provided a fix to this problem. But a year later, in another JPR paper, Bern and
Kil pointed out that the fix was wrong, and proposed a different fix that required the change of the search engine’s algorithm. This shows that the FDR estimation is very tricky, even the experts can sometimes get it wrong. 


In PEAKS, we used a new approach, called decoy fusion to solve this problem. 

Instead of mixing the target and decoy databases, we append a decoy sequence to each target protein.

So, after the fast search round, the protein shortlist will still contain the same length of target and decoy sequences. And the false hits of the second round will have the equal chance to be from the target and decoy sequences. This recreates the balance and can accurately estimate the FDR in the multiple-round search setting.


*The content of this post is extracted from "Practical Guide to Significantly Improve Peptide Identification Sensitivity and Accuracy" by Dr. Bin Ma, CTO of Bioinformatics Solutions Inc. You can find the link to the guide on this page.

15 comments:

  1. عزيزي العميل إذا كنت تبحث عن أفضل شركات التنظيف في الدمام ، فيجب عليك الاتصال بشركة التنظيف فيلات في الدمام على الإطلاق ، فالشركة تنتظرك الآن للقيام بكل ذلك
    شركة النجوم لخدمات التنظيف
    شركة نقل اثاث بجدة
    شركة كشف تسربات المياه بجدة
    نصائح لرائحة بيت منعشة

    ReplyDelete
  2. such a important information, i like your writing skills and everything about this blog just caught my attention.
    - Heley robert

    ReplyDelete
  3. Amazing blog and about beauty and personal care because everyday in fashion industry have new changes. Find out more about These: leather handbag manufacturer

    ReplyDelete
  4. yes this is right that this strategy is useful in many ways even proof of funds providers are also talking about it

    ReplyDelete
  5. having a rough estimation for any thing helps a lot even 100 cotton t shirt suppliers have estimate about the cotton usage

    ReplyDelete
  6. The moderator and admin teams at 808 teens are always on hand to assist with the enforcement of chat rules and safety precautions. Users have the ability to report messages and bullying, as well as ignore irritating users. Our list of features is constantly developing in response to Teen Chat customer demand, which will make the site even more enjoyable!

    ReplyDelete
  7. At Inleit Ingredients, we ensure that all of our products are in complete compliance with the highest food safety requirements by continuously improving the safety and inspection of our working process. food safety quality management system We are committed to adopting, adhering to, and maintaining a Food Quality and Safety Policy, with the satisfaction of our customers' requirements and expectations taking precedence above all other considerations.

    ReplyDelete
  8. "Common Pitfalls of FDR Estimation Part One" offers a comprehensive overview of the challenges of False Discovery Rate (FDR) estimation. It identifies common pitfalls researchers may face, provides practical examples, and offers clear explanations. The article's structured format and use of visual aids make it accessible to a wide audience, making it an excellent resource for those navigating FDR estimation complexities in their research.
    Nueva York de Divorcio en Período de Espera

    ReplyDelete
  9. I'm pleased to learn that you find the information enjoyable! If you have any particular requests or topics you'd like me to address or rephrase, please feel free to inform me. I'll make every effort to assist you.
    Working Capital , Invoice Discounting

    ReplyDelete
  10. Selection Bias: FDR estimation can be prone to selection bias if the subset of features or data points chosen for analysis is not representative of the overall dataset.

    Assumption Violation: Incorrect assumptions about the distribution or independence of features may lead to flawed FDR estimates, undermining the reliability of the results.

    Data Leakage: Inadvertent inclusion of information from the validation set into the training set can lead to optimistic FDR estimates, as the model may inadvertently learn patterns specific to the validation set.
    estate tax lawyer attorney corporate

    ReplyDelete
  11. "PEAKS Blog" is a comprehensive platform that offers a diverse range of content, catering to various interests. It covers a wide spectrum from insightful articles to entertaining reads, keeping readers informed and entertained. The blog's writing style is engaging, making it easy to connect with topics. Its commitment to quality and relevance makes it a valuable resource for those seeking informative and enjoyable content. Lawyers in new jersey

    ReplyDelete
  12. The article "Common Pitfalls of FDR Estimation – Part One" provides a comprehensive guide for researchers seeking accurate and reliable results in FDR estimation, highlighting potential pitfalls such as inappropriate statistical assumptions and inadequate control of confounding variables. New York State Divorce Lawyers

    ReplyDelete