Friday, April 26, 2013

Common ptifalls of FDR estimation part three


The third pitfall is also caused due to the over-emphasis on sensitivity.

There is another trend in database search software to re-score the peptide identification results by using machine learning. The idea is straightforward: After the search, we know what the decoy hits are. The algorithm should take advantage of it, and retrain the parameters of the scoring function to get rid of the decoy hits. With this effort, it will get rid of a lot of the target false hits as well.

The method is valid, except that it may cause FDR underestimation. This is because the target false hits are unknown to the machine learning algorithm. Therefore, there is a risk that the machine learning algorithm removes more decoy hits than the target false hits.
This overfit risk is well known in machine learning. A machine learning expert can reduce the risk but can never get rid of it. 

The solution to this pitfall number 3 is trickier.

The first suggestion: don’t use it. The philosophy here is that judges cannot be players. If we want to use the decoy for result validate, the decoy information should never be released to the search algorithm.

If this re-scoring method must be used due to the low-performance of some database search software, it should only be used for very large dataset to reduce the risk of over-fit.

Perhaps the best solution is the third one. That is, the retraining of the score parameters should be done for each different instrument type, instead of each dataset. This will gain much of the benefit provided by machine learning, but without the problem of over-fitting. Indeed, this third approach is what we do in the PEAKS DB algorithm.


*The content of this post is extracted from "Practical Guide to Significantly Improve Peptide Identification Sensitivity and Accuracy" by Dr. Bin Ma, CTO of Bioinformatics Solutions Inc. You can find the link to the guide on this page.

7 comments:

  1. هذا ما يريده جميع العملاء بشكل مستمر لأن الشركة توفر جميع خدمات مكافحة الآفات على مدار الساعة وطوال أيام الأسبوع ، مما يجعلها واحدة من أهم الشركات العاملة في مجال مكافحة الآفات. والقضاء.
    شركة مكافحة النمل الابيض بجازان
    شركة مكافحة حشرات بجازان
    شركة رش مبيدات بجازان
    افضل شركة رش مبيدات

    ReplyDelete
  2. I’m excited to uncover this page. I need to thank you for your time for this, particularly fantastic read!! I definitely really liked every part of it and I also have you saved to fav to look at new information in your site.
    Machine learning training in pune
    Machine learning classes in pune
    Machine learning course in pune

    ReplyDelete
  3. Found your post interesting to read. I cant wait to see your post soon. Good Luck for the upcoming update.This article is really very interesting and effective.
    - Jamescooper

    ReplyDelete
  4. Very simple and useful content. I am also wanted to write blog kindly guide me if my topic is geophyscial investigation then what should I do first and how will I create new and unique content on this topic

    ReplyDelete
  5. We have always been at the service of students at best paper help when it comes to assisting them with research papers, essays, and any other projects they require. You can get exceptional, on point, and assist students in achieving the highest possible grades in class. Our paper helper is unrivaled, and we can assure you that you will receive fantastic grades. Having online paper help makes school life easier and more manageable because quality assignments can be delivered on time, resulting in great grades. It's quite OK to seek the best paper help; in fact, you should do so whenever possible because the paper helper will save you time and help you flourish.

    ReplyDelete
  6. Found your post interesting to read. I think you should also write an article about Portable fire pump

    ReplyDelete
  7. security guard services, Los Angeles' most prestigious security guard company, where safety begins with protection and reliability begins with security guard company san francisco

    Officers are standing by to keep your belongings and lives safe and protected. We offer defence and prevention against any injury, haphazardness, or other crime. We are Fire Watch Guard Security Service, a reputable and dependable Los Angeles security firm.

    ReplyDelete