07
Oct
07

Pipe Dreams

Although most of you are probably already familiar with Yahoo pipes by now, and there have already been several bioinformatics, etc pipes created, I thought I’d add one more to the array. I made this pipe back in August to search through several of the bioinformatics journals I could think of, as well as a couple other mainstream science journals and preprint servers.

Daily Cup O' Joe

The pipe works by grabbing feeds from each of the journals and then filtering based on the title and/or description. The pipe includes feeds from Oxford Bioinformatics, Evolutionary Bioinformatics, Journal of Computational Biology, PLOS Computational Biology, BMC Bioinformatics, Nature, and Science (along with preprint feeds from several of the journals). To use the pipe, you simply specify the search term(s) you want to filter by. Running the pipe will show you what the current results for the specified search terms are, as well as give you the option to subscribe to those search terms. This way, if you want to be sent any new articles relating to “prediction,” for example, you can add the search to your news reader.


Yahoo pipes example search results

Although the pipe works pretty well for the most part, I’ve noticed that there are still a couple kinks. For one thing, the pipe has trouble fetching articles from PLOS. When trying to grab the feed, it outputs “error fetching pbiol.plosjournals.org/perlserv/?request=get-rss&issn=1553-7358&type=new-articles (302 Moved Temporarily)“. I’ve checked the feed, however, and it seems to be working fine. Another anomaly that I noticed is that sometimes the filters have trouble working together. The pipe is supposed to perform an “OR” search based on input to the title/description fields. In one instance I was getting more results when searching only the description field than when searching both the title and description field for the same search term. That was a couple of days ago, and I have not been able to reproduce the error since then.

All in all, the pipe is pretty nifty. You all are welcome to modify and use it as you please. Let me know if you have any suggestions :)


2 Responses to “Pipe Dreams”


  1. October 8, 2007 at 1:49 pm

    Dear Keith,

    Thank you for your interest in PLoS Computational Biology, and for taking the time to share the “kinks” you have been experiencing with our RSS Feed. We sincerely appreciate all feedback, and constantly work to improve access to open-access research.

    I am not too familiar with the Yahoo Pipe set-up, but I noticed that the error message you mention returns an incorrect feed URL, which might explain the issue:
    ** Wrong Feed URL (as indicated in Yahoo Pipe Error Message):
    http://pbiol.plosjournals.org/perlserv/?request=get-rss&issn=1553-7358&type=new-articles
    ** Correct Feed URL:
    http://compbiol.plosjournals.org/perlserv/?request=get-rss&issn=1553-7358&type=new-articles

    Additionally, when I tested “Daily cup o’ Joe” Pipes for the PLoS Computational biology feed with the text filter “heat shock”, it did fetch (3) relevant RSS results, and did not return an error message.

    On a separate note, there seems to be a parsing error when trying to retrieve feeds from Bio Med Central: “could not parse feed from: http://www.biomedcentral.com/bmcbioinformatics/rss/

    I hope this help resolve any issues you might be encountering, please do not hesitate to keep us posted on your progress!

    Celine Nadeau
    Web Producer – PLoS Journals
    cnadeau [at] plos . org

  2. 2 Keith
    October 8, 2007 at 2:29 pm

    Hi Celine,

    Thank you for the feedback. I believe the error is on the side of the Yahoo Pipes framework rather than on the side of the journals. I have gone back and tested the pipe again and verified the that it is in fact able to fetch the PLOS article feed. The error message does still come up on occasion, but this is likely due to a problem on Yahoos end.

    If you would like to see the exact error message, you can click on “view source,” which requires first logging into into a Yahoo account. From there, highlight the PLOS feed box and hit “refresh” several times. About half of the time yahoo will successfully retrieve the rss feed, and the other times it will fail, outputting the error message mentioned above. I’m not sure why this happens, but I believe it has something to do with the way Yahoo pipes is communicating with the PLOS servers. I will post a note on the yahoo Pipes forums and report the bug to them if it has not already been reported.

    As for the Bio Med Central feed, Yahoo pipes has been consistently failing to fetch the feed contents. I believe this may be because they are apparently using a non-standard rss form for their article feed, or have a redirect somewhere that yahoo (and myself) are not catching. What is interesting however is that Google reader *is* able to fetch articles from BMC. In any case I will let them know, and check with the pipes forums.

    Thanks for the help!
    Take care,
    Keith


Leave a Reply