Improving MEDLINE Query Precision With Pseudo Relevance Feedback : Improving MEDLINE Query Precision With Pseudo Relevance Feedback January 30, 2004
Jay Urbain
Advisor: Ophir Frieder
Improving MEDLINE Query Precision With Pseudo Relevance Feedback : Improving MEDLINE Query Precision With Pseudo Relevance Feedback Improving the precision of MEDLINE search results is an important challenge.
Prior investigations on improving retrieval performance using thesaurus and relevance feedback have produced mixed results.
The use of a mixture of IR strategies & utilities along with RF makes it difficult to determine the contributions of each technique.
We review prior work, and use a systematic method to evaluate best techniques & parameters for using pseudo RF on MEDLINE.
Our research showed an increase in average precision of 21.5% using pseudo RF, exceeding the results of similar efforts.
MEDLINE : MEDLINE Collection of more than 10,000,000 citations from more than 4,200 medical journals, indexed from a MetaThesaurus containing more than 19,000 Medical Subject Headings (MeSH) .
MeSH organized in a related hierarchy of categories and sub-categories.
Myocardial Ischemia (304,753)
Coronary Disease (140,720)
Angina Pectoris (33,938)
Angina, Pectoris, Variant (1,889)
Angina, Unstable
Syndrome X
Coronary Aneurysm
Coronary Arteriosclerosis
Coronary Thrombosis
Coronary Vasospasm
http://www.nlm.nih.gov/mesh/MBrowser.html
Sample MeSH : Sample MeSH
Adult
Cardiovascular Diseases
epidemiology
prevention & control
Diabetes Mellitus, Type II
epidemiology
…
UMLS MetaThesaurus : UMLS MetaThesaurus The MetaThesaurus is a comprehensive vocabulary that includes definitions MeSH terms [13].
Thesauri can have three types of relationships:
Synonym
Represents same underlying concept.
“cancer vs. carcinoma”
Hierarchical
Broader/narrower “is-a” relationship.
“Angiotensin Converting Enzyme Inhibitors vs. Captopril”
Related
Some other type of relationship
“hypertensive vs. anti-hypertensive”
Searching the MEDLINE Database : Searching the MEDLINE Database Requires knowledge of the hierarchical MeSH indexing scheme.
Even “expert” users tend to select MeSH terms that are too general.
Studies by Hersh, et al. [5] found novice users using VSM search can be as effective as experts using Boolean search.
IR systems are becoming essential tools for the practice of evidence based medicine [12].
Novice and expert users alike need to find relevant documents with the facts needed to make decisions.
Retrieval Strategies : Retrieval Strategies Vector Space Model (VSM)
Measure similarity by inner product, or Cosine angle between query & document vectors.
IDF = log(N/df) // N= total # of docs; df=# of docs with term
SC = ∑ (tfd*idf * tfq*idf) // dot product.
SC = ∑ (tfd*idf * tfq*idf)/nd // cosine normalized by document length.
Probabilistic
Probability estimate of the chance of a document being relevant to a query.
P(R | t1) = (num relevant with t1 / num relevant)
(num with t1 / all documents)
P(R | t1, t2, …, tn) = P(R | t1) x P(R | t1) x P(R | t2) x … x P(R | tn)
Relevance Feedback : Relevance Feedback Refine search queries based on prior relevance judgments.
Manual –
Relevant documents identified by the user.
New terms selected by the user or automatically.
Automatic “Pseudo” –
Assume the top-ranked documents are relevant.
New terms are selected automatically.
Prior Work : Prior Work
Experimental Methodology : Experimental Methodology Prior work used combinations of several IR strategies and utilities, making the identification of the contribution due to RF difficult.
Results from Hersh and Srinivasan, and others vary considerably.
Used a variation of the systematic method outlined by Lundquist, Grossman, and Frieder [7] for identifying optimal techniques to improve relevance feedback.
Our method include calibrating the number of top-ranked documents, the number of feedback terms used for relevance feedback, and adjusting the term weight.
Methodology – Experimental Steps : Methodology – Experimental Steps Evaluate baseline retrieval strategy and term weights.
Dot Product
Cosine
Probabilistic
Explore expansion using MeSH and title terms.
Identify optimal query expansion parameters:
Term relevance (term count, idf, term count and idf)
Number of top documents to include in term selection
Number of terms added to the query
Number of iterations
Data set, Sample Query : Data set, Sample Query Data set:
The TREC-9 training subset of the OHSUMED MEDLINE training collection: 54,710 MEDLINE citations from 1987 and 63 queries developed by the Hersh team at Oregon Health Sciences University [5].
Sample query:
Number: OHSU1
60 year old menopausal woman without hormone replacement therapy
<desc> Description:
Are there adverse effects on lipids when progesterone is given with estrogen replacement therapy
</top><br /><br /><br /><b>Sample MEDLINE Citation from Dataset </b> : Sample MEDLINE Citation from Dataset .I 48200
.U
87316326
.S
Obstet Gynecol Clin North Am 8712; 14(1):299-320
.M
Drug Administration Schedule; Endometrium/*DE/PA; Estrogens/AD/*PD; Female; Human; Menopause/*DE; Middle Age; Norethindrone/*AA/AD/PD; Support, Non-U.S. Gov't; Uterine Hemorrhage/ET/PA; Uterine Neoplasms/CI/PC.
.T
The effects of estrogens and progestogens on the endometrium. Modern approach to treatment.
.P
JOURNAL ARTICLE; REVIEW.
.W
The major hazard of postmenopausal cyclic estrogen therapy is endometrial hyperstimulation. The incidence of hyperplasia is dose dependent; the incidence of carcinoma is both dose and duration dependent. The risk of carcinoma is small. Invasive procedures such as endometrial biopsy to detect those patients with hyperplasia and carcinoma are unlikely to be cost-effective and have other disadvantages. …
.A
Whitehead MI; Fraser D.<br /><br /><br /><b>Methodology – Baseline </b> : Methodology – Baseline Phrasing*:
Phrasing techniques were not included in the study.
Initial testing with 2-word phrasing showed no significant increase in precision.
Significantly increased the size of the index, and index build time.
Stemming*
Standard Porter and Lovins stemmers significantly reduce precision in early testing.
*Would have also increased the number of variables in the study.
Environment:
The SimpleIR IR engine used in Information Retrieval classes at IIT was enhanced to include weighted terms, support for the tested information retrieval strategies, and modified to improve scalability.
All runs were performed on a 400 MHz Pentium III Dell laptop with 500M of memory.
<br /><br /><br /><b>Results - Baseline retrieval strategy </b> : Results - Baseline retrieval strategy Identify Document Term Weights
<br /><br /><br /><b>Results - Baseline retrieval strategy</b> : Results - Baseline retrieval strategy Identify Retrieval Strategy with Constant Relevance Feedback Parameters*
*3 additional terms from top 10 documents with one additional iteration, feedback term weight: termFrequency*idf*idf;
<br /><br /><br /><b>Expansion using MeSH and title terms</b> : Expansion using MeSH and title terms Query Term Expansion: MeSH terms, Title terms
*3 additional terms from top 10 documents with one additional iteration
<br /><br /><br /><b>Identify query expansion parameters </b> : Identify query expansion parameters Term Relevance (term count, idf, term count and idf)
3 additional terms from top 10 documents with one additional iteration, dot product SC, max(1, 0.1tf) document term weighting, query expansion with MeSH terms only<br /><br /><br /><b>Identify query expansion parameters </b> : Identify query expansion parameters Number of Top Documents to Include In Term Selection
3 additional terms from top documents with one additional iteration, dot product SC, max(1, 0.1tf) document term weighting, query expansion with MeSH terms only, term relevance measure of termFrequency*idf.<br /><br /><br /><b>Identify query expansion parameters </b> : Identify query expansion parameters Number of Terms Added To Query Terms*
*15 top documents with one additional iteration, dot product SC, max(1, 0.1tf) document term weighting, query expansion with MeSH terms only, term relevance measure of termFrequency*idf.<br /><br /><br /><b>Identify query expansion parameters </b> : Identify query expansion parameters Number of Feedback Iterations
3 terms selected from 15 top documents, dot product SC, max(1, 0.1tf) document term weighting, query expansion with MeSH terms only, term relevance measure of termFrequency*idf.<br /><br /><br /><b>Summary - RF Parameters </b> : Summary - RF Parameters Optimal parameters:
Select the 3 most discriminating terms
Use tf*idf term selection criteria
Select from the 15 most highly ranked documents
Use one feedback iteration.
These relevance feedback parameters yielded an average precision of .3590 over 11-points of recall and represented an increase of 21.5% over the average precision of .2955 achieved without any relevance feedback<br /><br /><br /><b>RF vs. No RF Precision </b> : RF vs. No RF Precision <br /><br /><br /><b>RF vs. No RF Precision at n Docs</b> : RF vs. No RF Precision at n Docs<br /><br /><br /><b>Discussion</b> : Discussion Pseudo RF using MeSH terms uniformly increased precision at all points of recall.
Results exceeded the improvement in average precision in similar studies by Hersh [21], Srinivasan [11], and Aronson [1], though they can’t be directly compared due to differences in underlying IR strategies and utilities.
Results indicate a broad based robustness of utilizing relevance feedback to improve precision.
Results and tuning parameters, may be subject to some bias introduced by the order in which we conducted our experiments.
By RF’s very nature of expanding queries with terms from top ranked documents, relevance feedback is bound to provide improvement since we are utilizing the criteria we have defined for relevance to identify “relevant.” terms.
<br /><br /><br /><b>Future Work</b> : Future Work Apply adaptive learning methods at each iteration of RF.
Restrict MeSH term expansion for highly ranked retrieved documents to MetaThesaurus concepts related to the initial query.
Expansion using Subject heading/sub-headings relations.
“fuzzy” term matcher utilizing machine learning.
Explore causes of rare diseases by looking for indirect links in different subsets of MEDLINE biomedical literature (Swanson [15]).
UMLS co-occurrence statistics.
Add page Score algorithm to help identify “gold standard” papers.
Full OSHUMED data set.
Automated weighting of feedback terms.
<br /><br /><br /><b>Jay Urbain</b> : Jay Urbain Professional:
7/99 to Present. Principal, Upstream Development LLC
8/03 to Present. Lecturer, Milwaukee School of Engineering, EECS.
9/00 to Present. Certified Instructor, Learning Tree International
7/97 to 7/99. VP Software, ThinkMed LLC
3/93 to 7/97. Senior Engineering Manager, Marquette Medical Systems
2/86 to 8/87. Senior Systems Engineer, Rockwell, Computer Vision Group
4/84 to 2/86. Senior Software Engineer, GCA Corp.
1/82 to 4/84. Software Engineer, Northrop Defense Systems
Academic:
Post Grad. Studies in Computer Science, IIT, Chicago
MBA, May ’97, UNIVERSITY OF WISCONSIN, Madison
MSEE, Computer Engineering, 1986, IIT, Chicago
BS Engineering, Computer & Info. Systems, 1981, UNIVERSITY OF ILLINOIS, Chicago
Certifications:
Sun Certified Enterprise Architect for Java Platform Enterprise Edition Technology
Sun Certified Programmer for the Java 2 Platform
Java Enterprise Certified Professional (Learning Tree International)
<br /><br /><br /><b>Old Stuff follows</b> : Old Stuff follows <br /><br /><br /><b>Retrieval Strategy – VSM</b> : Retrieval Strategy – VSM VSM represents each document and query as a vector of terms.
Measure similarity (SC) by taking dot product, or measuring angle (Cosine) between them.
Documents most similar to the query are deemed to be most relevant.
Each term in a document and a query is weighted by term frequency and inverse document frequency score (IDF).
IDF provides basic measure of how discriminating a term is in identifying a document
<br /><br /><br /><b>Retrieval Strategy – VSM</b> : Retrieval Strategy – VSM
Measure similarity (SC) by taking dot product, or measuring angle (Cosine) between document and query vector.
Each term in a document and a query is weighted by term frequency and inverse document frequency score (IDF).
IDF = log(N/df) // N= total # of docs; df=# of docs with term
SC = ∑ (tfd*idf * tfq*idf) // dot product.
SC = ∑ (tfd*idf * tfq*idf)/nd // cosine normalized by document length. <br /><br /><br /><b>Retrieval Strategy – Probabilistic</b> : Retrieval Strategy – Probabilistic Use probability estimate of the chance of a document being relevant to a query.
Odds a single term will be in a relevant document:
P(R | t1) = (num relevant with t1 / num relevant)
(num with t1 / all documents)
Probability for multiple terms:
P(R | t1, t2, …, tn) = P(R | t1) x P(R | t1) x P(R | t2) x … x P(R | tn)<br /><br /><br /><b>Retrieval Strategy – Issues</b> : Retrieval Strategy – Issues Small size of the query vector makes finding a higher percentage of relevant documents difficult since query vector contains only a few “relevant” terms.
In a MEDLINE application, it is difficult for the average user to identify relevant MeSH terms to facilitate high precision searches.
Relevance feedback can address both issues by adjusting the query vector through term expansion of highly relevant/discriminating terms.<br /><br /><br /><b>Relevance Feedback</b> : Relevance Feedback Automatically expand query by adding terms not in the original query, but in the ranked relevant documents.
The “more like this” link available with search engines allows user to select which documents are relevant.
In Rocchio’s approach:
VSM is used to rank documents.
After feedback from the user, relevant document vectors are added to the query, and non-relevant document vectors are removed:
<br /><br /><br /><b>Relevance Feedback</b> : Relevance Feedback Many extensions of Rocchio's algorithm have been proposed, like the Ide regular algorithm and Ide dec-hi algorithm.
Salton and Buckley found a variation of “Ide dec-hi” method, where relevant terms from top ranked doc’s are added directly to the queries, to be the most effective technique in their study.
<br /><br /><br /><b>Pseudo RF – Approach</b> : Pseudo RF – Approach
Assume the top ranking retrieved documents are relevant.
Identify most relevant terms (MeSH, title, abstract):
Term frequency
How effectively a term uniquely identifies a document (IDF).
Add most relevant terms to query.
Re-execute the expanded query.
Repeat process (if helpful).
<br /><br /><br /><b>Prior Work - Hersh</b> : Prior Work - Hersh Hersh, Price, and Donohoe [21] conducted a study assessing thesaurus-based query expansion using the MetaThesaurus.
OHSUMED test collection: 106 queries, ~350,000 MEDLINE ref’s.
Created MetaThesaurus concepts for each query.
Used SAPHIRE concept matching program to identify “best” terms
Used synonym, hierarchical, and related term information
Found all types of query expansion degraded aggregate retrieval performance as measured by recall and precision.
Some queries showed some improvement.<br /><br /><br /><b>Prior Work - Srinivasan</b> : Prior Work - Srinivasan Srinivasan demonstrated effectiveness of expanding MEDLINE queries with three expansion strategies:
expansion of MeSH terms.
expansion of document free text.
expansion of both MeSH and document free text terms.
Used subset of OHSUMED test collection of 75 queries and 2,344 MEDLINE documents.
Results showed significant improvement for all three strategies.
Showed that expansion of MeSH terms outperformed free text query expansion.
MeSH term expansion improved 11-AvgP by 14.4% over baseline of .5169.<br /><br /><br /><b>Prior Work - Aranson</b> : Prior Work - Aranson Aronson, et. al. [1] used the UMLS MetaMapTM program for associating UMLS Metathesaurus concepts with terms in the original query.
Query expansion included human assigned MeSH terms, along with terms through MetaThesaurus lookup.
Used probabilistic retrieval strategy.
Used same OHSUMED test collection as Srinivasan (75 queries and 2,344 MEDLINE documents.
Results compared favorable to automated relevance feedback and the results of Srinivasan (2.2% to 9.4% over baseline).
Authors suggest an optimal strategy is to use both UMLS Metathesaurus and some form of automated relevance feedback.<br /><br /><br /><b>Prior Work - Yang and Chute </b> : Prior Work - Yang and Chute Yang and Chute [14] utilized a least squared error technique to map query terms to MeSH terms in the MetaThesaurus.
They reported a 32.2% improvement in average precision over their baseline.
Difficult to compare results since they utilized a custom collection.
An additional concern is the risk of over-fitting their data by utilizing 88% of their data as the training set.<br /><br /><br /><b>Prior Work - Mao and Chu</b> : Prior Work - Mao and Chu Mao and Chu[8] use a variant on query expansion through the use of concepts and relations in the MetaThesaurus.
Define a phrase as consisting of word stems and “concepts.”
The similarity between two phrases is jointly determined by their conceptual similarity and their common word stems.
Concepts represent a UMLS concept identifier, e.g., “high body temperature” is a hyponym of “hyperthermia”, and “hyperthermia” is synonymous with “fever.”
Their results showed an increase of 16% in precision, but they only tested with one query. <br /><br /><br /><b>Prior Work - Mendonca and Cimino </b> : Prior Work - Mendonca and Cimino Mendonca and Cimino [23] investigated the utilization of co-occurrence of MeSH terms in MEDLINE citations.
Interested in terms associated with the search strategies optimal for evidence–based medicine to automate construction of a knowledge base.
Used UMLS MRCOC & MRREL table.
Found good specificity and sensitivity for co-occurrence of MeSH terms in MEDLINE citations.
These results lend support to the notion of utilizing MeSH terms from highly ranked documents as part of a query expansion strategy for relevance feedback.<br /><br /><br /></span>
</p>
</div>
</div>
<div id="DivProgressBar" class="modalWindow_old" style="width: 170px; float: left;">
<div style="float: left; background-color: #F6F6F6; padding: 10px; clear: both; width: 150px;">
<img src="http://www.authorstream.com/Images/loader.gif" alt="Loading" style="vertical-align: middle"
id="gif_image" />
<span class="verdana11r858585">Updating ....</span>
</div>
</div>
<!-- Footer Section Starts -->
<script type="text/javascript">
function openWindow()
{
window.open('http://www.authorstream.com/' + "ImportContacts.aspx",'',"left=0,top=0,width=700,height=600,status=1");
}
</script>
<div class="main">
<div class="footer">
<div style="float:right;padding-top:5px;padding-right:20px;width:960px;text-align:right;"><a href='http://www.authorstream.com/PowerPointForPeace/' target="_blank"><strong>Contribute to World Peace</strong></a></div>
<ul id="footerLeft">
<li id="Footer1_spanloggedOut"><span><strong>Account</strong><br /><br /></span></li>
<li><strong>RSS</strong><br /><br /></li>
<li><strong>Browse</strong><br /><br /></li>
<li><strong>Help & Info</strong><br /><br /></li>
<li><strong>Why authorSTREAM</strong></li>
</ul>
<ul id="footerRight">
<div id="Footer1_spanloggedOutJoinFree" style="width:100%;float:left;height:28px;">
<span style="float:left;"><li><a href='http://www.authorstream.com/Register/Presentation/Flemel-28551-Improving-MEDLINE-Query-Precision-Pseudo-Rele-Relevance-Feedback-Sample-MeSH-UMLS-MetaThesaurus-Sear-with-final-as-Entertainment-ppt-powerpoint/' title="Register">Join for Free</a></li>
<li>|</li><li><a href='http://www.authorstream.com/SignIn/Presentation/Flemel-28551-Improving-MEDLINE-Query-Precision-Pseudo-Rele-Relevance-Feedback-Sample-MeSH-UMLS-MetaThesaurus-Sear-with-final-as-Entertainment-ppt-powerpoint/' title="Sign In" >Sign In</a></li>
</span>
<span id="Footer1_pnlMyUploadedPPTGuest" style="float:left;display:none;"><li>|</li><li><a href='http://www.authorstream.com/MyUploaded-Presentations' title="Register">Your Uploaded Presentations</a></li></span>
</div>
<li><a target="_blank" title="Featured" href='http://www.authorstream.com/RSS/Type/Featured' >Featured</a></li><li>|</li><li><a title="Featured Audio" target="_blank" href='http://www.authorstream.com/RSS/Type/FeaturedAudio'>Featured Audio</a></li><li>|</li><li><a target="_blank" href='http://www.authorstream.com/RSS/Type/FeaturedAnimated' title="Featured Animated">Featured Animated</a></li><li>|</li><li><a target="_blank" href='http://www.authorstream.com/RSS/Type/LatestUploads' title="Latest Uploads">Latest Uploads</a></li><li>|</li><li><a target="_blank" href='http://www.authorstream.com/RSS/Type/MostLiked' title="Most Liked">Most Liked</a></li><li>|</li><li><a target="_blank" href='http://www.authorstream.com/RSS/Type/MostViewed' title="Most Viewed">Most Viewed</a><br /><br /></li>
<li><a href='http://www.authorstream.com/Presentation-Browse-All/Podcasts' >Podcast</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Featured' title="Featured">Featured</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Audio' title="Featured Audio">Featured Audio</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Animated' title="Featured Animated">Featured Animated</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Latest' title="Latest Uploads">Latest Uploads</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Liked' title="Most Liked">Most Liked</a></li><li>|</li><li><a href='http://www.authorstream.com/Presentation-Browse-All/Viewed' title="Most Viewed">Most Viewed</a></li><li>|</li><li><a href='http://www.authorstream.com/Category-presentations' title="Categories">Categories</a></li><li>|</li><li><a href='http://apps.facebook.com/authorstream/' target="_blank">Facebook App</a><br /><br /></li>
<li><a target="_blank" href='http://www.authorstream.com/About_us.aspx' title="About Us">About Us</a></li><li>|</li><li><a href='http://www.authorstream.com/Help' title="Help">Help</a></li><li>|</li><li>
<a href='http://www.authorstream.com/Faq' title="FAQ">FAQ</a></li><li>|</li><li><a href='http://www.authorstream.com/Contactus' title="Contact Us">Contact Us</a></li><li>|</li><li><a
href='http://www.authorstream.com/Termsandconditions' title="Terms and Conditions">Terms and Conditions</a></li><li>|</li><li><a href='http://www.authorstream.com/Privacypolicy' title="Privacy Policy">Privacy Policy</a></li><li>
|</li><li><a href='http://blog.authorstream.com/' title="Blog" target="_blank">Blog</a></li><li>|</li><li><a href='http://www.authorstream.com/Forum' target="_blank" title="Forum">Forum</a><br /><br /></li>
<li><a href="http://www.authorstream.com/how-it-works/">How it works</a></li><li>|</li><li><a href="http://www.authorstream.com/powerpoint-to-video/">PowerPoint to video, YouTube, iPod</a></li><li>|</li><li><a href="http://www.authorstream.com/powerpoint-to-blogs/">PowerPoint on blogs, websites</a></li><li>|</li><li><a href="http://www.authorstream.com/powerpoint-to-flash/">PowerPoint to Flash</a></li><li>|</li><li><a href="http://www.authorstream.com/present-live/">PresentLive</a></li>
</ul>
</div>
<div class="catchBuzz">Catch the <a href='http://www.authorstream.com/buzz/'>buzz</a> on authorSTREAM</div>
<div class="copyrighttxt">Copyright © 2002-2008 authorSTREAM. All rights reserved.</div>
</div>
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-211915-4");
pageTracker._initData();
pageTracker._trackPageview();
</script>
<!-- Footer Section Ends -->
<!-- Mark As Inappropriate Pop Up Div -->
<span id="upnlInAppr">
<div id="divFlag2" class="PopUpDvAll modalWindow" style="width: 430px; display: none;">
<span class="closeIcon"><a href="javascript:void(0);" onclick="PopupClose();">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 400px;">
<span class="headingwithLine"><span class="Orange_headingAll">Mark As Inappropriate</span></span>
<p>
Please select the category that most closely reflects your concern about the Presentation,
so that we can review it and determine whether it violates our Community Guidelines
or isn't appropriate for all viewers. Abusing this feature is also a violation of
the authorSTREAM Community Guidelines, so don't do it.<br />
<span id="lblFlagCmt2" class="stream_verdana11_r_signin_red"></span>
<br />
Mark as:<br />
<select name="ddlFlagIt2" id="ddlFlagIt2" onchange="SetInnappValue(this);" style="width:270px;">
<option value="">-Select-</option>
<option value="13">Mature (over 18 only) content</option>
<option value="14">Other Terms of Use violation</option>
</select>
<br />
<br />
Comments:<br />
<textarea name="txtFlag2Comments" rows="5" cols="50" id="txtFlag2Comments" style="width: 370px;"></textarea><br />
<span style="padding-top: 15px; float: left;">
<input type="submit" name="btnFlagIt2" value="Submit" onclick="return ValidateFlagComments('ddlFlagIt2', 'lblFlagCmt2', 'txtFlag2Comments');" id="btnFlagIt2" title="Submit" class="btn_all" style="width: 65px; color: #fff; cursor: pointer;" />
</span><span class="lnkbtnAll"><a href="javascript:void(0);" onclick="PopupClose();">
Close</a></span>
</p>
</div>
</div>
</span>
<!-- Mark As Inappropriate Pop Up Div Ends -->
<div id="MainProfileDiv" style="position: absolute; z-index: 100; left: 25; top: 56;
display: none;">
</div>
<!-- Progress Bar Pop Up Div Starts -->
<div id="divUpdate" class="modalWindow_old" style="width: 170px; float: left; z-index: 1000">
<div style="float: left; background-color: #F6F6F6; padding: 10px; clear: both; width: 150px;">
<img src="http://www.authorstream.com/Images/loader.gif" alt="Loading" style="vertical-align: middle"
id="Img1" />
<span class="verdana11r858585">Updating ....</span>
</div>
</div>
<!-- Progress Bar Pop Up Div Ends -->
<!-- Featured Pop Up Div -->
<span id="UpdatePanel1">
<div id="divMarkIt" class="PopUpDvAll modalWindow" style="width: 430px; display: none;">
<span class="closeIcon"><a href="javascript:void(0);" onclick="PopupClose();">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 400px;">
<span class="headingwithLine"><span class="Orange_headingAll">Featured</span></span>
<p>
If you like this presentation, flag it in the best fit category. (select from the
drop down below). Express your opinion in the Comments field. Your comments will
appear with the presentation on this page.<br />
<br />
<span id="lblFlagCmt1" class="stream_verdana11_r_signin_red"></span>
<br />
<span style="color: Red">*</span>Vote as:<br />
<select name="ddlFeatured" id="ddlFeatured" onchange="SetFeaturedValue(this);" style="width:270px;">
<option value="">-Select-</option>
<option value="10">Featured</option>
</select><br />
<br />
<span style="color: Red">*</span>Comments:<br />
<textarea name="txtFlagComments" rows="5" cols="50" id="txtFlagComments" style="width:370px;"></textarea><br />
<span style="padding-top: 15px; float: left;">
<input type="submit" name="btnMarkFeatured" value="Submit" onclick="return ValidateFlagComments('ddlFeatured', 'lblFlagCmt1', 'txtFlagComments');" id="btnMarkFeatured" title="Submit" class="btn_all" style="cursor: pointer; width: 65px; color: #fff;" />
</span><span class="lnkbtnAll"><a href="javascript:void(0);" onclick="PopupClose();">
Close</a></span>
</p>
</div>
</div>
</span>
<!-- Featured Pop Up Div Ends -->
<!-- RSS Feed Pop Up Div -->
<div class="PopUpDvAll" style="width: 430px; display: none;">
<span class="closeIcon"><a href="#">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 390px;">
<span class="headingwithLine"><span class="Orange_headingAll">Subscription of Rss Feed </span><span><img
src="http://www.authorstream.com/images/bullet.gif" width="5" height="7" alt="" style="margin-top: 5px;" /></span><span id="spnRSSHeader" class="smallHeading">Improving MEDLINE Query Precision With Pseudo Rele</span></span>
<p>
Feed URL:<br />
<span>
<input type="text" value="http://www.authorstream.com/Presentations/DaveGrewal-6213"
style="width: 340px;" class="txtBox" /></span><span><a href="#" class="lnkbtnAll"
style="margin-top: 5px;">Copy</a><br />
<br />
<br />
</span><span class="defaultBrowser">Open in Default Browser</span> <span style="margin-top: 6px;">
OR</span> <span class="selecttBrowser">Open in
<select style="width: 105px; margin-top: 0px!important; margin-top: 3px; padding: 1px;
margin-left: 10px; border: solid 1px #808080;">
<option>-Select-</option>
</select>
</span><a href="#" class="lnkbtnAll" style="margin-top: 15px; padding: 0px;">Close</a>
</p>
<span class="txtDvwithLine"><strong>You are viewing a feed that contains frequently
updated content.</strong> When you subscribe to a feed, it is added to the Common
Feed List. Updated information from the feed is automatically downloaded to your
computer and can be viewed in Internet Explorer and other programs. <a href="#" class="wordpress">
Learn more about feeds.</a></span>
</div>
</div>
<!-- RSS Feed Pop Up Div Ends -->
<!-- Podcast Feed Pop Up Div -->
<div class="PopUpDvAll" style="width: 430px; display: none;">
<span class="closeIcon"><a href="#">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 390px;">
<span class="headingwithLine"><span class="Orange_headingAll">Subscription of Podcast
Feed </span><span><img src="http://www.authorstream.com/images/bullet.gif" width="5"
height="7" alt="" style="margin-top: 5px;" /></span><span id="spnPodCastHeader" class="smallHeading">Improving MEDLINE Query Precision With Pseudo Rele</span></span>
<p>
Feed URL:<br />
<span>
<input type="text" value="http://www.authorstream.com/Presentations/DaveGrewal-6213"
style="width: 340px;" class="txtBox" /></span><span><a href="#" class="lnkbtnAll"
style="margin-top: 5px;">Copy</a><br />
<br />
<br />
</span><span class="defaultBrowser">Open in Default Browser</span> <span style="margin-top: 6px;">
OR</span> <span class="ituneBrowser">Open in iTunes</span>
</p>
<a href="#" class="lnkbtnAll" style="margin-top: 15px; padding: 0px;">Close</a>
</div>
</div>
<!-- Podcast Feed Pop Up Div Ends -->
<!-- WordPress Embed Pop Up Div -->
<div class="PopUpDvAll modalWindow" style="width: 430px; display: none;" id="ShowWordPress">
<span class="closeIcon"><a href="javascript:void(0);" onclick="PopupClose();">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 390px;">
<span class="headingwithLine"><span class="Orange_headingAll">WordPress embed </span><span><img
src="http://www.authorstream.com/images/bullet.gif" width="5" height="7" alt="" style="margin-top: 5px;" /></span><span id="spnEmbedHeader" class="smallHeading">Improving MEDLINE Query Precision With Pseudo Rele</span></span>
<p>
Feed URL:<br />
<span>
<input name="txtWordPress" type="text" value="[authorSTREAM id= Flemel-28551-Improving-MEDLINE-Query-Precision-Pseudo-Rele-Relevance-Feedback-Sample-MeSH-UMLS-MetaThesaurus-Sear-with-final-as-Entertainment-ppt-powerpoint pl= player by= Flemel]" readonly="readonly" id="txtWordPress" class="txtBox" ondblclick="SelectData(this);" onclick="SelectData(this);" style="width: 340px" /></span>
<span><a href="javascript:void(0);" class="lnkbtnAll" style="margin-top: 5px;" onclick="copy_Text();">
Copy</a></span>
</p>
<a href="javascript:void(0);" onclick="PopupClose();" class="lnkbtnAll" style="padding: 0px;
margin-top: 15px;">Close</a>
</div>
</div>
<!-- WordPress Embed Pop Up Div Ends -->
<div id="div1" class="modalWindow_old" style="height: 250px; width: 418px; display: none;">
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tr>
<td style="width: 80%; height: 40px; padding-left: 10px;">
<span class="Orange_headingAll">Send to Blogs and Networks</span></td>
<td align="right" style="padding-right: 10px;">
<img src="http://www.authorstream.com/Images/cross_small.gif" title="Close" style="cursor: pointer;"
onclick="PopupClose();" /></td>
</tr>
</table>
<div style="float: left; padding-left: 8px;">
<div id="divWildfirePost">
</div>
</div>
</div>
<!-- Customize Embed Pop Up Div -->
<div class="PopUpDvAll modalWindow" style="width: 680px; display: none;" id="ShowCustomizeEmbed">
<span class="closeIcon"><a href="javascript:void(0);" onclick="PopupClose();">
<img src="http://www.authorstream.com/images/icon_cross.gif" alt="Close" title="Close" /></a></span><br />
<div class="PopUpDvAllInner" style="width: 640px;">
<span class="headingwithLine"><span class="Orange_headingAll">Customize Embed </span><span><img
src="http://www.authorstream.com/images/bullet.gif" width="5" height="7" alt="" style="margin-top: 5px;" /></span><span id="spnCustomizeHeader" class="smallHeading">Improving MEDLINE Query Precision With Pseudo Rele</span></span>
<div class="CustomizeLeft">
<p class="bdrbottom" style="padding-top: 0px;">
<span class="lblleft">Color Theme</span> <span class="colorTheme"><span><a href="#"
class="colorGreen"></a></span><span><a href="#" class="colorGray"></a></span><span><a
href="#" class="colorOrange"></a></span><span><a href="#" class="colorBlue"></a></span>
<span><a href="#" class="colorBlack"></a></span></span>
</p>
<p class="bdrbottom">
<span class="lblleft">Selected embed Size:</span> <span class="lblright"><span>
<select style="width: 130px; margin-right: 10px;">
<option>481 x 480</option>
</select>
</span><span>H x W</span> </span>
</p>
<p class="bdrbottom">
<span class="lblleft">Navigation Settings:</span> <span class="lblright"><span class="radiobtn_CustomizeEmbed">
<input type="radio" checked="checked" name="NavigationSettings" /></span><span>Default</span><br />
<span class="radiobtn_CustomizeEmbed">
<input type="radio" name="NavigationSettings" /></span><span>On click</span><br />
<span class="radiobtn_CustomizeEmbed">
<input type="radio" name="NavigationSettings" /></span><span>Automatic</span><br />
</span>
</p>
<p class="bdrbottom" style="padding-bottom: 0px; border: 0px;">
<span class="lblleft">Show Border:</span> <span class="lblright" style="margin-bottom: 8px;">
<select>
<option>none</option>
<option>1</option>
<option>2</option>
<option>3</option>
<option>4</option>
<option>5</option>
</select>
</span>
<br />
<span class="radiobtn_CustomizeEmbed">
<input type="radio" checked="checked" name="Include" /></span><span>Include related
Presentation</span><span class="radiobtn_CustomizeEmbed"><input type="radio" style="margin-left: 5px;"
name="Include" /></span><span>Ddon't include related Presentation</span>
</p>
</div>
<div class="CustomizeRight">
<strong> Preview:</strong><br />
<img src="http://www.authorstream.com/images/player_preview.gif" alt="Player Preview" width="200"
height="171" /><br />
<a href="#">Click here to Preview in Original size</a></div>
<div class="yelloblock">
<strong>Copy Embed Code</strong> <em>(Code Changes as per options selected)</em><br />
<span>
<input type="text" style="width: 560px; padding: 5px;" class="txtBox" /></span><span><a
href="#" class="lnkbtnAll" style="margin-top: 5px;">Copy</a></span>
</div>
<p class="bdrbottom">
<span class="radiobtn_CustomizeEmbed">
<input type="checkbox" /></span><span>Save Settings for future embeds</span></p>
<a href="javascript:void(0);" onclick="PopupClose();" class="lnkbtnAll" style="padding: 0px;
margin-top: 15px;">Close</a>
</div>
</div>
<!-- Customize Embed Pop Up Div Ends -->
<input name="hdnStatus" type="hidden" id="hdnStatus" value="0" />
<input name="hdnTagsStatus" type="hidden" id="hdnTagsStatus" value="0" />
<input name="featuredIndex" type="hidden" id="featuredIndex" />
<input name="featuredValue" type="hidden" id="featuredValue" />
<input name="inappIndex" type="hidden" id="inappIndex" />
<input name="inappVal" type="hidden" id="inappVal" />
</div>
<script type="text/javascript">
<!--
Sys.Application.initialize();
Sys.Application.add_init(function() {
$create(Custom.UI.AutoCompleteBehavior, {"cssHoverItem":"autoCompleteHoverItem","cssItem":"autoCompleteItem","cssList":"autoCompleteList","id":"CustomExtender1","minimumPrefixLength":1,"separatorChar":", ","serviceMethod":"GetTags","serviceURL":"http://www.authorstream.com/Services/Test.asmx"}, null, null, $get("txtTags"));
});
// -->
</script>
</form>
</body>
</html>