NewsMePresentation

NewsMe:: 

NewsMe: A Case Study for Adaptive News Systems with Open User Model Preliminary Examination Paper 2007 Chirayu Wongchokprasitti IS PhD Student School of Information Sciences

NewsMe: 

NewsMe

NewsMe Overview: 

NewsMe Overview Personalized News Access System Feed the news that response to the user’s interest 82 RSS news feeds, 21 sources 8 News Topics Ranking the news Open User Model based system

NewsMe Interface: 

NewsMe Interface 4 News Sections Recent News Recommended News My Profile News History

User Feedback Method: 

User Feedback Method Add a news item to Tracked News Add a news item to Blacklist

User Model Manipulation: 

User Model Manipulation Update rating of news in user model

User Model Manipulation (Con’t): 

User Model Manipulation (Con’t) List all history of viewed news Update rating of news in user model

Learning User Models for News Access: 

Learning User Models for News Access The system uses a machine learning approach to build a simple model of each user’s interests. A similarity-based method achieves the balance of learning and adapts quickly to change interests while avoiding brittleness.

Learning User Models for News Access (cont.): 

Learning User Models for News Access (cont.) The purpose of the user model First, it should contain information about recently read events, so that stories which belong to the same thread can be identified. To allow for identification of news that user already knows. The k-nearest-neighbor algorithm (kNN) is used to achieve the desired functionality. Convert news contents to tf-idf vectors (term-frequency/inverse-document-frequency). Use the cosine similarity measure to quantify the similarity of two vectors.

Learning User Models for News Access (cont.): 

Learning User Models for News Access (cont.) Decay Function Freshness of news content is our issue. Freshness should decay exponentially day by day. Freshness of news remains a half after fed 7 days. is the initial freshness of news content. is a decay instance, which its value is around 0.099.

Study Design: 

Study Design 20 Participants Assign to be Information Analysts 2 News Topic: US and Business 2 Sessions, 3 stages per session Retrieved News: Nov 28th – Dec 12th, 2006 Google Notebook extension (http://www.google.com/notebook)

Implicit VS Explicit Feedback: 

Implicit VS Explicit Feedback Implicit feedback Assuming every news user read is a tracked news Explicit feedback Users add news items to their user model Tracked news as Positive sample Blacklist News as Negative sample

Hypotheses: 

Hypotheses Performance hypotheses are: H1: The open model system with user profile manipulation by users performs better than the open model system without them, H1.1: The open model system with explicit feedback generates results with better performance, and, H1.2: Users with explicit feedback system demonstrate higher task performance.

Hypotheses (Con’t): 

Hypotheses (Con’t) User Perspective hypotheses are: H2: Users prefer the user profile manipulation features in the open model system, H2.1: Users appreciate better in the system with explicit feedback, and, H2.2: Users appreciate the ability to control their profiles.

Preliminary Results: 

Preliminary Results The Ground Truth System Performance Analysis User Performance Analysis User Feedback Analysis

The Ground Truth: 

The Ground Truth F-measure defines as follows: Summary of news items in the study

System Performance Analysis: 

System Performance Analysis

System Precision @ First Screen: 

System Precision @ First Screen

System Precision @ 60: 

System Precision @ 60

System Precision @ 100: 

System Precision @ 100

News Items Manipulation vs. System Performance (Stage 2): 

News Items Manipulation vs. System Performance (Stage 2)

Tracked News Blacklist (Stage 2): 

Tracked News Blacklist (Stage 2)

Tracked News History (Stage 2): 

Tracked News History (Stage 2)

Blacklist Tracked News (Stage 2): 

Blacklist Tracked News (Stage 2)

News Items Manipulation vs. System Performance (Stage 3): 

News Items Manipulation vs. System Performance (Stage 3)

Tracked News Blacklist (Stage 3): 

Tracked News Blacklist (Stage 3)

Tracked News History (Stage 3): 

Tracked News History (Stage 3)

Blacklist Tracked News (Stage 3): 

Blacklist Tracked News (Stage 3)

User Performance Analysis: 

User Performance Analysis

User Precision: 

User Precision

User Avg. Rank of Selected Items: 

User Avg. Rank of Selected Items

User Feedback Analysis: 

User Feedback Analysis A two-way ANOVA was performed on the questionnaire data to examine significant differences in user answers by system and by stage. On the question 3, subjects indicated they trusted in system’s ability to find useful information for the US topic versus the Business topic in overall (p-value = 0.017). On the question 7, subjects indicated My Profile helps them to understand how the system finds useful news items for the US topic versus the Business topic in overall (p-value = 0.013).

Slide33: 

Open Model with explicit feedback did not outperform the baseline. The experiment indicates that without caution, user model manipulation not only benefit the performance but lower the output. Binary rating might not be a suitable way. Fuzzy rating is a good way to study further. Future Work

Q & A: 

Q & A