chi course06 4 23

Faceted Metadata for Information Architecture and Search CHI Course - April 24, 2006  Session I : Faceted Metadata for Information Architecture and Search CHI Course - April 24, 2006 Session I Marti Hearst, School of Information, UC Berkeley Preston Smalley & Corey Chandler, eBay User Experience & Design


Session I: Agenda : Session I: Agenda Intro and Goals (5 min) Faceted Metadata (15 min) Definition Advantages Interface Design using Faceted Metadata (40 min) The Chess Analogy The Nobel Prize Example Results of Usability Studies Software Tools Design Issues (15 min) Q&A (15 min)


Focus: Search and Navigation of Large Collections : Focus: Search and Navigation of Large Collections Image Collections E-Government Sites Shopping Sites Digital Libraries


Problems with Site Search : Study by Vividence in 2001 on 69 Sites 70% eCommerce 31% Service 21% Content 2% Community Poorly organized search results Frustration and wasted time Poor information architecture Confusion Dead ends "back and forthing" Forced to search Problems with Site Search


What we want to Achieve : What we want to Achieve Integrate browsing and searching seamlessly Support exploration and learning Avoid dead-ends, “pogo’ing”, and “lostness”


Main Idea : Main Idea Use hierarchical faceted metadata Design the interface to: Allow flexible navigation Provide previews of next steps Organize results in a meaningful way Support both expanding and refining the search


The Problem With Categories : The Problem With Categories Most things can be classified in more than one way. Most organizational systems do not handle this well. Example: Animal Classification otter penguin robin salmon wolf cobra bat Skin Covering Locomotion Diet


The Problem with Hierarchy : Inflexible Force the user to start with a particular category What if I don’t know the animal’s diet, but the interface makes me start with that category? Wasteful Have to repeat combinations of categories Makes for extra clicking and extra coding Difficult to modify To add a new category type, must duplicate it everywhere or change things everywhere The Problem with Hierarchy


The Problem With Hierarchy : The Problem With Hierarchy start fur scales feathers swim fly run slither fur scales feathers fur scales feathers fish rodents insects fish rodents insects fish rodents insects fish rodents insects fish rodents insects fish rodents insects fish rodents insects fish rodents insects fish rodents insects salmon bat robin wolf …


The Idea of Facets : The Idea of Facets Facets are a way of labeling data A kind of Metadata (data about data) Can be thought of as properties of items Facets vs. Categories Items are placed INTO a category system Multiple facet labels are ASSIGNED TO items


The Idea of Facets : The Idea of Facets Create INDEPENDENT categories (facets) Each facet has labels (sometimes arranged in a hierarchy) Assign labels from the facets to every item Example: recipe collection Course Main Course Cooking Method Stir-fry Cuisine Thai Ingredient Bell Pepper Curry Chicken


The Idea of Facets : The Idea of Facets Break out all the important concepts into their own facets Sometimes the facets are hierarchical Assign labels to items from any level of the hierarchy Preparation Method Fry Saute Boil Bake Broil Freeze Desserts Cakes Cookies Dairy Ice Cream Sorbet Flan Fruits Cherries Berries Blueberries Strawberries Bananas Pineapple


Using Facets : Using Facets Now there are multiple ways to get to each item Preparation Method Fry Saute Boil Bake Broil Freeze Desserts Cakes Cookies Dairy Ice Cream Sherbet Flan Fruits Cherries Berries Blueberries Strawberries Bananas Pineapple Fruit > Pineapple Dessert > Cake Preparation > Bake Dessert > Dairy > Sherbet Fruit > Berries > Strawberries Preparation > Freeze


Using Facets : Using Facets The system only shows the labels that correspond to the current set of items Start with all items and all facets The user then selects a label within a facet This reduces the set of items (only those that have been assigned to the subcategory label are displayed) This also eliminates some subcategories from the view.


The Advantage of Facets : The Advantage of Facets Lets the user decide how to start, and how to explore and group.


The Advantage of Facets : The Advantage of Facets After refinement, categories that are not relevant to the current results disappear. Note that other diet choices have disappeared


The Advantage of Facets : The Advantage of Facets Seamlessly integrates keyword search with the organizational structure.


The Advantage of Facets : The Advantage of Facets Very easy to expand out (loosen constraints) Very easy to build up complex queries.


Advantages of Facets : Advantages of Facets Can’t end up with empty results sets (except with keyword search) Helps avoid feelings of being lost. Easier to explore the collection. Helps users infer what kinds of things are in the collection. Evokes a feeling of “browsing the shelves” Is preferred over standard search for collection browsing in usability studies. (Interface must be designed properly)


Advantages of Facets : Advantages of Facets Seamless to add new facets and subcategories Seamless to add new items. Helps with “categorization wars” Don’t have to agree exactly where to place something Interaction can be implemented using a standard relational database. May be easier for automatic categorization


Information previews : Information previews Use the metadata to show where to go next More flexible than canned hyperlinks Less complex than full search Help users see and return to previous steps Reduces mental work Recognition over recall Suggests alternatives More clicks are ok only if (J. Spool) The “scent” of the target does not weaken If users feel they are going towards, rather than away, from their target.


Facets vs. Hierarchy : Facets vs. Hierarchy Early Flamenco studies compared allowing multiple hierarchical facets vs. just one facet. Multiple facets was preferred and more successful.


Limitation of Facets : Limitation of Facets Do not naturally capture MAIN THEMES Facets do not show RELATIONS explicitly Aquamarine Red Orange Door Doorway Wall Which color associated with which object? Photo by J. Hearst, jhearst.typepad.com


Terminology Clarification : Terminology Clarification Facets vs. Attributes Facets are shown independently in the interface Attributes just associated with individual items E.g., ID number, Source, Affiliation However, can always convert an attribute to a facet Facets vs. Labels Labels are the names used within facets These are organized into subhierarchies Synonyms There should be alternate names for the category labels Currently (in Flamenco) this is done with subcategories E.g., Deer has subcategories “stag”, “fawn”, “doe”


The Chess Analogy : The Chess Analogy


Analogy: Chess : Analogy: Chess Chess is characterized by a few simple rules that disguise an infinitely complex game The three-part structure of play Openings: many strategies, entire books on this Endgame: well-defined, well-understood Middlegame: nebulous, hard to describe Our thought: information navigation has a similar structure, and the middlegame is critically underserved.


The Opening : The Opening Usually exposes top-level hierarchy or top-level facets Usually also has a search component


The Endgame – Penultimate Pages : The Endgame – Penultimate Pages


The Endgame – Content Pages : The Endgame – Content Pages


The Middlegame : The Middlegame The heart of the navigation experience There is a strategic advantage to having a good middlegame Standard Web search doesn’t handle this well This is where the flexible faceted metadata approach can work best.


Example:Nobel Prize Winners Collection(Before and After Facets) : Example: Nobel Prize Winners Collection (Before and After Facets)


Only One Way to View Laureates : Only One Way to View Laureates


First, Choose Prize Type : First, Choose Prize Type


Next, view the list! : Next, view the list! The user must first choose an Award type (literature), then browse through the laureates in chronological order. No choice is given to, say organize by year and then award, or by country, then decade, then award, etc.


Using Hierarchical Faceted Metadata : Using Hierarchical Faceted Metadata


Slide36 : Opening View Select literature from PRIZE facet


Slide37 : Group results by YEAR facet


Slide38 : Select 1920’s from YEAR facet


Slide39 : Current query is PRIZE > literature AND YEAR: 1920’s. Now remove PRIZE > literature


Slide40 : Now Group By YEAR > 1920’s


Slide41 : Hierarchy Traversal: Group By YEAR > 1920’s, and drill down to 1921


Slide42 : Select an individual item


Slide43 : Use Endgame to expand out


Slide44 : Use Endgame to expand out


Slide45 : Or use “More like this” to find similar items


Slide46 : Start a new search using keyword “California”


Slide47 : Note that category structure remains after the keyword search


Slide48 : The query is now a keyword ANDed with a facet subhierarchy


The Challenges : The Challenges Users generally do not adopt new search interfaces How to show a lot more information without overwhelming or confusing? Most users prefer simplicity unless complexity really makes a difference Small details matter Next we describe the design decisions that we have found lead to success.


Usability Study Results : Usability Study Results


Search Usability Design Goals : Search Usability Design Goals Strive for Consistency Provide Shortcuts Offer Informative Feedback Design for Closure Provide Simple Error Handling Permit Easy Reversal of Actions Support User Control Reduce Short-term Memory Load From Shneiderman, Byrd, & Croft, Clarifying Search, DLIB Magazine, Jan 1997. www.dlib.org


Usability Studies : Usability Studies Usability studies done on 3 collections: Recipes (epicurious): 13,000 items Architecture Images: 40,000 items Fine Arts Images: 35,000 items Conclusions: Users like and are successful with the dynamic faceted hierarchical metadata, especially for browsing tasks Very positive results, in contrast with studies on earlier iterations.


Most Recent Usability Study : Most Recent Usability Study Participants & Collection 32 Art History Students ~35,000 images from SF Fine Arts Museum Study Design Within-subjects Each participant sees both interfaces Balanced in terms of order and tasks Participants assess each interface after use Afterwards they compare them directly Data recorded in behavior logs, server logs, paper-surveys; one or two experienced testers at each trial. Used 9 point Likert scales. Session took about 1.5 hours; pay was $15/hour


The Baseline System : The Baseline System Floogle (takes the best of the existing keyword-based image search systems)


Post-Interface Assessments : Post-Interface Assessments All significant at p<.05 except “simple” and “overwhelming”


Post-Test Comparison : Post-Test Comparison Faceted Baseline Overall Assessment More useful for your tasks Easiest to use Most flexible More likely to result in dead ends Helped you learn more Overall preference Find images of roses Find all works from a given period Find pictures by 2 artists in same media Which Interface Preferable For:


Software Tools : Software Tools


Flamenco (flamenco.berkeley.edu) : Flamenco (flamenco.berkeley.edu) Demos, papers, talks are online Nobel example uses this toolkit Open source software is now available! Requires Apache and a DBMS (MySQL) You format your data in simple text files (We may add XFML support later) Our programs convert to appropriate DBMS tables Check it out: http://flamenco.berkeley.edu


FacetMap (facetmap.com) : FacetMap (facetmap.com)


Commercial Implementations : Commercial Implementations (Not an exhaustive list) endeca.com siderean.com www.dieselpoint.com www.rawsugar.com


Design Issues : Design Issues


Small Details Matter : Small Details Matter With text, it’s very difficult to avoid a cluttered look Must carefully design visual details White space Font style and weight contrast Color that distinguishes and doesn’t clash BEFORE AFTER


“Breadcrumb” Design : “Breadcrumb” Design Chains should only be used within hierarchy Need to separate the facets This allows both expanding within a facet and removing one facet while retaining the rest of the navigation. incorrect correct


Checkboxes vs. Hyperlinks : Checkboxes vs. Hyperlinks People LOVE checkboxes in principle However, they are dangerous because, when ANDED, they lead to empty results which people HATE They also often have confusing semantics Combine AND, OR, keyword search, etc. See Advanced Search at eat.epicurious.com


Checkboxes vs. Hyperlinks : Checkboxes vs. Hyperlinks (Advanced search from epicurious.com)


Handling Disjunction (ORs) : Handling Disjunction (ORs) The faceted queries are really a combination of ANDs and ORs The facet hierarchies actually do this Example: select Animal > Feline AND Location >Continent > North America This actually does a query as follows: AND( OR (panther, jaguar, lion), OR (US, Canada, Mexico) ) Nevertheless, sometimes you want to select just a subset of a facet’s labels


Handling Disjunction (ORs) : Handling Disjunction (ORs) Using checkboxes with ORs can work However, if allowed everywhere they clutter the screen eBay shows how to do it: Focus on one facet Select multiple labels Treat as an OR Won’t get empty results


How many facets? : How many facets? Many facets means more choice, but more scanning and more scrolling An alternative (by eBay) initially show the few most important facets allow user to choose a label from one then show an additional new facet (next most important) The right choice depends on the application Browsing art history vs. shopping


Revealing Hierarchy : Revealing Hierarchy One approach (Flamenco): keep all facets present, show deeper level as you descend.


Revealing Hierarchy : Revealing Hierarchy Another approach (eBay): show only one level at a time; if a facet is chosen that has subhierarchy, show the next level as an additional facet. Example: In Shoes, user selects Style > Athletic Now show a new facet that shows types of Athletic shoes Hiking, Running, Walking, etc.


Reversibility : Reversibility Make navigation urls consistent and persistent This way the Back button always works Allows for bookmarking of pages


Choosing Labels : Choosing Labels Labels must be short – to fit! Tricky with terminology: “endoplasmic reticulum” Labels must be evocative It’s very difficult to find successful words Depends on user familiarity with the domain Use card-sorting exercises Associate synonyms with labels Beware the context of label use! The “kosher salt” incident


Creating Facets : Creating Facets Need to balance depth and breadth Avoid long “skinny” hierarchies Example from the Art and Architecture Thesaurus: 7 clicks before you get to anything interesting


Summary : Summary Flexible application of hierarchical faceted metadata is a proven approach for navigating large information collections. Midway in complexity between simple hierarchies and deep knowledge representation. Currently in use on e-commerce sites; spreading to other domains We have presented design issues and principles.


Session II: Agenda : Session II: Agenda Highlights from Session 1 (5 min) Interactive exercise (20 min) Evolution of IA at eBay (10 min) Demo of latest eBay design (5 min) Lessons learned at eBay (35 min) Discussion and Q&A (15 min)


Discussion : Discussion


Faceted Metadata for Information Architecture and Search CHI Course - April 24, 2006  Session II : Faceted Metadata for Information Architecture and Search CHI Course - April 24, 2006 Session II Marti Hearst, School of Information, UC Berkeley Preston Smalley & Corey Chandler, eBay User Experience & Design


Session II: Agenda : Session II: Agenda Highlights from Session 1 (5 min) Interactive exercise (20 min) Evolution of IA at eBay (10 min) Demo of latest eBay design (5 min) Lessons learned at eBay (35 min) Discussion and Q&A (15 min)


Highlights from Session I : Highlights from Session I


Terminology Clarification : Terminology Clarification Facets vs. Attributes Facets are shown independently in the interface Attributes just associated with individual items E.g., ID number, Source, Affiliation However, can always convert an attribute to a facet Facets vs. Labels Labels are the names used within facets These are organized into subhierarchies Synonyms There should be alternate names for the category labels Currently (in Flamenco) this is done with subcategories E.g., Deer has subcategories “stag”, “fawn”, “doe”


Interactive Exercise : Interactive Exercise Introduce yourself to 4-6 people near you that you don’t already know Exchange business cards and note: Region they live (e.g. Canada, Western Europe) Role (e.g. Information Architect, User Researcher) Number of years of experience


Interactive Exercise : Interactive Exercise Organize the business cards using a hierarchy assuming you are a talent recruiter in Montreal


Interactive Exercise: One way… : Interactive Exercise: One way… start IA Mgr Programmer NA EU Asia IA Mgr Programmer IA Mgr Programmer 10+ 4-9 1-4 … … 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 10+ 4-9 1-4 Business Card #1 Business Card #2 Business Card #3 Business Card #4


Evolution of IA at eBay : Evolution of IA at eBay Flat Structure (2000 and earlier) Clothing, Shoes & Accessories Shoes Women’s Shoes - Boots - Pumps - Sandals


Evolution of IA at eBay : Evolution of IA at eBay Issues with approach: Products had to be categorized in just one way. Ex: Where are all the red Women’s shoes? Adding more descriptors meant creating a deep and complicated category structure. Ex: Shoes > Women’s > Boots > Black > Size 8 Flat Structure (2000 and earlier) Clothing, Shoes & Accessories Shoes Women’s Shoes - Boots - Pumps - Sandals


Evolution of IA at eBay : + Product Facets (2001 – 2005) Clothing, Shoes & Accessories Shoes Women’s Shoes - Style (Boots, Pumps, Sandals…) - Size (6, 6.5, 7, 7.5…) - Color (Black, Red, Tan…) - Condition (New, Used…) Evolution of IA at eBay Added Facets (flat)


Evolution of IA at eBay : Evolution of IA at eBay Issues with approach: Encourages over-constrained queries (Values “ANDED” together) Placing facets behind dropdowns reduces the exposure of the values to the user Left-Navigation Placement is only used a minority of the time by users While effective within a product domain their still is a need for facets above that level Ex: Everything Coach makes that is Red. + Product Facets (2001 – 2005) Clothing, Shoes & Accessories Shoes Women’s Shoes - Style (Boots, Pumps, Sandals…) - Size (6, 6.5, 7, 7.5…) - Color (Black, Red, Tan…) - Condition (New, Used…)


Evolution of IA at eBay : Evolution of IA at eBay Faceted Metadata (May 2005 Magellan Test) Clothing, Shoes & Accessories Shoes Women’s Shoes - Style (Boots, Pumps, Sandals…) - Size (6, 6.5, 7, 7.5…) - Color (Black, Red, Tan…) - Condition (New, Used…) - Brand (Nine West, Coach…) Brands Coach Louis Vuitton Materials Cotton Leather Added Hierarchical Facets Moved to a Top Positioned Link Structure


Slide91 : Matching items


Slide92 : Matching items


Slide93 : Matching items


Slide94 : Matching items


Slide95 : Matching items


Slide96 : Matching items


Slide97 : Matching items


Slide98 : Matching items


Slide99 : Matching items


Slide100 : Matching items


Slide101 : Matching items


Slide102 : Matching items


Slide103 : Matching items


Slide104 : Matching items


Slide105 : Matching items


Latest eBay design is now live! : Latest eBay design is now live! Try multi-faceted search yourself with the launch of eBay Express in Spring 2006. See http://express.ebay.com for details.


Methodology : Methodology Qualitative: Rapid Iterative Testing & Evaluation (RITE) Method (2 days testing, 1 day to iterate design) n = 48 users (over 9 months) 10 versions of the design 3 domains: Shoes, TVs, and Collective Glass Quantitative: A/B Test on the live site for 3 weeks [n = 73k searches in test environment compared to current site]


Lessons Learned at eBay : Lessons Learned at eBay Data Design Facets Flexibility of Facets vs. Hierarchy Dependencies Presentation Integrating “browse” and “search” Control Placement Facet Presentation Breadcrumbs


Facets : Facets Lesson: Users desire facets above the domain Users also want… Brands (Coach, Louis Vuitton) Materials (Leather, Cotton)


Flexibility of Facets vs. Hierarchy : Flexibility of Facets vs. Hierarchy Lesson: Users expect multiple entry points into a domain (tickets under sports) Tickets?


Dependencies : Dependencies Lesson: Users understand result of removing a parent facet (dependent facets also removed)


Lessons Learned at eBay : Lessons Learned at eBay Data Design Facets Flexibility of Facets vs. Hierarchy Dependencies Presentation Integrating “browse” and “search” Control Placement Facet Presentation Breadcrumbs


Integrating “browse” and “search” : Integrating “browse” and “search” Lesson: “Parsing” feels natural to users (and the text in the search box is not sacred) athletic shoes


Integrating “browse” and “search” : Integrating “browse” and “search” Lesson: People browse using the facets more when they are not familiar with the domain


Control Placement : Control Placement Lesson: Controls placed along the top of the page are used more than when on the left side


Facet Presentation : Facet Presentation Lesson: Users stop using refinements when a) not useful, and b) item count low enough


Facet Presentation : Facet Presentation Lesson: Prominently showing 4 facets is sufficient (but prioritization is important)


Facet Presentation : Facet Presentation Lesson: Shifting columns doesn’t disturb people


Facet Presentation : Facet Presentation Lesson: Truncated list of values per facet is okay (users know how to access the rest)


Facet Presentation : Facet Presentation Lesson: Showing sample values help users understand facets and can expose breadth


Facet Presentation : Facet Presentation Lesson: Users often want to select multiple facet labels and are pleased when they can (treated as an OR by search engine)


Breadcrumbs : Breadcrumbs Lesson: Traditional breadcrumbs don’t work here


Breadcrumbs : Breadcrumbs Lesson: Users understand the idea of applying and removing facets using this modified breadcrumb without instruction


Lessons Learned at eBay : Lessons Learned at eBay Data Design Facets Flexibility of Facets vs. Hierarchy Dependencies Presentation Integrating “browse” and “search” Control Placement Facet Presentation Breadcrumbs


Discussion and Q&A : Discussion and Q&A Your chance to make a comment on the subject or ask a question of the presenters. Marti Hearst School of Information UC Berkeley Preston Smalley & Corey Chandler User Experience & Design eBay Marketplaces


Acknowledgements : Acknowledgements Flamenco Team Brycen Chun, Ame Elliott, Jennifer English, Kevin Li, Rashmi Sinha, Emilia Stoica, Kirsten Swearingen, Ka-Ping Yee This work supported in part by NSF (IIS-9984741) eBay Product Team Corey Chandler, Sam Devins, Elaine Fung, Jean-Michel Leon, Michelle Millis, Louis Monier, Michael Morgan, Hill Nguyen, Kenny Pate, Melissa Quan, James Reffell, Suzanne Scott, Seema Shah, Preston Smalley, Anselm Baird-Smith, Luke Wroblewski