Business Information – sources and search techniques


Presentation Description

Workshop given to the Scottish Law Librarians Group in Edinburgh, 5th March 2015


By: bhattishb (23 month(s) ago)

we are offering CRM solutions, business IT solutions https://ww

Presentation Transcript

Business Information – sources and search techniques:

Business Information – sources and search techniques Scottish Law Librarians Group, Edinburgh, 5 th March 2015 Edinburgh Training and Conference Venue Karen Blakeman RBA Information Services [email protected] karenblakeman Presentation available for a short time at : / This work is licensed under a  Creative Commons Attribution-Share-Alike License .

Slide 2:

Everything is changing very quickly – not all of the slides are printed here. Download the full set from / [Also slides for Google and other advanced search workshops available at Some advanced search techniques, lots of sources Practical sessions - exercises provided but feel free to do your own stuff Ask questions as we go along – the slides are just a framework and we can explore topics in more depth “Top Tips” from you at the end of the day Share with colleagues 09/03/2015 2

What’s new:

What’s new EU “right to be forgotten” More government information and documents supposedly being moved to - increasing amount of information going missing More official data being made available as open data including Companies House possible quality and accessibility issues Google knows best! - major changes to how it analyses and runs your search - rewriting your search is standard - results increasingly device dependent 09/03/2015 3

EU - so called “right to be forgotten” ruling:

EU - so called “right to be forgotten” ruling 09/03/2015 4 Mario Costeja Gonzalez Edition of Monday, January 19, 1998, page 23 - Newspaper - EU Court of Justice ruled that Google is a “data controller” under Data Protection legislation and must remove, if requested, links to information that is “inadequate, irrelevant .... or excessive” from search results on a person’s name.

EU - so called “right to be forgotten” ruling:

Information is NOT removed from the web, only the link to information in the search results Not automatic – subject has to apply to have links that point to specific information about themselves removed from the results Request will be assessed to see if the information is “ inadequate, irrelevant or no longer relevant, or excessive in relation to the purposes for which they were processed.” Not just Google – all search engines with an EU presence Only applies to searches conducted in the EU + Norway, Switzerland, Iceland and Lichtenstein 09/03/2015 5 EU - so called “right to be forgotten” ruling

EU - so called “right to be forgotten” ruling:

Google adds removal statement from all results for searches on personal names even if nothing has been removed. (Does not apply to famous people or celebrities!) Use non-European Google to see all results e.g., 09/03/2015 6 EU - so called “right to be forgotten” ruling

Remember - Google knows best:

Remember - Google knows best Google very kindly.... Goes to great lengths to personalise your results according to your search history, contacts, location, device, phase of the moon, the train, bus or tram you take to work and anything else it can think of Rewrites your search for you by leaving out some of your terms and looking for weird and wonderful alternatives Doesn’t bother you with everything that might be relevant Changes its algorithms on a regular basis to keep you on your toes Constantly conducts experiments on you to ensure that you don’t feel forgotten 09/03/2015 7

Private browsing - quickest way “un-personalise”search :

Private browsing - quickest way “un-personalise”search Chrome - New Incognito window Ctrl+Shift+N FireFox Ctrl+Shift+P Internet Explorer Ctrl+Shift+P Opera Ctrl+Shift+N Will not remove country personalisation Not search engine specific, built into the browser 09/03/2015 8

Slide 9:

Google rewrites your search Google now tells you which terms it has ignored (some of the time) “..” around terms, phrases, names, titles of documents does not always work To force an exact match and inclusion of a term in a search prefix it with ‘ intext :’ intext:agricultural occupational asthma Use Verbatim for an exact match search 09/03/2015 9

Slide 10:

09/03/2015 10 Let’s throw it into the soup and see if the cat licks it up

Google – missing terms:

Google – missing terms 09/03/2015 11

Google Verbatim:

Google Verbatim 09/03/2015 12

Choosing your search terms:

Choosing your search terms 09/03/2015 13

Slide 14:

Google does not search everything it has in its database two indexes: main, default index and the supplemental index supplemental index may contain less popular, unusual, specialist material supplemental index comes into play when Google thinks your search has returned too few results Verbatim and some advanced search commands seems to trigger a search in the supplemental index 09/03/2015 14

Slide 15:

09/03/2015 15 “Normal search” 1,555,500 Search after Verbatim is applied 35,500,000

Slide 16:

Google changes its algorithms several hundred times a year How Google makes improvements to its search algorithm - YouTube 09/03/2015 16

We are all Google’s lab rats:

We are all Google’s lab rats Just Testing: Google Users May See Up To A Dozen Experiments Mostly minor effects on search but sometimes totally bizarre results Google decides that coots are really lions google -decides-that-coots-are-really-lions/ Update on coots vs. lions update-on-coots- vs -lions/ 09/03/2015 17


Hummingbird Not just an update but a completely new way of analysing your search No longer concentrating on keywords in isolation Tries to make “sense” of your search and put it into context, natural language queries, uses what others have searched and clicked on Announced 26 th September 2013 but had already been implemented for about a month Constantly changing – all bets are off when it comes to predicting what your results will look like 09/03/2015 18

Slide 19:

09/03/2015 19 Google wants to rank websites based on facts not links - 28 February 2015 - New Scientist

Slide 20:

09/03/2015 20

And then on another day...:

09/03/2015 21 And then on another day... : 09/03/2015 22

And one of many wrong Quick Answers submitted to me by delegates at a recent conference:

And one of many wrong Quick Answers submitted to me by delegates at a recent conference 09/03/2015 23 Many thanks to Philip Stirups for the example. About 24 hours after taking this screen shot Google corrected the error.

Waitrose Caversham opening times New Year’s Day:

Waitrose Caversham opening times New Year’s Day 09/03/2015 24 Google used the standard opening times in its answer, not the seasonal opening times

Slide 25:

Essential search commands especially when looking for government information and data sets 09/03/2015 25

Think file format:

Think file format PDF for research documents, government reports, industry papers ppt or pptx for presentations, tracking down an expert on a topic xls or xlsx for spreadsheets containing data Use the advanced search screen or the filetype : command UK waste vegetable oil energy generation regulations filetype:pdf UK waste vegetable oil energy generation regulations filetype:ppt UK waste vegetable oil energy generation regulations filetype:pptx UK waste vegetable oil energy generation filetype:xls UK waste vegetable oil energy generation filetype:xlsx 09/03/2015 26

Google site::

Google site: site: to search within a site or type of site waste vegetable oil energy generation Glasgow waste vegetable oil energy generation Glasgow waste vegetable oil energy generation Glasgow Can exclude sites using -site: agricultural occupational asthma UK – Does NOT search inside databases or protected areas Can combine with filetype: waste vegetable oil energy generation Glasgow filetype:pdf 09/03/2015 27

The best-laid plans of researchers often go awry:

The best-laid plans of researchers often go awry 09/03/2015 28 : 09/03/2015 29

Date/Any time:

Date/Any time Restrict your results to information that has been published within the last hour, day, week, month, year or your own date range Search tools , Any time and select an option  [does NOT work with Verbatim] 09/03/2015 30


daterange: Date/Any time menu option can not be used with Verbatim Use daterange : command instead Uses Julian date format (fractions omitted) Julian Date Converter, and information on Julian date format) is at http :// Convert your dates to the Julian format and then, omitting the fractions, copy and paste them into your search Syntax for example articles between September 1 st and November 1 st 2014 on the roll out of universal credit universal credit announcement daterange:2456901-2456962 Then apply Verbatim to the results page 09/03/2015 31

Slide 32:

09/03/2015 32

Slide 33:

09/03/2015 33 Run your search and then apply Verbatim

The Scottish Government :

The Scottish Government 09/03/2015 34

Welsh Government :

Welsh Government 09/03/2015 35 : 09/03/2015 36

The Northern Ireland Assembly :

The Northern Ireland Assembly 09/03/2015 37

Oh joy - NOT!:

Oh joy - NOT! 09/03/2015 38 More UK information vanishes into GOV.UK

Where’s the information gone to?:

Where’s the information gone to? List of departments, agencies and public bodies at “Home pages” on GOV.UK Data and information may still be on the old websites Data may have been moved to Information may have been sent to Or information may have been “lost” Don’t rely on just GOV.UK search – use Google/Bing site: command combined with filetype: if appropriate 09/03/2015 39

"Yes Minister" The Skeleton in the Cupboard (TV Episode 1982) - Quotes - IMDb :

"Yes Minister" The Skeleton in the Cupboard (TV Episode 1982) - Quotes - IMDb 09/03/2015 40 James Hacker : [ reads memo ] This file contains the complete set of papers, except for a number of secret documents, a few others which are part of still active files, some correspondence lost in the floods of 1967... James Hacker : Was 1967 a particularly bad winter? Sir Humphrey Appleby : No, a marvellous winter. We lost no end of embarrassing files. James Hacker : [ reads ] Some records which went astray in the move to London and others when the War Office was incorporated in the Ministry of Defence, and the normal withdrawal of papers whose publication could give grounds for an action for libel or breach of confidence or cause embarrassment to friendly governments. James Hacker : That's pretty comprehensive. How many does that normally leave for them to look at? James Hacker : How many does it actually leave? About a hundred?... Fifty?... Ten?... Five?... Four?... Three?... Two?... One?... *Zero?* Sir Humphrey Appleby : Yes, Minister. [Add “transfer to GOV.UK” to the list of excuses]

Slide 41:

09/03/2015 41 VAT Mini One Stop Shop – Service Assessment | Data at GDS An example of the rigorous nature of GOV.UK page development VAT MOSS pages are now live without any of the concerns and criticisms having been addressed

Slide 42:

09/03/2015 42 All is not well with some of the pages on GOV.UK

Slide 43:

09/03/2015 43 Land Registry open data Land Registry database moved to the Open Data web site : 09/03/2015 44 Land Registry data often goes missing. I know that 10 months ago the sold price for number 90 was listed as £ 185,000 and for 2012.

Slide 45:

09/03/2015 45 Data doesn’t show up via the Land Registry Open Data interface either.

Title document:

Title document 09/03/2015 46 Data IS present in the title document.

Missing data:

Missing data 09/03/2015 47 Error report filed with the Land Registry - still waiting for a response Why might a property/price paid not appear in the data? Seems not that uncommon according to discussion boards – usually data entry error (but the above example was in the open data sets until a few months ago ) Absence of price – gift of property or purchase of a share Impractical to calculate price e.g. bulk purchase of properties Commercial transactions Raw data files downloaded and searched and data for number 90 is missing

Why some data goes AWOL:

Why some data goes AWOL 09/03/2015 48 Variable Pitch Uses public electricity microgeneration data

Slide 49:

09/03/2015 49 Variable Pitch Virginia Station is the hydroelectric installation at Windsor Castle – no data!

FoI request generation data for Virginia Station:

FoI request generation data for Virginia Station 09/03/2015 50 https:// : Launched in 2010 Not all legislation has been updated Pending updates are flagged 09/03/2015 51

UK Parliament

UK Parliament 09/03/2015 52

Monitoring progress of legislation :

Monitoring progress of legislation 09/03/2015 53

They work for you :

They work for you 09/03/2015 54

WhatDoTheyKnow - Freedom of Information (FOI) requests :

WhatDoTheyKnow - Freedom of Information (FOI) requests 09/03/2015 55 : 09/03/2015 56

Wayback Machine :

Wayback Machine 09/03/2015 57

Company information :

Company information Ownership, directors, structure, share price, accounts, activities, news Availability depends on type of company sole trader partnership private limited liability e.g. Ltd, LLP companies with publicly traded shares e.g. PLC CIC (Community Interest Company), CIO (Charitable Incorporated Organisation – England and Wales) business size (large, medium, small) country state or region within the country 09/03/2015 58

UK Companies Act 2006:

UK Companies Act 2006 Some of the main changes memorandum of association now only needs to state that the founders of the company wish to form a limited company directors no longer need to apply to withhold their private address, for example if they believe they may be targeted by activists. Can now choose to supply a “service address” requirement to have a company secretary removed UK Companies House details of registration and filing requirements search for companies for free, some information free, accounts and other documents priced free Disqualified Directors search 09/03/2015 59


Exemptions For UK small and medium sized enterprises May prepare and file abbreviated accounts Must fulfil at least two of the following conditions to qualify 09/03/2015 60 Exemptions not available to public companies, banking, insurance or shipping companies Small Medium Turnover not exceeding £6.5m £25.9m Assets/balance sheet not exceeding £3.26m £12.9m Average number of employees 50 250

UK filing requirements:

UK filing requirements Small companies Medium companies Large companies Balance sheet Abbreviated balance sheet with notes Full balance sheet Full balance sheet signed by a Director Profit and loss Not required Abbreviated profit and loss account Full profit and loss account Turnover Not required Not required Required Director report Not required Required Required and signed by an officer of the company Notes to the accounts Not required Required Required 09/03/2015 61

Legal Forms & Company Extensions :

Legal Forms & Company Extensions A Guide to Legal Forms for Social Enterprise A Guide to Legal Forms for Business - Company Extensions and Security Identifiers [does not appear to be updated] Types of business entity Company Formation Survey Doing Business - Measuring Business Regulations - World Bank Group 09/03/2015 62

Office of the Regulator of Community Interest Companies :

Office of the Regulator of Community Interest Companies 09/03/2015 63

Mutual societies - Financial Conduct Authority :

Mutual societies - Financial Conduct Authority 09/03/2015 64

UK Companies Beneficial Ownership Register :

UK Companies Beneficial Ownership Register http :// 09/03/2015 65 Will be implemented in the Small Business, Enterprise and Employment bill http://

Charity Commission – does not include Scotland:

Charity Commission – does not include Scotland 09/03/2015 66

The Charity Commission for Northern Ireland :

The Charity Commission for Northern Ireland https :// / 09/03/2015 67

OSCR Scottish Charity Regulator :

OSCR Scottish Charity Regulator http :// / 09/03/2015 68 Scotland's charity regulator (OSCR) plans to publish charity accounts http :// /

CAF Charity Trends :

CAF Charity Trends http:// 09/03/2015 69 Many thanks to Andrew Oakes for the information and link to this site Only covers England and Wales

Slide 70:

UK Companies House Database still at http :// / documents £1 each 09/03/2015 70

Company documents:

Company documents Companies House charges £1 per document and there can be lots of them - decide which ones you really need The list of company documents may be all you need to see 09/03/2015 71

Slide 72:

09/03/2015 72 Free Companies House data to boost UK economy - Press releases - GOV.UK

Companies House free data:

Companies House free data Bulk data – all or nothing Large daily files available as zipped files No support provided – you’re on your own! 09/03/2015 73

Companies House free data:

Companies House free data 09/03/2015 74 Each file within the zipped file is a separate document. Note the uninformative file names!

Company Check:

Company Check 09/03/2015 75

Company Check:

Company Check 09/03/2015 76

Company Check Dashboard:

Company Check Dashboard 09/03/2015 77


DUEDIL 09/03/2015 78 Sign-up required (free of charge), some information priced


DUEDIL 09/03/2015 79

DUEDIL group chart:

DUEDIL group chart 09/03/2015 80 Now only available as part of a subscription

Official company information – other countries:

Official company information – other countries Lists of official company registers Official Company Registers Company registration around the world Companies House Overseas registries links US – listed companies SEC Edgar IDEA – Interactive Date Electronic Applications Canada - listed companies SEDAR 09/03/2015 81

Examples of problem areas for the researcher:

Examples of problem areas for the researcher Switzerland only banks, insurance companies and companies traded on the Swiss stock exchange are required to provide financial statements Cayman Islands “ The Registrar of Companies can only release the name and type of company, its date of registration, the address of the registered office and the company’s status. Disclosing any other information is prohibited unless requested by a law enforcement agency .” British Virgin Islands “ A company is not required to file its register of directors, register of members, register of charges or an annual return with the BVI Registrar of Corporate Affairs.” 09/03/2015 82

OpenCorporates :

OpenCorporates 09/03/2015 83

OpenCorporates :

OpenCorporates 09/03/2015 84

European Business Register:

European Business Register Gateway to official company information in 28 European countries UK distributer is GBRDirect – priced service 09/03/2015 85

7side :

7side Priced service UK companies and documents, directors information, insolvency reports Credit reports International registry and documents service 1 hour registry and documents service translation service available have agents "on the ground", can quickly tell you what is available in that country and what may not be available 09/03/2015 86 Several of the workshop delegates use this service and had very positive feedback on it.

ICIJ Offshore Leaks Database :

ICIJ Offshore Leaks Database 09/03/2015 87 ICIJ Releases Offshore Leaks Database Revealing Names Behind Secret Companies, Trusts | International Consortium of Investigative Journalists

VAT number validation and EU VAT rates:

VAT number validation and EU VAT rates 09/03/2015 88 VAT number validation VIES EU VAT rates - essential if a seller is subject to the new VAT MOSS rules

Google street view:

Google street view As well as for personal use, useful for assessing location of a business (but remember out of date images) 09/03/2015 89

Social media & professional networks:

Social media & professional networks 09/03/2015 90

Slide 91:

09/03/2015 91

Official statistics:

Official statistics OFFSTATS UK National Statistics Publication Hub / [now redirects to ] Office for National Statistics Official Statistics in Scotland Welsh Government Statistics & Research Welsh Assembly Government StatsWales Eurostat European Union Open Data Portal 09/03/2015 92 : 09/03/2015 93

Google Public Data Explorer:

Google Public Data Explorer One of Google's best kept secrets! Public data sets made available by Eurostat, World Bank, IMF, CSO Ireland, OECD, ITU, some national statistics offices (but not ONS), and many more. Source and date updated given. Charts and charting options can highlight oddities and missing data Look at the charts to see if there is a sudden change in the trends. 09/03/2015 94

Google Public Data Explorer Minimum Wage:

Google Public Data Explorer Minimum Wage 09/03/2015 95 Some countries are missing e.g. Germany

Eurostat - Minimum Wage:

Eurostat - Minimum Wage 09/03/2015 96

Slide 97:

09/03/2015 97

Datamarket :

Datamarket Open portal to datasets worldwide and market research Creates visualisations of the data 09/03/2015 98 No longer available. DataMarket has been acquired by Qlik

Statista :

Statista “The Statistics Portal for Market Data, Market Research and Market Studies” 60,000 topics from over 18,000 sources some information free, registration (free) required Chart of the day 09/03/2015 99


Zanran Zanran – great for data in tables, charts and graphs 09/03/2015 100

Slide 101:

09/03/2015 101 The Guardian

And finally..... Per capita consumption of mozzarella cheese (US) correlates with Civil engineering doctorates awarded (US) :

And finally..... Per capita consumption of mozzarella cheese (US) correlates with Civil engineering doctorates awarded (US) 09/03/2015 102

authorStream Live Help