1 Comprehensive literature searching in the Digital Age using the Pearl Harvesting Information Retrieval Theory Robert Sandieson Western University London, Ontario Canada
2 Outline Introductions Context of present day information retrievalTraditional search methods Pearl Harvesting Information Retrieval Theory Pearl Harvesting comprehensive search filters and Search Thesaurus Developing your own Pearl Harvested comprehensive search filter
3 Information ExplosionInformation is exploding in many research areas (Fraser & Dunstan, 2010; Pandita & Singh, 2015; Pautasso, 2012) For example, searching with the terms wellbeing OR “well being” in PsycINFO
4 One of the responses to information overload was the Cochrane Collaboration in medicine (Smith, 2010) and subsequently the Campbell Collaboration in the social sciences The systematic reviews and meta-analyses produced by these and other organizations are extremely helpful HOWEVER they depend on traditional ways of literature searching to gather their information
5 Traditional Search Methods
6 The current methods of searching were developed before or at the beginning of the digital era and are strained to deal with the current amount of information being produced.
7 Subject headings OR descriptors OR index terms- database indexers assign terms from database thesaurus Indexers are strained to keep up; MEDLINE had over 800,000 new articles indexed in 2016 Inconsistent coding by database indexers (Wilczynski & Haynes, 2009) Indexing is not cross referenced Subject Searching
8 Keywords are entered into a text search Deriving keywords is usually done intuitively Recent methods of being systematic are extremely cumbersome Inconsistent use of search terms across meta-analyses and systematic reviews (Valentine et al., 2010 ) Text Searching
9 Citation Tracking / Ancestral Searching or Forward TrackingA relevant article is located and its references are checked for further relevant articles A relevant articles is located and articles that have cited it are checked for relevancy The Silo Effect: Authors tend to cite papers within their own area e.g., a behaviourist would not cite cognitive psychology and vice versa Therefore, a wide variety of studies could be missed Citation Tracking / Ancestral Searching or Forward Tracking
10 Traditional search methods are strained to keep up with the current information explosionThey may provide a representation of the research which may or may not be accurate One study showed that with a more extensive literature search the conclusions of a meta-analysis was contradicted (Minor & Raju, 2004) New approaches to information retrieval are necessary to improve the capacity to comprehensively locate studies in this new information age
11 Pearl Harvesting Premise: Searching the literature is an act of communication
12 The Information Seeker
13 Authors
14 Database Indexers
15 Digital communication is facilitated by linguistic identifiers
16 Digital communication is facilitated by linguistic identifiersLinguistic identifiers provide the pathway to connect with the ideas authors talk about If we can understand the language, i.e., the linguistic identifiers, used by authors and database indexers Then we can digitally communicate with their ideas, i.e., locate their conversations i.e., locate their research studies These linguistic identifiers become the search terms in a comprehensive search Digital communication is facilitated by linguistic identifiers
17 Comprehensive Collections of Search TermsLinguistic Identifiers are words, phrases, or acronyms The identifiers of interest here are ones can locate unique citations that no other linguistic identifiers can find These, then, become the unique search terms A comprehensive collection of unique search terms comprises a Comprehensive Search Filter A Comprehensive Search Filter locates a comprehensive list of citations on a topic Comprehensive Collections of Search Terms
18 The comprehensive search begins once the unique search terms are assembled
19 Examples of Pearl Harvested Comprehensive Search Filters
20 Autism Comprehensive Search Filterautis* OR "pervasive developmental disorder*" OR "pervasive developmental delay*" OR "pervasive developmental disab*" OR Asperger* OR "ASD and ADHD" OR Rett* OR "Fragile X" OR "childhood schizophrenia" Autism Comprehensive Search Filter
21 Intellectual Disabilities Comprehensive Search Filter"mental* retard*” OR "mental* disab*" OR "mild disab*" OR "moderate* disab*" OR "severe* disab*" OR "profound* disab*" OR "multiple disab*" OR "intellectual disab*”OR “intellectually disab*” OR "developmental* disab*" OR "substantial disab*" OR "cognitive disab*" OR "mild handicap*" OR "moderate handicap*" OR "severe handicap*" OR "mental handicap*" OR "multiple handicap*" OR "profound handicap*" OR "developmental handicap*" OR "developmental delay*" OR "delayed development" OR "mentally delay*" OR "intellectually delay*" OR "mentally impair*" OR "intellectually impair*" OR "cognitive impair*" Intellectual Disabilities Comprehensive Search Filter
22 Fetal Alcohol Spectrum Disorder Comprehensive Search FilterFAS OR FASD OR "prenatal-alcohol exposure" OR "fetal alcohol syndrome" OR "fetal alcohol spectrum disorder*" OR "foetal alcohol syndrome" OR "fetal alcohol effect*" OR "alcohol-related disorder*" OR "alcohol-related neurodevelopmental disorder*" OR ARND OR "alcohol related birth defect*" OR "maternal alcohol abuse" OR "maternal and paternal alcohol abuse" OR ND-PAE OR "alcohol exposed child*" Fetal Alcohol Spectrum Disorder Comprehensive Search Filter
23 pharmacoepid. OR prescrib. OR prescript. OR "drug. therappharmacoepid* OR prescrib* OR prescript* OR "drug* therap*" OR therapeutic OR medication* OR pharmaceutic* OR pharmacy OR pharmacies OR oral OR intravenous OR nasogastric OR rectal OR milligram* OR microgram OR micrograms OR dose OR doses OR dosage* OR administer* OR chemotherap* OR adjuvant OR pill* OR tablet* OR lozenge* OR suppositor* OR injection* OR infusion OR "drug utilization" OR "adverse effect*" OR "Drug Prescriptions " OR "Pharmaceutical Preparations" Pharmacology Comprehensive Search Filter (adapted from PubMed search blog)
24 Meta-analysis/Systematic Reviews Comprehensive Search Filter (MEDLINE)"data synthesis" OR "evidence synthesis" OR metasynthesis OR meta-synthesis OR "narrative synthesis" OR "qualitative synthesis" OR "quantitative synthesis" OR "realist synthesis" OR "research synthesis" OR "synthesis of evidence" OR "thematic synthesis" OR metaanaly* OR meta-analy* OR "systematic map*" OR "systematic overview*" OR "systematic review*" OR "systematically review*" OR "bibliographic search" OR "database search" OR "electronic search" OR handsearch* OR "hand search*" OR "keyword search" OR "literature search" OR "search term*" OR "article reviews" OR "literature review" OR "overview* of reviews" OR "review literature" OR "reviewed the literature" OR "reviews studies" OR "this review" OR "scoping stud*" OR "overview study" OR "overview of the literature" OR meta-ethnograph* OR meta-epidemiological OR "data extraction" OR "meta-regression" Meta-analysis/Systematic Reviews Comprehensive Search Filter (MEDLINE)
25 Meta-analysis/Systematic Reviews Comprehensive Search Filter (PubMed)(data synthesis[all] OR evidence synthesis[all] OR metasynthesis[all] OR meta-synthesis[all] OR narrative synthesis[all] OR qualitative synthesis[all] OR quantitative synthesis[all] OR realist synthesis[all] OR research synthesis[all] OR “synthesis of evidence”[all] OR thematic synthesis[all] OR metaanaly*[all] OR meta-analy*[all] OR systematic map*[all] OR systematic overview*[all] OR systematic review*[all] OR systematically review*[all] OR bibliographic search[all] OR database search[all] OR electronic search[all] OR handsearch*[all] OR hand search*[all] OR keyword search[all] OR literature search[all] OR search term*[all] OR article reviews[all] OR literature review[all] OR “overview of reviews”[all] OR review literature[all] OR “reviewed the literature”[all] OR reviews studies[all] OR this review[all] OR scoping stud*[all] OR overview study[all] OR meta-ethnograph*[all] OR meta-epidemiological[all] OR data extraction[all] OR meta-regression[all] OR narrative review[all]) Meta-analysis/Systematic Reviews Comprehensive Search Filter (PubMed)
26 Too few search terms are used, potentially missing studiesSearch terms that aren’t necessary may result in volumes of information that need not be reviewed A common pattern across Meta-analyses and Systematic Reviews
27 TERMS: child developmental disorders pervasive, speech disorders, developmental disorder, PDD, language delay, communication disorder, speech disorder, childhood schizophrenia, Kanner*, asperg* PHCSF: autis* OR "pervasive developmental disorder*" OR "pervasive developmental delay*" OR "pervasive developmental disab*" OR Asperger* OR "ASD and ADHD" OR Rett* OR "Fragile X" OR "childhood schizophrenia Comparison of Pearl Harvested Comprehensive Search Filter to search terms of Systematic Reviews Sinha, Silove, Wheeler, and Williams (2006). Auditory integration training and other sound therapies for autism spectrum disorders: A systematic review.
28 SEARCH TERMS: systematic review. OR systematic literature reviewSEARCH TERMS: systematic review* OR systematic literature review* OR meta-analys* PHCSF: "data synthesis" OR "evidence synthesis" OR metasynthesis OR meta-synthesis OR "narrative synthesis" OR "qualitative synthesis" OR "quantitative synthesis" OR "realist synthesis" OR "research synthesis" OR "synthesis of evidence" OR "thematic synthesis" OR metaanaly* OR meta-analy* OR "systematic map*" OR "systematic overview*" OR "systematic review*" OR "systematically review*" OR "bibliographic search" OR "database search" OR "electronic search" OR handsearch* OR "hand search*" OR "keyword search" OR "literature search" OR "search term*" OR "article reviews" OR "literature review" OR "overview* of reviews" OR "review literature" OR "reviewed the literature" OR "reviews studies" OR "this review" OR "scoping stud*" OR "overview study" OR "overview of the literature" OR meta-ethnograph* OR meta-epidemiological OR "data extraction" OR "meta-regression" Comparison of Pearl Harvested Comprehensive Search Filter to search terms of Systematic Reviews Belbasis, Savvidou, Kanu, Evangelou, and Tzoulaki (2016). Birth weight in relation to health and disease in later life: an umbrella review of systematic reviews and meta-analyses.
29 Take the citations produced by a single search termSubtract the citations produced by all other potential search terms Are there any citations left which are relevant to the search intent? If so, that term is retained in the Synonym Ring SEARCH PROCEDURE: (Term A) NOT (ALL the other potential terms) Determining the necessity of a Search Term: The Boolean Subtraction Procedure
30 Place your screenshot hereUsing pre- developed Comprehensiv e Search Filters Google: The Pearl Harvesting Search Thesaurus
31 Population Comprehensive Search FiltersNeurodevelopmental Disorders Autism Learning disabilities (general, reading, mathematics, non-verbal), \ Intellectual and developmental disabilities Other exceptional/at risk Children Gifted Hearing Impairment Vision Impairment Fetal Alcohol Syndrome Students at risk for dropping out of school Developmental Ages of students/children Elementary school aged children Cultural Differences Native North American Digital Natives Educators Preservice teachers (i.e., teachers in training) Population Comprehensive Search Filters
32 Intervention Comprehensive Search FiltersApplied Behavior Analysis Inclusive Education Functional Mathematics for students with Intellectual Disabilities Online Learning Videogaming Information Overload Pharmacological Terms for Searching the research literature Second language learning/immersion programs Intervention Comprehensive Search Filters
33 Study Design Comprehensive Search FiltersQualitative Research terms Systematic Review/Meta-Analysis Study Design Comprehensive Search Filters
34 Pearl Harvesting methodology and Comprehensive Search Filters are used globally - as reported by: 30 organizations and universities, 21 graduate theses, 11 published systematic review protocols, 16 journal articles/books/conference presentations. The Pearl Harvesting Search Thesaurus wiki is viewed by approximately 15,000 people a year, from over 50 countries.
35 Using a pre-developed Comprehensive Search FilterGo to the Pearl Harvesting Search Thesaurus Choose and copy a Comprehensive Search Filter (also called a Synonym Ring) Paste it into a search field of a research database, such as PsycINFO or ERIC (note it won’t work in Google or Google Scholar) Search Review citations
36 How to develop a Pearl Harvested Comprehensive Search Filter
37 Find text sources where there is a conversation about the topic of interestFind relevant citations – the pearls Harvest potential linguistic identifiers and convert them to search terms Test the necessity of each of potential search term Test the sufficiency of the Comprehensive Search Filter How to develop a Pearl Harvested Comprehensiv e Search Filter (overview)
38 References in meta-analyses and Systematic ReviewsSearch for Systematic Reviews on the topic using the Pearl Harvested Comprehensive Search Filter (Synonym Ring) for Systematic Reviews 1. Find text sources where there is a conversation about the topic of interest
39 2 . Find relevant citations – the pearlsFind a wide range of articles on the same topic A final comprehensive search depends on gathering as many variants of terms used for the same topic as possible 2 . Find relevant citations – the pearls
40 Authors use a wide variety of terminology to talk about the same topicTerms vary over time, across theories, and cultures Perform a content analysis of the articles’ title, abstract, and indexing terms Determining which linguistic identifiers/terms are appropriate search terms can be tricky Going to a thesaurus may not help as the terms there may not be part of the conversation 3. Harvest potential search terms and render them for further processing
41 The appropriate level of language is importantToo general: “special education” is too general and not relevant when looking for articles on specific populations of children with disabilities Too specific: communication OR behaviour are too specific for finding information of autism per se i.e., these are symptomatic terms not diagnostic terms 3. Harvest potential search terms and render them for further processing
42 E.g., review,, systematic, analysis, synthesisTerms that are relevant but have numerous possible interpretations; therefore too broad E.g., review,, systematic, analysis, synthesis These are important but can be made more precise through Word Sense Disambiguation A single polysemic term can be made more precise by finding specific phrases using the same term E.g. disability => Intellectual disability OR severe disability OR developmental disability This omits citations such as physical disability TIP: AVOID using the NOT command to disambiguate Polysemic terms
43 Use double quotations for phrasesThis will increase the precision of the final search E.g., in ERIC “learning disab*” is not the same as learning disab* Use truncation function (*) to ensure all permutations of a word are included E.g., child* = child OR children Student* = student OR students 3. Harvest potential linguistic identifiers and convert them to search terms
44 4. Test the necessity of the Search TermsAre all the potential terms necessary for the Comprehensive Search Filter? Terms that are not necessary may produce many non-relevant citations Use the Boolean Subtraction Procedure 4. Test the necessity of the Search Terms
45 Boolean Subtraction ProcedureTake the citations produced by a single search term Subtract the citations produced by all other potential search terms SEARCH PROCEDURE: (Term A) NOT (ALL the other potential terms) Are there any citations left which are relevant to the search intent? If so, that term is retained in the Comprehensive Search Filter Boolean Subtraction Procedure
46 5. Test the sufficiency of the Comprehensive Search FilterCan the Comprehensive Search Filter locate as many citations on the topic as possible? Does the Comprehensive Search Filter contain a comprehensive list of unique search term or are any missing? 5. Test the sufficiency of the Comprehensive Search Filter
47 5. Test the sufficiency of the Comprehensive Search FilterCompare with other attempts to search comprehensively on the topic E.g., other meta-analyses, systematic reviews, other search strategies Does the Comprehensive Search Filter use more, necessary search terms than other search strategies? Does the Comprehensive Search Filter need less search terms than other search strategies (i.e., is it more precise)? Does the Comprehensive Search Filter find all the citations found by other strategies? 5. Test the sufficiency of the Comprehensive Search Filter
48 5. Test the sufficiency of the Comprehensive Search FilterIf other search terms are found, test them for their necessity using the Boolean Subtraction Procedure If they are found to locate topic relevant citations that the Comprehensive Search Filter misses then they are added to the Comprehensive Search Filter 5. Test the sufficiency of the Comprehensive Search Filter
49 Creating your own Comprehensive Search Filter
50 Choose a specific topic to develop a Comprehensive Search FilterLocate systematic reviews on the topic Analyze the title, abstract and index terms of references in the review looking for potential search terms for the topic Task for Today
51 Ask: is the term part of the conversation of the topic? is the term too general (not relevant) is the term too specific is the term too general (relevant) and needs to be disambiguated?
52 Creating your own Comprehensive Search FilterRender the potential search terms using double quotations and * Test for necessity using the Boolean Subtraction Procedure Test for sufficiency by comparing with other systematic reviews; see if PH Comprehensive Search Filter locates all its articles If new term found test it with the Boolean Subtraction Procedure Creating your own Comprehensive Search Filter
53 Thank you Robert Sandieson [email protected]If there are groups of people working on the same task feel free to work in groups Take your time and go through carefully Feel free to consult me Thank you Robert Sandieson