Grey Literature in Research
Grey literature plays a vital role in research, providing valuable insights beyond traditional academic sources. Including grey literature in reviews helps reduce bias, offers diverse perspectives, and ensures a comprehensive analysis. This type of literature encompasses unpublished reports, working papers, official publications, and more, offering a rich source of information for systematic reviews and policymaking.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Chwilio llenyddiaeth lwyd Grey Literature Searching Cyfarfod Canolfan Casglu Tystiolaeth Bangor 7 Ebrill 2016 Bangor Evidence Synthesis Hub (BESH) meeting 7thApril 2016 Dr Beth Hall Gwasanaeth Llyfrgell ac Archifau, Library and Archives Service
Beth yw llenyddiaeth lwyd?/ What is grey literature? Dyma rai enghreifftiau: Adroddiadau; papurau gwaith; theses a thraethodau hir; newyddlenni; cyhoeddiadau swyddogol a llywodraethol; papurau cynadleddau; data heb ei gyhoeddi (o ohebiaeth bersonol ymchwilwyr eraill); data treial clinigol na arweiniodd at gyhoeddiad; protocolau; canllawiau; adroddiadau am y farchnad; papurau gwyn; posteri a chyflwyniadau cynadleddau; adnoddau ystadegol; safonau; patentau; manylebau technegol; trydar; data Facebook Here are some examples: Reports, working papers, theses and dissertations, newsletters, official and governmental publications, conference papers, unpublished data (from other researchers personal correspondence), clinical trial data that did not lead to a publication, protocols, guidelines, market reports, white papers, conference posters and presentations, statistical resources, standards, patents, technical specifications, tweets, Facebook data A yw hi'n fuddiol eu rhannu yn "ymchwil heb ei gyhoeddi" ac "ymchwil a gynhyrchir gan ymarferwyr" Is it useful to categorise into unpublished research and practitioner-generated research ? Haddaway, Neal R., and Helen R. Bayliss. "Shades of grey: two forms of grey literature important for reviews in conservation."Biological Conservation191 (2015): 827-829. Haddaway, Neal R., and Helen R. Bayliss. "Shades of grey: two forms of grey literature important for reviews in conservation."Biological Conservation191 (2015): 827-829.
Pam y dylech ei chynnwys yn eich adolygiad? / Why include it in your review? Reduce positive publication bias Reduce publication lag Wider variety of information essential for a systematic review Voices who don t publish in academic sources, such as patients, practitioners, students Information from industry Material which has not been peer- reviewed but is authoritative Can be longer and more detailed than journal articles May have local content not in mainstream publications Good source for data and statistics Particularly important in the area of policy Lleihau tuedd o ran cyhoeddi cadarnhaol Lleihau oedi cyhoeddi Amrywiaeth ehangach o wybodaeth -yn hollbwysig i adolygiad systematig Dal lleisiau nad ydynt yn cyhoeddi mewn ffynonellau academaidd, fel cleifion, ymarferwyr, myfyrwyr Gwybodaeth o ddiwydiant Deunydd sydd heb ei adolygu gan gymheiriaid ond mae'n awdurdodol Gall fod yn hirach a manylach nag erthyglau cyfnodolion Gall fod chynnwys lleol sydd ddim mewn cyhoeddiadau prif ffrwd Ffynhonnell dda i gael data ac ystadegau Yn arbennig o bwysig ym maes polisi
Arweiniad yn Llawlyfr Cochranear gyfer adolygiadau systematig o ymyriadau / Cochrane and grey literature searching Cochrane Handbook for Systematic Reviews of Interventions http://handbook.cochrane.org/ Conference abstracts and other grey literature have been shown to be sources of approximately 10% of the studies referenced in Cochrane reviews Publication bias clearly is a major threat to the validity of any type of review, but particularly of unsystematic, narrative reviews. Obtaining and including data from unpublished trials appears to be one obvious way of avoiding this problem. The inclusion of data from unpublished studies can itself introduce bias. The studies that can be located may be an unrepresentative sample of all unpublished studies. Unpublished studies may be of lower methodological quality than published studies
Arweiniad o'r Ganolfan Adolygu a Lledaenu / CRD and grey literature searching Systematic Reviews: CRD s guidance for undertaking reviews in health care https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf Internet searching can be a useful means of retrieving grey literature, such as unpublished papers, reports and conference abstracts. Identifying and scanning specific relevant websites will usually be more practical than using a general search engine such as Google . Internet searching should be carried out in as structured a way as possible and the procedure documented Research groups and other experts as well as manufacturers may be useful sources of research not identified by the electronic searches, and may also be able to supply information about unpublished or ongoing research. Contacting relevant research centres or specialist libraries is another way of identifying potential studies. While these methods can all be useful, they are also time consuming and offer no guarantee of obtaining relevant information After a thorough and systematic search has been conducted, and relevant studies have been identified, topic experts can be asked to check the list to identify any known missing studies
Arweiniad o'r Sefydliad Rhagoriaeth mewn Gofal Cymdeithasol / SCIE and grey literature searching SCIE systematic research reviews: guidelines http://www.scie.org.uk/publications/researchresources/rr01.pdf 8. Reviews should incorporate the perspectives of user and carer groups and people from black and minority ethnic communities, and include other diversity issues and perspectives as far as possible. Research literature that covers these views and topics may not be available. It is therefore expected that searches will include grey literature and user testimony to capture these perspectives. Where available evidence does not demonstrate this coverage, the review report should highlight these omissions as this is an important issue. 146. In addition to database searches as detailed below, search strategies should be designed to capture user testimony and grey literature (literature which has not been formally published) e.g. King s Fund and Joseph Rowntree Foundation literature. Grey literature can also be found on Social Care Online. 152. Where the review includes interventions that might be either classed as educational or offered in an educational setting (e.g. some interventions aimed at young people such as teenage parents), The British Education Index (BEI) and Educational Resources Information Centre (ERIC) may be useful additional databases. Depending on the topic of a review, there may be other relevant databases, including those offering grey literature. Examples include CommunityWise, NSPCC Inform, DrugData. Some independent research bodies may also be good sources of material, e.g. Alcohol Concern, Centre for Research in Ethnic Relations, Joseph Rowntree Foundation, etc. Use of databases should be strategic and topic-related: there may be costs associated with their use, and the time taken in searching, retrieving and screening is itself a cost.
Arweiniad gan Grp Cydweithio i gael Tystiolaeth Amgylcheddol / Collaboration for Environmental Evidence and grey literature searching Guidelines for Conducting CEE Evidence Reviews http://www.environmentalevidence.org/information-for-authors 4.1.2 Many organisations and professional networks make documents freely available through their web pages, and many more contain lists of projects, datasets and references. Often, reports referred to on a website will be provided if an organisation is contacted. The list of organisations to be searched is dependent upon both the subject of the SR and any regional focus. Stakeholders should be consulted at the planning stage and asked to suggest relevant organisations. Many websites have a search facility but their functionality is limited. A formal, repeatable search strategy should be employed but generalised guidance is difficult to provide. If feasible, hand searching of specific sources and visits to institutions (e.g. libraries and museums) may be useful in identifying further relevant studies or datasets. Keep records and fully report on the extent of your search. 4.1.3 The Internet can be a useful tool for identifying unpublished and ongoing studies. Careful consideration must be given to the design of the search in order to ensure that it is as focused and specific as possible; where this is not done, searching the web can be a time- consuming task, with relatively little useful data being returned.
Arweiniad gan y Gynghrair Tystiolaeth Ddefnyddiol / Guidance from the Alliance for Useful Evidence The Alliance for Useful Evidence champions the use of evidence in social policy and practice. Learning from Research: Systematic Reviews for informing policy decisions http://www.alliance4usefulevidence.org/assets/Alliance-FUE-reviews-booklet- 3.pdf Searches of the internet can be broader as they are not restricted by the limits of a particular database; however, they are likely to generate a lot of non-relevant material. Some search engines, such as Google Scholar, are restricted to academic work, but they could miss the large number of research papers that are not formally published (sometimes referred to as grey literature ). The other disadvantage of internet searches is that it is more difficult to record the search method in a way that others can reproduce. Despite this, internet searches can provide a useful double check of a more specific search strategy and identify important grey literature that cannot be found in established databases. A more focused approach is to search specific websites that are likely to include relevant studies. These might include the websites of specific research centres, research associations or government departments.
Arweiniad gan y Ganolfan EPPI / EPPI-Centre and grey literature searching Useful advise in the following book available in the Deiniol library: Gough, D., Oliver, S. and Thomas, J. eds., 2012. An introduction to systematic reviews. Sage. Particularly Chapter 6: Brunton, Ginny, Claire Stansfield, and James Thomas. "Finding relevant studies."
Ffynonellau: cyffredinol Sources: general COPAC WorldCat OpenGrey (SIGLE, EAGLE, INIST-CNRS, OpenSIGLE, GreyNet International) BASE - the Bielefeld Academic Search Engine Instiutional Repositories: OpenDOAR Mailing lists, social networks, industry contacts, personal contacts Conference Proceedings are included in ISI (via Web of Science) Theses databases (ETHOS, www.theses.com) Agricola agricultural, food, environmental sciences, nutrition Campbell Collaboration (Crime & Justice, Education, International Development, and Social Welfare) ELDIS (international development issues) ERIC (educational research) also includes grey literature
Ffynonellau: iechyd Sources: health NHS Evidence Individual Organisations including patient groups, special interest groups, charities, NICE, World Health Organisation . New York Academy of Medicine http://www.greylit.org/ King's Fund Library Several different clinical trial registries Social Care Online The Healthcare Management Information Consortium (HMIC) database (no subscription at Bangor) The National Technical Information Service (NTIS) (US Department of Commerce): (available via a subscription but the Public National Technical Reports Library (NTRL) is free if you register) PsycEXTRA, a companion database to PsycINFO (no subscription at Bangor) UK Clinical Research Network Study Portfolio National Guideline Clearinghouse (USA) Agency for Healthcare Research and Quality (AHRQ) (USA) Health Evidence (McMaster University) effectiveness of public health interventions http://www.healthevidence.org/ Bibliographic databases that include conference proceedings
Defnyddio Google / Using Google Gellwch ddefnyddio chwilio am ymadroddion gan ddefnyddio dyfynodau Os rhoddwch ddyfynodau o amgylch y gair mae'n diffodd cyfystyron a gwirio sillafu "site:" chwilio am wefan neu barth penodol filetype: chwilio am fath o ffeil arbennig intitle: chwilio yn y teitl yn unig Defnyddiwch gymorth chwiliad uwch Google http://www.powersearchingwithgoogle.com/c ourse/aps/skills Beth na allwch ei wneud yn Google: Nid yw blaendoriad/nod-chwiliad wedi ei gefnogi Mae gan Google uchafswm o 150 gair ar gyfer chwiliadau Dim ond y 1000 canlyniad cyntaf fydd Google yn eu dangos Nid yw chwiliadau cymhleth gyda nythaid o dermau e.e. (gwyddoniaeth NEU dechnoleg) A (cymru neu cymraeg) yn gweithio You can use Phrase searching using speech marks If you put speech marks around a word it turns off synonyms and spell checking Search site: searches a particular website or domain Search filetype: searches for a particular filetype Search intitle: searches only in the title Use the Google advanced search help http://www.powersearchingwithgoogle.com/cour se/aps/skills What you can t do in Google: Truncation/wildcard searching is not supported Google has a 150 word limit for searches Google shows only the first 1000 results Complex searches with nested terms e.g. (science OR technology) AND (wales or welsh) - do not work
Defnyddio Google Scholar / Using Google Scholar Haddaway NR, Collins AM, Coughlin D, Kirk S (2015) The Role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE 10(9).doi:10.1371/journal.pone.0138237 used alone for systematic review searches. Rather, it forms a powerful addition to other traditional search methods. In addition we advocate the use of tools to transparently document and catalogue GS search results to maintain high level of transparency and the ability to be updated, critical to systematic reviews. Google Scholar can find much grey literature and specific, known studies, it should not be Bramer WM, Giustini D, Kramer BM, Anderson P (2013). The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Systematic Reviews 2(115). http://doi.org/10.1186/2046- 4053-2-115 Although its coverage and precision are acceptable, GS, because of its incomplete recall, should not be used as a single source in SR searching. A specialized, curated medical database such as PubMed provides experienced searchers with tools and functionality that help improve recall, and numerous options in order to optimize precision.
Chwilio ar wefannau sefydliadau / Searching organisations websites Use the search box on the homepage, does it search the whole contents of the site? If available, navigate to the publications/documents/outputs section Find out when the document or webpage was last updated If there is a contact for an information manager at the organisation get in touch with them Defnyddiwch y blwch chwilio ar y dudalen hafan, a yw'n chwilio holl gynnwys y safle? Os yw ar gael, ewch i'r adran publications/documents/outputs Chwiliwch i weld pa bryd gafoddy ddogfen neu'r we-dudalen ei diweddaru diwethaf Os oes cyswllt ar gyfer rheolwr gwybodaeth y sefydliad, cysylltwch 'runigolyn hwnnw
Problemau gyda chwilio llenyddiaeth lwyd Problems with grey literature searching Anodd i gael hyd iddynt, anodd i lywio o amgylch gwefannau'r sefydliad, ddim yn gallu dangos protocol chwilio tryloyw Cael gafael ar y testun llawn (e.e. theses UDA) Heb eu hadolygu gan gymheiriaid - ansawdd amrywiol - angen arfarnu: awdurdod, cywirdeb, ymdriniaeth, gwrthrychedd, dyddiad, arwyddoc d Gall fod yn benodol yn ddaearyddol Wedi'i fwriadu'n aml i gynulleidfa fechan neu arbenigol Caniat d/cyfeirio i'w ail-ddefnyddio Hard to find, hard to navigate organisation s websites, can t demonstrate a transparent search protocol Getting hold of the full text (e.g. USA theses) Not peer reviewed variable quality need to appraise: Authority, Accuracy, Coverage, Objectivity, Date, Significance May be geographically specific Often intended for a small or specialised audience Referencing/permission for re-use
Cofnodi eich chwiliad llenyddiaeth lwyd: bod yn dryloyw Reporting your grey literature searching: being transparent Cochrane handbook: List grey literature sources, such as internal reports and conference proceedings. If journals are specifically handsearched for the review, this should be noted but handsearching done by the authors to help build the specialized register of the CRG should not be listed because this is covered in the standardized description of the register. List people (e.g. trialists or topic specialists) and organizations who were contacted. List any other sources used, which may include, for example, reference lists, the World Wide Web or personal collections of articles.
Mae'n cymryd gormod o amser! It s all too time consuming! Time and budget restraints require the review author to balance the thoroughness of the search with efficiency in use of time and funds and the best way of achieving this balance is to be aware of, and try to minimize, the biases such as publication bias and language bias that can result from restricting searches in different ways Cochrane Handbook 6.1.1.2 Minimizing bias Dylech flaenoriaethu eich ffynonellau, cyfyngu'r ymdriniaeth: pwy yw eich budd-ddeiliaid? Pa fath o lenyddiaeth y mae gennych ddiddordeb ynddi? Byddwch yn glir ynghylch eich meini prawf cymhwysedd Gofynnwch i'r arbenigwyr cyn i chi ddechrau Gofalwch fod gennych gynllun a byddwch yn drefnus e.g.: Prioritise your sources, narrow the scope: who are your stakeholders? What type of literature are you interested? Be clear on your eligibility criteria Ask the experts before you start Have a plan and be methodical e.g.: Godin, Katelyn, et al. "Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada." Systematic reviews 4.1 (2015): 1. Godin, Katelyn, et al. "Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada."Systematic reviews4.1 (2015): 1.
Offer/canllawiau i'ch helpu Tools/guidance to help Canadian Agency for Drugs and Technologies in Health (CADTH) Grey Matters: a practical tool for searching health-related grey literature Haddaway, Neal R. "The Use of Web-scraping Software in Searching for Grey Literature." Grey Journal (TGJ)11.3 (2015) Web scraping .has been developed in the private sector for business purposes, but it offers substantial benefits to those searching for grey literature. By building and sharing protocols that extract search results and other data from web pages, those looking for grey literature can drastically increase their transparency and resource efficiency. Defnyddiwch beiriannau chwilio wedi'u haddasu os ydynt ar gael / Use custom search engines if they exist e.g. https://library.carleton.ca/find/government-information Cipio cyfeiriadau: awgrymir eich bod yn defnyddio un o'r offer newydd fel Mendeley neu Zotero / Capturing references: suggest you try one of the new tools such as Mendeley or Zotero
Datblygiadau/at beth ydym ni'nanelu/beth allwn ni ei wneud? Advances/where are we heading/what can we do? Data agored/data mawr Data cyfryngau cymdeithasol "Crafu'r we" / gweithio gydag APIs Cymryd rhan mewn cael gafael ar lenyddiaeth lwyd: cyhoeddi, archifo a'i wneud yn hygyrch Deall y berthynas rhwng ymchwil a pholisi: pa fath o ddeunyddiau sy'n cael eu cynhyrchu, a'u defnyddio, gan bwy Cefnogi datblygiad technolegau newydd i adalw llenyddiaeth lwyd Open Data/Big data Social media data Web scraping / working with APIs Involvement in securing the grey literature: publishing, archiving, and making accessible Understanding the relationship between research and policy: what kinds of material are being produced, and being used, by whom Support the development of new technologies for retrieval of grey literature Lawrence, Amanda, et al. "Collecting the Evidence: Improving Access to Grey Literature and Data for Public Policy and Practice." Australian Academic & Research Libraries(2015): 1-20. Raamkumar, Aravind Sesagiri, Schubert Foo, and Natalie Pang. "More Than Just Black and White: A Case for Grey Literature References in Scientific Paper Information Retrieval Systems." Digital Libraries: Providing Quality Information. Springer International Publishing, 2015. 252-257. MacDonald, Bertrum H., et al. "How Information in Grey Literature Informs Policy and Decision-Making: A Perspective on the Need to Understand the Processes." Grey Journal (TGJ) 11.1 (2015).
