Searching for sources

How to Evaluate and Select Literature

ana.lacerda@mail.utoronto.ca — Tue, 06 Feb 2024 19:12:11 +0000

How to Evaluate and Select Literature ana.lacerda@ma… Tue, 02/06/2024 - 14:12

Skills Categories:

Academic writing, Reading skills, Research skills

Skills Keywords:

Annotated bibliographies, Literature reviews, Reading critically, Searching for sources

This Resource Page will help you:

Understand the purpose of having inclusion and exclusion criteria for evaluating literature
Learn the main steps to form inclusion and exclusion criteria for source evaluation
Develop your own inclusion and exclusion criteria for selecting literature

Roadmap presenting the fours steps to prepare a literature review or an academic paper and indicating you are currently in step 2, part 3, evaluating and selecting literature.

Introduction

As you begin searching for sources for your assignment, you might notice that there are many available sources – sometimes too many. After doing an initial search, it is important to establish evaluation criteria to select only relevant and high-quality sources.

This page will provide you with practical strategies to develop inclusion and exclusion criteria and help you make informed decisions when selecting the sources for your assignments.

If you haven't yet started looking for sources or your search hasn't been satisfactory, please refer to our resource page on How to Find Sources for Assignments.

What are Inclusion and Exclusion Criteria?

Inclusion and exclusion criteria are parameters used to determine which sources are relevant for inclusion in a literature review or research project.

Inclusion criteria specify the characteristics that must be present in the studies for them to be considered, such as type of participants, interventions, outcomes, or study designs.

Exclusion criteria are the factors that disqualify studies from being included, like date of publication, language, or lack of peer review.

Why Do You Need Inclusion and Exclusion Criteria?

It is very important to have clear inclusion and exclusion criteria for your literature because they:

Ensure the quality of your selected literature
Reduce the number of sources you need to read during your source searching phase

Ensure that the literature is relevant to your research focus and appropriate in scope
Increase consistency in your search methodology

Using the RADAR Model and Other Criteria to Evaluate Literature

The RADAR model (Mandalios, 2013) is a framework used for evaluating sources. It provides a structured approach to help you assess important characteristics of a source, such as relevance to your research and accuracy of information. It's designed to help you make informed decisions on which sources to include or exclude. Below is what each component of the RADAR model stands for:

R: Relevance

A: Authority

D: Date

A: Accuracy

R: Rationale

In the following sub-sections, we will guide you to develop inclusion and exclusion criteria using the RADAR model and other elements to help you address these two crucial questions for source selection: 1) Can I trust this source? And 2) Is the source content relevant to my topic and scope?

Step 1: Can I Trust this Source?

The first step is to evaluate the reliability of your searched source. For this purpose, you can use the “Authority,” “Accuracy” and “Rationale” criteria from the RADAR model.

Authority

Assess the qualifications and expertise of the author or organization responsible for the source.
Evaluate the publishers to ensure the credibility of the source.

Guiding questions:

What type of work is this? (e.g., scholarly article, policy document, blog, news, etc.)
Where is this work published? (e.g., type of publication, credible publisher)
Is this work published in a peer-reviewed journal?
Who are the authors?
How is the author's expertise specifically relevant to the research topic? (Note: In certain fields or theoretical approaches, lived experience matters more than formal qualifications, and can be seen as a form of expertise.)
What other works does the publisher present?

Accuracy

Focus on the reliability and truthfulness of the information in the source.

Guiding questions:

Are the authors’ claims and arguments sufficiently supported by reliable evidence?
Is there any unverified and biased information included in the source?
How was the information gathered and presented?

Rationale

Determine the authors’ purposes and objectives.

Guiding questions:

What is the purpose of this publication? Why was it published?
To inform, teach, sell, entertain, or persuade?

Step 2: Is the Source Content Relevant to my Topic and Scope?

Once you have established that your source is reliable, you can determine the relevance of the source to your research topic and scope. For this purpose, you can use the “Relevance” and “Date” criteria from the RADAR model, along with other criteria listed below, to form your inclusion and exclusion criteria.

Relevance

Determine how well the source addresses your specific research needs.

Guiding questions:

Does the source have information relevant to my research topic?
To what extent does the source fit into the scope of my topic?

Date

Determine how current the source is for your topic.

Pay attention to the year when the source was published and set a timeframe (e.g., the most recent 10-20 years). IMPORTANT: Depending on your research topic, you might need to set different timeframes. For example:

If your topic is on a particularly current or recent issue (e.g., the use of Artificial Intelligence in the classroom; post-Covid issues in education), you may want to include more recent publications.
If your assignment is about a historical perspective on a topic (e.g., how the idea of critical pedagogy has evolved over time), you may want to include older works.

Guiding questions:

When was the source published?
In what way is this relevant for my topic?
Is the source up-to-date, or does it include outdated information?

Tip: Do not discard a source only because it’s from a few decades ago and regardless of your research topic. You may want to keep older sources if they have been very influential in the field. Many influential works in education and psychology aren’t recent, such as the works of renowned scholars Paulo Freire on critical pedagogy (1960s) or those of Jean Piaget on developmental psychology (1920s-1970s).

Language

Check what language the authors used for their work. For example, you may want to exclude sources written in languages for which it is difficult to arrange a reliable translation

Methodology

In some cases, you may want to include studies that employ a specific methodology (e.g., qualitative or quantitative studies only) or research designs (e.g., longitudinal studies).

Location & Setting

You may want to include only studies conducted in certain settings. This can refer to

geographical locations (e.g., North America)
specific contexts (e.g., universities).

Tip: When you find too many sources on your topic, you may need to narrow the scope of your topic (e.g., from schools in Canada to schools in Ontario). In the opposite case, you may need to expand the scope of your topic (e.g., from schools in Ontario to Canada).

Participants Profile

You may set criteria for participant demographics depending on the scope of your topic, such as

Education level (e.g., only participants from post-secondary institutions)
Gender
Age group
Cultural / geographical background
Other specific criteria (profession, family status, financial status, etc.)

Theoretical Framework

You may want to include works adopting a specific theoretical or conceptual framework. For instance, you may only include studies framed by sociocultural theories, a focus on gender, Indigenous perspectives, etc.

Next Step

For the next step after source evaluation, please refer to our resource page on How to Organize Literature.

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services

How to Find Sources for Assignments

ana.lacerda@mail.utoronto.ca — Fri, 17 Nov 2023 20:01:25 +0000

How to Find Sources for Assignments ana.lacerda@ma… Fri, 11/17/2023 - 15:01

Skills Categories:

Academic writing, Research skills

Skills Keywords:

Library navigation, Literature reviews, Searching for sources

This Resource Page will help you:

Understand why you should search for sources through the library system .
Search for appropriate sources that fit your assignment requirements.
Learn strategies to make your search more efficient.

Before You Start

Most of the content on this page has been summarized based on a Quercus self-paced learning module developed by the University of Toronto Libraries titled Library Research Skills.

Roadmap presenting the fours steps to prepare a literature review and indicating you are currently in step 2, part 2, finding sources for your assignment.

What are Scholarly and Peer-reviewed Sources?

You may receive assignment instructions that require you to find sources labelled as “scholarly”, or “academic”, or “peer-reviewed”. You might wonder what these terms mean:

Scholarly or Academic Sources

Scholarly or academic sources are materials that have been created by experts in a particular field of study. They are characterized by their depth of research, rigorous analysis, and adherence to high academic standards. These resources contribute to scholarly discussions and are often published in journals, books, or conference proceedings. Examples include research articles, academic monographs, and conference papers.

Peer-reviewed Sources

Peer-reviewed sources are scholarly materials that have undergone a thorough evaluation process by experts in the same field before publication. This evaluation ensures the quality, accuracy, and credibility of the content. Peer-reviewed resources are essential for academic research as they provide reliable information that has been validated by peers with expertise in the subject area. For a deeper look at the peer review process check Peer Review from the University of Toronto Libraries .

Source Searching Roadmap

This resource is designed to guide you in efficiently locating the appropriate sources for your assignments. The roadmap below gives you an overview of the steps involved in searching for sources. Each step is then explained in detail in the individual sections of this resource page.

Illustrated roadmap for research with three steps: 1. Select search tools, 2. Choose keywords, 3. Find sources. Unsuccessful searches loop back with advanced strategies; successful ones move to source evaluation. A tip reminds to track searches.

1. Select Suitable Search Tools

You might wonder why Google isn`t a suitable search tool. Here are some reasons:

Although Google can help you start searching for information, you should follow the assignment instructions if they require you to search for scholarly sources.
There is no guarantee that the sources you find on Google are credible, or that they were written by actual experts in your field. Anyone can publish anything outside of scholarly books and articles!
As graduate students, you should be using multiple search tools in conjunction with each other, including databases and U of T’s Library search. You’ll find more information about this below.

Most scholarly sources you will be requested to search are journal articles and scholarly books. You can learn more about other types of sources here: Exploring sources: Library Research Skills (utoronto.ca)

Types of search tools

Library Search: The LibrarySearch is the main search box on the University of Toronto Libraries websites. You will need to log in with your UTORiD to view most sources you find. Watch this U of T Libraries video for basic instructions on how to search the general library online system.

Subject-specific databases: Databases help you search for articles more efficiently. Most databases specialize in a specific subject and have specific features that let you find the best sources for your discipline faster.

What are the Differences Between Search Tools?

Watch the OISE Library’s video below (2mins 45sec) to learn about it.

What is a database?

Databases are storehouses of articles that have been collected together in one place, often because of their common subject focus; for example, an education database such as ERIC will only index (i.e., contain records) from journals in the field of education. You'll find your results are -- hopefully! -- more relevant to your research interests, so there's less need to sift through a scattered, drawn-out results list. You can search databases by keywords, article title, journal title, and/or author, and use various limiters (e.g., date, language, topics).

The database hierarchy is similar to the Netflix hierarchy, with journals = TV series, volume/issue = season, and article = episode.

Taken from What are Articles & Databases? - Research Help Guide - Research Guides at Toronto Metropolitan University Libraries (torontomu.ca)

Which Databases Contain Education and Psychology Scholarly Articles?

The OISE library has created a page with several education and psychology databases. Starting from the OISE Library main page, you can click on Education Databases and use the filters to choose the database that better fits your topic. Watch this short video to learn more about Education Databases.

What about relevant databases for my department/program?

The OISE Library has created library research modules for each OISE department. The modules are designed to support self-directed learning on the fundamentals of library searching as well as specific databases and resources customized to the different research areas in each of OISE's four departments. These modules are hosted in Quercus on a self-enroll basis.

Library Resources for Applied Psychology and Human Development(APHD)
Library Resources for Curriculum, Teaching and Learning (CTL)
Library Resources for Leadership, Higher and Adult Education(LHAE)
Library Resources for Social Justice Education (SJE)
Library Resources for Master of Teaching (Research 1/CTL7006)

2. Select Search Keywords

What are “keywords”?

Keywords are individual words or concise phrases representing the core concepts of a research topic or document. In academia, they help categorize and locate information. Keyword searching is easy but imprecise.

For example, if you want to write an assignment on second language learning for adult learners, relevant keywords could be:

Second Language Acquisition; Adult Learners; Language Learning Strategies; Communicative Approach; Language Proficiency; Cultural Competence; Pedagogical Techniques, etc.

How to select appropriate keywords

Brainstorm synonyms and related terms to enhance your search strategy. Check out the video ‘How do I Brainstorm Synonyms and Related Terms to Enhance my Search Strategy? from the OISE Library.
Use a planning worksheet to brainstorm keywords for core concepts and related terms.

3. Search for Sources

Once you have identified a few appropriate keywords for your topic, start using them in the database of your choice and check your search results.

Use Database Filters: After performing an initial search, use filters available in the database to refine your results. Filtering by peer-reviewed status, publication date, or subject area can help you sift through results more effectively and find sources that are most relevant to your academic needs.

Adjust Search Scope: If your initial search yields too many irrelevant results, consider narrowing your search to specific fields within the database, such as the article title, abstract, or author keywords, rather than a full-text search. Conversely, if you're finding too few results, expanding your search to include full text might uncover hidden gems.

Keep Track of Your Searches: You will likely perform multiple searches with different keywords. Saving and tracking your searches will therefore be useful. You can use this worksheet.

4. Apply Advanced Search Strategies

Research is an iterative process. If your initial searches don't yield the desired results, revisit your search terms and strategies. See below advanced search strategies to help you find the sources you need.

Search phrases in quotations

Phrase searching is a type of search command that can be used to narrow search results. When your keyword is made of two or more words, search the phrase in quotations, e.g., “elementary school” or “post-secondary education” to search for separate words as an exact phrase.

Use Boolean operators

Read this page for a detailed explanation of Boolean operators

Use truncated words

Truncation allows you to expand your results by searching for multiple endings of the same root word using an asterisk*.

Example:  Search canad* to find Canada, Canadian, or Canadians.

Watch this video (external page) to learn more about truncation.

Still not sure what all of these strategies look like in practice? The video tutorial below shows you how to use search strategies to find sources in Education Source, one of the main education databases.

Education Source

5. Next Steps

Once you have found some potentially useful sources, you will need to take a closer look and and evaluate whether they are relevant to your topic, credible and useful for your arguments and ideas. To learn more about critical reading, check our resource page on How to Read Academic Articles. To learn more about evaluating sources, read our resource page How to Evaluate and Select Literature.

As you read, evaluate and go through your selection process, you will also need to save and organize your sources (i.e., the documents, their citations, and the relevant content) to easily and quickly retrieve the information you need. To learn more about organizing your sources, read our resource page on how to organize literature.

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services

How to Select Assignment Topics

ana.lacerda@mail.utoronto.ca — Mon, 23 Oct 2023 22:17:53 +0000

How to Select Assignment Topics ana.lacerda@ma… Mon, 10/23/2023 - 18:17

Skills Categories:

Academic writing, Learning strategies, Research skills

Skills Keywords:

Annotated bibliographies, Library navigation, Literature reviews, Proposals, Searching for sources

This Resource Page will help you:

Understand the importance of selecting an appropriate assignment topic.
Learn effective strategies for selecting a topic.

Roadmap presenting the fours steps to prepare a literature review and indicating you are currently in step 2, part 1, selecting the assignment topic.

Introduction

Your course instructors might ask you to write a ‘midterm paper’ or ‘final paper’, and they will often require you to choose your own assignment topic.

For example, in a course about Second Language Teaching Methodologies, your instructor might provide the following guidelines:

“You can write a research-oriented position paper. From an educational researcher perspective, position yourself in relation to and critically discuss one of the fundamental methodologies, areas, concepts or issues presented in this course. Aim to support your position on this topic with scholarly research, examples, and any other resources you deem appropriate.”

Once you’ve understood the assignment instructions, the next thing to do is choose your paper topic.

What are the Features of an Appropriate Assignment Topic?

Selecting and developing your own topic helps you improve your thinking and research skills. As you think about topics, it is important to find a balance between what you want to explore and what your assignment instructions ask for.

A common challenge for students is finding topics that are focused. When your topic is too broad, the feedback you might receive from your instructor is “to narrow it down”.

Narrowing down a topic means defining

a specific focus of inquiry (e.g., a teaching strategy, a framework, a curriculum reform, an educational policy, a counselling approach)
a context (e.g., geographical context, education setting, target population)
a purpose (e.g., what is the impact of an approach, what are the causes of an issue, how was a policy decided, contrast/comparison of methodologies, solutions to a problem)

When you focus your topic (AKA “narrow it down”), you dig deeper into a specific part of it, which makes your research more detailed and doable. Focusing your topic is important to establish clear boundaries to what you will be reading and writing about.

If your topic is too broad: 

You may find too many sources and feel overwhelmed
You may struggle deciding what concepts to include or omit
The information you find will be too general
The result might be a general assignment that fails to respond to the specific assignment instructions.

Let's compare some broad and narrowed-down assignment topics to highlight the difference:

Strategies for Finding Your Topic

MUST DO!

Before you choose your assignment topic, review the assignment instructions.

Pay attention to any specific requirements or constraints outlined in the instructions, including the types and number of sources you are required to use. This can help you identify the boundaries and expectations for your topic. You can learn more about understanding assignment instructions here.

Draw from your Personal and Professional Experience

Our unique background can lead to fresh perspectives on existing topics or inspire entirely new research questions. Think about your personal or professional experience related to the theme of your course with an inquisitive eye. Ask yourself critical questions about these experiences in relation to what you are learning.

Here is some inspiration for you:

If you have teaching experience, think about how concepts you're learning in your course might apply to your teaching context.
Think about a problem / issue you have encountered in your personal /professional life. Can you seek information to better understand it or solve it?
Think about some behaviors / attitudes that you or people you know have. Can you find explanations for them through the concepts you’re learning?
Think about broader social issues or education-related issues in your culture. Can you find any relationships to your course contents?

Engage with the academic community

Talk about your assignment topic ideas with your peers and instructors in class.

Outside of class, unleash your curiosity and be proactive! Attend workshops, seminars, university events, and other academic activities. You might discover new areas of interest or research angles. You can also receive valuable guidance and suggest areas of research that align with our academic goals.

Keep a list

While taking your courses, keep a list of interesting topics and questions.

Inspiration can strike at any moment, so jot down your ideas whenever they come to you.
Where to find inspiration for ideas?

Review course outlines and topics and see what interests you
Take notes of interesting points made by your professors and colleagues in and outside of class
Think about your professional /personal experience and write down issues and questions that might align with course content

Brainstorming and Mind Mapping

Do Preliminary Literature Search

If you have a general idea about your topic and are in the process of refining it, you should do some preliminary research and find relevant literature that will help you narrow down and refine your topic.

If your topic is very broad (e.g., Constructivism and learning), start from general sources like

Specialized encyclopedias
State-of-the-art articles
Literature reviews
Meta-analysis articles

These are goldmines for identifying the main concepts, trends, research gaps and scholars around your topic of interest.

Look for recent literature, if possible, to learn about the most current knowledge and questions in your field of study.

Note that the literature you read and search can be relatively broader than the topic we will eventually choose. For example, our review might reveal that there is a growing body of literature on the “benefits of project-based learning in primary education”. This could inspire us to investigate the effectiveness of project-based learning specifically in science education.

For more information on how to search for appropriate sources check How to Find Sources for Assignments. To learn more about literature reviews, check What is a Literature Review and How to Prepare a Literature Review.

Our Tips

Find a topic that interests you! Graduate courses allow you to explore areas of study that are meaningful to you. Consider assignments as an opportunity to advance your own learning interests.

Start early! Finding the right assignment topic is often an iterative process and can take some time. This means that your first topic idea might not be your final topic as you go back and forth between your topic selection strategies, discussions with your instructor and literature search.

Be proactive! Discuss your ‘half-baked’ ideas and/or outline with your instructor, and ask for specific feedback and constructive advice. Show your instructor that you have done some preliminary work instead of asking them, “can you please help me find a topic?”

Next Steps

Once you finalize your assignment topic, it’s time to search for sources on the library databases using different combinations of keywords. Check our resource page on searching for sources.

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

Mindmaster (free mindmapping tool)

On Campus Services

Book a writing consultation (OSSC)

Library Research Skills

ana.lacerda@mail.utoronto.ca — Sat, 22 Apr 2023 20:25:20 +0000

Library Research Skills ana.lacerda@ma… Sat, 04/22/2023 - 16:25

Skills Categories:

Research skills

Skills Keywords:

Annotated bibliographies, Citations, Library navigation, Literature reviews, Searching for sources

This Resource Page will help you:

Familiarize with the Library Research Skills Quercus course offered by UofT Libraries.

The Library Research Skills Quercus-based course is divided into six modules.

The modules cover: an introduction to the University of Toronto Libraries; how to begin your research; how to find a research topic; how to search for and evaluate sources; how to cite sources properly; how to use library tools such as databases and catalogs effectively; how to get help from library staff; and how to access specialized research materials such as archives and rare books. Each module contains a mix of instructional texts, videos and short quizzes.

Our Tips

Although this resource is on Quercus, you do not need to login to access the page. Your participation is private, and you will not be graded on any of the activities offered through these learning modules. Just click the link below and start your research journey!

Searching for sources

How to Evaluate and Select Literature

Table of Contents

This Resource Page will help you:

Introduction

What are Inclusion and Exclusion Criteria?

Why Do You Need Inclusion and Exclusion Criteria?

Using the RADAR Model and Other Criteria to Evaluate Literature

Step 1: Can I Trust this Source?

Step 2: Is the Source Content Relevant to my Topic and Scope?

Next Step

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services

How to Find Sources for Assignments

Table of Contents

This Resource Page will help you:

Before You Start

What are Scholarly and Peer-reviewed Sources?

Source Searching Roadmap

1. Select Suitable Search Tools

Types of search tools

2. Select Search Keywords

What are “keywords”?

How to select appropriate keywords

3. Search for Sources

4. Apply Advanced Search Strategies

5. Next Steps

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services

How to Select Assignment Topics

Table of Contents

This Resource Page will help you:

Introduction

What are the Features of an Appropriate Assignment Topic?

Strategies for Finding Your Topic

MUST DO!

Our Tips

Next Steps

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services

Library Research Skills

This Resource Page will help you:

Our Tips

Help & Additional Resources

Related Resource Pages on ASH

Additional Resources

On Campus Services