Critical Analysis: Finding the Good Stuff

Originally published in the Learning and Teaching weekly newsletter (internal publication) at Saskatchewan Polytechnic. 

Conducting a literature search can leave you feeling overwhelmed by the large number of results. You enter your keywords into a library database to conduct a search, hoping to find relevant articles, but instead retrieve 1,000s of articles, news items, videos, images, etc. that you have to sift through to find the perfect resources. When this process involves research literature, you wonder how you will be able to determine what is a trusted, authoritative source of information? That is where critical appraisal or critical thinking comes in. Continue reading to learn more about how you can find the good stuff.

Your search string is the first step in finding materials effectively and efficiently.

•         What do you want to learn more about?
•         What keywords can you string together to improve relevancy?
•         What limitations can you add to focus your search?

Don’t be afraid to use the suggestions. For example, if you are looking for post-secondary content, and the database suggests “post-secondary education or post secondary education or college or university or universities” use it or adjust as appropriate. If the date of an item matters, limit your search to a particular date range. Brainstorm the topic. Better yet, contact the library for assistance. They have many tips and tricks that can help you become a super searcher.

Once you have completed your search string, quickly scan the materials, starting with the title. Does the title give any clues about its usefulness? If you are looking for a post-secondary article on education in Canada; titles that mention K12 education in Australia can immediately be skipped. Your next step is to review the keywords used by the author or supplied by the publisher. If the article looks promising, review the abstract, introduction and conclusion for alignment with your own argument, thesis statement or hypothesis. Revision of your search strategy may be necessary at several points within this process. The most important takeaway is that research is an iterative process. It may surprise you to know that library staff adjust their searches many times before they perfect their search string.

Once you have fully explored the search process, select the articles that you wish to thoroughly read and review. One option is to quickly read an article for content (keep or delete), and then follow up with further investigation. This is where critical appraisal comes in. Critical appraisal is the process of assessing and interpreting research evidence for relevance, validity, reliability, and applicability. Critical appraisal is reliant on a balanced assessment of both the research process, as well as the results presented in the paper. Here are a couple of examples:

•         is the research rigorous?
•         have the author(s) outlined all of the potential limitations of research?
•         do the results seem likely based on the research process?

Critical appraisal means that you are deciding which works to include after you have completed a thorough analysis. Remember, you are the best judge of what you need, and what you are looking for, so concentrate on tweaking your search string to narrow the focus of your results. Capture the key points and then synthesise that information into your work.

Critical analysis also applies more broadly to the concept of critical thinking. Grigg (n.d.) notes that critical thinking “is about problem-solving, problem-posing, developing sound arguments and simply, making good decisions.  In turn, [critical thinking] pedagogy is about developing teaching strategies that deepen the quality of our students’ capacities for sound reasoning across the curriculum” (para. 1). In other words, critical analysis of research literature can be applied to pedagogy. Think about how you can help students to explore topics of interest, seek evidence that supports their argument and will further develop their academic writing and presentation skills (Grigg, n.d.).

Want to learn more about critical analysis – check out the Search Skills research guide.

References and Recommended Further Reading

Grigg, L. (n.d.) Critical thinking pedagogy in the classroom: Dr. Lance Grigg [Blog post]. University of Lethbridge. https://www.ulethbridge.ca/education/story/4082#:~:text=In%20short%2C%20CT%20is%20about,Dr

Saint Mary’s University. (2018, May 7). Critical analysis guide. https://studio.smu.ca/ac-resources/critical-analysis-guide

Saskatchewan Polytechnic Library. (n.d.). Search skills [Library research guide]. https://saskpolytech.libguides.com/searchskills/select

Western University. (2012, Apr 12). How to read a scholarly article [Video]. YouTube. https://www.youtube.com/watch?v=3SmOq6gENPM&feature=youtu.be

 

Sharing is Caring!

I reviewed the YouTube video 'OER Discoverability and Sharing', presented by Erin Fields (CARL).  Erin began this discussion by stating that there is no 'one size fits all' solution as discoverability can be problematic.  There are many systems and many tools, with several aggregators and various repositories.

In order to make an item open to the largest possible audience, you must consider accessibility, inclusivity, file format, as well as the licence assigned to the item.  You can help to make items truly open by: describing visuals, transcribing videos, describing hyperlinks, making text readable, and finally structuring the page using headings and other style conventions.

Other design considerations include making sure that the individual can edit the file, and that they will not require special software (i.e.: Adobe) to work with the item.  Open file formats include: HTML, ODF, RTF, SVG, PNG and CSV.  PDF can be open, if you use that text format that allows for OCR. 

When sharing items, it is important to have a standardized metadata which will allow for discovery on repositories and aggregators, and in turn allows for the creator to collect impact metrics.  Include: source type, author, title, date, URL, subjects, abstract, duration/length, audience, and copyright or licence information.  If you wish to archive your item, WikiMedia is a good choice, for archiving and aggregating, OER Commons, Open Textbook Library is an aggregator of resources and MASON OER Finder is an aggregator of repositories.  So many choices and so many decisions that need to be made before you even begin creating your OER project.

Collecting metrics are useful for grant applications, promotion and merit (universities), and for your personal records.  You can review views/downloads/saves as well as ratings of your project - dependent on what tool you use.

As for marketing and promotion: share news through your institution and professional associations, along with your personal social media channels.  Consider looking for relevant hashtags, and promote on academic social media channels as well.

Further resources:
https://open.ubc.ca/access/toolkits-access/oer-accessibility-toolkit/ - OER accessibility toolkit
http://guides.library.ubc.ca/open-education/sharing - Sharing your repository

Resources

The third module in the Instructional Design course from Library Juice Academy covered the creation of learning outcomes and the use of supplemental materials.  

Learning materials fall into several categories:

• The traditional commercially published textbook model.  Many publishers are making their packages more lucrative by including learning objects, such as PPT, test banks, instructional videos, etc.  These additional items leave me wondering if they take teaching out of instruction and out of the institution.  The instruction materials covered by the publisher rely on use of the textbook from start to finish.  The instructor becoming dependent on what is available from the publisher and at the will of updates/edits/adaptations that are made by said company.
• Library resources is a viable alternative, removing many copyright restrictions.  Library resources can be embedded in class materials through a permalink.  When an instructor used a permalink to share content, the library can capture accurate usage statistics.  Also, providing references increases awareness of this type of information and how it can be used as search clues for retrieving content.
• OER is another option that is gaining traction in post-secondary institutions.  OER allows for more flexibility and versatility of instruction, as the instructor can pick and choose what works for them and their teaching activities. 

This module included some basic information about accessibility: 

• Visual - screen readers, considerations of font and colour variances 
• Hearing - save chat transcript, consideration synchronous vs. asynchronous courses and how that might effective the needs of your students.  Ask yourself, what arrangements need to be made in advance?  

A good source for further information on this topic is: https://webaim.org.  This is a very important topic and I encourage you to delve deeper into it.   

Online Instruction for Adult Learners

Diane Zerr and I presented our online module for the Adult Learning and Teaching program here at Saskatchewan Polytechnic at the Saskatchewan Health Libraries Association AGM.  It was an opportunity to showcase how important the learning cycle is, and the need to ensure that learning outcomes are clearly tied to assessments.  Program Overview

classroom size

I am especially proud of this paper, as I had set a goal for myself to submit it before my contract at the University of Saskatchewan finished.  I managed to get it in, with 2 hours left to spare!  It discusses my use of flipped teaching in three different engineering classrooms in a lecture theatre or a computer lab with varying number of students in each class.

So here it is, my third scholarly, peer-reviewed article:

Maddison, T.  (2015).  A Matter of size: Flipping library instruction in various engineering classrooms.  Issues in Science and Technology Librarianship, Number 82(Fall 2015).  doi: 10.5062/F4QV3JJ5.  http://istl.org/15-fall/refereed2.html

Know Thyself

I taught four sections of the same class spread out over 2 weeks.  When I have the opportunity to teach the same class more than once, I often notice a pattern of my energy levels and also the speed of the session.  The first session goes extremely well.  I am excited and interested in sharing the materials that I have prepared, although, I tend to be the most nervous with this first class.  The second and third class I work out the kinks and each session tends to be better than the one before it.  The last one is okay, usually I have good energy, but I usually end up finishing sooner than any of the other sessions. 

Note to self: When scheduling a peer evaluation, schedule it for one of the middle sessions and preferably not on a Friday when that class is the last class that students have that day!

what a difference a day makes

Last Friday, I had the (self-declared) worse class ever.  I had looked over the notes for the previous class from last fall and thought they looked a bit boring .  I decided that I would use the same format but integrate several active learning techniques throughout the instruction.  I was prepared to have some fun.

Wrong!  The class refused to participate and I mean refuse.  I would pause for 7 seconds.  I paused for 15 seconds.  I made eye contact with the students, cracked a couple of jokes about participating in library instruction and nothing.  I was done early.  I was flustered and I couldn't think of what else I should talk about as they were obviously not interested in learning about the library.

The next class that I taught was yesterday.  I was more prepared than Friday (for fear of repeating events) but I relied on the same format.  This class participated from the moment I walked into the class.  We were chatting before the professor arrived.  They answered my questions, contributed their own thoughts and we discussed the benefits of several different search methods.  I had fun!  Lots of fun!  This is the type of class that I excel at teaching.

So what did I learn?

1. I need to have a more detailed conversation with the professors every year.  I need to ensure that the students have been given their research assignment in advance of my class.  Together, we need to discuss what are the learning outcomes for the class and share feedback from the previous year's class.
2. I need to be a bit more assertive when asking for answers.  The Health Science folks pick people at random to answer questions.  I have never done this, but I need to start.
3. I will start preparing a back up plans for situations when the students are not participating
4. And finally, I think I will begin each class with the statement "I have two classes prepared.  One that will be delightful but requires you to participate.  You will learn a lot and will likely be very engaged in the materials that I have to share.  The other will be completely boring and will require nothing on your part."  Well, perhaps not that exactly - but something like that!

context

Yesterday I asked for some sample search terms from my class in Biological Engineering.  My plan was to use their term as a thesaurus search in Engineering Village.  One student suggested 'cherry stoner'.  We tried that with no luck.  I asked the student if there was another term that he thought we could use.  We then tried 'cherry pitter' instead which was equally unsuccessful. 

Try, try again -  I thought we could search more broadly first and then move to more specific results after so we tried pitter and then stoner.  Context is everything in a search.  I hadn't intended on looking up information for the term stoner, but was very relieved that we were searching in Engineering Village and not in Google. 

It became the perfect opportunity to talk to the students about the benefits of using a highly specialized database.

and then this happened

I had a lesson in 'thinking on your feet' yesterday.  I was asked to provide a library orientation session for new students in a graduate program.  The professor asked me to follow the outline that I used last year - which caused me to assume that I would again have an hour for more presentation. 

Looking at the notes, I thought my previous session looked a bit boring, so I thought I would try and integrate a couple of active learning techniques as well as an ice-breaker activity.  I worked away at the content and thought that I had a really neat lesson planned for the class.

I arrive at the class and the professor states that he has some comments to make first and then it will be my turn.  I should have at least 20 minutes.

Yikes!  I planned how I could adapt the session while he gave his presentation.

I made a quick joke about my abbreviated session and then delivered key pieces of information that I thought they would require.  I wasn't able to add any active learning techniques to the class, but there was a lot of thoughtful questions at the end.  The energy in the room (including mine) led me to believe the session was well received.

Medium Rare

Beef Up Your Backchat
This session looked at using Audience Response Systems to engage your audience while providing the instructor with the formative assessment piece.  The presenter started with a poll using www.govote.at that could be accessed through your Smartphone or tablet.  While some of us were doing that, she asked the rest of the audience for verbal feedback on her question.  I thought she handled this transition very nicely as she was able to continue the conversation appropriately for those who chose to participate online and those who didn't.

Definitions:
ARS - are clickers without the hardware, so text or Twitter and can be used on desktop computers or mobile devices
Backchat - refers to the side conversations or discussions that happen in a classroom
Formative - immediate and ongoing feedback

Free tools:
Poll Everywhere, limit of 40 users
Mentimeter, no limit of users, but can only provide multiple choice questions
Socrative, limit of 50 users
And here is a tool that are not free:
Top Hat, free for professors, students pay - but cheaper than a clicker

Suggestions:

• use what the professor is using wherever possible; students know how it works and you don't have to waste time providing instructions
• question design should be based on specific learning goals
• use ARS at 20 minute intervals
• hide answers until ready to reveal
• test the system
• address 'code of conduct' with students for open-ended responses

Challenges:

• technology and access
• anonymous feedback vs. social media
• consider having a print alternative for students who do not wish to participate publically

Up, up and away

The first session that I attended on Thursday was entitled 'Beyond the Database Demo', which focused on discourse analysis.  I again was intrigued by the clever title yet I did not fully grasp the concept.  Unfortunately I was a bit late for the session, so I think I missed some vital information, but the presentation in its entirety is available at http://ir.lib.uwo.ca/wilu/wilu2014/Presentations/15/.

The speaker identified 4 main problems that students' experience with the research process: lack of focus (I need 5 documents, but I don't care which 5 they are), lack of synthesis (how their research is part of the larger concept, topic), problems with citing or format and the result.

She then moved on to discuss how discourse analysis could be applied to research if we view scholarship as a conversation.  She gave two examples:

1. there are rules for all genres (writing an email to a friend is different than writing an email to a business associate) 
2. for English language learners, we give them writing examples for them to analyze and figure out language, phrasing, format, etc. so that they can learn to write in a similar way.

Application of concept in information literacy courses:

• In Nursing Ethics, used the Robert Latimer case and asked students to compare the two perspectives, looking at:
•  bias
• word choice
• language
• how the statement is construction
• information included, or left out
• what is the message the writer is giving
• how can we check / verify the information
• In a multiple session course:
• uses an article abstract to discover type (peer-review, popular, trade)
• moves onto a discussion of discourse, use of language, etc.
• concentrates on style of writing, i.e.: APA style manual
• academic integrity and plagiarism (https://www.softchalkcloud.com/lesson/serve/ebQzqKONlMJt12/html)

Follow Up - here are two articles that I want to read...

• Bizup, J. (2008). BEAM: A rhetorical vocabulary for teaching research-based writing. Rhetoric     Review, 27(1), 72-86.
• Ganski, K. http://ganski.wordpress.com/2013/03/28/flipping-freshman-composition-library-instruction/

Do Less, Accomplish More!

Today I had the opportunity to provide an information literacy session for participants in a summer school of sorts hosted by the College of Engineering.  As I was typing up my lecture notes, it occurred to me that I had a total of five activities and essentially my whole 60 minute segment one active learning method after another, yet, I probably accomplished more in that hour than I normally do in an one-shot instructional session. 

Here is a list of my activities:

• Be My Google – class brainstorming activity [activity courtesy of Amy Fyn of Bowling Green University]
• Question: What is human powered transportation? If I ask that question and you are Google, what types of results will I retrieve
• Begin to evaluate results, then compare results with Google and Google Scholar
• Look Up - class brainstorming activity [again courtesy of Amy Fyn who refers to this activity as Thumball]
• I randomly selected participants instead of throwing a ball around the room
• Question: Why don't people commute by human powered transportation methods?
• Discover - small group brainstorming activity
• Based on your project and group solution; come up with a list of keywords, sources of information, dates, topics and potential authors
• Pick a spokesperson from their group and share with the class the results from this activity.
• Further the activity by using a discovery system or database
• Keyword Challenge
• http://21cif.com/rkitp/challenge/Keyword_Challenge/KeywordChallenge.swf  
• In class, we did the Earthquake challenge together

Post-Assignment Assessment

If you are so lucky to have a library information literacy session that is directly related to an assignment, why not have students fill out a post-assignment assessment.  The purpose of this form of assessment is to note the affect of instruction on the student's ability to complete the assignment.

The assessment would ideally be handed out to students when they hand the assignment into their professor.

Think about how much you could learn from this form of assessment!!!

Hello, my name is....

Last week I taught 7 RCM 300 information literacy classes.  In the last session, a colleague of mine performed a peer evaluation of my instruction. 

One of the techniques that I use continuously in my courses is group work, of which there was no shortage in this particular class. 

Immediately following the class, my colleague offered me a brilliant suggestion stating that she uses  similar active learning techniques in her instruction.  She recommended that I start each class, that has a group work component in the instruction, with a brief meet-n-greet ice breaker.  Having students briefly introduce themselves to the people sitting next to them and offer one tidbit of previously unknown information benefits the class discussion in two ways: first, it avoids awkwardness at being forced to answer questions in a group situation and second, it sets up the expectation that students will be expected to contribute throughout the class.

I love this idea!  This ice breaker will be an excellent addition this activity will make to future instructional sessions.

"slow down, you move to fast..."

I learned a valuable lesson today: slow down!  When expecting students to follow along with a  hands-on activity make sure to give clear instructions, such as, what URL they should start with and then how to proceed from there.  And most importantly - wait for them to get there!

Also never underestimate the benefit of having a co-pilot at any session with a hands-on component.  It was very useful to me to have my colleague co-present our session on RefWorks today.  Not only were we given the opportunity to brainstorm the topic in advance, but also we had the chance to learn from one another in action.  We could also circulate the room and discover which students were on the right page and which students needed a bit of further assistance!

When does 7 seconds seem like an infinity?

Again, speaking with the organizational development specialist (ODS), I recounted a tale from my earlier instructional session where I became very aware of something I had learned in the Library Leadership Development Program.  According to the team player assessment tool, I am a communicator.  Those that know me, would surely agree!  One of the traits of being a communicator is that silences make me incredibly uncomfortable, and I will typically jump in with a response if the silence becomes too long in my mind.  In my instruction session I noted that several times I was to quick to jump in with an answer and rescue my students from the silence.  The ODS suggested that I count to 7 in my head to see if the question is answered by a participant. 

I plan to try that on Monday in my next session.... but I am sure it will seem like an infinite amount of time for me, the communicator!

Biology 301

Well this was an interesting project.  As I posted earlier, I wanted students to compare and contrast two articles in my instructional session, but I am not able to distribute copies of these articles to the class due to copyright restrictions.  Because of this I started a LibGuide for the class in which I could embed links to the articles that would only be available to authenticated users. 

Today, I wanted to finalize my LibGuide and prepare for tomorrow's instruction session, but as luck would have it, the LibGuide's site was down for most of the day.  Consequently I spent the day creating back up materials preparing for the worst (which is a good idea anyways, just not how I thought I would spend my day).

Thankfully tonight, it is back up and running so I was able to complete the LibGuide.  Here is it: http://libguides.usask.ca/biology301.  The LibGuide includes a link to my presentation that I shared through Prezi, as well as a link to an evaluation tool that I created through a tool provided in-house.

Now, let's just cross our fingers that it all works tomorrow!

Copyright restrictions require creativitiy

On August 27th, I attended a new faculty orientation at the University of Saskatchewan.  One of the speakers in the afternoon shared an active learning technique that I knew would be perfect for integration into a library instruction session.  The speaker gave one side of the classroom a syllabus created for an undergraduate class while the other side of the classroom was given a syllabus for a graduate class.  Once you had the syllabus in hand, students were instructed to pair up with someone on the opposite side of the classroom and discuss the layout, the content, etc. 

ah ha!  THE perfect activity for comparing and contrasting scholarly journals with popular magazine... Except....

With copyright restrictions, instructors are not allowed to distributed multiple copies of a journal article within a classroom without prior approval from the copyright holder; even if you have the students return the copies at the end of the class.  I know this because I was on the copyright/usage rights communication team.  So what can I do?

Well, I ended up embedding the links to two articles into a LibGuide for the course.  I will instruct students to click on the links and retrieve the article that way - thus compiling with copyright.

If there were no computers in the classroom... now that would be an exercise in creativity!

Improve Instruction Assessment - notes from a webinar

On January 11th, I attended an ALA workshop on improving your library instruction assessment. At first I thought that 90 minutes was much too long and I wondered how I would remain interested, but before I knew it the session was over and I wondered where the time went. The instructor was fabulous - interesting to listen too, insightful and knowledgeable! She also integrated active learning techniques into the session, which is truly impressive as there were more than 200 attendees in this online class.

The workshop discussed ways in which to incorporate simple classroom assessment tools into a one-shot library instructional session; such as, polls, quizzes or questionnaires and one-minute papers.

Begin with the end in mind or vice versa. Your learning objectives can dictate the type of assessment questions that you ask; or you can begin with exploring what you want your students to be able to demonstrate upon completion of the course, and then create your learning objectives from those evaluative statements. One thing is for certain; planning is key to any successful training session. You need to identify what you hope to achieve and then assess whether or not the students have been successful.

Another key point made by the instructor: if you are not going to evaluate your materials or your delivery; then do not bother to evaluate the student's learning... it will be a waste of every one's time. You must be prepared to review the results of the assessment and make changes. Teaching is a cyclical process.
- plan
- deliver
- evaluate
- improve
- plan
- deliver

Information literacy skills - notes for a conference presentation

Well, I am finally getting back to my notes from the WILU conference, so expect two blog entries today as I report on the last two sessions that I attended at the conference.

On June 3rd, I attended a session entitled "New Horizons: Information Literacy Skills of Incoming University Students", which was presented by an academic librarian and an M.L.I.S. candidate from the University of Alberta.

From their research findings, they identified several key areas where students struggled with information literacy skills upon entering an academic institution.
- search strategies
- Boolean operators
- understanding of academic journals and the publication process
- use of databases

The strengths they identified were:
- evaluation of the resources
- ethical use of information
- print resources

Implications of their findings:
- students entering university with underdeveloped literacy skills
- there is a gap between curricular mandates and learning outcomes
But
- there is significant potential to develop information literacy skills at university
- there is also an opportunity for academic librarians to provide these programs