Pages

Tuesday, December 31, 2013

Canada

Canada By the end, I was enjoying this book.

Canada by Richard Ford came to me highly recommended. It took me a while to get around to reading it, but during that time the anticipation grew, and I was looking forward to this as a Christmas treat. Which is partly why this book was such as disappointment. By the end I was starting to enjoy it, but the flaccid first half ruined it for me. There was a good book in here if a decent editor had got hold of it before publication.

Disappointing. 6/10.


Tuesday, December 24, 2013

A Ghost Story for Christmas

Ghost stories and Christmas go together like Salmonella and mayonnaise. So I've written my contribution to this long tradition over at MicrobiologyBytes:

By 3 o'clock the light had almost gone from the room and the candles were already lit. Antonie van Leeuwenhoek had been working all d o'clock the light had almost gone from the room and the candles were already lit. Antonie van Leeuwenhoek had been working all day in his draper's shop, pausing only briefly to return to his house for a short lunch with his wife Barbara. His eyes were tired and the threads on the linens became finer year by year as the weaving machinery improved. But without a close examination of each sample of cloth it was impossible to know what was a fair price to pay and how much his customers would be willing to part with for it. He rubbed the square of linen between his finger and thumb, trying to gauge the evenness of each thread, to predict how hardwearing this sample would be. In fatigue and frustration he cast the cloth aside onto his work bench.

more



Wednesday, December 18, 2013

Mastery learning - my struggle

Mastery learning Having had our shiny new biological sciences curriculum approved for 2014, one of the main things occupying me at the moment is planning the new module I am convenor of - BS2000: Research Skills.

Our curriculum redesign had several guiding principles. Reducing the overall number of modules and student assessment load were two of them. Partially as a consequence of that, previous key skills modules have been absorbed into subject modules. The Research Skills module is the exception to that principle as it is designed to equip students for the challenge of their final year research project. As I know from long experience, getting students to engage with anything that smacks of "skills" rather than biology is difficult, and I know exactly what approach I would like to take to overcome that problem: mastery learning. Consequently, I was very interested in a new paper, which describes almost exactly the approach I would like to take with BS2000:

Lesley J. Morrell: (2013) Use of Feed-forward Mechanisms in a Novel Research-led Module. Bioscience Education. DOI: 10.11120/beej.2013.00020
Abstract: I describe a novel research-led module that combines reduced academic marking loads with increased feedback to students, and allows students to reflect on and improve attainment prior to summative assessment. The module is based around eight seminar-style presentations (one per week), on which the students write 500-word ‘news & views’ style articles (short pieces highlighting new results to a scientific audience). Students receive individual written feedback (annotated electronically on the work), plus an indicative mark, on their first submitted report. For subsequent reports, only a subset is marked each week, such that each student receives feedback on two further submissions. Simultaneously, they have access to written feedback on their peers’ reports (a total of two reports per student enrolled on the module). Students are encouraged to read and apply the general and specific messages from all the feedback to their own subsequent work (using it as feed-forward). At the end of the module, students self-assess their eight submissions and select the two they believe are their best pieces to put forward for summative assessment. Combining data from three cohorts, student attainment increased throughout the module, with higher marks for the two chosen reports than for the two marked reports or their first report. Students selecting previously unmarked reports also showed a greater increase in their mark for the module than students selecting reports that had previously received a mark. Module evaluation forms revealed that the students found access to feedback on others’ work helpful in shaping their own assignments.


But there's a problem. BS2000 will have over 350 students on it, not 32 as described in this paper. Much as we would like to, the team of five co-convenors would not be able to cope with the workload of the formative assessments in this model. There are plenty of American models for mastery learning with large student cohorts (such as: The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. (1984) Educational Researcher, 13(6), 4-161), but they mostly fall back on online testing and MCQs, which are not suitable for the skills we need to develop in this module (critical appraisal, ethics, information literacy). So I'm stuck, and how to square this circle is the problem the module team and our advisers have got to figure out over the next few months. Right now our proposed solution is group working, effectively dividing 350 by five, but it's not clear to me if that's the best way to get students who are still a year away from a final year project to engage with skills development.





1 If you're looking for someone to blame for Udacity, there's a direct lineage from this paper to the present day.


Thursday, December 12, 2013

Employability modules

University of Leeds employ ability module specifications:


BIOL1223 Career and Professional Development for Life Scientists
Module summary
This module will help students with their next career steps by introducing them to the careers available to bioscience graduates, the skills employers are looking for and how to achieve them. It will provide students with a competitive edge in the employment market by helping them to develop the necessary skills to seek and compete for work experience, internships and graduate employment. Comments from students who completed the module include…. "The opportunity to reflect on personal qualities and employability is invaluable. It has helped me realise what I need to do" "It was really useful. I definitely feel more prepared for job applications" "I would recommend it very highly. It’s made me think early what I want to do". The module will provide advice on when and how to start looking for work or work experience plus making applications and succeeding at interviews for work from a basic level. The module will take students on a journey, starting with the need for career and life planning, to exploring what an ideal graduate is and exactly what employers expect of today’s graduates, through to the career choices, routes and opportunities currently open to graduates and bioscientists in particular. During the module students will learn about and develop the range of skills (teamworking, networking, seeking opportunities, presentation skills) needed to successfully secure and succeed in the world of work. They will also learn how to evidence these skills. Students will learn about the importance of building up personal and employability skills in addition to bioscience-related qualifications and knowledge. Students will also learn how to applications and be interviewed for their job/career of choice to help them towards successfully progressing into the world of work. Students will learn about securing work/work experience via networking and ‘the hidden jobs market’ too. Whatever their future work experience or career plans this module aims to help students move forward into their first work experience and/or graduate job/role.
Objectives
The objective of the module is to help undergraduate students to reflect on their future career/life plans and to prepare for work and/or work experience, plus to learn and develop the key skills that are expected by graduate employers in today’s world of work – be that work in academic or other research and development laboratories, in a business environment or in the field.
Learning outcomes
On completion of this module students should be able to:
  1. Devise and maintain a career/life plan
  2. Research their own career interests, values and motivations
  3. Understand what is meant by the term an ideal graduate and know who employs bioscientists after graduation
  4. Describe the key skills that employers look for in new graduate recruits
  5. Identify their own current employability skills strengths and weaknesses
  6. Identify opportunities to develop their employability skills during their time at Leeds
  7. Identify common team roles and how they themselves fit best within a team
  8. Be able to establish effective working relationships within a team
  9. Write and maintain a curriculum vitae and personal statement
  10. Complete a job application form (on paper or online)
  11. Prepare for a job interview and/or assessment centre
  12. Present themselves positively in a job interview and/or assessment centre
  13. Learn how to network effectively in person and online
  14. Feel more confident about being able to secure future work opportunities (including work shadowing, short-work internships and/or longer work placements)
  15. Manage in a professional manner relationships with others in a work environment
Skills outcomes
Students will:
  • Learn how to plan their next career/life steps and to reflect and build on their plans.
  • Recognise the value of self-reflection in career/life planning (e.g. students will be able to reflect on their own interests, values and motivations) and learn to feed-their reflections forward to influence their own career/life plans.
  • Learn about the employability skills and professionalism that are required in today’s workplace.
  • During their own time, learn how to self-assess and rank their own employability skills strengths and weaknesses.
  • During their own time, identify and engage with curricular and extra-curricular activities that will help them to develop the bioscience and employability skills that today’s employers require.
  • Learn about teams and team-working and identify how they might fit and work best within a team/project group.
  • Learn how to positively present themselves to others in writing (e.g. via a CV, personal statement or online job application) and in person, via networking, a more formal interview/assessment centre or presentation
  • Learn about the hidden jobs market and how to network and positively communicate their skills and career plans to others, in person and in writing and practice working effectively in a team.
  • Learn to work alone to gather information related to their future career plans from a range of literature and online sources.
Syllabus
  • Lecture/workshop (1 hour): Career/life planning: what are your next steps - what is an ideal graduate, what careers are open to graduates and bioscientists in particular, what factors can influence your career/life plans e.g.: degree, interests, values, motivations and skills.
  • Lecture/workshop (1 hour): Who might employ you and how, when and where to make contact with potential employers?
  • Workshop (2 hours): The skills that employers want in new graduate-level recruits
  • Workshop (2 hours): Team-working roles and skills (students to complete a team-working test e.g. The Belbin Test, or an equivalent, before this session)
  • Workshop (2 hours): The job application process
  • Workshop (2 hours): The job interview process
  • Workshop (2 hours): Effective networking
  • Workshop (2 hours): The value of work experience
  • Workshop (2 hours): The work experience search
  • Workshop (2 hours): Preparing for work experience, politics, fitting in, asking for help/information
  • Seminar (2 hours): Feedback/question and answer session
(20 hours in total)

Teaching methods
Delivery type Number Length hours Student hours
Lecture 10 2.00 20.00
Independent online learning hours 18.00
Private study hours 62.00
Total Contact hours 20.00
Total hours (100hr per 10 credits) 100.00

Private study:
20 hours: Students will be expected to spend 1 hour following each lecture/workshop completing the relevant sections of a career development workbook/flow chart.
18 hours independent online learning: Students will be expected to complete online: self-awareness tests, a teamworking-related test e.g. the Belbin test and CV and online job application form; they will also be expected to use the University websites, including LeedsforLife, skills@thelibrary and Leeds University Union websites to help with identifying opportunities for their own employability skills development.
42 hours: Students will be expected to:
a. Visit the University Careers Centre and learn about its website and other resources that could help them with their future work experience/career/life plans e.g. learn from guidance materials re. CV writing, interview skills and how to prepare for assessment centres.
b. Research potential employers online, on paper and in person via careers or networking events or fairs and via their own personal networks too
c Explore who might offer them work experience (shadowing, an internship or placement) or an actual job in the future
d. Develop targeted job applications and prepare fully for course related presentations, interviews, assessment centres and networking opportunities.
Progress monitoring

Student attendance at lectures will be monitored in line with standard faculty practice.

To ensure satisfactory progress throughout the module: Students progress/performance on formative and summative tasks will be monitored by academics during and after each workshop session and any struggling students identified so that additional advice/support can be provided where necessary:
  1. Formative tasks and worksheets will be devised for each of the workshops and students will be provided with immediate constructive feedback on their performance either by their peers and/or the group academics/facilitators.
  2. Post-workshop online quiz scores will be monitored by academics. Students will receive immediate online feedback based on their responses to the quizzes. However, struggling students will be identified from their poor scores and provided with additional feedback/advice where necessary.
  3. Start and end of module improvements in CV writing, LinkedIn profile completion, interview and presentation techniques will be monitored by peers and academics.
  4. Students will be required to spend time monitoring/reflecting on their own progress each week by completing a careers direction workbook based on workshop content, online revision and private reading sessions. The workbooks will also be monitored by academics firstly in week 3. At the end of the module. General feedback will be provided too.
  5. Students will be encouraged to seek help outside of the module too and perhaps to sign up to the alumni and development mentoring scheme so that they can ask for help from people perhaps already working in a role in which they are interested in getting moving towards.
Module manager and lecturers will be notified of students failing any formative or summative tasks/assessments, so that additional support can be provided. Conditional release of support material, based on completion of each week's post-workshop online test or workbook worksheet, will ensure student involvement.

Methods of assessment
Coursework

Assessment type Notes % of formal assessment
Oral Presentation Group presentation; individuals to work in groups of 4 and all to contribute equally to the final presentation 35.00
Reflective log Students to reflect on their own career/life plans via completion of a flow chart or career direction/skills development workbook 25.00
Online Assessment Online job application (to include completed CV and personal statement) 40.00
Total percentage (Assessment Coursework) 100.00




BIOL2223 Employment, Career planning and Professional Development for Life Scientists
Module summary
This module will equip students to explore their next career steps by introducing them to the careers available to bioscience graduates, the skills employers are looking for and how to achieve them. It will provide an opportunity for students who have not yet developed their career plans to reflect on their future goals, hone their employability skills and develop the necessary skills to seek and compete for work experience, internships and graduate employment. Comments from students who completed the module include…. "The opportunity to reflect on personal qualities and employability is invaluable. It has helped me realise what I need to do" "It was really useful. I definitely feel more prepared for job applications" "I would recommend it very highly. It’s made me think early what I want to do". The module will provide advice on when and how to start looking for work or work experience plus making applications and succeeding at interviews for work. The module will take students on a journey, starting with the need for career and life planning, to exploring what an ideal graduate is and exactly what employers expect of today’s graduates, through to the career choices, routes and opportunities currently open to graduates and bioscientists in particular. During the module students will learn about and develop the range of skills (team working, networking, seeking opportunities, presentation skills, commercial awareness) needed to successfully secure and succeed in the world of work. They will also learn how to evidence these skills. Students will learn about the importance of building up personal and employability skills in addition to bioscience-related qualifications and knowledge. Students will also learn how to prepare applications and be interviewed for their job/career of choice to help them towards successfully progressing into the world of work. Students will learn about securing work/work experience via networking and ‘the hidden jobs market’. Whatever their future work experience or career plans this module aims to help students move forward into their first internship and/or graduate job/role.

Objectives
On completion of this module, students should be able to reflect on their future career/life plans, to prepare for work and/or work experience, plus to learn and develop the key skills and attributes that are expected by graduate employers in today’s world of work – be that work in academic or other research and development laboratories, in a business environment or in the field.

Learning outcomes
On completion of this module students should be able to:
  1. Devise and maintain a career plan, identifying the skills, opportunities and activities needed to meet their plan
  2. Reflect, research and report on their own career interests, values and motivations with respect to at least one employment setting
  3. Understand what is meant by the term an ideal graduate and know who employs bioscientists after graduation
  4. Describe the key skills that employers look for in new graduate recruits
  5. Identify their own current employability skills strengths and weaknesses
  6. Identify opportunities to develop their employability skills during their time at Leeds
  7. Identify common team roles and how they themselves fit best within a team
  8. Be able to establish effective working relationships within a team
  9. Write and maintain a curriculum vitae and personal statement/covering letter
  10. Complete a job application form
  11. Prepare for a job interview and/or assessment centre
  12. Understand what makes a good presentation
  13. Present themselves positively in a job interview and/or assessment centre
  14. Learn how to network effectively in person and online
  15. Be able to reflect on their own progress and development e.g. delivery and reflection on their own presentation
  16. Feel more confident about being able to secure future work opportunities (including work shadowing, short-work internships, longer work placements, graduate employment)
  17. Manage in a professional manner relationships with others in a work environment
Skills outcomes
Students will:
  • Learn how to plan their next career/life steps and to reflect and build on their plans.
  • Recognise the value of self-reflection in career/life planning (e.g. students will be able to reflect on their own interests, values, motivations, skills and the contributions they make) and learn to feed their reflections forward to influence their own career/life plans and development.
  • Learn about the employability skills and professionalism that are required in today’s workplace.
  • During their own time, learn how to self-assess and rank their own employability skills strengths and weaknesses.
  • During their own time, identify and engage with curricular and extra-curricular activities that will help them to develop the bioscience and employability skills that today’s employers require.
  • Learn about teams and team-working and identify how they might fit and work best within a team/project group and articulate (in writing) their progress in team working and the contribution they make.
  • Learn how to positively present themselves to others in writing (e.g. via a CV, personal statement or online job application) and in person, via networking, a more formal interview/assessment centre or presentation
  • Learn about the hidden jobs market and how to network and positively communicate their skills and career plans to others, in person and in writing and practice working effectively in a team.
  • Learn to work alone to gather and present written information related to their future career plans and the company/employer they wish to apply to.
Syllabus
  • Lecture/workshop (1 hour): Career/life planning: what are your next steps - what is an ideal graduate, what careers are open to graduates and bioscientists in particular, what factors can influence your career/life plans e.g.: degree, interests, values, motivations and skills.
  • Lecture/workshop (1 hour): Who might employ you and how, when and where to make contact with potential employers?
  • Workshop (2 hours): The skills that employers want in new graduate-level recruits
  • Workshop (2 hours): Team-working roles and skills (students to complete a team-working test e.g. The Belbin Test, or an equivalent, before this session)
  • Workshop (2 hours): The job application process (including presentation skills)
  • Workshop (2 hours): The job interview process
  • Workshop (2 hours): Effective networking
  • Workshop (2 hours): The value of work experience
  • Workshop (2 hours): The work experience search
  • Workshop (2 hours): Preparing for work experience, politics, fitting in, asking for help/information
  • Seminar (2 hours): Feedback/question and answer session
(20 hours in total

Teaching methods
Delivery type Number Length hours Student hours
Lecture 10 2.00 20.00
Independent online learning hours 18.00
Private study hours 62.00
Total Contact hours 20.00
Total hours (100hr per 10 credits) 100.00

Private study:
20 hours: Students will be expected to spend 1 hour following each lecture/workshop completing the relevant sections of a career development workbook/flow chart.
- 18 hours independent online learning: Students will be expected to complete online: self-awareness tests, a team working-related test e.g. Profiling for success and CV and online job application form; they will also be expected to use the University websites, including LeedsforLife, skills@thelibrary and Leeds University Union websites to help with identifying opportunities for their own employability skills development and to use iDECIDE for their own career planning.
- 42 hours: Students will be expected to:
a) Visit the University Careers Centre and learn about its website and other resources that could help them with their future work experience/career/life plans e.g. learn from guidance materials re. CV writing, interview skills and how to prepare for assessment centres.
b) Research potential employers online, on paper and in person via careers or networking events or fairs and via their own personal networks too
c) Explore who might offer them work experience (shadowing, an internship or placement) or an actual job in the future
d) Develop targeted job applications and prepare fully for course related presentations, interviews, assessment centres and networking opportunities.

Progress monitoring
To ensure satisfactory progress throughout the module: Students progress/performance on formative and summative tasks will be monitored by academics during and after each workshop session and any struggling students identified so that additional advice/support can be provided where necessary:
  1. Formative tasks and worksheets will be devised for each of the workshops and students will be provided with immediate constructive feedback on their performance either by their peers and/or the group academics/facilitators.
  2. Start and end of module improvements in CV writing, LinkedIn profile completion, interview and presentation techniques will be monitored by peers and academics.
  3. Students will be required to spend time monitoring/reflecting on their own progress each week by completing a careers direction workbook based on workshop content, online revision and private reading sessions. The workbooks will also be monitored on an on-going basis and formally (as part of the assessment) at the end of the module. General feedback will be provided too.
  4. Students will be encouraged to seek help outside of the module too and perhaps to sign up to the alumni and development mentoring scheme so that they can ask for help from people perhaps already working in a role in which they are interested in getting moving towards.
Module manager and lecturers will be notified of students failing any formative or summative tasks/assessments, so that additional support can be provided. Conditional release of support material, based on completion of each week's workbook worksheet, will ensure student involvement.

Methods of assessment
Coursework
Assessment type Notes % of formal assessment
Group Project Team working 0.00
Presentation Group presentation (groups of 4) and individual reflection/log of the team work and their own contribution to the process and outcomes. All members to contribute equally to the final presentation 35.00
Reflective log Students to reflect on their own career/life plans via completion of a flow chart and skills development workbook on a weekly basis 25.00
Assignment Job application (to include completed CV and personal statement) and 1000 word review of company /employer 40.00
Total percentage (Assessment Coursework) 100.00








Monday, December 09, 2013

Can you trust MCQs?

Blackboard logo With rising student numbers and increasing pressure on feedback turnaround times it is almost inevitable that we will see the increased use of online MCQs, at least for formative if not for summative assessment. But can you trust MCQs?

I recently stumbled across Blackboard Grade Centre Item Analysis Discrimination. No, I hadn't seen this before either.


So is this the answer to construction of good (valid) online MCQs?

Blackboard's (sketchy) description of how the tool works is that "Discrimination correlates question performance with overall performance on the test". Hang on - if the categories are not independent, is this analysis valid? The Discrimination score tells you whether good students (those who score highly on the test) do better than poor students, or whether poor students (who get low scores on the test) answer the question correctly. Is this a valid basis for judging the value of a question? In my hands, the tool also fails to calculate scores for some questions for reasons not given. So is it better than nothing? Well, possibly. Is it a valid analysis of question utility? You decide.

Get ready for lots more MCQs in the coming years. And keep lying awake at night worry about using MCQs. In the future, we will all be Sebastian Thrun.




Thursday, December 05, 2013

BioSET project update

BioSET logo Time for a quick update on our BioSET project.

We have had two meetings recently. The first one on 13th November was a progress update on employability case studies. Much of the discussion at this meeting centred on the pressures on final year students. As a result, it was decided to widen the number of participants by inviting any interested second year students to join the project.

On 4th December we met again and welcomed a number of second year students who were interested in finding out more. One of the outcomes from this meeting was the decision to hold a blogging workshop on 11th December, so I'll be writing more after that.

Recent blog posts from participants:

The first Case Study and Pioneering Xerte territory

Xerte Web Design and Science Writing Case Study Part I





Tuesday, December 03, 2013

Why *do* I still blog?

Thinking The month's crunching of blog stats coupled with a bit of end of year retrospection reveals a familiar pattern of declining readership, which is mirrored almost exactly by an increase in followers on various social media sites. This pattern is occurring on not just one but many established blogs written by different people on many different topics, so it's a general trend, which now prompts the question, is it still worth spending the time to blog?

Although there is some value in simply writing for writing's sake without thinking about a readership, for most blogs, having clear goals and objectives makes the time consumed by blogging more worthwhile. If your goals are measured purely in terms of eyeballs, then the answer may well be that blogging is no longer worth while. But I have always regarded this blog as a sort of online notebook, and the readership and discussion which comes by virtue of carrying out the activity publicly, although welcome, is almost incidental. As a personal repository, this blog is worth more to me than my sharecropping activity on social networks, although that is also valuable.

For my longest running blog, readership has now declined to a fraction of its former level, and if this were the only criterion, writing it would no longer be worth while. But the fact is that my publication schedule forces me to read the current research literature in that area, something that would almost certainly get squeezed out if I stopped writing it. One of my guiding principles in blogging is to add value to whatever I am discussing, not simply reblog or point at information. Blogs such as the Economist Explains are beacons in that regard, and I attempt to emulate them. That surely has to be a worthwhile activity.

The Guardian notes that most academics are blogging for professionals peers, rather than for the public in any general sense. The same article also notes the growing risk of being a publicly-identifiable academic blogger. This is a further part of the diminishing returns for blogging.

Is blogging still worthwhile? Just, but it's a close run thing.


See also:



Monday, December 02, 2013

Mobile phones make you unhappy and stupid

Mobile Or do they?

Usual caveats apply, i.e. only one study, but more importantly consider cause and effect - do mobile phones make students unhappy or do unhappy students use mobile phones more .... ?


The relationship between cell phone use, academic performance, anxiety, and Satisfaction with Life in college students. (2014) Computers in Human Behavior, 31, 343-350
While functional differences between today’s cell phones and traditional computers are becoming less clear, one difference remains plain – cell phones are almost always on-hand and allow users to connect with an array of services and networks at almost any time and any place. The Pew Center’s Internet and American Life Project suggests that college students are the most rapid adopters of cell phone technology and research is emerging which suggests high frequency cell phone use may be influencing their health and behavior. Thus, we investigated the relationships between total cell phone use (N = 496) and texting (N = 490) on Satisfaction with Life (SWL) in a large sample of college students. It was hypothesized that the relationship would be mediated by Academic Performance (GPA) and anxiety. Two separate path models indicated that the cell phone use and texting models had good overall fit. Cell phone use/texting was negatively related to GPA and positively related to anxiety; in turn, GPA was positively related to SWL while anxiety was negatively related to SWL. These findings add to the debate about student cell phone use, and how increased use may negatively impact academic performance, mental health, and subjective well-being or happiness.





Tuesday, November 26, 2013

What does ratemyprofessors.com actually rate?

"For almost a century, attempts have been made to evaluate the teaching effectiveness of instructors in higher education. Much of this effort has centred on an attempt to create instruments that would allow students to make this assessment. These instruments have varied over both place and time, and have created one of the longest lasting debates in higher education. ... One source stated that there were close to 3000 articles published on student evaluation of teaching (SET) from 1990 to 2005 alone. Published findings on the topic are so voluminous that many researchers have been using the method of meta–analysis, in which the case is not a subject but an entire published article. Nevertheless, little agreement has been reached on key points. ... One of the difficulties of studying SET has been finding adequate and appropriate large samples, because of the confidential and anonymous nature of the inventories for both students and instructors. An alternative has become available online. Increasingly popular databases, such as ratemyprofessors.com, uloop.com, koofers.com, myEdu.com and studentsreview.com, offer large and easily accessible sources of data. Although these sources are tempting, a question remains about their validity. ... ‘Does ratemyprofessor.com really rate my professor?' "

ratemyprofessors.com - sponsored by Wonga

What does ratemyprofessors.com actually rate? (2013) Assessment & Evaluation in Higher Education doi: 10.1080/02602938.2013.861384
Abstract: This research looks closely at claims that ratemyprofessors.com creates a valid measure of teaching effectiveness because student responses are consistent with a learning model. While some evidence for this contention was found in three datasets taken from the site, the majority of the evidence indicates that the instrument is biassed by a halo effect, and creates what most accurately could be called a ‘likeability’ scale.


More on this exciting field of research





Sunday, November 24, 2013

The murky world of academic anayltics

Tinfoil hat I've been involved in a number of discussions this week about the use and abuse of metadata. The reactions have been mixed. Many people seem unaware of the potential dangers in this area, some of them who clearly should be better informed. Others are sanguine about the prospects. The final group have their tinfoil hats clamped securely over their ears - obvious nutjobs.

Or are they?

Before you decide, read this recent article in The Atlantic: Your Job, Their Data: The Most Important Untold Story About the Future.

One of the things I have been thinking about this week is how much metadata is captured by academic systems such as Blackboard, Echo360 and Panopto. Most people seem to be blissfully unaware of this, or simply don't care. Stephen Walker's post Lecture Capture and Data Capture did a good job of raising the key issues. We know what use commercial services such as Amazon, Facebook and Google make of this data - but what about academia? Is "Do No Evil" the watchword or do uses such as performance management - of students or staff - take precedence?

There is an easy way to deal with this problem - clear and transparent institutional policies on these issues. But more often than not, policies are lacking, either through omission or commission, resulting in loss of trust. The University of Manchester Policy on the Recording of Lectures and other Teaching and Learning is exemplary in this regard and should be followed by all institutions:
3.2 Prior to the beginning of each Semester, teaching staff will be informed that recordings will be made if their teaching and learning activities take place in a location containing automated lecture capture technology. Staff not wishing to have their sessions recorded should respond to this stating 'Opt out'.

3.6 Recordings will not be used for staff performance management purposes.

Such clear policies allow both staff and students alike to concentrate on learning rather than spending time wondering what Big Brother will do with their personal data.


Update: Helpful link via Martin Hawksey:
Prinsloo, P., & Slade, S. (2013, April). An evaluation of policy frameworks for addressing ethical considerations in learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 240-244). ACM.



Wednesday, November 20, 2013

#ITFocusWeek - What's wrong with "lecture capture"? This:

Today I'm taking part in a Question Time panel about lecture capture technology. I'm cast comfortably in the role of curmudgeonly old git who doesn't like it. The reality is slightly more complicated.

YouTube

Everyone likes online videos. I do, that's why my YouTube channel is closing in on 1.5 million views. At least, I like good online video. But what does that mean? It certainly doesn't mean the lowest common denominator of talking head video - plonking yourself in front of a webcam and droning on because that's the easiest thing to do. And it certainly doesn't mean any video longer than five minutes. All of which means that the idea of "lecture capture" is wrong headed.

Alexa.com reports that people average 17 minutes per day on YouTube and watch 12 videos, that's 85 seconds per viewing. Here's a graph which shows YouTube's own data for "audience retention", i.e. the length of time people pay attention to online videos before stopping or skipping ahead. What do you think this data says about the concept of "lecture capture"?

YouTube Audience Retention

Ah, you say, but people sit on their sofas watching David Attenborough with rapt attention for an hour. Well first of all, you're not David Attenborough. Second, you don't have 1% of 1% of the budget or the time that the BBC or Sky spent on making that program. But most important of all, you don't understand the difference between lean forward and sit back media. The interactivity of an integrated control bar in online videos banishes the passivity of watching linear video on a big screen such as a television and encourages use of fast forward skipping. Online, all attention spans are short.

The Atlantic recently ran a piece titled Lectures Didn't Work in 1350—and They Still Don't Work Today. It's hard to know where to start listing what's wrong with this article. First, it's not about higher education or about lectures at all, it's about primary and secondary education where lectures (rightly) don't feature as a mode of instruction as far as I'm aware. But even if you apply it to higher education, the whole piece conveniently ignores the rather obvious fact that lectures have indeed worked since 1350 - that's why students pass rather than fail. It is simple minded twaddle to simply ignore this. And in an age where technology increasingly threatens to distance us from each other, the physicality of the lecture room increases its impact and its value. Sitting in front of a recording erodes that valuable link between staff and students.

So what's my problem with lecture capture technology? The very name. The idea that a lecture can be captured and bottled is laughable and shows a fundamental misunderstanding. This technodeterministic language cements the idea of technology replacing rather than augmenting pedagogy, the wrong-headed notion that disruptive technologies kill older technologies. This is demonstrably false - TV didn't kill radio, and radio didn't kill newspapers, etc. Instead, in the real world we have the Lindy Effect - the longer a technology has been around, the longer we can expect it to survive. Lectures are here to stay, so the question is, how do we use new technologies to augment and improve lectures, not to replace them?

And that's where the use of technologies such as Echo360 and Panopto come in useful. We use them to produce short (less than 5 minute - look at the graph) high impact videos covering the threshold concepts that students struggle with. And we use those recordings to augment and improve the vital lecture experience. Give 'em the old TED razzle dazzle. More importantly, we make these technologies available to students and we challenge them to make their own videos explaining threshold concepts. Instead of sitting passively in front of recorded lectures students become active partners in transmitting knowledge. And we stop pretending that technology solves problems. Technology doesn't solve problems, people do. But technology can cause problems, such as increasingly passive learning. Let's not let that happen. Let's encourage the use of online video. Let's stop talking about lecture capture.

But don't take my word for it:




Tuesday, November 19, 2013

What happens when an economist measures teaching quality?

... probably not what you're expecting.

"Although assessment exercises are nowadays a widespread practice in educational systems, we lack an effective understanding of what is the actual feedback effect on the overall efficiency (macro level) of the system and on the individual level (micro level). Given the lack of any sort of incentive to perform better in teaching activity, I formulated the hypothesis that this effect is null (at least for the Italian academic system), implying that the overall quality of teaching which is supplied does not change over time. Even worse, I claimed that the same is true at the individual level. ... This study ... shows that the evaluation of teaching quality might not have been conducive in improving it. On the one hand, it is possible that feedback from the questionnaire was not used effectively, and on the other (much more plausible) that the incentive system is not working properly. This last argument might entail dramatic consequences for the Italian academic system (and in general for European education systems), which tends to be increasingly research-oriented and neglects teaching duties. At this point one crucial question remains open: does it make any sense to continue carrying out teaching quality assessment when it does not enhance teaching performance? The tentative answer is ‘no’, at least until some incentives are introduced for providing better teaching or a more efficient teaching performance management is established."



Bianchini, Stefano. Feedback effects of teaching quality assessment: macro and micro evidence. Assessment & Evaluation in Higher Education (2013): 1-15. doi: 10.1080/02602938.2013.842957
Abstract: This study investigates the feedback effects of teaching quality assessment. Previous literature looked separately at the evolution of individual and aggregate scores to understand whether instructors and university performance depends on its past evaluation. I propose a new quantitative-based methodology, combining statistical distributions and transition probabilities matrices, to take into account the dynamics of teaching quality over time both at the macro and the micro level. Using a three-year longitudinal panel from an Italian university, it is shown that evaluation exercises do not impact future teaching performance at either the university level or the individual level.



Monday, November 18, 2013

Blogging 2013

The LSE Impact of Social Sciences blog asked me to write a guest post about the blogging session at #solo13. So I did, and they have just published this:



As academic blogging becomes mainstream, science communication must facilitate depth and breadth in online discourse.

The SpotOn London conference, formerly called Science Online London, has just run for the sixth time. Held at the British Library and sponsored by Digital Science, The Wellcome Trust and a number of other high tech, high profile sponsors, SpotOn London (or #solo13 as it is more widely known) is both the flagship event for current science communication and collaboration and represents the cutting edge of developments in the way science is being conducted. The three official strands at the meeting, science communication and outreach, online tools and digital publishing, and science policy, reflect the interest and enthusiasms of those involved.

I have been fortunate enough to be involved with all six Science Online London meetings. Although I was not physically present at the first Science Blogging conference at the Royal Institution in 2008 I spent a day following the event online - my first exposure to how new media were changing academic life. The feeling of excitement around the emerging tools - Friendfeed (precursor of the present day Facebook interface), Twitter, informal video livestreams (from people's phones and of varying degrees of reliability), and aggregation of discussions from blogs, Flickr, etc - had sufficient impact to shape my thinking about and approach to online communication, as well as my own academic practices.

As a grizzled veteran, #solo13 inevitably didn't have the same impact for me that some of the earlier meetings did. This is just part of the natural cycle of exposure - adoption - commonplace with emerging technologies. What was noticeable at #solo13 was the emergence of a younger demographic of PhD students and science communicators experiencing it all for the first time. I don't intend to try to cover all the content and discussion from this packed two day event in a single post, but I would like to describe some of my reactions to the "Using blogs and other online forums for communication between scientists" session held on the second day. You can watch a video recording of the whole session online:


As I commented during the session, I'm a serial blogger who couldn't stop if I wanted to. In fact, the only major change I could envisage in my blogging practice might be to obscure my identity when writing about certain topics - more of this later. I currently maintain five active blogs. My personal blog, Science of the Invisible http://scienceoftheinvisible.blogspot.com , is where I write about my educational research and professional life, but also share more personal (but not intimate) content such as recipes and commentary. MicrobiologyBytes is my oldest continuous blog, and writing about microbiology online helps me stay up to date for my teaching and my personal interest in science communication now that I am officially no longer "research active" in this field (which is a reflection of grant income rather than the end of my scholarship). I am the Internet Consulting editor for Annals of Botany, responsible for social media policy, and our blog is very much a team effort aimed at promoting the content in the journal, but also awareness of plant science to a more general audience. @leBioscience is an online shop window, the 21st century equivalent of the staff newsletter we used to distribute in print 20 years ago. My student support blog is both a content management system and an authoring tool for advice and guidance to the students I teach, and this growing personal repository of information saves me endless amounts of time by allowing me to refer frequently asked questions to this resource rather than to have to answer each one anew. In addition to this core, project blogs and grant blogs come and go with fixed lifespans as dissemination tools and archives for particular projects. You get the idea....

With this history, it was hard for me to resist a gentle smile at the panel member who introduced themselves with the line "I've been blogging for two months", but at the same time, it was encouraging that there is more of an acceptance of blogging as an important academic tool. Unlike the Science Blogging 2008 meeting, there is now a feeling of establishment rather than revolution about blogging - part of the process of scholarship, but particularly with regard to individual development rather than as an institutionally-owned process. Few academic institutions actively encourage their staff to blog and there remains a justified wariness of the possible consequences of being judged to have frittered away valuable time which could have been "better" spent on traditionally mainstream academic activities. Across a wider sector, Marketing still retain control of blogging in many organizations and we have still to fully realize the benefits that can arise from reflective cross-institutional communication and knowledge exchange which can arise from a blogging culture. At a simple level, it would be easy for institutions to encourage more staff to blog by simply providing the tools - for example, a site-wide installation of Wordpress available to all. Moving beyond that to a culture where blogging is the norm rather than is much more complex and not as easy to achieve. In a time of financial stringency, senior academic management will need to provide the lead in such matters in order to enable this sort of change.

There is also something of a pervasive feeling of weariness around blogging. There appear to be some bloggers who don't enjoy the process but do it out of a sense of obligation or not wanting to miss out. Blogging is still a peripheral rather than a core (paid) activity for most. For me, blogging is fun (although not always enjoyable, and not as much fun as teaching, with which it overlaps) - blogging is not my day job, so it's something I'm often doing in the early hours of the morning or in snatched minutes between meetings if I can get my brain in gear. If the reward is not financial nor much institutional credit for most, what keeps academic bloggers going? One common experience at the meeting was of colleagues and students who read your blog secretly, then come up to you at some point and say "I read your blog". It doesn't count as REF impact, but this peer acknowledgement is a significant personal reward. If your institution does not provide the platform, other sites such as the Huffington Post, The Guardian and New Statesman all accept guest posts, and if you have the time and talent to get your writing accepted by such sites it will certainly attract attention within your institution and from future employers.

The issue of commenting on blogs was also discussed in the session. For most blogs, comments are dead and any conversation is now based around is now retweets and reposts to social media sites rather than a threaded stream on the original site. The same is also true of online commenting on scientific papers - on the PLOS and Nature websites for example. There is plenty of discussion going on about the content, but it is distributed across many networks rather than being focussed into an easily accessible thread. In spite of a relaxed attitude to this by some, in my opinion this is a problem as it encourages superficial commenting at the expense of more in depth conversation. Sharing on Facebook is a good thing but it is not a substitue for a good discussion thread on a thorny issue. In science in particular, putting your head above the parapet and being seen to criticise the work of senior scientists is still a risky business, even when such comments are made in neutral tones and intended as a positive contribution to ongoing work.

Overall, I came away from this session feeling buoyed up by the discussion. A blog is not a book or a newspaper, but it is more than a Twitter account or a Facebook page. Blogs are still the centre of serious online academic communication but there is still a long way to go until the Republic of Blogs is established and academic blogging moves from being a spare time activity to a mainstream output. I'm delighted to have been asked to contribute this guest post to the LSE Impact blog and I intend to expand my guest blogging activity as time permits. Sleep is over rated anyway.






Wednesday, November 13, 2013

Why Are We Bowling Alone?

Bowling Alone The most influential book I have read in the past 10 years is Robert D. Putnam's Bowling Alone (Simon and Schuster, 2001. ISBN 0743219031). I'm aware of some of the criticism which has been leveled at Putnam's work, but for me, as for many others, Bowling Alone provides a description of the society I see around me. The problem I've always had, and which Putnam does not fully address, is why has there been the erosion of social capital that he describes?

Last weekend I read Thinking, Fast and Slow by Daniel Kahneman (Penguin UK, 2012. ISBN 0141033576). Kahneman's book does not have the impact of Putnam's and while not undermining the overall corpus of his work, comes across as rather superficial in many ways. But one aspect that Kahneman discusses definitely struck a chord, the issue of money priming:
“Reminders of money produce some troubling effects. Participants in one experiment were shown a list of five words from which they were required to construct a four-word phrase that had a money theme (“high a salary desk paying” became “a high-paying salary”). Other primes were much more subtle, including the presence of an irrelevant money-related object in the background, such as a stack of Monopoly money on a table, or a computer with a screen saver of dollar bills floating in water. Money-primed people become more independent than they would be without the associative trigger. They persevered almost twice as long in trying to solve a very difficult problem before they asked the experimenter for help, a crisp demonstration of increased self-reliance. Money-primed people are also more selfish: they were much less willing to spend time helping another student who pretended to be confused about an experimental task. When an experimenter clumsily dropped a bunch of pencils on the floor, the participants with money (unconsciously) on their mind picked up fewer pencils. In another experiment in the series, participants were told that they would shortly have a get-acquainted conversation with another person and were asked to set up two chairs while the experimenter left to retrieve that person. Participants primed by money chose in the exto stay much farther apart than their non-primed peers (118 vs. 80 centimeters). Money-primed undergraduates also showed a greater preference for being alone.”
Reading this instantly bought Bowling Alone to my mind. I spent a happy/unhappy 24 hours reflecting on this new knowledge (depending on how you want to look at it). Sadly, the next day I read Michael Kraus's post, A Social Priming Finding with Direct Replications, which is another take on the recently discussed "Replication Problem" in psychology.
"Social priming researchers are faced with the public perception that our field lacks methodological rigor. In the past, the dominant reaction has been to simply downplay these concerns and assure people that our findings replicate."
Together with the rather glib tone of Thinking, Fast and Slow, this has somewhat undermined my confidence that I have finally found a reason we are bowling alone. But I can't shake the gut feeling that affluence must be at least part of the explanation for the accelerating enclosure of public goods and the continued decline of social capital.

 

Tuesday, November 12, 2013

Time is of the essence

ScienceGist I don't think there was a single session at #solo13 where time was not mentioned as an obstacle to getting more of what we want.


ScienceGist (tagline: "Simplifying science") aims to demystify scientific papers by publishing easily understandable summaries. The project is the outcome of a hackday earlier this year. It's very early days for ScienceGist, and there's a limited range of content there at present, for example, Plant Biology such as this.

I like the idea of ScienceGist a lot. But there's a problem. The site is in competition with Wikipedia, reddit, Buzzfeed, etc, and compared to those sites it is a minnow. Brain Kelly quoted this yesterday:

"as Cameron Neylon had said in the “Wikimedia UK Annual Review 2012-13″:
“If you’re serious about ensuring public engagement in your research then you need to make damn sure your work can be incorporated into Wikipedia. Wikipedia is the most important engagement channel for your research."
(The Wikipedia Editing Workshop Session at the SpotOn 2013 Conference)

So far I have only reposted content I have written for other sites. Since both are published under a CC license this isn't a problem, but if I wasn't already writing this content for another site, would I have time for ScienceGist, good as it is?

One thing I've learned about curation is to find a model that works for me and not to try to do everything. In cutting back to a single channel, popularity has to be a key feature. I really like ScienceGist, but they've chosen a hard furrow to plough.



Monday, November 11, 2013

Badge Sex at #solo13

Blendology badges Yes, that's got a "ge" on the end of it. And if your name is Badge, this post isn't what you think (I hope).

At #solo13, Blendology (rather than Bellendology which I originally typed) supplied interactive badges that you could tap to show who you had interacted with and which sessions you attended. In addition to uncertainty surrounding the protocol of staring at ladies chests while trying to read their name badges, since the lanyards were a little long, these badges introduced the additional element of trying to introduce a device into the groin area of someone you've never met before. Certainly an ice breaker. Stephen Curry came up with the idea of "badge sex", reminding me of the epidemiology practicals we run for students. (Before too many people apply to do degrees at this university, it should be noted that no actual sex is involved in these practicals, although cherries are.) The outcome of all this tapping is a digital record of of interactions - which someone else memorably referred to as Science Grindr. Here's mine:


Although better than cardboard business cards, I'm not sure my individual data is of much use - in the sense that it doesn't add a lot to knowing someone's Twitter handle. Some of the names I don't recognize are Blendology staff "pre-taps". Hmm, I understand why they want to do that but it doesn't do much for data integrity. The overall collected data is probably much more interesting - were strands distinctive or isolated, which sessions were the most/least popular, etc.

Inevitably there were concerns raised about privacy. As far as I'm concerned, I made no secret of the fact that I was at this event by blogging and tweeting about it, so I don't have a problem with the data being published. The badges were opt-in at this event, and since privacy has a lot to do with choices, I have no great concerns.

So is this the future of conferences? Nah, let's just RFID microchip everyone to see who eats the most cake. 




#solo13 versus #solo14

I'm back from the SpotOn/Science Online conference where I facilitated a workshop on dark social:

Slides | Storify

This was my fifth Science Online conference in a row, and while it didn't have the impact for me of some of the previous events, it was still well worth while. Noticeably, the next generation have arrived, which is good. At the same time, SpotOn/Science Online feels like it is in need of a refresh and it is time to try something new to capture the impact of the earlier events.

Where would I like to see SpotOn go next? Ironically, I'd like to see it go all the way back to its 2008 roots and become much more focused online. The online component was the revolutionary element of the first event, but it is now standard for all major conferences. I would like to see the SpotOn conference as a one day midpoint/highlight of a month long online event.

It would be a huge mistake to try to capture all the activity in one site. Rather the focus should be on encouraging contributions across a distributed network of media - blogs, Twitter, Google+ hangouts, YouTube, ScienceGist, Wikipedia, etc, etc - conference as aggregator. Two weeks of online activity, with themes, campaigns and targets, one day f2f event (a sort of AGM), then another two week follow up to capitalize on the momentum rather than dissipating it all the Monday morning after the meeting. Build an online community of mutual support.

What say you?




Saturday, November 09, 2013

SpotOn London 2013: The Dark Art of Dark Social: Email, the antisocial medium which will not die #solo13

I'm at SpotOn London 2013 today, doing this:

The Dark Art of Dark Social: Email, the antisocial medium which will not die
Email newsletters go back to the dawn of the Internet but have been neglected in recent years with the rise of shiny social networks. But eTOCs have seen off RSS and email will not die. Why else does Twitter encourage users to share tweets via email? The evidence is clear that message dissemination via social networks only reaches a relatively small proportion of the potential audience and that most link sharing occurs via private channels such as email (hence “dark social”). The Silk Road bust and the unmasking of Dread Pirate Roberts reveals how difficult it is to maintain anonymity online. Public discussion of science by scientists, e.g. via online comments, is limited by the high risk nature of the putting your head above the parapet in the hierarchical merit structure of science. Yet discussion of published science via private channels continues unabated. The rebirth email on mobile devices and the contribution of social media to information overload is causing a rethink about the utility of public versus private channels and we are seeing the rebirth of email newsletters and mailing lists. This workshop looks at the case for and against dark social, practical aspects such and tools such as how to use blogs as content management systems for email generation and asks you to embrace the dark side.




Wednesday, November 06, 2013

The Great Email Opportunity #solo13 #solo13dark

Email At #solo13 I'm running a workshop on email: The Dark Art of Dark Social: Email, the antisocial medium which will not die. Yesterday I gave a short presentation about using email with students, but I've been thinking about email for the past year as part of my current interest in Dark Social. As a result, I was interested in a new post on AllThingsD, The Big Email Opportunity.

I don't particularly want to be in the position of being the public apologist for email, but I don't agree with Nick Morris that email is evil - although you could make a case that some of the people who use it are. More charitably, I think most of them are misguided rather than evil, and if high powered academics don't know the difference between Reply and Reply To All, who am I to correct them? What I particularly like about Robert Abbott's post is that it shifts the focus away from users onto the software. He is absolutely right to point out that email systems have not moved on since the 1990's, with the worst offended of all being Microsoft Outlook. (Man, it feels so retro bashing Microsoft).

A little dab of A.I. and almost all of what people feel to be the "problems" of email go away. Short of that, decent software that reminds the user to include a subject line (helpfully suggesting a suitable offering from the context of the thread) and automatically attaches the correct attachment for them would be an improvement. (Yeah, remember the 90s? That's how it was kids.)  Email has to change, and the sooner the better. Long Live Email! It ain't going away, so let's get it right.




Tuesday, November 05, 2013

Students and Email

I'm just about to give this talk as part of our monthly PedR meeting. I'll be giving a more research-focused version of this at Solo13 on Saturday. 

Video:




Monday, November 04, 2013

Getting Evo-Devo: Concepts and Challenges for Students

Getting Evo-Devo

Useful article for anyone involved in teaching these fundamental concepts.
"Evo-devo content presents students with new conceptual challenges and potential difficulties in attempting to understand evolution. For example, while several evo-devo concepts rely on the supporting concept (SC) of conserved gene networks that operate in a variety of developmental contexts, many students hold that each trait of an observed phenotype is the result of the expression of a single gene. Improving strategies for teaching evo-devo will benefit from an inventory of concepts appropriate for undergraduates, a learning progression toward their mastery, and a description of their attendant conceptual difficulties."

Hiatt, A., Davis, G.K., Trujillo, C., Terry, M., French, D.P., Price, R.M., & Perez, K. E. (2013) Getting to Evo-Devo: Concepts and Challenges for Students Learning Evolutionary Developmental Biology. CBE-Life Sciences Education, 12(3), 494-508.
Abstract: To examine how well biology majors have achieved the necessary foundation in evolution, numerous studies have examined how students learn natural selection. However, no studies to date have examined how students learn developmental aspects of evolution (evo-devo). Although evo-devo plays an increasing role in undergraduate biology curricula, we find that instruction often addresses development cursorily, with most of the treatment embedded within instruction on evolution. Based on results of surveys and interviews with students, we suggest that teaching core concepts (CCs) within a framework that integrates supporting concepts (SCs) from both evolutionary and developmental biology can improve evo-devo instruction. We articulate CCs, SCs, and foundational concepts (FCs) that provide an integrative framework to help students master evo-devo concepts and to help educators address specific conceptual difficulties their students have with evo-devo. We then identify the difficulties that undergraduates have with these concepts. Most of these difficulties are of two types: those that are ubiquitous among students in all areas of biology and those that stem from an inadequate understanding of FCs from developmental, cell, and molecular biology.





Friday, November 01, 2013

Mobile phone images and video in science teaching and learning

Mobile me The educational use of mobile devices has had some attention recently in terms of recording field work and also as voting devices, but considering that the majority of students now own these powerful handheld computers, it is surprising that we have not given them a more central place in education. For all out ego-centric technodeterminism, mobile technologies have had far more impact in developing countries than here. Which is why it is not surprising to see papers such as this, but how long will it be until we stop trying to control technology in education and bring it fully into play?


Mobile phone images and video in science teaching and learning. (2013) Learning, Media and Technology doi: 10.1080/17439884.2013.825628
This article reports a study into how mobile phones could be used to enhance teaching and learning in secondary school science. It describes four lessons devised by groups of Sri Lankan teachers all of which centred on the use of the mobile phone cameras rather than their communication functions. A qualitative methodological approach was used to analyse data collected from the teachers' planning, observations of the lessons and subsequent interviews with selected pupils. The results show that using images and video captured on mobile phones supported the teachers not only in bringing the outside world into the classroom but also in delivering instructions, in assessing students' learning and in correcting students' misconceptions. In these instances, the way the images from the mobile phone cameras supported students' learning is explained using a variety of approaches to understand how images support learning.







Tuesday, October 29, 2013

The network as proxy

Cover A new paper in BJET discusses the use of social network analysis as a proxy for academic performance. Unfortunately, it doesn't refer to our earlier work in a this area (Badge, J. L., Saunders, N. F., & Cann, A. J. (2012). Beyond marks: new tools to visualise student engagement via social networks. Research in Learning Technology, 20).


Reviewing the differences in size, composition and structure between the personal networks of high- and low-performing students. Br J Educ Technol (1 November 2013), doi:10.1111/bjet.12110
Abstract: An interesting aspect in the current literature about learning networks is the shift of focus from the understanding of the “whole network” of a course to the examination of the “personal networks” of individual students. This line of research is relatively new, based on small-scale studies and diverse analysis techniques, which demands for more empirical research in order to contextualize the findings and to meta-analyze the research methods. The main objective of this paper is to review two research questions posed by a previous British Journal of Educational Technology contribution by Shane Dawson in order to know whether the differences in personal network composition impact on the performance of students. The two questions were defined by Dawson as follows: (1) Are there significant differences in personal network composition between high- and low-performing students? and (2) Do high-performing students have larger personal networks than their low-performing peers? In addition, the “clustered graphs” method used in this study allows the inclusion of the structural analysis of personal networks. In doing so, a new research question is addressed: (3) Are there significant differences in personal network structure between high- and low-performing students? This paper tries to answer these questions in the context of two undergraduate, inter-university and fully online courses, and two different technology-enhanced learning environments (a virtual learning environment and a personal learning environment) where interactions took place indirectly through shared resources. The results show that the network behaviors of high- and low-performing students' are strongly correlated, and that high-performing students developed larger personal networks than low performers.


The analysis in this paper (wisely) stops at correlation. But correlation is enough to get you into trouble.
"I spent 24 hours last weekend at the Isle of Wight literary festival, only in its second year but already thriving. I went to a session on spies with the writers Roger Hermiston and Professor Richard Aldrich. Aldrich has written a fascinating book about GCHQ. He said that so much of what we do now is on the electronic record that we live in a world that offers neither secrecy nor privacy. You might imagine we still have a secret ballot. Yet analysis of our buying patterns in the supermarket can, these days, reveal how we vote to a likelihood of 87%-90%." Guardian
I think it's fairly clear where learning analytics is heading, from the descriptive to the presumptive. At a time when I'm considering deciding what to buy from Sainsbury's based on a random number generator, I'm not sure what I think about these developments, except that I know I'm not sanguine about them. I believe this sort of data has real societal value, whether it be catching terrorists or supporting student learning, but the present situation we are slipping into is unacceptable. Yesterday I spent an hour interrogating Blackboard to extract data about submissions to pre-lab quizzes. I was interested in how far in advance of the practical classes students had submitted their answers, but the data buried in Blackboard tells me much more than this - not only facts such as IP addresses but far more subtle patterns. Even a simple visual inspection without software tools reveals that the majority of (but not all) students submit their answers to the quizzes multiple times. Why? Because having submitted once and seen the feedback, they then resubmit (sometimes multiple times) to ensure the mark recorded by Blackboard is as high as possible. We could infer all sorts of things from observing this behavior (such as the fact that students are engaging with the assessment part of this exercise rather than the feedback). This is the pattern followed by most students, but what about those students who only submit once, then move on - how should we respond to them? And most importantly, are these students aware of the information that their online activity patterns are revealing? Exactly how evil are the NSA, GCHQ, Blackboard?

There is a way to square the circle. Transparency. Unlike the present situation, we need to make people much more aware of what data is being collected and the implications of this. That's where the NSA went wrong. We in higher education should not follow the same path.





Monday, October 28, 2013

Grokking the visual web

BuzzFeed I've put a fair amount of effort into getting under the skin of the visual web over the past few years. I've played with Pinterest, without success, and put this failure down to the largely non-visual nature if much of what I work on. But now that the visual web is a billion dollar trend driven by mobile devices, giving up is simply not an option.

So I'm trying again, and this time it's BuzzFeed. If you haven't encountered BuzzFeed, that's because you're old. BuzzFeed is like reddit for .... even younger people. And BuzzFeed is the hottest property online right now by some distance. When ze Frank went to BuzzFeed I didn't get it. But now I do. Yet BuzzFeed vexes me. So much trivia. And yet, when you dig a little deeper, so much potential. Strip away the froth and BuzzFeed is the place to be. Why? Because visuals drive the mobile web (typing on phone keyboards is so last year), and BuzzFeed is about visual if it's about anything. But dig a little deeper. Longform has real substance, fact checking and debunking the media, or the subtle political. Still think it's all froth? Well there's science there too. But not just science, BuzzFeed is pioneering new formats, such as reporting on scientific papers in the style of a teen romance magazine photo story. Everybody wants in. Since BuzzFeed declared "The Internet loves a list", even the Times Higher is getting in on the act.

For Open Access Week, Alun wrote 6 Discoveries You Can Read In Open Access Journals. (Yes, please up vote it!) And now it's my turn, so I've been trying to grok BuzzFeed the only way I know, by immersion:
The Wisdom of The Wire
Why The Wire (2002-2008) is the greatest TV series ever made.

You Spent Money On What? 11 Improbable Scientific Papers [NSFW]
Science is a laff. No, really, it is. Millions of pounds get spent on research into… improbable events. Here are some of my favourites.

Even such trivial immersion as this has given me the vital insight into BuzzFeed that I needed. And I like it. Why? It feels exactly like Twitter did when I first started using it, i.e. entirely trivial at first sight, but when you scratch the surface, you realize the power of the interface and what it is capable of. And the main thing is it capable of is exploiting the mobile device driven visual web. I recommend it to you.

So will I be using BuzzFeed on a regular basis. No. Here's why: I find it difficult to compose in the visual medium. I am happy to consume visual media but in terms of output I'm a words person. Something like Medium.com suits my style much better, for example, Medium's take on the visual web, or indeed, this post. Which is a shame, because Medium is rather backward looking in terms of current technologies. Or possibly, like Ev William's previous offerings, Medium is so far ahead of its time in the vanguard of the backlash against social media that we just can't see it yet.